Commit Graph
6432 Commits
Author SHA1 Message Date
Ryan Houdek fc00a31aee Frontend: Remove errant header 2023-04-15 18:37:53 -07:00
Ryan Houdek 27c03f98d8 Merge pull request #2580 from Sonicadvance1/glibc_fault_ci
CI: Adds glibc faulting testing
2023-04-15 15:07:09 -07:00
Ryan Houdek bfd606ec3d Merge pull request #2552 from Sonicadvance1/glibc_docs_and_ci
Docs: Adds programming concerns documentation
2023-04-15 03:36:21 -07:00
Ryan Houdek 9823a64164 CI: Adds glibc faulting testing
This is based on our regular CI runner file with a couple of things
stripped out.

- gvisor tests removed because they cause our CI machines pain with
  tmp/shm mounts
- Thunks disabled since glibc fault testing is incompatible with it
- ARMEmitter tests removed since we don't want to test vixl.

All glibc conversion PRs will need to be merged before this and then
this needs to be rebased.
2023-04-15 03:18:09 -07:00
Ryan Houdek 72483ea21d Docs: Adds programming concerns documentation
Explaining FEX's memory concerns.
2023-04-15 02:29:31 -07:00
Ryan Houdek 1a91d849f0 Merge pull request #2591 from Sonicadvance1/new_jemalloc
Add in jemalloc glibc hooking again
2023-04-14 13:32:11 -07:00
Ryan Houdek 6d4cef723a Merge pull request #2595 from Sonicadvance1/fix_thread_cancel_fault_test
Threads: Disable glibc allocator fault testing with `exit`
2023-04-14 13:24:04 -07:00
Ryan Houdek 1c2fd72c84 Merge pull request #2590 from Sonicadvance1/drm_v6.2
Update drm headers to v6.2
2023-04-14 13:20:07 -07:00
Ryan Houdek 77f2378080 docs: Adds jemalloc document 2023-04-14 13:16:22 -07:00
Ryan Houdek 1cc9f2107d Add in jemalloc glibc hooking again
We still need to hook glibc for thunks to work with
`IsHostHeapAllocation`.
So now we link in two jemalloc allocators in different namespaces.

As usual we have multiple heap allocators that we need to be careful about.

1. jemalloc with `je_` namespace.
  - This is FEX's regular heap allocator and what gets used for all the
    fextl objects.
  - This allocator is the one that the FEX mmap/munmap hooks hook in to
     - This mmap hooking gives this allocator the full 48-bit VA even in
       32-bit space.

2. jemalloc with `glibc_je_` namespace.
  - This is the allocator that overrides the glibc allocator routines
  - This is the allocator that thunks will use.
  - This is what `IsHostHeapAllocation` will check for.

3. Guest glibc allocator
  - We don't touch this one. But it is distinct from the host side
    allocators.
  - The guest side of thunks will use this heap allocator.

4. Host glibc allocator
  - #2 replaces this one unless explicitly disabled.
  - Always expected to override the allocator, so this configuration
    isn't expected.

Already tested this with Dota Underlords to ensure this works with
thunks.
2023-04-14 13:16:22 -07:00
Ryan Houdek b979b339fc Threads: Disable glibc allocator fault testing with exit
FEX might not ever make it to its cleanup routines when exit is used. So
make sure to disable the fault checking if exit is used.

Opened #2594 to track this investigation.
2023-04-14 13:10:14 -07:00
Ryan Houdek 46e5343a0e External/drm: Update to v6.2 2023-04-14 13:07:52 -07:00
Ryan Houdek d519883dbe StructPackVerifier: Pass on Aligned attr 2023-04-14 13:07:52 -07:00
Ryan Houdek beb2c36fc2 drm: Define __user define which drm headers use. 2023-04-14 13:07:52 -07:00
Ryan Houdek f45ea1e0f6 Merge pull request #2600 from Sonicadvance1/fix_stringconv_allocate
StringUtils: Stop allocating TrimTokens
2023-04-12 03:50:20 -07:00
Ryan Houdek 92593162b0 StringUtils: Stop allocating TrimTokens
Use a string_view instead of a fextl::string so this stops allocating
memory (stack in the cases I have seen).

Fixes #2562
2023-04-12 03:34:58 -07:00
Ryan Houdek 307158d425 Merge pull request #2601 from Sonicadvance1/move_fexloader
FEXLoader: Move to Tools folder
2023-04-12 03:33:02 -07:00
Ryan Houdek 5c62ea21f4 Merge pull request #2598 from Sonicadvance1/stop_leaking_avx
FEXCore: Stop leaking AVX configuration state
2023-04-11 19:57:58 -07:00
Ryan Houdek 73caa6725f Merge pull request #2597 from Sonicadvance1/stop_sending_uninitialized_data
FEXServerClient: Insert missing padding in message packet
2023-04-11 19:57:50 -07:00
Ryan Houdek b0b23abedd StructVerifier: Update include path 2023-04-11 16:56:53 -07:00
Mai f55a653e99 Merge pull request #2599 from Sonicadvance1/andn_usage
OpcodeDispatcher: Move usages of `And(Not(` to Andn
2023-04-11 19:53:44 -04:00
Ryan Houdek 4cb35d2f5e Fix header includes from the move 2023-04-11 16:44:06 -07:00
Ryan Houdek 7e4334c98a FEXLoader: Move to Tools folder
This has confused people for quite a long time thinking these are unit
tests.

First step, move it without any changes. Only Renamed.
2023-04-11 16:37:21 -07:00
Ryan Houdek 0d0b99f344 OpcodeDispatcher: Move usages of And(Not( to Andn
Fixes #2199

Very few uses actually, we were pretty good at this already.
2023-04-11 15:35:12 -07:00
Ryan Houdek 44e06185b7 FEXCore: Stop leaking AVX configuration state
The dispatcher was saving AVX state even though FEX doesn't support it
currently. This is due to it checking for the config option rather than
the HostFeatures option.

The `EnableAVX` config option is supposed to be used to inform FEXCore
if we want AVX disabled or not when the host supports the feature. In
this case it is universally enabled because we haven't encountered any
games that have issues with AVX state being saved with signals. (We know
they exist, we just don't have configurations for them).

The HostFeatures option `SupportsAVX` is the option that is supposed to
be getting used for determining if the runtime AVX feature is enabled.
This also had an issue though that this was **also** always enabled if
running on an x86 host with AVX, or an ARM host with SVE2-256bit.
It was then disabled if the config option was disabled; But, since
FEX-Emu doesn't support AVX fully yet, we need to ensure this isn't yet
enabled.

But this only solves half the problem. In order for our CI to test AVX
features before fully supporting AVX, it needs to be able to enable AVX
so that the CPU state is correctly saved.

So we need to change the default configuration option to be false, and
have CI enable it for the tests that matter before AVX is fully
implemented.
2023-04-11 15:21:32 -07:00
Ryan Houdek 3f89cf1512 FEXServerClient: Insert missing padding in message packet
This was sending uninitialized data across the wire.
2023-04-11 14:27:40 -07:00
Ryan Houdek 18154183ad Merge pull request #2585 from Sonicadvance1/jemalloc_ptr_cost
Allocator: Remove pointer indirection overhead
2023-04-11 10:47:24 -07:00
Ryan Houdek 49abe8afb5 Allocator: Remove pointer indirection overhead
Every time we are calling a function in `FEXCore::Allocator::` this is a
pointer indirection. Which means on x86 it is always a `call [rdi]` and
on AArch64 it is a `ldr x17, [x0]; blr x17;`.

Instead of doing this, use inline functions in the header that call the
correct allocation function directly. This function gets inlined and is
no longer an indirect call.

When compiling with jemalloc, we forward declare the jemalloc function
definitions so we don't have to pull in the entire jemalloc interface in
to the public header definitions.
2023-04-11 10:29:35 -07:00
Ryan Houdek 7916281ee7 Merge pull request #2593 from Sonicadvance1/cpp_optparse_update
cpp-optparse: Update to latest optparse
2023-04-11 09:25:28 -07:00
Ryan Houdek d7bc0370ee Merge pull request #2592 from Sonicadvance1/remove_fwrite
fextl::fmt: Remove fwrite usage
2023-04-11 03:32:06 -07:00
Ryan Houdek c5da0e7ac1 Merge pull request #2596 from Sonicadvance1/fix_stringconv_handlers
StringConv: Convert to conversion functions that don't use std::string
2023-04-11 03:31:58 -07:00
Ryan Houdek 0e007d2724 StringConv: Convert to conversion functions that don't use std::string
`std::stoul` and `std::stroull` take a std::string which was converting
the string_view to a std::string first. Causing glibc fault testing to
catch this since not much uses this.

These will be added to the documentation.
2023-04-10 18:56:05 -07:00
Ryan Houdek 63b31d54c4 cpp-optparse: Update to latest optparse
Changes std::set and std::map usage over to fextl.

I missed pushing this change before.
2023-04-10 18:12:18 -07:00
Ryan Houdek 466edf7744 fextl::fmt: Remove fwrite usage
fwrite allocates some backing memory for buffering outputs which FEX
can't track.

Switch to using `fileno` to get the fd from the FILE and write directly.
This will need to be changed for llvm-mingw support but that will come after
this.

This will be added to the documentation that we can't use fwrite.
2023-04-10 17:43:47 -07:00
Ryan Houdek 0bb59ade53 Updates jemalloc
Needed to change some symbol names due to proper jemalloc namespacing
now.
2023-04-10 16:21:33 -07:00
Ryan Houdek c03ed529e6 Merge pull request #2582 from Sonicadvance1/ccache_thunks
Thunks: Enable ccache if available
2023-04-10 15:57:23 -07:00
Ryan Houdek f806ca688c Merge pull request #2564 from Sonicadvance1/more_glibc_allocations
More glibc allocation removals.
2023-04-10 15:57:05 -07:00
Ryan Houdek 48531e1dd2 FEXLoader: Use the std::FILE print handlers again
Just to match prior behaviour.
2023-04-10 15:40:51 -07:00
Ryan Houdek 1864a1d3b5 fextl::fmt: Add print with std::FILE* handler
This is to match prior behaviour, but untested if fwrite itself
allocates any memory so far.
2023-04-10 15:38:55 -07:00
Ryan Houdek e98a46aa5f Review comments 2023-04-07 17:01:53 -07:00
Ryan Houdek 96e9b5cf51 FEXServer: Fixes some missing fextl usage
Not sure how this was working when the type declaration didn't match.
2023-04-07 17:01:53 -07:00
Ryan Houdek 265c918d90 Move fextl::String_from_path to the only usage in FEXConfig
Ensures that people won't be tempted to use this elsewhere.
2023-04-07 17:01:53 -07:00
Ryan Houdek 76a5e4e66e ELFContainer: Removes some c_str 2023-04-07 17:01:53 -07:00
Ryan Houdek dce6389d87 FHU: Convert relative/absolute check to string_view 2023-04-07 17:01:52 -07:00
Ryan Houdek 46b306e861 Config: Remove to_string usage 2023-04-07 17:01:52 -07:00
Ryan Houdek 32d7fae373 GdbServer: Convert to_string usage 2023-04-07 17:01:52 -07:00
Ryan Houdek 47e5096676 Syscalls: Remove usage of std::string 2023-04-07 17:01:52 -07:00
Ryan Houdek f25b0461d0 HarnessHelpers: Remove std::to_string usage 2023-04-07 17:01:52 -07:00
Ryan Houdek 11402b637a fextl: Remove now unused string_from_string 2023-04-07 17:01:52 -07:00
Ryan Houdek a9c27646a0 FEXRootFSFetcher: Remove usage of fextl::string_from_string 2023-04-07 17:01:52 -07:00