There's no longer a distinction between AArch64 and x86 and everything
effectively falls under "Common" now. This means flattening the entire
structure just cleans it up.
NFC. (Although instcountCI will update because of a couple pointer
offsets changing)
Fixes crash in thunks that use callbacks, introduced in #5148.
The dispatcher would call the syscallhandler to get the VDSO thunk
callback. But due to reordering initialization, the VDSO thunk would
have not been loaded at that point. This would cause thunks that use
callbacks to crash with a nullptr exception.
Instead, defer the thunk callback pointer loading until the thread
starts executing, and load the pointer in to our thread state's pointer
struct instead.
Didn't get caught in my initial test sweep since I didn't run a Wine
game with thunks.
This wasn't handling negatives correctly which was causing xalia.exe to
assert. Disable for now rather than further changing logic, with a TODO that it
should get fixed in the future.
Steam started taking advantage of newer fcntl commands, in particular
`F_CREATED_QUERY`. This was causing our fcntl handler for 32-bit
processes to assert out.
Fixes the handler so it passes all other fcntl commands forward, just
like the Linux kernel does. Splits the 32-bit handler to explicitly
ignore a handful of commands just like the Linux kernel.
Fixes an assert that Steam was hitting.
Prevent suspension when flushing thread state to the WOW64 context
during BTCpuSetContext as suspension itself performs a flush. If
suspension were to occur halfway through FlushThreadStateContext then
the fact that the WOW64 context has been dirtied would be missed and the
new context would be partially overwritten with stale data from the FEX
thread state. Additionally if the context were to be dirtied by
suspension at any point during context merging in BTCpuSetContext the
input context could be lost.
Fixes occasional crashes when starting AC4.
Handling this safely requires blocking all compilation for the duration
of the read, as fault-based tracking doesn't work when wine's unix side
itself is the one performing the write into the RWX region (we cannot
catch unix faults).
Fixes a startup crash in Persona 5.
We have run out of EmulatorData slots on ARM64EC, so introduce this for
less-frequently accessed TLS data for which the additional cost doesn't
matter.
Execstack enables `READ_IMPLIES_EXEC` and we were accidentally
decoupling our expected persona due to the reordering of initialization.
This was causing the smc-1-dynamic test to fail because it's testing
execstack and our VMA tracking wasn't returning read-write VMAs as
executable once these decoupled.
Easy fix.
This allows the context and parent thread objects to be created earlier,
allowing the VDSO and ELFCodeLoader mapping functions to have a thread
object for tracking memory mappings through the regular guest routines.
This means that we no longer need to do any form of deferred handling
for code caching as all the state is ready early in the initialization
process.
A little bit of care needed to be taken to ensure we still close the
ELFCodeLoader's FDs later and that VDSO unmapping happens before tearing
down the parent thread, but overall this is mostly just passing the
InternalThreadState object around as normal.
I couldn't find any functional regression from this change alongside
code caching, but it would be good for @neobrain to double check this.
FEX argument will no longer exist in `/proc/<pid>/cmdline` after this
PR. Ensure this tool still works.
After #5097 is merged, one can use regular `pidof` for Linux
applications, but this is still useful as it finds Wine applications
running with FEX as well.
Theoretically, another thread could run after the reprotection but
before the invalidation, write some code and jump to it but end up
jumping to old code as the invalidation is yet to occur. Fix this by
locking the invalidation mutex, invalidating and only then reprotecting.
Opcode handlers are written with the assumption that LoadSource will not
touch flags and this would be an annoying assumption to change. As this
is such an edge case anyway just don't defer flags and force a load of
the saved value before _TelemetrySetValue (which are implicitly saved
before it).
Fixes the following snippet in upc.exe:
AND word ptr [ESP + ECX*0x1 + 0x80000000],DX
BTR CX,DX
ADC CX,word ptr SS:[EAX + ECX*0x1 + 0x80000000]
Windows will populate Size/Address with those of the underlying free
region. This is important for cross-process free operations, where the
the 'before' callback is still called after the memory is freed so
querying the size of the section (which is not freed) fails.
For now, a simple directory structure with filename based mapping is
used:
LOCALAPPDATA/cache/<main image id>/{<dll 1 image id>, <dll 2 image id>}
All cache blobs are loaded once at the start for simplicity, this will
also be useful in the future as metadata validation for settings etc can
be done just once.
Before compiler/architecture checks, we check:
- `sizeof(void*) == 8`: 64-bit systems have an 8-byte void pointer,
whereas 32-bit systems (should) have a 4-byte pointer; since 32-bit
hosts are completely unsupported might as well check for it just in
case
- Only Windows and Linux are supported, so let's add an early check for
that as well.
Signed-off-by: crueter <crueter@eden-emu.dev>
I may have gotten carried away.
- I missed some stuff for end parenthesis because I accidentally
searched within project files instead of the entire directory (so some
thunk/test/windows stuff was missed), cleaned those up.
- `INTERFACE`, `PUBLIC`, `PRIVATE`, `RUNTIME`, `LIBRARY` should be on
the same line as the target name. (I should really invest in making a
style guide...)
- Some short statements were unnecessarily split across multiple
lines--cleaned those up
- Made a common `LinkerGC` module that applies gc-sections etc. to a
target in Release mode
- Usually for functions you want to have something on the first line,
e.g. `FILES`/`DIRECTORY` for install, or the target/a positional
argument, etc etc. Not always though, notably for some custom_command
calls
TODO:
- What's with the `list(APPEND LIBS...)` stuff? It's used really
inconsistently, sometimes not at all, sometimes it looks like there're
duplicates? A more thorough cleanup is in order there.
Signed-off-by: crueter <crueter@eden-emu.dev>
Most constants don't need to be padded for relocations. So now that
these have all been audited, switch to defaulting to NoPad to reduce
verbosity.
The number of constant that need to be explicitly padded are now marked
and with all the prior changes, this allows bisecting if something has
gone wrong.
- Do compiler/architecture checks EARLY, don't waste time doing random
configuration stuff if the user can't even compile in the first place
- MSVC is unsupported, I assume? So add a check to disallow. There's
literally no MSVC or MSC_VER checks anywhere, so...
- Rather than using the MSVC architecture definitions, use our own
`ARCHITECTURE_arm64` et al. Hijacking existing "standard" definitions
is a very bad idea. Also makes it more readable in CMake
- Change the x86 host check to `x86|amd64`. Some systems still refer to
themselves as x86 despite being 64-bit for... reasons, and I saw one a
very long time ago that referred to it as amd64. This should
basically never come up, nor is it really relevant given that FEX is
for arm64... but it kinda annoyed me so whatever.
TODOs:
- Should we check `CMAKE_SIZEOF_VOID_P (equal) 64`? I don't think anyone
is even trying to compile this thing on armv7 or older, but might as
well? maybe?
- What's the status of *BSD, Solaris, macOS? Technically macOS does
support Wine, not sure about the others.
Signed-off-by: crueter <crueter@eden-emu.dev>
Just my brain spinning as I try and determine what is causing some
hanging. Seems to be WINE specific so might not even be in FEX code.
Good to have some more documentation so when I read this again I don't
need to make some more logic deductions.
Port of #5159
Find modules are preferred for pkgconfig/otherwise non-CMake libraries.
Let's use that here for xxHash.
Signed-off-by: crueter <crueter@eden-emu.dev>
Trivial variables like SRCS, NAME, etc. actually do more harm than good.
They *will* make your IDE mad, and are also less readable. Remember:
verbosity is not a bad thing! Usually
Also: did a few tiny cleanups that I missed from my `endpara` PR.
Signed-off-by: crueter <crueter@eden-emu.dev>
Command names shouldn't be uppercase. A lot of LSPs scold you if you do
this, and apparently it's the official recommendation from the CMake
team to absolutely never, ever do this.
Signed-off-by: crueter <crueter@eden-emu.dev>
The Race:
1. A Reader sets `READ_WAITER_BIT` (Bit 15) and sleeps on the High 16 bits (`Futex+2`).
2. Writer A unlocks. It clears `READ_WAITER_BIT` (in Low 16 bits) and `WRITE_OWNED` (in High 16 bits).
3. Writer B immediately steals the lock. It sets `WRITE_OWNED` but preserves the now-cleared `READ_WAITER_BIT`.
4. The Reader, checking `Futex+2`, sees `WRITE_OWNED` is set. Since it cannot see that Bit 15 was unset (as it is watching High 16 bits), it assumes its wait signal is still valid and sleeps.
5. Writer B unlocks. It sees no `READ_WAITER_BIT` and wakes nobody. Deadlock.
The Fix:
Move `READ_WAITER_BIT` to Bit 30 (High 16 bits).
Now, when Writer A clears the flag, the High 16 bits change value which will prevent the wait from occurring within WaitForAddress
- Some CMake LSPs have aneurysms when you put the end parenthesis on a
different line. Annoying? Yes, but this is all we can really do about
it for now.
- `set`, `option`, and `message` should not have spaces before their
opening parenthesis.
Signed-off-by: crueter <crueter@eden-emu.dev>
- `AddObject` doesn't need an additional Type parameter since it's
already assumed to be an object library
- The static and shared libraries don't need any explicit compile
options, as the object already handled this.
- They also don't need to be linked to FEXCore_Base. Object libraries
already handle that for us since the symbols already get pulled in
anyways.
- The object library doesn't need an output name. Only the user-facing
libraries do
Signed-off-by: crueter <crueter@eden-emu.dev>
Rather than individually adding `${CMAKE_SOURCE_DIR}/Source` as
an include directory to each target, just use `include_directories` once
in the Tools directory and each subsequent target will have this
propagated down.
Also removed a seemingly unnecessary `-I` in LinuxEmulation--maybe
needed? But I can't test compilation right now as I don't have an ARM
development environment on hand for the next day or two.
Signed-off-by: crueter <crueter@eden-emu.dev>
CMake has had the `MINGW` builtin to describe MinGW targets since at
least version 3.2, so it can safely be used. This variable is also set
for the MSYS2 environments, so CLANGARM64 also correctly sets `MINGW`.
Note that this depends on https://github.com/FEX-Emu/jemalloc/pull/11.
Signed-off-by: crueter <crueter@eden-emu.dev>
Windows caches can only really work safely if they are exclusive to a
prefix due to the state for things like sharing modes that is tied to
the per-prefix wineserver and codemaps encoding prefix-local paths for
system DLLs.
Recent CPUs do nop fusion with the following instruction, this gives the
CPU the best chance to do fusion with something that actually does work.
Very trivial, doesn't do this for the more complex handling below these
as counting the number of moves before nop emitting is messy.
Instead of just a trivial pad being on or off, support a tri-state
on/off/auto where on will always pad, off will never pad, and auto will
pad only if code caching is enabled.
Further augment this by allowing a byte-width to be passed in, which can
be used with pointers to force a 48-bit VA width to only ever pad to
three instructions, reducing the common worst-case situation from 4
instructions to 3. This works because we're not going to expose a VA
width larger than 47-bit to the guest.
Fixes the handful of use-cases that explicitly chose their NOP padding,
and a bug in Arm64Relocations.cpp where it was incorrectly asking to not
receive padding even though it requires it.
and use it to commonise volatile metadata code (adding WOW64 image
voltmd support in the process). This will be extended to track mapped
images for code caching.
When relocations are loaded, all immediates read from memory are checked
against the relocation map and transformed into an appropriately
sign-extended entrypoint-relative variant of the specific operands
addressing mode. As almost every case of an unhandled relocation will
lead to a later, likely harder to debug, crash at runtime just bail out
early if any such cases are encountered. Note that while this
handles/detects all cases of relocated immediates, if relocations were
applied to instructions themselves (occurs in some malware variants)
these would be missed without any errors reported.
When compiling code at runtime there is no harm to including jumps to
different sections within a multiblock, when enforcing as such would
introduce a lookup cost for every decode invocation (or some caching).
However when compiling offline as each cache blob is tied to a specific
library these boundaries should be enforced.
In order to support code caching of 32-bit libraries, any library-base
relative relocations on the guest must be transformed into FEX
relocations so e.g. absolute jumps or loads refer to the correct
location when the library is loaded at a different base address.
I noticed that we weren't actually testing 32-bit displacement on
64-bit. This is an edge case on 64-bit where you can have 32-bit
displacement WITHOUT it being RIP relative. This is very uncommonly used
but is technically a possibility where 64-bit code can access an
absolute address in the lower 2GB.
With negative addresses you could technically access canonical kernel
addresses in the lower 2GB if they were mapped in userspace, but since
they aren't this is untestable (FEX also doesn't handle canonical kernel
addresses correctly anyway).
`MemoryData.asm` accidentally tested this but it didn't test both load
and store sides. So let's be a bit more stringent on this.
When loading code caches, these constants get patched up for the new guest
address. The new value may be larger than the original, so the padding bytes
ensure the maximum of 16 bytes of encoding space is always available.
Closing the original stdout is how we signal to a larger compatibility
tool (in practice pressure-vessel) that either we are ready, or we have
crashed: when every process that held the write end open has closed it,
the compatibility tool sees EOF on the pipe. If the FEXServer inherits
stdout and holds it open, then that state will never be reached.
steamrt/tasks#870
Signed-off-by: Simon McVittie <smcv@collabora.com>
The ELF tracking thing has an expectation that only portions of ELF
files that are described in the program headers will be mapped
executable. This doesn't hold true as programs will remap random
portions of ELF files as executable. In the case that this occurs, don't
assert out and instead print a warning.
This was discovered as Node.js remaps a portion of itself executable
that isn't described as such in the program headers. I also have a local
unittest that exposes the same problem. I had discovered same problem in
some other program with #5038.
Also fixes a bug where sometimes completely anonymously mapped
executable sneak in and cause a crash, which is kind of silly.
Always enables FEXServer log thread when built for Steam so that clients
can be controlled with `STEAM_FEX_LOG=1`. FEX logs will then always go
to FEXServer and those can get directed to wherever pressure-vessel
chooses.
This is fairly simple. Needs to be installed alongside FEXServer, so
that it can start it in a portable config.
- Starts a FEXServer
- Tells pressure-vessel when FEXServer is ready
- Keeps FEXServer alive with the `watch_fd` as long as the process lives
- Listens for pressure-vessel to be shutting down
- Exits once pressure-vessel exits, also letting FEXServer shutdown if
no FEX instances are alive.
When the realpath of a program path can't be resolved, we weren't setting
the config option. This was cascading to be a crashing in codemaps where
it was unconditionally using the optional value (with assert checks),
and causing things to crash.
Pass in the path that can't be resolved to work around a crash in PV
that can happen.
This was a requested feature. To make sure that FEXServer is running and
managed by a parent process, we need to have a way to tell FEXServer to
keep alive without any FEX clients. The best way to do this is to pass
FEXServer a Pipe (like FEX does when a client starts it), but instead of
FEXServer signaling to FEXInterpreter that it's ready. FEXServer listens
to the pipe to see if the management process is still alive.
The expectation here is that the management process passes FEXServer the
read end of a pipe, and when the management software is done (or gets
killed by the kernel!) then the write end of the pipe is closed, and
FEXServer naturally closes (As long as there's no FEX processes
remaining).
Checks to see if the package is installed first before trying to use it.
Fixes an issue where fresh users don't have this package installed and
the script fails.
This will be useful to fix the mutex locking mess that occurs currently
when forks occur. Instead of needing to be /very/ meticulous with many
futexes, we can instead have working threads shared_lock this one, then
when a fork occurs just only have the forker themselves unique_lock and
let the readers drain out. Since it's write-priority it'll happen quite
quickly, letting the fork get in and out relatively easily.
This is going to take some massaging to get the frontend and FEXCore to
a place that this works but we can get this simple change in early.
Saves power and responds faster. Pass in the atomic to `WaitPred` with
the predicate checking if the buffer has been flushed yet. Same
behaviour as previous code but more efficient on our hardware.
This simplifies the loop a bit and moves the non-predicated exact
matching version to use the predicated version.
We will need a predicated version for the next commit.
Default to enabled because this is the config we expect by default.
In the future will get some benchmarking in various games like
Assassin's Creed, and Call of Duty.
Managing code maps in FEXServer rather than in FEXInterpreter makes it
easier to handle multiple concurrent processes sharing code caches for
the main executable and libraries.
A few games were generating "Can't handle adddress size".
I implemented 0x67 prefix handling for CMPSOp and SCASOP and improved
the error messages for the remainder. This will implement the address
modifier on 64bit systems, and keep issuing an error on 32bits.
Shared code buffer support introduced the concept of having a single
GuestToHostMaps shared across many threads. In the common case all
threads will share one however if e.g. a resize recently occured and
specific thread is yet to compile any code with the new codebuffer it
will still use the old GuestToHostMap. The current invalidation
approach handles this by repeatedly calling erase for every single
thread's GuestToHostMap, even if it is repeated. An accumulator is used
to ensure when two threads share a map, the L1/L2 cache entries in the
second thread will still be invalidated even if the the iteration for
the first thread removed them from the map.
Unfortunately this is incredibly slow in cases with many threads, as
a significant number of redundant map lookups and L1/L2 cache erasures
on threads that never even observed a given block can occur. Solve this
by introducing a two-pass model:
- First, all active codebuffers (and their associated GuestToHostMaps)
have their entries invalidated for the given range, these codebuffers
are tracked internally within FEXCore. It is at this point that delinking
callbacks are ran.
- Second, each thread will have its caches invalidated. But rather than
naively invalidating the L1/L2 caches for every invalidated block for
every thread, threads now track on their own what specific entries
have been potentially fetched into their L1/L2 caches. This is
aided by GuestToHostMap now tracking the pages each block touches. (an
inverse CodePages so to speak).
We currently rely on the frontend to keep track of threads and then
iterate over all threads to perform per-codebuffer operations. However
as codebuffers are shared between many threads (the common case is a
single code buffer across all) this ends up being inefficient. Introduce
a list of codebuffers to solve that (new codebuffers are very rare, so a
vector is plenty fine here for erasing invalid weak refs).
The sender might provide all requested message bytes but no FD. The receiver
interface has no simple way of indicating this scenario yet, so just assert
out for now to ensure it never happens in the first place.
If needed, this can be changed to return a new error code to indicate partial
read in the future.
This allows using read_some for incoming messages with an optional FD.
Doing so fits the purpose of read_some more closely, which is to read *any*
non-empty amount of data.
Adds it to the VDSO handling, it's not necessarily a VDSO function but
it behaves as such as it is in every single process. This means we get
to reuse the mapped page for every process when thunks are built,
shaving a page out of 32-bit processes.
Also, fixes a bug in guest VDSO symbol loading where clang sticks all
symbols in to `.dynsym` where gcc sticks them in to `.symtab`. Search
both. This effectively meant the couple of guest VDSO symbols were
always failing to get found, causing us to allocate yet another page on
32-bit. So effectively three pages stolen.
This also means we can remove the Linux specific X86HelperGen stuff from
FEXCore, only passing a single "VDSO" function pointer to the backend
for the dispatcher. Once again moving the Linux stuff to the frontend is
good.
Fixes an assert about about untracked noexec code `NoExec
instruction in entry block: FFFFE000` whenever thunk callbacks were
used.
Fixes a bug in guest VDSO symbol loading where clang sticks all
symbols in to .dynsym where gcc sticks them in to .symtab. Search
both. This effectively meant the couple of guest VDSO symbols were
always failing to get found, causing us to allocate yet another page on
32-bit. So effectively three pages stolen.
Peeled out of #5039
FEX_CONFIG_OPT can only be used as a standalone statement, which is
inconvenient for config values that are only used once. The new functions
(e.g. Get_DUMPIR()) can be used in conditions or other expressions.
Enables memcpy optimization of 80bit floats on reduced precision.
Also uncovered a bug where if we had done 80bit memcpy
optimization, we wouldn't have properly stored the 80bits.
This was caught by the existing tests when we enabled the optimization.
This is going to be necessary for the offline compiler work.
Also allow FEXGetConfig to print the same registers in the correct
format for easy fetching.
This allows using read_some for incoming messages with an optional FD.
Doing so fits the purpose of read_some more closely, which is to read *any*
non-empty amount of data.
Previously this was overriding the cached value in the local Value object.
The global SilentLog variable never got updated, so a stale value would be
used.
This is only ever used on the path where we'd want to handle something
like this (in EmitEntryPoint()), so we can just extend the long handler
type instead of introducing a new type to handle this.
At runtime, glxtest is mapped as follows:
0x000055fd9a030000 0x000055fd9a034000 0x4000 0x0 r--p glxtest
0x000055fd9a034000 0x000055fd9a038000 0x4000 0x3000 r-xp glxtest
0x000055fd9a038000 0x000055fd9a039000 0x1000 0x6000 rw-p glxtest
0x000055fd9a039000 0x000055fd9a03a000 0x1000 0x6000 rw-p glxtest
The problem here is that the last two sections can't be distinguished solely
by their mmap parameters. This would cause the wrong base address to be
inferred for the last mapping. To fix this, we can be more permissive by
allowing multiple candidates to be returned.
In practice, this only affects non-code sections, so it's not a big issue
either way.
This is the only usage of LSE atomics that isn't the fetch variety.
[This article](https://www.phoronix.com/news/Linux-6.18-ARM64-Atomics-Issue)
reminded me that this was a thing and that I should double check the IR.
This was the only IR operation remaining that still didn't use the fetch
variety. Convert it over to the fetch to avoid the expectation that it
can be a "remote atomic". Change is going to fall in to noise, but might
as well as be consistent.
While this worked great for the singular unit test. I remembered thatour
pool allocator returns the minimum working size asked for but will
return larger sizes if exact fitment couldn't occur.
Because we are dealing with guard pages, we need to return the full
buffer size to the "client" so they can tell the frontend where the
guard page actually lives. Otherwise the JIT will tell the frontend the
guard page is at the end of the requested size, blow past the limit,
and fault in a completely different location.
With a bit of logging I saw in a multithreaded environment that we were
basically always getting a larger requested buffer while Steam was
starting up.
Primary fix here is returning the current CPU index in function 01h.
Intel Quartus uses this alongside affinity setting to check if all cores
can be used for its calculation. Since we had hardcoded apicid 0 here,
it assumed to only have one core and never generated worker threads.
Additional fix for apicid size. This is the size of the bitmask required
for apic ids, we weren't calculating this correctly at all. This mask is
a "maximum" number of APICs that the CPU reserves in power of two.
Say the core supports 256 APICs, but the processor only supports 16, or
any other combination.
Turns out I was reading six year old code for Wine's implementation for
SRWLocks. It actually /doesn't/ use WAIT_BITSET in their implementation.
It's still write-priority but it's actually significantly slower than I
was expecting due to futex queue usage and some other implementation
details.
Instead of using Wine's implementation, use win32's Wait/Wake on address
functionality and reuse all our other mechanism for implementing this
futex. This grants us our regular low-overhead codepath that I tested on
Linux, while the fallback is the only "slow" path. This also allows us
to still support a pseudo `WAIT_BITSET` code-path that reduces
stampeding even on Win32. The reader side just waits on the upper-half
of the futex (the writer bits) and the `WaitOnAddress` means only the
exact match address will be woken. We also get the regular
reader<->writer hand-offs working.
While this path still uses the futex
queue, the majority of the time our mutexes get acquired in the WFE loop
already, so it's a significant win.
Dark Souls Remastered before:
```
$RDLck Time: 4.531100 ms/second (0.04 percent)
$WRLck Time: 2.122560 ms/second (0.02 percent)
```
after:
```
$RDLck Time: 1.441620 ms/second (0.01 percent)
$WRLck Time: 0.963720 ms/second (0.01 percent)
```
When the JIT CodeBuffer overflows, we will now catch accesses to the
guard page and longjump while restarting the JIT with a larger buffer
request.
Fixes#4877
ARM64 branches have fairly small relative distances they can encode.
These can be +-1MB, or even +-32KB. The largest relative branch is
+-128MB, which we already set as an upper limit of our block JIT cache
size.
We have for a long time just compiled these without checking with the
expectation that things just happen to work. We didn't hit the asserts
so it was relatively low priority. Apparently now with Steam and a
MaxInst limit of 5000, we are now hitting an assert where we are
encoding too large of a range.
Implement support for long jumping from anywhere in the JIT for when a
long jump tries to be encoded and fails, allowing us to restart the JIT
at any moment. This is implemented as a long jump when this singular
feature could have gotten away with some sort of invasive check and
early exit path for two reasons. For one, that would be even more
invasive, effectively doing try-catch logic manually. And two, the next
step is supporting JIT buffer overflow for when our block size heuristic
fails.
This next step will mandate longjump on SIGSEGV (with cooperative
interaction with the frontend) from effectively /anywhere/ in the JIT.
One of the design goals of the CodeEmitter is that every code emission
function doesn't do a size remaining check to allow the compiler to do
some very effective optimization of emitting code blocks to memory (and
it works!).
But we lose the ability to sanely size check. When writing the emitter I
knew we were going to need to write this cooperative guard page handler,
and we're finally at a point where it needs to be done. This will be in
the next PR although.
When MAP_FIXED is used, if it was larger than the VMA region it was
trying to fit in to, then it would overallocate, corruption memory
adjacent to the VMA region. This was due to a typo in the LiveRegion
range checking.
Fix the typo, add a unittest that tries to overallocate space. Would
assert out without this bug fix.
In the case that overlapping `MAP_FIXED` mmap functions were used, we
were incorrectly tracking the full mapped regions size as new
allocation. We instead need to track which pages have already been
previously allocated and only track those. Would behave like FEX was
running out of memory, but we were just mapping the same location many
times.
Adds a unittest to track this.
Now that our Lookup cache mutex is no longer recursive, we can safely
use a shared_mutex instead. The problem with a c++ std::shared_mutex is
that it doesn't guarantee any form of priority, so tens of thousands of
read-locks per second can cause a writer to never acquire the lock, or
take too much time.
The bad news is that C++ doesn't provide us a primitive with
write-priority, so we need to construct our own that is still compatible
with Linux futex. So this is what we do.
- Windows: Uses an SRWLock instead.
- Only way for WINE to provide us a futex fallback that priorities
write-priority without stampeding.
A couple things here, we were never returning the last searched element,
either the last or first depending on search direction.
Also the backward scan would return incorrect indexes in some cases.
Also scanning beyond its page bounds.
Additionally some minorly incorrect assertions.
Adds a new unit test that ensures that we can allocate in to every
location, and that we get the correct indexes back. Also allocated
within guarded pages to ensure it doesn't read outside the bounds.
Fixes a spurious crash in Ender Magnolia.
Just helps when an unhandled ESR occurs, it was always a case of needing
to go in to the ARM ARM to decode it which was a bit of a pain. Add a
textual representation of it.
Fixes crash that occurs in applications that use both GL and Vulkan,
Like UE5 Vulkan native games. Fixes Ender Magnolia.
The issue here is that UE5 loads libGL first, which initializes our
libGL thunks, setting its X11Manager's functions.
It then loads libvulkan, which calls our oninit constructor, which
because of the symbol conflict, calls in to the libGL thunk's host
functions to reinitialize its function pointers, never initializing the
Vulkan X11Manager's functions. It would then crash as soon as an X11
function was used.
Give them unique symbol names so we don't accidentally look up the
incorrect symbol.
QEmu 10.2 is going to expose MIDR with Apple's vendor ID with variant 0.
That's the best they can do because they don't can't pin threads to
particular cores. So give a string for it, and detect it in the fit
script.
This reverts commit e1a45a2720, reversing
changes made to bd7edd8651.
The change rendered pressure-vessel non-functional on muvm-based setups
like Fedora Asahi Remix.
The InterpretAsFloat was never properly made use of. There's a couple of issues
that are fixed more easily with this gone, so lets remove it.
If there's a specific optimization that requires this, we can bring it back
at a later time. This should not have any effect on the current code generation.
This was missed before, where the non-repeating strings instructions
were still using TSO even when the memcpy/set config option was
disabled. Make sure it listens to the config option and disable TSO in
those instances.
Noticed this while profiling Dishonored, and WINE's `sse2_memmove`
function was showing up as a high amount of CPU time. This is due to
them using non-repeating string operations on the header and tail of
their memmove to align to 16-byte.
With this fixed, it causes the game to go from ~62FPS to ~67FPS,
becoming bottlenecked by x87 emulation instead of memmove. Doing about
23 million soft-float operations per second, because it needs full
precision to remove some flickering artifacts.
- Cache miss counts
- Useful for determining if L2 cache or dynamic cache could help
- Cache read/write lock contention times
- Useful to see if threads are blocking each other on contention
- Read lock is the case where a read-lock is beneficial, even if we
currently use a write lock.
- JIT count
- Useful to see if any new JIT blocks are generating
On top of #4951 because it fiddles with the cache stuff.
rpmalloc is currently very aggressively configured which causes
significant reductions in resident memory over jemalloc.
In Bayonetta's title screen it went from 963MB down to 834MB resident.
With our flags being optimized, this does even less than when it was
introduced. It's a hack, people are tinkering with it thinking it'll do
something. Get rid of it.
L1 cache residency can get quite large. Solution, start out small and
scale quickly on L1 cache misses but L2/L3 cache hits.
Some stats on L1 cache residency change:
- Teardown: 40MB -> 16MB (40%)
- Ender Lilies: 79MB -> 32MB (40.5%)
- Death Stranding: 186MB -> 93MB (50%)
- Steam: 75MB -> 7MB (9.3%)
The cost of this option is effectively free in our JIT. It changes a
single LDR to be a single LDP, which on Cortex CPUs cost the same. We do
this by moving the L1 pointer mask in to the CPUState object, making it
dynamic so it lives next to the L1 pointer. We then use that directly
rather than having the hardcoded value.
The lookup cache does a little bit of additional tracking and heuristics
to determine when the current L1 cache should increase or decrease in
size. From 128KB to 16MB per thread, allocating the full VA range as
previously.
Once the heuristic determines that L1 should be increased, it simply
changes the max and the L1 pointer size to compensate, the kernel will
fault in whichever pages are necessary.
Decreasing the size is a little bit more complex, as we want to madvise
the resulting L1 range to ensure we don't have that memory as resident
anymore. Same heuristic but going in the opposite direction otherwise.
Tends to be the case that L1 cache increases a bit on loading screens
then backs down once in-game.
These heuristic values are exposed for increasing and decreasing because
while I think I've picked reasonable values, we will likely need some
more fine tuning over time. Kind of expert user toggles at that point.
Based on #4940 as a base which needs to be merged first.
Full tracked stats from steam as an example of where we are:
```
Total (1000 millisecond sample period):
JIT Time: 0.486630 ms/second (0.00 percent)
Signal Time: 0.065880 ms/second (0.00 percent)
SIGBUS Cnt: 38 (38.160780 per second)
SMC Cnt: 0
Softfloat Cnt: 0
FEX JIT Load: 0.004585 (cycles: 552510)
Total FEX Anon memory resident: 368 mB
JIT resident: 95 mB
OpDispatcher resident: 38 mB
Frontend resident: 8 mB
CPUBackend resident: 624 kB
Lookup cache resident: 0 (null)
Lookup L1 cache resident: 7 mB
ThreadStates resident: 460 kB
Unaccounted resident: 217 mB
```
This mode has been broken for a long time because it's mostly untested.
Barriers, and backpatching while slow have proven that they work.
Maintain the one TSO path, at least until all ARM hardware gains support for
x86-TSO memory model mode.
Test behaviour themselves not changed at all, just data moved or
aligned.
For tests that aren't explicitly testing out SMC behaviour, we were
accidentally relying on some aggressive SMC tracking by mixing data and
code in the same page. To fix this just align the test's data to the
next page boundary which means FEX's SMC tracking won't get triggered
since it is no longer living in the same page.
This has been a thorn for a while, so just get rid of it. We obviously
still have ASM tests that still exist that /do/ rely on SMC, and those
are still expected to work.
If called back-to-back, the compiler can't optimize the object creation
resulting in multiple indirections. Save the creation and pass it
around, allowing the compiler to merge loadstores, and remove redundant
loads.
NFC
The file offset of a file mapping doesn't necessarily match its address
offset in virtual memory from the base file mapping. Indeed, most libraries
violate this assumption.
Now that the MappedResource::FirstVMA reliably identifies the base memory
mapping for a given library (even when that library is mapped multiple times),
this can easily be fixed.
PE/ELF binaries are sometimes mapped multiple times in the same process.
If this happens, there is no longer a unique base virtual address per file.
This breaks assumptions required for code caching: Any time a file mapping
is created, FEX must be able to unambiguously determine the base virtual
address of the mapped library.
This becomes possible by creating a separate MappedResource each time an
ELF header is re-mapped.
We want to take advantage of 16-byte single-copy atomicity. Which I am
relying on, but didn't codify it the first time.
Additionally add comments to explain that new members should be added to
the end to allow tools time to gain support gradually. This will allow
me to add new members without fully breaking mangohud, they'll just not
display the new information until support is added.
We're not guaranteeing backwards compatibility, just an attempt not to
constantly churn the format unless necessary. This way if we do break
compatibility, the tool will have an upper bound on supported versions
before needing to rewrite code.
Currently FEX doesn't properly support partial decoded instructions,
which behave slightly differently than full noexec or invalid
instruction decodings. Before this commit we didn't even have a way to
detect the difference.
Primary difference is that the faulting RIP is the beginning of
instruction decode, while the fault address is the first byte that
couldn't be fetched due to memory permissions. This shows up as a
difference between the RIP in mcontext and si_addr in siginfo in the
Linux signal handler.
Right now just change the log so we can determine if we need to support
this edge case.
This saves a whole bunch of memory. Cutting `Just Cause 2`'s title
screen from 1132MB anonymous FEX memory down to 438MB. 629MB in L2
alone.
L2 is primarily a means to reduce overhead in map queries, so it's all
about performance. But because it consumes a lot of people it's kind of
hard.
One idea is that the L2 lookups can be moved to shared data structures,
since we already pull the shared lock when doing an L2 lookup this is
already halfway there.
Side note, we're using unique locks even with read-only code paths
which we can't use the shared lock because this terrible recursive
mutex!
Instead of outright changing L2 behaviour and potentially wrecking
havoc, add a config option for now so testing can happen over time.
before:
```
Total FEX Anon memory resident: 1132 mB
JIT resident: 60 mB
OpDispatcher resident: 97 mB
Frontend resident: 37 mB
CPUBackend resident: 500 kB
Lookup cache resident: 629 mB
Lookup L1 cache resident: 108 mB
ThreadStates resident: 436 kB
```
after:
```
Total FEX Anon memory resident: 438 mB
JIT resident: 62 mB
OpDispatcher resident: 56 mB
Frontend resident: 22 mB
CPUBackend resident: 496 kB
Lookup cache resident: 0 (null)
Lookup L1 cache resident: 109 mB
ThreadStates resident: 436 kB
```
Trivial fix, if a symlink gets deleted between checking if it is a
symlink versus getting a path of it then it resulted in a crash.
Happened periodically for me.
Every time I see this recursive mutex I glare at it. Remove the last one
so that we no longer need to deal with it.
The only reason why this recursive mutex still existed today was because
it is fairly intertwined with the ContextImpl and tracing it all was a
pain.
Peel back the layers and follow the idiom to have ContextImpl pull the
write mutex when requiredand pass it through by reference to ensure it stays alive.
This allows us to entirely give rid of the recursive nature of the
mutex, which means that `FindBlock` can eventually be switched over to a
read-lock to improve multiple threads reading the caches at the same
time.
I didn't do that exercise since that can be followed up in a subsequent
PR.
Use getuid() instead of geteuid() when determining FEXServer socket names.
Ensures setuid binaries (like chrome-sandbox from Discord) connect to their parent
user's FEXServer instance instead of trying to spawn a separate server.
Fixes connection errors when running applications that spawn setuid children.
Fix a typo in unittests/ASM/Secondary/09_XX_07.asm which caused it to
jump back to the test_32bit label instead of test_64bit.
On Arm systems with FEAT_RNG support, a significant amount of time may
be required before successive uses of RNDRRS. This is because it returns
a random number with fresh full entropy, and it can take a while to
collect the new entropy. That time may be hundreds or thousands of
instructions, so by jumping back to test_32bit the 64-bit test will
alway fail because an RNDRRS has been executed too recently.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
This captures the remaining FEX allocations that /aren't/ coming from
JEMalloc, allowing us to separate our mapped regions versus just
jemalloc allocations.
With some additional naming in jemalloc (which I'm not adding here) this
gets us interesting results:
```
Misc resident: 54 MiB
JEMalloc resident: 208 MiB
```
So 208MB of active jemalloc allocations in this particular case. These will be able to be tracked in heaptrack-like applications if careful.
This should let us target down whatever live allocations we're keeping
large amounts of data around if possible.
These don't require being bound to class state directly, and so the list
management can be completely opaque to the outside (also means less
rebuilding if these change)
With the gather overflow fixes in place, I've been having this running
for a while. Now that we just kicked out a release, enable AVX even on
32-bit.
We'll need to eventually create a list of games that explode with AVX
enabled, but that same list would match what happens on real x86 hosts,
so there can be some collaboration there.
The frontend did a quirky widening check which was accidentally working
in this case, but it is supposed to be for the couple of GPR handling
AVX instructions.
Correct the implementation to use the correct register size for FMA.
Turns out a simpler way was added to the docs at some point and I never
noticed.
Before:
text data bss dec hex filename
4159895 1471360 4336824 9968079 9819cf Bin/FEX
After:
text data bss dec hex filename
4157159 1471360 4336824 9965343 980f1f Bin/FEX
Just a trivial rearrangement, Drops the struct down to 40 bytes.
We can also make some functions take it by const reference so we
aren't churning some copies.
Before:
text data bss dec hex filename
4160559 1471360 4336824 9968743 981c67 Bin/FEX
After:
text data bss dec hex filename
4159927 1471360 4336824 9968111 9819ef Bin/FEX
SHM stats and CallRet stack was accidentally using system mmap which
means it would take VA space from the guest. Ensure it uses the
Allocator helpers so that doesn't happen.
Gives 32-bit guests 8-ish megabyte of VA space back.
Avoids actively doing this wonky thing where we're passing
iInvalid all over the place to mean variable alignment depending
on store element size or GPR size.
Makes using the API a little more visibly straightforward and makes
cases where alignment matters more explicit.
Will allow external tools to track how much memory FEX allocates.
Necessary since we can't use traditional memory usage tools to track FEX
memory allocation independently of guest allocations. Plus most tools
like heaptrack hook allocation symbols, which break under jemalloc.
Using this information I can see with Steam loaded with my library that
FEX consumes ~825MB. Total process resident memory is 1204M, accounting
for around 379MB being used by steam itself. This is /relatively/ close
to my desktop running steam at around 261MB. There's a bit of variance
due to what Steam chooses to do at startup.
This tracking will be the first step towards seeing where our memory
usage is going.
rbit is useful in generic algorithms.
`MaskGenerateFromBitWidth` is only really useful for SSE4a, but
implemented in the OpcodeDispatcher is rough, so add an operation.
This isn't actually used anywhere in the IR header, so we can
move it to where it's actually used.
Now the IR interface header doesn't have anything related to the
independent passes in it.
Reduces a bunch of noise related to the register classes and hoists them
out so that converting the classes over to enums should be fairly
straightforward.
This allows easily moving the default case out of the switch, making it
easier for compilers to warn about missing values in switches if any
enum members are added in the future but aren't added to the
formatters.
Instead of having this sort of odd indirection through a struct type,
we can add support for defining custom enums in the IR description.
This lets us both get strong typing (and allow for weak typing, should
any enum in the future need it), without needing a struct for a basic
value type.
Even then, if we do need a struct for anything in the future, then
we still allow strong typing for values themselves while allowing
them to be used in various ways.
Allows applications running entirely under Linux to use the same
extended volatile metadata as Windows.
For example `FEX_EXTENDEDVOLATILEMETADATA=iw4sp.exe\;0xe9da0-0xe9ec7`
this configuration works for both wow64 and Linux to disable the TSO
emulation on the memcpy routine in that game that consumes around 80% of
CPU time in TSO emulation.
Works with Linux native games as well of course.
This adds a new mode X87StrictReducedPrecision.
The strict reduced precision is like the reduced precision but adds extra checks,
like the currently implemented nan and snan propagations.
Fix for __builtin_issignaling() test of SPEC2017 classify test.
These get stored based on the current rotation of TOP. So we need to be
a bit careful with how we do this storing. A smidge of overhead, but
nothing unexpected.
This allows Known_Failures files to contain either:
- Partial paths (existing): Test_X87_F64/SomeTest.asm (affects all configs)
- Full paths (new): jit_1/Test_64Bit_X87_F64/SomeTest.asm (specific config)
For example: Mark only the jit_1 (-n 1) configuration as failing:
jit_1/Test_64Bit_X87_F64/Memcopy_int_F64.asm
Two bug fixes here.
- We weren't reporting a correctl ErrorRegister. Needs to contain
PF_INSTR at least.
- We weren't setting siginfo->si_addr in the case of `FaultToTopAndGeneratedException`
Fixes Mafia 2 (Classic)
We need to read the current x87 TOP location and rotate the values when
saving and restoring the context state on Linux. Wasn't visible in the
ASM tests since we read the FEX's CPU state directly there.
Still creates a copy of FEXInterpreter from FEX for downstream projects
to have some time to get off the old name. Creating a symlink is kind of
a pain in cmake so just doing an install copy is easy.
We were paying a large cost per Literal type that we can special case
for the two class of instructions that use a 64-bit literal.
If we packed this would get to a further 62 bytes but probably not worth
it.
This is entirely unused as FEXServer only ever reads the environment and
other configs.
There was /technically/ a weird conflict with FEXServer arguments but
that was an accident if it ever happened.
x87 80-bit loads, both BCD and regular tword were loading 128-bits of
data when reduced precision was enabled. This was an oversight from the
previous fix a while ago.
Adds a specific reduced precision test for this, and updates the current
test to ensure stores are still tested as well.
This is now cached internally in the X87 pass, and all operations
outside of that that use it are rare so can afford loading/storing
directly from context after flushing x87 regs.
Flushing other regs is not necessary, and breaks any ConvertNZCVToX87 use
which relies previously saved NZCV values as the flag-setting NZCV op after
the save could trigger a flush of NZCV.
We had quite a bit of dependencies that were currently being indirectly relied upon.
We can forward declare the relevant ones and add the includes for the ones that are
explicit.
We don't really need to make this potentially return a null pointer when we could always return a valid instance
that would just happen to be empty if unused.
We already have an equivalent header within FEXCore that's header only,
so we can adapt it to conform for both cases, allowing for removal of
one of them.
Moves the IR include into one of the more specific headers, which avoids dumping the IR header into any core bits that use the interface.
Also uncovered a missing header guard.
We definitely don't want the boolean null test to be able to be implicitly converted
(e.g. to an int or whatever else).
For example a non-explicit bool operator allows for silly things like:
NonMovableUniquePtr<...> ptr;
// ...
auto k = 5 + ptr;
to build without issue, which we should really force the user to be explicit about if it's *really* a desired behavior.
We can reduce includes, such as logging by specifying a concrete size for logging levels,
allowing the enum to be forward declared. We can also move FillHeader into the cpp file,
allowing the syscalls header to be removed.
Profiling shows this is by far the hottest path, as every DX call will
hit this (as arm64ec functions in a vtable). L2 hits are much rarer and
L1 almost never hits since that is looked up inline in most cases.
The dispatcher/block linker will handle this, but if the instruction
following a POPF flag doesn't otherwise trigger one of those the
interrupt would be missed.
Unity games crash with TSO disabled due to the SPSC GfxDevice
ThreadedStreamBuffer read/write pointer updates missing acq/rel
semantics. Rather than attempting to use heuristics to match cases like
this, which could be overzealous and hit more accesses than necessary, just
target the problem directly as this is consist across 32/64 bit Unity versions
for at least the past 10 years. Gate this behind the existing Unity mono
hacks to avoid false-positives in non-Unity games.
nasm doesn't let you encode all of these but we had already supported
them in FEX.
It's a bit of a weird edge case behaviour, since it's supposed to be
sign extended a 32-bit value in to a 64-bit register but with prefixes
(or lack of) it can be a move with zero-extend or a 16-bit insert.
Make sure we're actually testing these edge cases.
Fix SoftFloat IsNan - custom detection matches IEEE754 semantics.
Sets Invalid Operation flags properly for NaN comparisons.
Fixes: GCC-C-execute-ieee-fp-cmp-8l test
__builtin_isunordered() now returns correct values for both NaN and normal operands
This isn't used outside of the IR emitter. Plus, IsBlockExit() is already a more general interface to use,
since it handles the fragment exit case as well.
Since all of the information comes from the Dispatcher, we can have the dispatcher
provide that information. This way we can also eliminate a bunch of now-redundant
public interface members and simplify the config setup within InitCoreImpl().
Conveniently, this also allows making all members of the dispatcher non-public.
The name itself is already qualified with SignalDelegator, so this can reasonably be outside the class itself.
This also allows for forward declarations of the config struct
One set of tables for 128-bit and one set of tables for 256-bit.
This one took a bit longer since I needed to convert a few handlers over
to `Bind`. With this all of our x86 tables are costexpr so they end up
in RO mapped memory which is great.
This matches kernel behaviour for how brk works. I initially implemented
this as a way to speed up some early applications that relied heavily on
brk. We're long past that time and we should match Linux behaviour
instead.
This fixes issues where applications (FEXLoader really) can see we've
allocated a larger granule and breaks self-hosting. There's no reason to
be smarter around this, if applications want more perf then they'll
switch to a better allocator than brk.
Also need to make sure that after the brk region has been reserved, to
unmap its initial mapping to allow future mmap syscalls to overwrite it.
We need to reserve it early to ensure the region initially exists and we
don't accidentally map other things in that space. Then it is up to the
guest if they don't want to overwrite it.
Implied the maximum size that brk could grow to. Which isn't correct, it
is the current maximum size mapped which is page size, versus the
current brk offset which is byte ranged.
A wild use case of union over a variant because we don't want to
increase the encoding size from 128-bit to 256-bit (because of padding).
The type of operation is encoded with the table operation type, so a
variant is unnecessary and we get to keep the 128-bit encoding.
This allows us to have "recursive" x86 table descriptions. But in
reality this is going to only be one layer deep. As this will allow the
Frontend decoder to select instruction encodings based on arch bitness
once the tables are generated correctly.
Noticed some runners had generated a .fex-emu/Config.json, run the tests
in portable mode to ensure they don't pick up a spurious config option.
FEXLinuxTests still ran non-portable because it's required for the thunk
tests.
A prevalent pattern in the FEX codebase is to compute some data and store it
in a maybe_unused variable that's only ever passed to LOGMAN_THROW_A_FMT.
Besides few exceptions, we never compute expensive data in the macro
arguments themselves, so we can remove a lot of code noise by unconditionally
evaluating the condition even in assertion-disabled builds.
When loading an ELF file (usually PT_EXEC), we would have BSS regions
that ended up in the high pages of the address space. We would then
attempt mapping BRK after whatever the highest address ending up being.
This was problematic because we used `MAP_FIXED` which means that the
BRK region could cross in to 64-bit address space, overwrite VDSO,
overwrite the stack, vsyscall, maybe a couple of other things.
Instead of letting that happen, reorder some of the logic so that in
`LoadElfFile` will ensure the full `BRK_SIZE` is allocated (or return
zero) and then we can ensure we're never overwriting other various
memory regions.
This doesn't fix two fundamental issues that FEX has with brk:
* If BRK gets mapped below the ELF, that should mean it isn't mapped at all
* Linux-isms, doubt this breaks new software
* We still map the full 8MB, which isn't correct
* I'm working on fixing this issue, which is why I encountered this.
Only wired up for wow64 and arm64ec. Gives more granular control over
TSO enabling and disabling. Matches arm64ec volatile metadata except
with one more additional feature that whole modules can be disabled at a
time.
Once we know the mapped size of files in Linux then we'll be able to do
the same thing there, but there's not a full mechanism wired up for that
yet.
This nearly gets FEX's TestHarnessRunner to be self-hosting inside of
FEX. The only thing blocking it currently is that our SBRK emulation
reserves the whole region, when it should be "soft-reserved" and mmap
with MAP_FIXED_NOREPLACE can override it. Plus an assert in
OpcodeDispatcher preventing any 32-bit code from running from a 64-bit
process.
In the most simple terms, gdt and ldt are setup to be unique per thread,
and modify_ldt then modifies that thread's ldt entry. On thread
creation, these values get inherited as a copy.
This allows installation of 32-bit code entries, which with the previous
PRs merged allows the code to attempt jumping to that 32-bit code entry.
It then will immediately explode with an assert in our OpcodeDispatcher.
We can't allow 32-bit code jumping yet until our OpDispatcher/X86Tables
allows runtime selection of 64-bit and 32-bit code entries which is
still a ways away.
With the assert removed and the SBRK code handling hacked out,
/technically/ the TestHarnessRunner can run some code, albeit anything
that changes behaviour between bitness is completely incorrect.
These x87 f64 reduced precision operations don't use FCW so we don't
need to load it from the context. So just remove loading it. This falls
within noise while benchmarking.
Effectively NFC today while we don't allow installing LDT entries, but
is required to have signal handlers execute in the correct bitness if
32-bit code segments are enabled.
Nothing too crazy, just that four segments are stuffed in to the
`REG_CSGSFS` data value, and then CS/SS gets restored on restore.
FS and GS are ignored in this instance because edge case behaviour where
those don't get reset on signal entry, so if they get changed then it is
on the userspace application to fix it.
Gets another change out of my stashes.
Similar to the previous far jmp, if the CS changes operating mode then
things will still explode with other FEX asserts. But this gets another
change out of my stashes.
These instructions go hand-in-hand obviously so they get implemented as
a pair.
Taking this very slowly because this is very fickle code. The frontend
needs to manage GDT and LDT, but before we get there, we need to
actually add support for LDT in the backend. Split the segments to two
arrays so the JIT can actually update their cached values correctly.
Still treats GDT and LDT as mirrors like how the JIT previously did (By
it ignoring the selector's TI bit).
While most applications will just create ALL_ACCESS handles which
support the additional operations FEX needs over the regular windows syscall
(usually just QUERY_INFORMATION to lookup the thread object), it is
valid for them to pass in the minimal set required for each operation
and windows/wine will reject any other operations (e.g. querying the TEB
base on a handle with only the SUSPEND right). Windows permits
duplicating handles to extend their permissions given the process itself
has such permissions, so do that as necessary for the operations FEX
needs.
In the terminate case, if we don't return early then we will delete the
FEX thread object but the actual terminate call that follows will
fail, leaving the thread in a bad state.
We don't need to be informed of our own mappings, avoids a deadlock that
occurs if the ThreadCreationMutex lifetime is extended when creating a
thread.
Hashmaps are super expensive and there's no reason not to use a vector - we
already have compact block IDs so we don't benefit from the sparseness. Huge win
for very little effort.
Spotted when profiling FEX. CondJump() in the JIT was almost 4% of our time (?!)
and all because of map slowness. Easy fix.
Difference at 95.0% confidence
-0.0196494 +/- 0.00194956
-3.92827% +/- 0.389753%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
..and be consistent about signedness.
clang doesn't seem able to do this itself.
Difference at 95.0% confidence, n=100
-0.272326% +/- 0.242353%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We are onyl wanting to test the FTW, which is 16-bit but we were reading
a whole 32-bits. The upper 16-bits was ending up in the reserved
section, which is undefined as to what it contains. In FEX it'll contain
0 but on a Zen 4 CPU it contains 0xFFFF.
Only read the bits we care about.
now obsolete!
Results for the whole series are excellent:
Difference at 95.0% confidence
-3.97603% +/- 0.254656%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Augment IREmitter to inline constants as we generate code, rather than
needing a later clean up pass. This replaces the last function of ConstProp.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now executable page tracking is implemented, this limit is technically
unnecessary and effectively never hit in normal code. Keep a reasonable
limit however to avoid accidentally inlining tail calls and exploring
dead branches in obfuscated code.
We were failing to save the entire 64-bit register, cutting off the top
32-bits which contain SS and CS. This ...happened to not cause issues
but is a bug that I encountered with far jump handling.
This patch does cover up all v4l2 syscalls in i386. It just works
well with the poor v4l2 driver of wine32.
Signed-off-by: Changwei Miao <chanthmiao@outlook.com>
In b8dd5d95b ("OpcodeDispatcher: optimize X87FTWTag"), we optimized
X87FTWTag using an efficient Morton interleave operation. Here, we do the
inverse, optimizing SetX87FTW using an efficient Morton deinterleave
operation.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
I don't think these are worth it, and also currently they don't trigger ever.
n=100:
Difference at 95.0% confidence
-0.00245961 +/- 0.00139573
-0.524468% +/- 0.297615%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is expensive and only needed for spilling, so only do it for spilling. This
complicates the RA a bit but speeds us up on average since most blocks
don't spill. Total results of this change (including the prep commits that
slowed things down temporarily):
Difference at 95.0% confidence
-1.71952% +/- 0.455996%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
if we spill for SRA, we don't need/want to execute this code path. this will be
load bearing by the end of this series.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We had quite a few instructions that we weren't constraining with this,
now the instruction itself will show up in the failure output more immediately
instead of failing on the internal implementation function if anything is
incorrectly passed through.
Only the last prefix byte is retained when multiple are set. We were
accidentally generating a mask.
Additionally with 64-bit code, the legacy segment prefixes don't
overwrite if FS or GS have been set. So no weird behaviour where FS/GS
is set, a legacy prefix is used for padding, and then it "ignores" a bad
prefix by ignoring only the latest one.
This has the Frontend and OpcodeDispatcher select their operating mode
depending on the incoming code segment long-mode flag.
Adds some asserts since currently it is unexpected if the configuration
changes at runtime.
This is fairly straightforward for an initial setup but isn't fully
fleshed out.
Right now FEX's x86 tables aren't setup in a way to support choosing a
different instruction decoding depending on runtime operating mode
change, so that would break in interesting ways.
Primarily this just gets FEX setup to start piping the operating mode
through from the frontend to the backend. This is a long term task, so
it is going to take a long time to iron out all the issues.
Previously we were only storing the 32-bit base address which isn't
actually how segment descriptors work.
In reality segment descriptors are 64-bit descriptors that are laid out
in a particular layout depending on the 4-bit type value. In reality we
only care about code and data segment layouts since the rest are
bonkers.
Describe these descriptors correctly and setup a default code descriptor
for the operating mode that FEX is starting in.
This option is free and only enabled if the config option is set. Enable
it always at build time so that users can pick it up without enabling
the full gpuviz/tracy paths.
Fixes#4124
clang feature checking can't check beyond MIDR, so compiling for a
specific cortex version means compiling SVE on these CPUs that disabled
them.
Just detect the particular situation in-which SVE isn't inside
proc/cpuinfo and is one of the snapdragon cores that are supposed to
support SVE. Then compile for Cortex-a78 instead.
If we have more constants than registers, something will be rematerialized. Use
a simple round-robin heuristic to pick instead of the better-but-slower approach
with RA. This is a heuristic to reduce JIT time with minimal impact on code
quality. In Instcountci, the only impact is a block in oblivion only increasing
instruction count by 0.2%. And moves of constants are free for cycles at least
on Firestorm, so this isn't where we want to spend piles of JIT time anyway.
Difference at 95.0% confidence
-0.00138911 +/- 0.00104724
-0.418608% +/- 0.315587%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is slightly worse for x87 blocks since we can't share constants between the
x87 and the main code, but otherwise should be comparable and this avoids an
expensive remapping operation.
Difference at 95.0% confidence
-0.00474273 +/- 0.00119189
-1.40908% +/- 0.354114%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Due to us only enabling the CPUID extension in the case that the host
hardware supports SHA or not, this has actually been largely unused now.
Also the only hardware that doesn't support the crypto extension has
been some old Pi hardware and some other things we don't really care
about.
This code was a phenomenal reference point for implementing the SHA
versions of the instructions and would have been significantly more
difficult to implement had this not been available. Kudos to @lioncash
for having written it!
But now as we are no longer utilizing it, it is time to remove it.
This matches the systemd path, no more _32 and _64 versions, just
binfmt_misc.
Having a mixed install where one program does one architecture and
another is weird and unsupported anyway.
The constraints introduced by shared code buffers make supporting
calls with the previous layout impossible. The main additional constraint
imposed by call-ret that if a host location is ever pushed onto the
call-ret stack, then it must forever be a valid jump target. While
this is reasonable in the: unlinked, direct linked, unlinked,
direct linked case; it's almost impossible to achieve in the: unlinked,
indirect linked, unlinked, direct linked case while ensuring
all backpatching cases are valid with the current approach.
To solve this introduce an additional layer of indirection, jump thunks,
these are emitted at the end of a multiblock and are used to handle the
two cases of calling the initial linker, and calling an indirect linked
block. Initially at the ExitFunction location a branch/call to a unique
jump thunk will be emitted, which will have the code layout:
00: b 0x8
04: br TMP1
08: ldr TMP1, <Shared exit linker>
0c: blr TMP1
10: HostCode
18: GuestRIP
20: CallerOffset
If a direct link can be performed, then the initial branch/call to the
jump thunk can be linked/unlinked to point to the jump thunk in a
single 32-bit atomic operation. For an indirect link, the HostCode
member is updated with a 64 bit atomic operation, and then a 32 bit
atomic operation is used to replace the branch at 00 with a load of
HostCode. Indirect unlinks are done by placing back the b 0x8 at 00.
Safety:
(1)
Sequential link (e.g. one waiting to lock, one locked and linking):
Linking is idempotent, would just rewrite the same data atomically.
(2)
Simultaneous link or simultaneous delink:
Impossible due to LookupCache locking.
(3)
Simultaneous link and execute:
(3.1)
Direct link: Either the direct link is observed at the thunk
callsite, or it is not observed and the linker is entered - this is
then just (1).
(3.2)
Indirect link: Either the branch at 00 in the thunk is observed
to be replaced with an ldr, in which case the modified HostCode
must be observed due to the cache flush. Alternatively the branch
replacement isn't observed and it's just (1).
(4)
Simultaneous unlink and execute:
(4.1)
Direct link: Either the jump to the jump thunk is seen, which must
be in its base unlinked state with the branch at 00 as that would
be inserted by any previous indirect unlink. In such a case the
linker would just be entered, giving (5). Alternatively the modified
jump isn't seen and it calls the original host code (which is fine).
(4.2)
Indirect link: If an ldr is seen at 00, then the rest of that sequence
will function fine as HostCode is left untouched. If a branch is seen
at 00, then it will just call the linker giving (5).
(5)
Sequential unlink then link:
Unlinking restores the callsite and jump thunk to their original
contents (aside from a modified HostCode). Linking then works as
usual.
This is made slightly awkward by the many potential orderings of blocks
and desire to support both fallthrough jumps and calls without additional
branches.
Allocate the stack with uncommited guard pages on either side, if
any faulting accesses to these occur then the call-ret SP value is
reset to the default and execution resumed.
This can't be handled fully within FEXCore due to the frontend-specific
handling of guard pages. Frontends can populate this at init time and
are expected to handle setting the CPUState field and register as approriate.
The host stack pointer can't be reused since explicit bounds checks
would be far too expensive, and on-stack signals prevent implicit ones
using guard pages from working.
Since CodePages is now a member of the guest to host map, which could
be replaced when JITing ARM code, any additions to it must be moved after that.
Additionally there is no benefit marking code pages for invalidation at all if
they are never added to the cache as in the single-step case.
This does technically prolong the window of an existing race where guest code
modifications could be missed, however this is unlikely to cause issues and didn't
prior.
Prevents invalidations being missed under the following circumstances:
Thread A JITs block A into the global codebuffer, adding the guest to host
mapping to its CodePages, thread A is then killed.
Thread B then performs SMC on block A. An exception will be triggered but
as CodePages was stored per-thread, and thread A is now killed when all
threads are iterated over by the frontend to perform invalidations it
will be missed.
The accumulator is introduced to handle the case where multiple threads
have the same code entry in their local caches but share the same codebuffer.
Consider a thread C in the above example that also has block A in its cache,
without an accumulator, when invalidating thread B the entrypoint of A is erased
from the shared guest to host map. So when C is invalidated, the local cache entry
for A is not removed since it was removed from CodePages when invalidating B.
Consider a page with two blocks in it, A and B. Block A performs SMC on B then A.
With the previous logic, the SMC write of B would unprotect the page and then
the inline SMC touching A would not be detected and a single-step would not be forced.
Forgot in #4697 to take into account that there are still parts of the
emitter that qualify with its own namespace.
This also removes those where applicable to be more in line with the rest
of the cases.
These were actually encoded incorrectly, and I noticed they changed in
PR #4670 for some reason. So I dove in to the nasm source to figure out
what it takes to encode these sanely rather than with raw bytes, turns
out it's fairly trivial, just annoying to track in their parser through
iwdq->mem_offset->MEM_OFFS->qword.
Should remove the change in #4670 once it rebases.
Now that the PF flag isn't using popcount, this is a win across the
board if the hardware supports it.
Been a while since I last looked at this, added a new instcountci file
to show the improvement.
Ignore any additional jemalloc configuration options, can come from
`/etc/malloc.conf` or even the `MALLOC_CONF` environment variable.
Ensure nothing can override our options.
There is a ELF note for this but currently clang doesn't support a
`no-gcs` flag. The best we can do is check if the kernel has GCS enabled
for the current process and early exit.
Then continue to use the kernel's locking functionality to disable it if
the guest happens to try, ensuring safety.
Unifies the interface, so that there's no need for a stark difference.
Previously, calling the single/double variant didn't require explicit template
arguments, but the half-float version did, which is inconsistent.
Technically it also made the interface more cumbersome to use in the event the
element size isn't able to be determined as a constant ahead of time.
If multiblock discovers a codepath with an `int 0x80` then it would
crash the emulator even if it never gets executed. Ensure that this
ERROR_AND_DIE_FMT instead just gets handled as an UnhandledOp to ensure
the Core early terminates the block.
Found by having steamwebhelper spuriously crash when it hit this.
Adds a simple unittest to ensure discovery doesn't break again.
Part of waitpkg is the TPAUSE instruction. This instruction gives an
RDTSC deadline to go in to a low power sleep mode with the CPU.
Semantically we can't implement umonitor and umwait with ARM's exclusive
monitor implementation, but a nop implementation is sane. Just need to
make sure to clear the pre-req flags.
This lowers power consumption of UE5 games since their job handler now
goes to a tpause based implementation instead of a `pause` spinloop
implementation.
Technically we can end up doing null pointer dereferencing here since checks were being
chained with || instead of &&. So we can just separate the checks out.
Avoids an additional layer of indirection for callbacks. Passing them
around deep into instruction decoding logic doesn't provide much benefit
seeing as there will always be one frontend object per thread.
If a multiblock contains a call instruction, we know at the point
of compilation that the instruction after that call will likely be
jumped to at some point. Avoid redundant recompilation by tracking
such cases and including an entrypoint for that instruction in the
multiblock aswell.
Rather than relying on wine-specific alphabetical behaviour, that
has since been changed. Rely on their addresses being sorted which
is more stable behaviour also present in Windows.
Buffers are tied to the lifetime of their owned flag, and as that
is a member of PoolBufferWithTimedRetirement we must always unclaim here.
Avoids the need to manually remember this quirk (which was forgot for the
temporary compilation buffer in JIT.cpp) at every use-site.
FEX should only set fpx_sw_bytes.magic1 with FP_XSTATE_MAGIC when
avx is enabled. Otherwise it may cause segfault in ntdll::save_context,
which requires access to extended xstate info if magic1 equals FP_XSTATE_MAGIC.
Signed-off-by: Changwei Miao <chanthmiao@outlook.com>
WINE uses this to determine TSC frequency and because it doesn't say
`tsc` on ARM devices, it was ignoring TSC and instead using CPU maximum
frequency.
This was causing Horizon to think the TSC ran at whatever the max
frequency of a core was (1.8Ghz to 2.6Ghz depending?) This was causing
all of Horizon Zero Dawn's physics to run at slower than real time
speeds because our 1Ghz (on Orion) TSC is significantly lower than the
max clock speeds of the cores.
This is still a bug in Wine that it is using the maximum CPU clock speed
in the case of current_clocksource not being TSC, but that's a battle
for a different time.
Turns out Bayonetta hammers SINCOS, our splitting the operation is
actually harming the performance of games that heavily use FSINCOS. We
instead can actually combine the operation which improves performance.
Not enough to get the game running full speed consistently on my Radxa,
but good numbers in my microbenchmark.
```
Test, Total Cycles, Total Runs, Cycles Average, Internal Loops, Average cycles per internal, per/second
64-bit:
Before:
FSIN, 2691031290, 50000, 53820.6, 1000, 53.8206, 18580.237319
FCOS, 2719397120, 50000, 54387.9, 1000, 54.3879, 18386.428239
FSINCOS, 5586917530, 50000, 111738, 1000, 111.738, 8949.478801
After:
FSIN, 2669959250, 50000, 53399.2, 1000, 53.3992, 18726.877573
FCOS, 2740942260, 50000, 54818.8, 1000, 54.8188, 18241.901965
FSINCOS, 3189472870, 50000, 63789.5, 1000, 63.7895, 15676.571659
80-bit:
Before:
FSIN, 24702939380, 50000, 494059, 1000, 494.059, 2024.050629
FCOS, 19127131020, 50000, 382543, 1000, 382.543, 2614.087808
FSINCOS, 40386785260, 50000, 807736, 1000, 807.736, 1238.028719
After:
FSIN, 24869980710, 50000, 497400, 1000, 497.4, 2010.455922
FCOS, 19131849590, 50000, 382637, 1000, 382.637, 2613.443084
FSINCOS, 38329985570, 50000, 766600, 1000, 766.6, 1304.461749
Improvement 64-bit: 1.75x
Improvement 80-bit: 1.05x
```
Only a minor improvement at 80-bit precision since cephes doesn't provide a combined sincos operation, but the f64 implementation is significantly improved, allowing 75% more operations per second.
Disabled in the simulator because we can't easily support pairs of
vector registers being returned.
for constant function where we don't have a leaf. this isn't fully general but
we can't do better without a more general post-RA optimizer. i'm not inclined to
do that unless/until we get hot blocks demonstrating its value (that we can
compare against the JIT time hit of the heavier-duty optimizer.)
however this special case we can (and should) optimize for now.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The cmake_configure_woa*.sh scripts will automatically install any required
cross-toolchains required to enable either ARM64EC or WOW64 builds of FEX,
and it will initialize the current build folder appropriately.
For advanced uses, a shell can be opened by running `nix-shell WineOnArm/shell.nix`,
which will make the toolchain available via environment variables. This also
generates a meson crossfile for building VKD3D or vkd3d-proton.
The cmake_enable_flt.sh script will automatically install the required
cross-toolchains required to build FEXLinuxTests, and it will reconfigure
the current build folder appropriately.
For advanced uses, a shell can be opened by running `nix-shell FEXLinuxTests/shell.nix`,
which will make the toolchain available via environment variables.
The cmake_enable_libfwd.sh script will automatically install any required
cross- toolchains and development headers required to enable library
forwarding, and it will reconfigure the current build folder appropriately.
For advanced uses, a shell can be opened by running `nix-shell LibraryForwarding/shell.nix`,
which will make the toolchain available via environment variables.
this makes it a lot easier to turn a long division into a non-long division,
just by nulling out a source.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
this will eliminate an annoying special case in post-RA opts.
No difference proven at 95.0% confidence
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Previously, the first attempt to look up a FEXInterpreter would always
fail due to a missing path separator.
Additionally, fallback lookup now uses /proc/self/exe to find a path relative
to the FEXBash executable. The previous use of FindContainerPrefix does not
seem to be required anymore in current Steam versions.
This is now relative to the time of the first message. Furthermore, display
precision is limited to milliseconds (which are actually zero-padded now!).
Wine does some minimal attaching, peeking, poking, and detaching to have
a different process inspect another's TEB region. Ubisoft's launcher
does this to check if a debugger is attached and rejects it in the case
that it is.
While this implementation isn't all-encompassing, it is good enough for
this family of games.
This allows changes to CMAKE_INSTALL_PREFIX to be automatically picked up and
propagated properly. Previously, you had to update 4 variables in 3 files by
hand to do so.
Apparently I completely missed that cpuid, xgetbv, syscall,
l{u,}{div,rem} were failing to hit their optimized cases for inlining
and avoiding 128-bit software divide.
The divisions are a clear performance regression for 64-bit applications
since that is the only real way to do a 64-bit division on x86, I added
those specifically because it sped up games.
CPUID depends heavily on the game, since some games use that as a
serialization instruction fairly heavily.
XGETBV is trivial since it matches behaviour of CPUID (and is basically
an extension of it).
Syscall inlining can save a decent amount of time, again heavily depends
on game.
Adds multi-inst tests for all of these situations so that once it gets
fixed (Apparently broken once RCLSE got stripped out), we can see that
they keep working. Obviously tests couldn't have existed in instcountCI
before since we didn't support multi-instruction tests.
now that the algebraic/folding opts are gone, we can do this in one pass for a
2.5% speedup:
N Min Max Median Avg Stddev
x 50 0.44474704 0.46750433 0.45455258 0.45446569 0.0044727894
+ 50 0.43149892 0.45173984 0.44252267 0.44295575 0.0045621814
Difference at 95.0% confidence
-0.0115099 +/- 0.00179263
-2.53263% +/- 0.394447%
(Student's t, pooled s = 0.00451771)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
No longer needed.
The total difference from the beginning of this series (all the prep work to
make this change possible) plus this commit is a modest 0.4% win.
N Min Max Median Avg Stddev
x 100 0.4472467 0.46646308 0.45708424 0.45713057 0.0040838243
+ 100 0.44707586 0.46581227 0.45479448 0.45509309 0.0037548573
Difference at 95.0% confidence
-0.00203748 +/- 0.00108734
-0.445711% +/- 0.237862%
(Student's t, pooled s = 0.00392279)
...in addition to a net deletion of 144 lines of code.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
these can't work due to architectural limitations. they could be ported to
post-RA passes, I think, but having them here now is not helping anything and
they're in the way.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Due to interactions between file times and the lack of granularity in
futimesat, if we don't remove the nanoseconds then this test can flake.
Easy enough.
combined results on node from this and the previous commit:
N Min Max Median Avg Stddev
x 50 0.44433582 0.46988457 0.45344824 0.45320041 0.0043288623
+ 50 0.4385365 0.46615359 0.45026179 0.44996216 0.0045742233
Difference at 95.0% confidence
-0.00323825 +/- 0.00176704
-0.71453% +/- 0.389903%
(Student's t, pooled s = 0.00445323)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is changes the interface of CodeBuffer to that of a partially persistent
data structure based on reference counting:
- Exactly one CodeBuffer is now designated as "active", which means data can
be *appended* to it
- Lossy modifications to the active CodeBuffer will not invalidate any data
in use by other threads, which enables save sharing across threads
- Instead, such lossy modifications trigger a new "version" of the data in
the modifying thread. Old versions of the CodeBuffer persist as read-only
data for use by the other threads.
- The other threads can update their version of the CodeBuffer. This will
decrease the reference count and eventually trigger deallocation of the
old version
This data isn't really a cache, since the JIT is directly responsible of
writing its contents. Instead it be considered the source to populate the
L1/L2 caches from.
Furthermore, splitting off this data allows it to be shared across threads
in the future without affecting L1/L2 caches.
This has been a bug that we have technically lived with ever since SMC
tracking was introduced. The problem boils down to the fact that memory
management syscalls from multiple threads can race our SMC tracking.
This was only uncovered due to recent changes in the Steam client where
downloading games has more aggressively started reallocating memory.
This causes Steam to oversubscribe the CPU by a small margin, causing
threads to context switch more heavily during memory management.
The strace that finally managed to capture this:
```
41574 munmap(0xba84e000, 724992 <unfinished ...>
<...>
41227 mmap(NULL, 540672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -3, 0 <unfinished ...>
<...>
41574 <... munmap resumed>) = 0
<...>
41227 <... mmap resumed>) = 0xba87b000
```
While FEX's tracking linearly was:
```
mmap, 0xba87b000, 0x84000, 0x3, 0x22, 0xfffffffd, 0x0
munmap, 0xba84e000, 0xb1000
```
The way munmap and mmap perfectly interleave while getting context switched meant that the kernel's view of munmap then mmap didn't match our view of mmap completing first then munmap happening afterwards.
The kernel/strace is obviously the correct view in this instance.
This all comes down to how these threads are racing the VMA tracking
mutex after the syscall happens and not guaranteeing sequential
consistency that matches the kernel's view.
The only way to correct this sanely is to extend the locking period to
also encompass the syscalls getting executed. This is a bit tricky since
the VMA tracking needs to ensure that the lock is no longer held once
ThreadManager invalidation occurs so a callback to do the syscall
operation is about the only sane approach here. Luckily we now have
fextl::move_only_function.
Fixes consistent crashes with Steam game downloads (and maybe some
chromium crashes?)
FEXServer's logger performs such self-moves for the log pipe posix_descriptor
of short-lived clients. This resulted in a double-close previously, which
could interfere with file operations on other threads (typically crashing
FEXServer in effect).
some obvious opportunities to improve here! although mostly I want this to
regression test my constprop rework.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
as a simple post-RA peephole. much much easier to do post-RA than pre-RA.
This isn't a post-RA /pass/ in the traditional sense... it's done while
assigning registers to coalesce the passes over the IR, since we pay per-pass
and we can merge the walks over the IR.
Closes: #4480
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This matches behaviour with the x86 host runner that RSP isn't
guaranteed to be set to a valid memory location. Make sure when running
tests on ARM that it gets the same behaviour.
lots of instructions only exist for RA, so RA can garbage collect them before
post-RA passes (including the JIT) deals with them. this simplifies our life
now, and makes post-RA passes a LOT simpler for little cost.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Remove blows up because of use tracking, but we can do a much simpler version
for post-RA and elide lots of checks from trying to make Remove more general.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now that we can just set registers directly, we can simplify RA a lot. All the
Map/Unmap nonsense - it all goes away. We just assign registers as we go and
everything clicks into place naturally.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This sideband is now unused, registers are encoded directly in the IR. So we can
garbage collect all this code for quite some savings.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Beyond the actual registers allocated, there are two pieces of sideband data we
store in the RAData object:
* # of spill slots (explicitly)
* whether RA has run (implicitly by the existence of RAData)
We want to get rid of RAData, so we'll move these to the header.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
when we need to materialize instructions inside RA without having a
corresponding OrderedNode* source, we want to just pass a OrderedNodeWrapper
with an encoded register. generate appropriate builders so this is possible.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
this will let us encode registers directly inside OrderedNodeWrapper, rather
than pointers to OrderedNode *. that will let us speed up RA & onwards.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
There's no reasonable way to keep this around without adding significant
complexity to RA. This series prefers to drop complexity from RA, lessening the
need for validation in the first place.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Avoid hardcoding the file name. Upstream Clang currently expects the aarch64 variant of
compiler-rt. Changing this to use arm64ec in the file name could be problematic in the future,
if the MinGW toolchain gains support for ARM64X. Querying Clang for the correct file name is
the most forward-compatible approach.
nothing else does this, and it complicates upcoming refactor to move away from
IR builder helpers doing IR dereferencing.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Makes it more easily reflect what this functions are actually doing and
add a couple lines of documentation to help with the inherent opaqueness
of these.
NFC, just helps my brain wrap this more easily.
pthreads allows us to check if mutex/rwlock is currently locked by the
calling thread. This can give us some safety in code expecting locks to
be in place, allowing us to find programming bugs.
info files for the test runner like Disabled_Tests, Known_Failures,
etc, receive not just the filename but the test name (which is the test name
and potentially its running config).
Increase the timeout of gcc tests to 30secs.
With fortifications enabled, glibc long jump has some additional checks
in place that break because we do a stack pivot. The only way around
this is to do our own long jumps. Luckily this is trivial.
Fixes#4558
2025-05-06 15:29:42 -07:00
1199 changed files with 106501 additions and 79028 deletions
set(X86_32_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/toolchain_x86_32.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting i686")
set(X86_64_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/toolchain_x86_64.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting x86_64")
set(X86_DEV_ROOTFS"/"CACHEFILEPATH"Path to the sysroot used for cross-compiling for i686 and x86_64")
set(DATA_DIRECTORY"${CMAKE_INSTALL_PREFIX}/share/fex-emu"CACHEPATH"global data directory")
set(X86_32_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_32.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting i686")
set(X86_64_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_64.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting x86_64")
set(X86_DEV_ROOTFS"/"CACHEFILEPATH"Path to the sysroot used for cross-compiling for i686 and x86_64")
set(DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
set(HOSTLIBS_DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
* A manager that coordinates access to the CodeBuffer used for compiling new code across threads.
*
* The CodeBuffer is managed as a partially persistent data structure:
* - Exactly one CodeBuffer is now designated as "active", which means data can be appended to it
* - Lossy modifications to the active CodeBuffer will not invalidate any data in use by other threads (which is what enables save CodeBuffer sharing across threads)
* - Instead, such lossy modifications trigger a new "version" of the data in the modifying thread. Old versions of the CodeBuffer persist as read-only data for use by the other threads.
* - The other threads can update their version of the CodeBuffer. This will decrease the reference count and eventually trigger deallocation of the old version
*/
classCodeBufferManager{
public:
// Get the CodeBuffer that was most recently allocated.
// This is the only CodeBuffer that data may be written to.
fextl::shared_ptr<CodeBuffer>GetLatest();
// Allocate a new CodeBuffer with geometric growth up to an internal maximum.
// Subsequent calls to GetLatest will point to the returned buffer.
Loaded 100 of 1199 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ryan Houdek