With the prior approach, backwards jumps into existing blocks would
explore the overlapping part rather than splitting the block, generating
needless code and wasting time decoding. Similarly, the current block
wouldn't be split when it is extended to overlap with a pending jump target.
Solve this by tracking blocks in a sorted vector and splitting existing blocks
on jumps when appropriate, in order to avoid any possibility of overlapping
blocks, which would break the lookup, misaligned and zero instruction blocks
are disallowed.
This was causing FEXServer to look in to global installed paths and
local paths for things when FEXServer was started.
Ensure it listens to FEX_PORTABLE so this doesn't occur.
This also requires us to scan both data directories and config
directories to find them.
When #2722 implemented this initially and #4271 switched over to signed
int16_t there was assumptions made that int16_t was a reasonable
trade-off in encoding size versus needing to deal with 8-bit values
being too small in some cases.
In the common case we are almost always encoding 8-bit values because
instructions are typically linear (and less than 15-bytes in size), but
16-bit was chosen because optimizing JIT and multiple instructions that
don't cause exceptions can add up to larger than 8-bit.
Instead of hardcoding 16-bit values, implement a variable length integer
class where ~96.8% of values are 8-bit encoded, and the remaining 3.19% are encoded using 16-bit.
Due to some constraints that #4271 put in place, we can basically
guarantee currently that branch targets are within 16-bit. The VL class
does support 32-bit and 64-bit as well so if we change behaviour then
nothing needs to change.
Some stats when running Sonic Mania with multiblock enabled.
Encoded integers: 3,504,907
Encoded 8-bit: 3,393,095 (96.8%)
Encoded 16-bit: 111,812 (3.19%)
Encoded 32/64-bit: 0
Encoded Size: 3,615,181 bytes (3.44MiB)
Fixed encoded size: 7,007,604 bytes (6.68MiB)
Definitely worth using and saves the headache of large RIP/PC offsets
causing problems.
A major limitation of iostream is that you can't have reads or writes
with a safe interrupt. Instead rewrite the interface with Linux ppoll so
that these can be safely interrupted with a signal and return early.
When the alt-stack gets overflown then it is hard to see what went wrong
since the TLS variable is no longer accessible.
Protect the first page that contains the TLS variable.
Fixes#4320
When multiple threads simultaneously SIGBUS on the same address, one of them
will perform the backpatching while the other will detect the backpatched
instruction sequence and hence report the SIGBUS as "handled".
This typo broke the instruction detection logic: The second thread would
assume the source of the SIGBUS was unrelated to TSO emulation and hence
report the signal as unhandled (generally triggering program abortion).
In practice, this problem did not manifest as FEX does not currently share
CodeBuffers between threads.
For some reason steamwebhelper is setting a zero length environment
variable. This was causing an assert to be raised early as the web
helper was starting up.
Just stop trying to memcpy the zero length string, gets steamwebhelper
working in the steam beta client again
Anything less than three pages can't be used for FEX allocations due to
VMA implementation details. Plus we may have reduced a single page
reservation to zero with the prior ObjectAlloc size reservation.
When small regions were being used for VMA allocations (less than 64
pages), this function was truncating the result to zero. Resulting in
incorrect `LiveVMARegion` size calculations. It would calculate that the
FlexBitSet consumes zero bits of space, even though it needs to use at
least 2, or 3 if we actually want to allocate anything from that
LiveVMARegion.
This was noticed in this PR because our VMA region tracking is being
used more, which has a more likely chance to have small VMA regions for
allocating from. Cause a 1page allocation to try and use a 3 page VMA
region for allocation, but failing because the FlexBitSet size wasn't
calculated correctly.
- Page layout:
- [0x0, 0x1000): struct LiveVMARegion
- [0x1000, 0x2000): FlexBitSet<uint64_t> UsedPages
- ^ This space wasn't allocated/mprotected due to the size not
calculating correctly.
- [0x2000, 0x3000): Memory for allocation
When running on a system with a 48-bit VA, if FEX does any allocations
between us reserving the upper 128TB and the application running, then
/technically/ we are intersecting with the application's memory region
in the lower 47-bits.
This didn't typically result in any problems due to how ASLR works, but
if we did any large allocations (like #4291 wants with 128MB VMA region)
then these typically get pushed higher in the VA space.
Again not usually a problem, but if you happen to be running an
application that is using MAP_FIXED with hardcoded addresses then this
can stomp over FEX-Emu memory causing problems.
This is what happens with Wine, it reserves the upper-32MB of its 47-bit
VA space, which is /highly/ likely to stomp on FEX memory. In-fact it
likely occurs all the time, we just got lucky with whatever it was
clobbering wasn't used at the time.
On 39-bit VA systems this isn't a problem because the mmap fails
outright with a warning message from WINE.
Because we are already reserving the upper 128TB of VA space, instead
just always enable our allocator and use the regions that were reserved.
We need to be a little bit careful to ensure we don't accidentally
allocate more memory post-reservation but that just requires a small
adjustment to our unique_ptr and constructor for the 64BitAllocator.
This means /all/ FEX-Emu allocations will be in the upper 128TB VA space
when running 64-bit applications on a 48-bit VA system. Which is kind of
nice.
Fixes WINE in #4291 when the allocator stats are bumped to 128MB per
process.
These parsing strings are tiny, less than 64 bytes all the time. Just
stack allocate the buffer. Makes it safer to use during extenuating
circumstances as well, like SIGBUS and SIGSEGV.
Nasm between the versions of 2.16.03 and 2.15.05 starting changing the
operand order of the instruction on us. Hard code both operand encodings
to ensure coverage.
Fixes#4296
String parsing was adding a newline but then we failed to use it. I
don't think it caused any issues considering how infrequent instant
profiler objects are used.
Placing optimization hints everywhere interferes with debugging of
RelWithDebInfo builds, since the debugger won't be able to reliably
inspect variables or control flow. These hints are better placed on an
individual basis after identifying bottlenecks in a profiler.
Just four new *at variants of the xattr syscalls.
This will also let us use the *at variants for the non-at versions but I
didn't implement that optimization because this is brand new.
With multiblock enabled, host code generated from guest code with a
lower address may be placed after host code generated from guest code
with a higher address in a multiblock. As each guest RIP reconstruction
entry is always relative to the one before it the offset needs to be
signed to allow this.
This avoids both the generation of multiblocks that cover massive spans
of guest code, which causes issues for both context reconstruction
overflowing the RIP offset and attempting to decode branch targets
in unmapped memory regions.
Once support for querying mappings from the FEX frontend is in place this
limit could be increased if necessary, but this seems fine for now.
'add [rax], al' is almost never seen in actual code so the assumption
can be made that we are most likely trying to explore garbage code and
that this will never be hit. If it is then code will be generated at
that point (where Entrypoint == true).
Includes tests and instcountci files and tests.
When the x87 optimizations were implement, we missed
optimizing different addressing modes. This commit addresses this issue.
Discussed in #4252.
Currently FinalInstruction causes only to the currently decoding block
to be terminated, but that is not enough as both MaxInst and
DefaultDecodedBufferSize are global limits that apply across all blocks
within a multiblock.
Noticed this as I was scrolling through some cmake. Usually this doesn't
matter as we declare `BUILD_SHARED_LIBS` as False/Off, but this can
technically be overridden even when we don't want to.
Updates the two definitions of `add_library` that was missing the static
qualifier to ensure they generate the code we want.
The thread termination callback can be called for other threads in the
process, not just the current one, in which case we cannot call DeinitCRT.
Deinitializing the CRT of another thread would be awkward so just skip that
and accept the small leak for now.
It's faster to load the f80 sign mask from our named vector constants
than synthesizing the values. Changes a 4 instruction sequence to
synthesize to be 1 load.
The IR stores elementsize, where the json was wanting number of
elements. While the IR Emitter function declaration always wanted
element size. This was causing us to do a little dance from ElementSize
-> Number of elements -> ElementSize. Just pass the ElementSize directly
instead of this bogus little dance.
Make sure to pass the clone3 arguments all the way to the fork handler
so it can check the flags. Currently nothing I know of uses fork plus
the new clone3 flags, but it would be hard to see without any logging.
Brought up in #4225 where it had issues with Openat2 which was added in
5.8.
The main driving force around minimum kernel version requirement is that
the lowest kernel version in our CI is 5.15. A benefit to this choice is
that this is an LTS release, which is also what Ubuntu 22.04 is
shipping.
Once the single CI machine is fixed to ship something newer then the
next logical choice would be kernel 6.1 which is also LTS, but until
then just lift it to 5.15. This version was released in October 2021,
and is supported by the kernel developers until 2026. Our previous
minimum of 5.0 was released in March 2019, so a two year leap here.
This removes the openat2 workaround that was necessary to pass our CI
since it is no longer necessary.
Some early FEXServer startup log failures weren't getting printed
correctly. They were going through the LogManager but before FEXServer
setup, or even stderr/stdout logman setup. So they were just getting
written to -1 and failing.
Fixes#4155
Most of this table ignores REX.W, but two encodings change behaviour
based on REX.W. These two encodings are PEXTRD/PEXTRQ and PINSRD/PINSRQ.
For every other instruction encoding, they will ignore REX.W, but FEX
was requiring that they didn't have REX.W encoding. I had special cased
this in the past by adding PALIGNR, but that didn't handle any of the
other instructions.
We can't just handle REX.W in the OpcodeDispatcher and remove the two
special cased instructions because these vector operations also interact
with instruction prefix 0x66 which changes the operating size to 16bit
with regular instructions.
So instead just generate all listings of instructions with REX.W being
zero and one and install handlers in all cases.
No need to extract the subregisters out before operating on them since
the long division and long remainder IR operations correctly zero/sign
extend the incoming sources as necessary. Saves a couple of
instructions.
This avoids having to do the symlink chasing in GetEmulatedFDPath, since
the kernel does it for us. On top of that, with a merged RootFS
setup, this will correctly handle symlinks from user directories into
the RootFS, fixing wine on Fedora.
The immediate offset masking was at the completely wrong offset when I
wrote these handlers. No idea how I managed to mess those up so badly.
Should fix at least some of the issues with #4216
Factorio hammers its drawSprite() function and ends up cpu bound under FEX.
Unfortunately, its hot blocks seem to be translated pretty optimally :-/
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
- Parse the shebang line properly (use FHU::ParseArgumentsFromString
which is the same code the loader uses)
- Make native-interpreter shebang files work by deferring to the kernel
in that case (previously, they'd get executed through the loader and
it would choke on the architecture of the interpreter)
- Do not use the RootFS-prepended path when executing shebang files. The
loader will prepend that anyway when looking it up, but it needs the
bare guest path so it can pass it as an argument to the interpreter,
which (since it's emulated) will do the lookup through the RootFS.
With a merged RootFS, all binaries are executed through the RootFS. When
executing a binary that is actually a native binary, we want to do so
outside the RootFS. Handle this by stripping the RootFS prefix in that
case.
If a RootFS symlink links to an absolute path within the RootFS, we need
to strip the RootFS prefix. This would not normally happen with a plain
RootFS, but it can happen if /proc is mounted within the RootFS.
If there's a symlink to / within the RootFS, don't attempt to follow it,
since that will end up trying to look up the empty string within the
RootFS (which is not legal). Just return the symlink.
To locate whether a path is in the emulated list, EmulatedFDManager::OpenAt()
attemps to resolve the path. realpath() ends up calling readlinkat() on
every path component, which is a lot of syscalls for every open()
variant syscall. It also makes interaction with the rootfs complex and
error-prone.
There's a much easier way to do this: We just open the file without
emulation and check its real path via get_fdpath(). This is just one
readlink() syscall per open, instead of one per path component. If the
file turns out to be emulated (uncommon case), we swap out the fds.
This also decouples EmulatedFDManager from guest path resolution
entirely, so it will never fall out of sync with the RootFS logic.
This is going to get used by gdbserver soon for ensuring memory accesses
are fault safe, because it tries to read outside of correct memory
bounds at times.
It is not an error that pread returns /less/ than what was requested. In
fact it's very common for the Linux kernel to return less than the data
requested from procfs.
procfs keeps coming back to bite this function, previously it was fstat
returning size of 0 which it hit. Now it only feeds data as much as it
wants per loop. In particular /proc/self/maps would only read ~3k bytes
on my system, but not be complete.
To fully fix the issue, always make sure to keep reading until there is
either an error OR zero is reached!
if we want to replace a node with one of its sources, we need to zero extend if
the source is 64-bit and the destination is 32-bit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
When an SMC trap happens: reconstruct the context before the SMC write
then compile the write as a single instruction block to reduce it to
regular SMC. SMC where the writing instruction is the instruction being
patched will hit the signal handler at most twice: the 1st will trigger
the write to be compiled as a single instuction block, the 2nd will
detect inline SMC of a single instruction block and then just take the
usual invalidate+reprotect+continue step, avoiding a potential infinite
loop of recompilation.
guest instruction
Single instruction blocks need to be treated specially when inline SMC
is detected, the frontend only needs to reprotect RWX and invalidate
caches then continue execution as side effects from the SMC shouldn't be
seen until the instruction executes.
Frontends need to detect this in order to handle SMC within the current
block (inline SMC) differently to regular SMC which can just reprotect
and continue.
As section permissions are set on the unix side we don't get a
protection callback for them, workaround this by iterating over
the sections of all executables after mapping and tracking the RWX
ones.
When set - either via POPF or a thread context operation - the trap flag
raises a single step exception after the execution of each instruction.
As e.g. a JUMP instruction with TF set will raise an exception at the
jump target. Handle this on the FEX side by storing both the flag itself
(in bit 0) and a 'block exceptions' flag (in bit 1, inverted). Each
generated block when TF is set is then forced to a single instruction
with logic to raise the exception at the start. Initially after setting
TF exceptions are blocked, then at the start of the block they are
unblocked so that after the instruction executes an exception is raised
at the start of the next block.
This is the command used when the `k` argument is passed to gdb. There
is nothing to do once this is received other than "kill" as quickly as
possible. The absolute way to ensure this is using SIGKILL.
No way to do a `r` command after `k` yet, but might be possible.
Seems the unused mutex locking is able to cause some hangs according to #4198
Hard to tell why, but might as well as get rid of that potential
pitfall.
if an application is using `exit` then it is usually a faulting
condition rather than cleanly exiting. When cleanly exiting
applications will typically use `exit_group` instead.
`exit` is useful to quickly cause a single thread to exit in a
multi-threaded environment as well, where `exit_group` will take down
the entire process group.
FEX had implemented this in a way that would do a double Stop signal,
cascading to a crash. When tied in to a crash handler, this could get
caught in a weird way.
This /should/ fix#4198, but I can't confirm locally. It looks like in
that issue that the steam install is slightly buggered (as evident by
missing srt-logger and steam-runtime-identify-library-abi).
This is a bug regardless so fix it and create a unittest. If it doesn't
fix the user's bug, then we have another workaround that will definitely
solve it.
clone3 was added in Linux 5.3 but our minimum spec is 5.0. Additionally
the Raspberry Pi 5 kernel seems to complain about clone3 for some
reason?
Just use clone instead of clone3
Tests all the instructions that are affected by FCW PC (or not!)
Only missing tests are fsincos (More easily tested with just fsin and
fcos), and fpatan
According to the documentation for x87 FCW precision control, this only
affects fadd*, fsub*, fmul*, fdiv*, and fsqrt. FEX was incorrectly
reducing precision for all x87 operations.
Precision is ignored for the following x87 ALU operations:
- fabs
- fscale
- fprem{1,}
- fcos
- fsin
- ftan
- fyl2x
- fyl2xp1
- fpatan
- fsincos
- Plus any operations just doing data movement and conversions
Next commit adds unittests to ensure this is correct for each
instruction.
Now that all the threading behaviour has been correctly separated/moved
to the frontend, these functions serve no purpose.
- Instead of using RunUntilExit, all threads can use `ExecuteThread`
directly, since there's nothing special about the primary thread now.
- This also removes the public function definition of `ExecutionThread` since that was only used for threading logic.
- Instead of using an exit handler, just do the same cleanup after
`ExecuteThread` has returned.
- Just make gdbserver is cleaned up early if it exists since it may
want to send some things to the connected gdb instance before
threads are exited.
These are all frontend constructs with mostly deprecated constraints.
WaitingToStart isn't used anymore, Running is effectively always true
(and behaviour has changed that if a thread is alive, it's running).
The only one that remains is `ThreadSleeping` which is only handled in
the frontend, and there was some conflation between ThreadSleeping and
Running which was hard to gauge. So delete `Running` and
`WaitingToStart`, but move `ThreadSleeping` to the frontend.
Now that most of the thread tracking is in the frontend, change this
over to building the thread execution handler on the parent thread.
Removes a memory allocation/free pair, and removes the copy of each
variable in the child thread.
We were using this variable for two things, letting the frontend signal
to the backend that it wants to start executing once the thread is
created, and also for handling thread pausing. These two features are
conflated with one another and actually makes things more confusing.
- Move StartRunning/StartPaused to the frontend, because its a construct
that only needs to exist in the frontend
- Adds a FEX::HLE::ThreadStateObject CV for handling pausing, which only
needs to exist for gdbserver
FEXCore hasn't been returning anything other than EXIT_SHUTDOWN for a
long time, so this ended up just moving data around for no reason.
This isn't going to be used for further GdbServer work anyway, so just
completely remove it.
This is a Linux construct, move it to the frontend.
This is going to need some changes in the future since exit_group and
exit syscalls are supposed to behave differently than how FEX implements
it. For now just move it to the frontend.
GDB has two ways to read the registers. One way is reading the full
GDBContextDefinition, which matches the layout in `BuildTargetXML`.
The other way is to read the individual elements out of
GDBContextDefinition.
These two code paths were independently implemented. Instead generate in
one location and use in either location.
NFC
This currently happens in two locations, so split it out.
There's some behaviour here where if the TID isn't found, then it
returns the ParentThread of the process. This is working around a bug in
either FEX's gdbserver or binaryninja. Leave it currently before we
figure out what's wrong.
NFC
Previous `S AA` logic is legacy for non-multithreaded applications. This
newer command gives more information about what occured and in what
thread id.
Alloc::OSAllocator uses a TLS variable of the thread object so it can
use a forkable mutex plus a deferring signal section. This was setup
when the FEXCore "ExecutionThread" function is called, which is a bit
awkward and is an artifact from when the thread creation was mixed
between the frontend and the backend.
Instead let the frontend inform the backend when to install the TLS
variable.
This is one step required to make GdbServer work correctly again since
the thread initialization and pausing is awkward today.
This was working around an edge case in the GdbServer where a thread was
getting created while the process was shutting down. This edge case is
getting removed so get rid of it.
Previously, running ctest with BUILD_TESTS=OFF would discover and execute
leftover tests from a previous build. This change ensures CTestTestfile.cmake
gets regenerated so that ctest will see an empty test list in that case.
ltrim had the issue that it would always consume the left-most character
even if it wasn't whitespace. So `FEXLoader` would turn in to
`EXLoader`, even without any whitespace in the string.
Adds a test to ensure this doesn't occur again.
this is a pain to track and, it turns out, buys us virtually nothing on flagm
systems. rip it out.
this fixes a bug with failing to set in all the right places.
on non-flagm systems there's a slight instcountci impact, but that is mostly
mitigated by the earlier patches in the series. so overall a wash there but
worth it for making the codebase easier
to reason about.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Ensures reads don't go past the end of the page boundary.
SVE masked loads can make this more effective but `VLoadVectorMasked`
isn't setup to be efficient for this case yet.
Abstract sockets have one limitation: they are bound to a network
namespace. Chromium/CEF sandboxes using a new netns, which breaks
connecting to the FEXServer.
To work around this, use and try *both* abstract and named sockets. As
long as either the filesystem or the network is unsandboxed, things will
work. If both are sandboxed, there isn't much we can do... but at that
point we shouldn't be reinitializing the FEXServer connection anyway
since the FS should be available on FEXInterpreter startup.
Needed by Discord, part of the Chromium sandbox code. The warning still
triggers because Chromium asks for CLONE_VM on x86_64, but that can be
safely ignored (CLONE_FS is the one that matters).
Apparently Chromium/CEF can chroot or otherwise sandbox the filesystem
away before forking and checking for directory FDs, making /proc
inaccessible, which means we can't stat it for our inode check, breaking
the hiding.
So, double down on things and do what Chromium does: open an fd to /proc
ahead of time, so that continues to work. Then we use it to update the
inode of our RootFS fd instead, and finally, also do the /proc fd itself
to hide that one too.
We also don't need to check the st_dev of /proc more than once, since
that's not expected to change anyway.
Fixes cefsimple.
NFC
Finally converts the IR operations themselves to store the OpSize for
the IR operation size and element sizes.
This also finally, FINALLY, converts that remaining `_Constant` helper
to stop using a size field that is specified in bits rather than bytes
like all the other IR op handlers. That thing was so confusing and now
it's gone.
Chromium/CEF has code that iterates through all open FDs and bails if
any are directories (apparently a sandboxing sanity check). To avoid
this check, we need to hide the RootFS FD. This requires hooking all the
getdents variants to skip that entry.
To keep the runtime cost low, we keep track of the inode of
/proc/self/fd/<rootfs fd> (note: not the RootFS inode, the inode of the
magic symlink in /proc), and first do a quick check on that. If it
matches, then we stat the dirfd we are reading and check against the
procfs device, to complete the inode equality check.
As an extra benefit, this also fixes code that tries to iterate and
close all/extra FDs and ends up closing the RootFS fd.
The push operation already truncates the result, there's no need to bfe
it. Noticed this while cleaning up in #4134. Removes one instruction for
16-bit and 32-bit pushf instructions.
these are used in places:
* bsf/bsr
* pcmpblabla
* x87 fild
In all cases we explicitly check for zero and change the behaviour accordingly.
So weaken the IR op to let us optimize. No sense checking twice.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Allows the AVX128 implementation to use the same implementation as the
128-bit SSE implementation, because it works per 128-bit lane just like
SSE. Is a minor optimization.
vblendvps, vblendvpd, vpblendvb all broke because of failing to zext the
128-bit register correctly.
vinsertps broke because it was accidentally using the wrong
implementation.
Extends each of their unittests to handle these cases.
Fixes fxtract by returning the correct values for 0.0 and -0.0. We moved the split of fxtract into _sig and _exp, to the opcode dispatcher, to ease some comparisons.
Also removed the IR node F80XTRACTStack which is not needed anymore.
This was brought up by #3831 but I finally got the courage to look at
the hard problem.
Although I'm only tackling half of the problem with this PR, which is
that FEXLoader needs to strip the rootfs path from the executed path if
it begins with the rootfs, plus some changes to the surrounding code.
The primary concern here is that when an application has been executed
under FEX, specifically through binfmt_misc, then FEX needs to prepend
the full rootfs path otherwise Linux can't find the program.
Additionally execveat with an FD will resolve a full path to the rootfs.
So past FEX's initial setup, we need to strip off the rootfs path to
provide an "absolute" path that is visible to the guest application
later. Which is kind of funny since we have a `RootFSRedirect` function
which did the exact opposite. This was due to legacy problems in the
original ELFLoader that couldn't handle symlinks correctly, which has
since been resolved, so that no longer needs to exist.
There was also some weirdness in `GetApplicationNames` where the passed
in argument list was modifying Args[0] and then saving the Program as
well. Which I just got rid of. Also stopped passing in the arguments by
value because....why did I write it like that?
In InterpreterHandler we now need to check if we can open the path
inside the rootfs or fallback without it. Plus I had to change the
shebang handling so it stopped prefixing the rootfs AGAIN. Took the time
to change the shebang handling there so it stops creating string copies
and instead just generates views.
Overall this fixes a fairly major flaw with how we were representing
`/proc/self` to the application, which was breaking wine since it would
prefix the rootfs multiple times, which was weird.
It doesn't address the remaining problem in #3831, which is that
applications can still see some of the leaky abstractions with symlinks
through the rootfs, but I want to get at least this step in.
FEX's ASM unitests had the problem that they were copy and pasted
templates and MemoryRegion was copied in to almost all tests.
Very few tests actually use the MemoryRegion they were asking for and
instead used none, or the hardcoded memory regions that the
TestHarnessRunner provides.
This is entirely a sed replacement and minor fixups plus reverts for the
few tests that actually use the region asked for.
When writing `FEX_bugs/tls_vector_element.asm` I had to switch to using
GS segment instead of FS segment because the unittests didn't correctly
restore FS after running. This is because GS is unused on Linux
applications, but FS would become broken and break glibc cleanup on
shutdown.
Now that xbyak has been updated to v7.09, it now supports
{rd,wr}{fs,gs}base which allows us to save and restore the segments
correctly. This lets us drop in TLS tests in to unittests more easily
now.
Ensured this works by modifying the test temporarily to use fs instead
of gs again, seeing it crash before HostRunner changes, and work after
HostRunner changes.
Fixes#4104
The instruction definition only allows these instructions to load/store
94 or 108 bytes, not affecting any bytes afterwards. This is a bit
awkward because 80-bit x87 registers are stored at the end.
FEX has an optimization today where it uses overlapping loads and stores
for the first seven x87 registers, and a split loadstore for the final
register. This ensures that we get the correct data while reducing the
number of loadstores.
We didn't have a unittest in place to ensure we only ever write the
correct amount of data, so changes like in #4107 which look correct from
an initial glance, would have resulted in broken behaviour.
This unittest ensures both that the instructions don't try to access
beyond the end of the page, and also ensures that they don't overwrite
subsequent data. Making sure that potentially broken behaviour doesn't
make its way in.
The size of the input_event struct differs between 32 bits applications
and 64 bits applications. To deal with this, the kernel implements a
compat variant for the input syscalls, but it's only enabled for 32 bit
processes.
In libkrunfw we're introducing a prctl that enables a 64 bit process to
request the kernel to enable the compat variant for the input syscalls.
This commit makes use of that interface for enabling/disabling the
compat input variant as required.
The visible effect is that input devices such as gamepads work properly
on emulated 32 bit applications.
Signed-off-by: Sergio Lopez <slp@redhat.com>
Just requires a thread pointer check to be fixed in FEXCore.
This doesn't need unique pages to exist for the file mapping and can be
shared since it's readonly mapped.
These had completely forgotten to load segment prefixes on the element
loadstore operations. This had caused a crash on the Linux version of
"Halls of Tormet" (steamid 2218750).
This game has TLS vector data which it was loading with a movhps and hit
this path.
fixes incorrect signs on FPREM. in turn should fix end-to-end failures logging
into Steam.
Thank you to Sergio Lopez for tracking down the JavaScript fail, and Ryan for
finding the bug.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
every pattern costs us JIT time, but not every pattern is doing anything.
especially with the flag rework of 2023-2024, a lot of patterns just don't make
sense anymore. constant folding in particular isn't too useful now.
only instcountci change is AAD which i'm contractually forbidden from caring
about.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
hash tables have high overhead! but individual blocks are small, so the
quadratic search is _much_ faster in practice. knocks 8% off node.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is an interesting vdso implementation because it only exists in
x86-64 with kernel v6.11. For Arm64 the implementation is likely to land
in a couple kernel versions from now.
Some differences with vdso_getrandom versus the regular getrandom
syscall
- Has two additional arguments, opaque_state and opaque_len
- Expects the userspace to mmap/munmap this opaque state structure
- Lets userspace query information about this structure upfront
- If the opaque data structure isn't provided then it falls back to
regular SYS_getrandom
With the "glibc" implementation, we can tell the interface to allocate a
single page that gets unused (otherwise glibc ends up not using the
interface), and fallback to the regular SYS_getrandom.
In the case of the vdso interface, currently the arguments just get
passed through.
Keeping this as a WIP until the ARM64 kernel patches land and I can
actually test the things. Running the "glibc" path works today with the
selftest, but the vdso path is currently completely untested.
Between this version and our previous version v1.3.278 this adds a few
extensions.
- VK_EXT_device_generated_commands
- VK_EXT_depth_clamp_control
- VK_AMD_anti_lag
- VK_KHR_pipeline_binary
These extensions are almost immediately getting picked up by mesa and
the D3D layers, so we want to support them as soon as possible.
Once again updated as described in #2076. This is starting to take more
time as more features and extensions get added to Vulkan. We probably
want to update the `DefinitionExtract.py` script at some point to
contain all the tertiary data in some json so everything gets generated
automatically. Not doing that today but something to think about.
Fixes#4083
From prep commit 511103ee5dcc9474b0b7468c05f61ce10fea4393.
Now that the frontend is setup, we can remove the temporary TLS
variables and use the ThreadObject directly.
Moves from FEXCore to FEX::HLE. Also moves the ThunkFunctions that get
exposed to a namespace to make it more obvious that these are
thunkhandlers rather than just static functions.
With the new systemd support, we had forgotten to wrap this check in an
arm64 check. I had done an install and broke my x86 machine.
Add the check back so we stop breaking x86 machines...again. Took me
only about three hours to find the issue this time.
Global rootfs doesn't support a named rootfs (only local supports that).
So when it is a global rootfs option just give the full path to the
config path.
Just needs some special handling to ensure the full path is retained for
global paths.
A reimplementation of #4067 that doesn't hardcode paths and routes the
global data path through FEXCore.
Using select initially was a mistake, it has issues with FD limits and
is less efficient due to fd_set. It's recommended to switch select using
over to polling, so do it.
we had a bug with max of nans. since x86 behaviour is weird, add a unit test to
try all the weird things. this fails on main without AFP
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We were creating a copy of the FEXCore::Core::CPUState object when we
didn't need to. We can pass the host thread's CPUState frame through to
the creation handlers since it's read-only (so modify it to be const).
We then just move the RAX and RSP setting to /after/ the CreateThread
handling instead of before.
This reduces stack usage from ~1392 bytes to ~80 bytes.
NaN and signed zero handling was busted. Pretty nasty.
Fixes the following Vulkan CTS test run under emulation without thunking with a clang-built Mesa:
dEQP-VK.spirv_assembly.instruction.graphics.float_controls.fp32.generated_args.denorm_nmax_nan_flush_to_zero_vert
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
EnvLoader::Load was generating strings on the stack and then copying
them to the FEXCore::Config state. FEXCore::Config::Set supports
string_views directly which will emplace in to the map directly.
This improves this functions stack usage from ~4464 bytes to ~288 bytes.
Since the frontend has changed to informing the backend if AVX is
supported, there is no reason to feed that configuration back in to
SignalDelegator from the backend.
Instead inform the SignalDelegator directly in the frontend instead of
this now weird round-about path.
This was a legacy interface that has long lost its implementation and
has long lost its use. This interface was similar to an older
Dolphin-Emu interface that was useful in that project, but isn't really
useful in a world with hundreds of threads and processes running at the
same time.
Previously reported as some old CPU without AVX and SSE4 and other
things.
Start advertising as something more modern that actually shipped with
AVX2 and other features. Should help some modern libraries that do bad
family and model checks rather that CPUID features checks.
Also removes the silly `(ES)` tag from CPU-Z.
Also moves generation in to a constexpr function that can actually range
check these 4-bit and 8-bit values.
Only doing the single table for review purposes. Once reviewed I will
hammer out the remaining tables.
Similar to #3320, most of the OpcodeDispatcher tables can be consteval
and made to be a compile time constant. This just requires shuffling the
code slightly. The idea is to get almost all of the table setup out of
the `InstallOpcodeHandlers` function and instead only install the
handlers that change based on 32-bit or 64-bit, just like the x86 tables
we also did.
Upstream has rejected this flag which would have let FEX close the gap
in functionality between binfmt_misc interpreters and PT_INTERP
interpreters for how `/proc/exe` is handled. Since we are unable to
change their opinions, just remove the code from FEX since it's never
going to happen.
Removes a little bit of confusing behaviour in execveat and
filemanagement handling. Leaving us with only the regular confusing
behaviour of trying to track accesses to `/proc/exe` instead.
This was causing problems in a typical use case of piping stdout/stderr
to another application. So if FEXInterpreter was started, with stdout
piped to another application, and FEXServer wasn't running so a new
instance needed to be started. FEXServer would write a null to every
pipe then close it. This includes the stdout that is being piped to the
new application!
This writing was legacy behaviour before FEX was properly listening to
POLLHUP and is no longer necessary. Just remove the writing completely
to fix this issue.
Easiest test case was just `./Bin/FEXInterpreter `which ls` / | xxd | head -n 3`
Returned:
```
$ ./Bin/FEXInterpreter `which ls` / | xxd | head -n 3
00000000: 0000 0000 6269 6e0a 6269 6e2e 7573 722d ....bin.bin.usr-
00000010: 6973 2d6d 6572 6765 640a 626f 6f74 0a64 is-merged.boot.d
00000020: 6576 0a65 7463 0a68 6f6d 650a 6c69 620a ev.etc.home.lib.
```
That first null shouldn't be there.
CEF on Windows patches these before FEX is even loaded, so this needs to
be setup as early as possible using some direct syscalls which skip any
such patches.
No functional change here.
- CoreRunningMode enum and variable wasn't used anymore.
- Code was moved to the frontend
- CustomCPUFactory wasn't used anymore
- All special signal handling and various features were moved to
TestHarnessRunner
- We also don't want to support actual custom CPU cores.
- TestHarnessRunner just runs as a host runner if compiled on an
x86-64 device if vixl sim isn't enabled now.
- Removes the Core config option entirely.
- Moves VDSOPointers struct to the frontend
- Every use of this lives in the Linux frontend instead now
This is one of the most expensive headers in FEX, averaging 1.2 seconds
of compile time per include.
We include this header 31 times around the project. Remove
the five instances where it is unused to help this issue. Next step
would be to make the header lighter if possible.
Since `HadError` is a lambda that handles FD closing this was a double
FD close. Just move the xxhash state freeing in there as well and remove
the FD and hash freeing. Fixes the double free.
Like how FEXCore disabled vixl linking, make sure we aren't accidentally
compiling and including it when vixl stuff is disabled. Noticed this in
the build logs.
We have been relying on the
`YesIKnowImNotSupposedToUseTheGlibcAllocator` fault avoidance for a long
while now. The plan was to rewrite the symbol fetching in the future to
avoid this since VDSO is kind of special anyway.
That time is now, I have had this VDSO symbol parsing code living in a
different project for a few months now and it is working great.
The basic things here are that the Linux kernel provides the VDSO
mapping base pointer through the auxv value `AT_SYSINFO_EHDR`. We then
need a minimal ELF parser that /only/ parses the dynamic symbol header
(and accompanying string header).
Then it's a simple case of walking the symbol table and recording the
pointers. Confirmed this fetches all the correct symbols (As I've been
using it for a while already I already knew it worked.)
This lets us stop allocating memory through the glibc allocator due to
dlopen.
`Extract` was never set to false even if the user said not to overwrite.
Ensure that if the user decided not to overwrite an existing folder,
that it returns false so it correctly doesn't overwrite the folder.
When the shift amount is >= 16-bytes then we need to zero the register.
We had a bug where we were assigning `Result.High` to itself, which
effectively made the top 128-bits of the ymm register not modify itself.
Adds a unit test to ensure that doesn't happen again.
more efficient to pool the output of constant folding and such. but we need to
avoid that becoming accidentally quadratic, so rework that too
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The previous approach assumed that ntdll wouldn't have been patched
before FEX was loaded, but that isn't necessarily the case thanks to
CEF. This takes the same approach used by xtajit which wine recently gained
support for wine. The overhead to this isn't ideal but most syscalls
aren't directly done through x86 code anyway and in the future FEX could
forward unpatched thunks to their ARM variants at JIT time.
it's only really load bearing for pf/af, which is handled as a global flag opt
now. this mitigates some of the compile time hit from globalizing flag opts.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
another global CFG-based optimization -- if we know that the raw PF is already
1-bit we can skip parity evaluation, saving work with floating point compares.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This works fine on real Windows and is relied on by wine as SystemCall
is set to 1 in KUSER_SHARED_DATA, which causes the ntdll thunks to use
it over `syscall`
The pointer is allowed to be null or garbage if the number of nfds
passed in is zero.
Unify the x86 and x86-64 implementations to ensure consistency.
Fixes Warhammer 40k: Relics of War
Because we already pooled the _Constants, there's no benefit to also pooling inline constants. the robin map to do so just adds extra overhead for no benefit - drop it.
without multiblock, shaves around 2% off node. with multiblock, a bit less than
1% but still a win.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
These will differ depending on which runner they are running on. Just
disable them but keep the instructions around so show how bad rdtscp is,
but how good rdtsc is.
When I implemented TSC scaling originally, I chose a scale factor of 128
because it basically covered the range of devices we cared about without
going too high. I also only tested devices that had a TSC scale factor
from 19.2Mhz to 34Mhz. Turns out there is hardware that also has a 48Mhz
cycle counter, which cause them to effectively have a 6.1Ghz cycle
counter, which is kind of absurd.
Instead of a fixed scale, just calculate the amount of scaling we need
to get >= the minimum threshold of 1Ghz. This will change the shift from
7 to 5 or 6 for the faster cycle counter devices.
Of course if someone wants to know the scale factor they can still use
cpuid function 15h to know it.
Fixes#4026
Seccomp is a relatively complex feature that was added to Linux back in
2005, and was further extended in 2013 to support BPF based protections.
Once seccomp is enabled, you can no longer disable seccomp but
additional protections can be placed on top of existing seccomp filters.
Additionally seccomp filters are inherited in child processes, which
ensures the process tree can't escape from the secure computing
environment through child processes.
The basis of this feature is a shim that lives between userspace and the
kernel at the syscall entrypoint.
In "strict" mode, seccomp only allows read, write, exit, exit_group, and {rt_,}sigreturn to function.
When in "filter" mode, a BPF filter is run on syscall entrypoint and
returns state about if the syscall should be allowed or not. Multiple
filters can be installed in this mode, all of which get executed. The
result that is the most restricted is the action that occurs at the end.
There are some significant limitations in filter mode that must be
adhered to which makes executing this code inside of kernel space a
non-issue and effectively limits how much cpu time is spent in the filters.
Although these filters are free to do basically anything with the
provided data, just can't do any loops.
FEX needs to implement seccomp because there are multiple applications
using the feature, the primary one being Chromium which some games embed
without disabling the sandbox. WINE also uses seccomp for capturing
games that do raw Windows system calls. Apparently Red Dead Redemption
is one of the games that requires this.
While FEX implements seccomp, it is not yet all encompassing, which is
one of the reasons why it isn't enabled by default and requires a config
option.
**seccomp_unotify is not implemented**
This is a relatively new feature for seccomp which lets the seccomp
filter signal an FD for multiple things. Luckily Chromium and WINE don't
use this. This will be tricky to implement under FEX since it
requires ioctl trapping and some other behaviour
**ptrace isn't supported**
One feature of seccomp is that it can raise ptrace events. Since FEX
doesn't support ptrace at all, this isn't handled. Again Chromium and
WINE don't use this.
**kill-thread not quite correct**
This isn't directly related to seccomp but more about how we do thread
shutdown in FEX. This will require some more changes around thread state
tracking before fully supporting this. Chromium and WINE don't use this.
kill-process also falls under this
Features that are supported:
- Strict mode and seccomp-bpf mode supported
- All BFP instructions that seccomp-bpf understands
- Inheriting seccomp through execve
- This means we serialize and deserialize the calling thread's
seccomp filters
- An execve that escapes FEX will also escape seccomp. Not much we
can do about it
- TSync - Allowing post-mortem seccomp insertion which allows threads to
synchronize seccomp filters after the fact
Features that are not supported:
- Different arch qualifiers depending on syscall entrypoint
- Just like our syscall handler, we are hardcoded to the arch that the
application starts with
- user_notif
- ptrace
- Runtime code cache invalidation when seccomp is installed
- Currently we must ensure all syscalls go through the frontend
syscall handler
- Runtime invalidation of code cache with inline syscalls will get
fixed in the future.
This currently isn't enabled by default because of the minor feature
problems that haven't been resolved. Currently the Linux Kernel's test
application works for the features that FEX supports, and WINE's usage
can be handled by FEX. Chromium's sandbox doesn't yet work with this PR,
but it only fails due to features unrelated to seccomp.
Having this open for merging now so we can work to resolve the remaining
issues without this bitrotting.
This must have happened during a refactor or something, but since we're
making a copy of the function pointers, it would have only gotten the
version /prior/ to loading host VDSO symbols.
Moves the VDSO thunk definition setting to the end after VDSO symbol
definitions in order to get host VDSO symbols working again.
Two primary things here:
- Remove the static `GlobalDelegator`
- Move the thread_local SignalDelegator::ThreadState information
directly in to ThreadStateObject
Having the ThreadStateObject and the SignalDelegator information
disjoint was confusing but was required when we didn't have any object
in the frontend that could have its own independent data. Since we fixed
this with the `ThreadStateObject` type we can now move this over.
The `GlobalDelegator` object is now instead stored in
`ThreadStateObject` instead.
Instead of using a thread_local variable, we now just consume 8-bytes of
the signal alt-stack since the kernel gives us that information about
where it lives. This then converts all the thread_local usage to use
either the passed in CPU state if it exists, or fetching it from the
alt-stack offset.
Very minor changes in behaviour here, will help when trying to improve
FEX's behaviour around signals.
The Ultra-class SoCs are two Max-class SoCs connected via
Apple's fabric, and thus use the same core revisions as the
Max-class SoCs for both big and LITTLE cores.
Signed-off-by: James Calligeros <jcalligeros99@gmail.com>
These are a Linux construct and should live here. Removes a weird
passthrough API from FEXCore and keeps it in the frontend instead.
This isn't even typically allocated in a real setup, as it's only a
fallback for if VDSO isn't loaded.
The CallbackReturn function stays in FEXCore because it would have
caused an API in the other direction instead.
This variable can't be constexpr initialized since it requires linker
fix-ups, which changes it in to a global static initializer instead.
While this benign as it doesn't allocate any memory, just move it next
to its single use. Removes the static initializer that was there for no
reason.
This is useful for some people.
This doesn't allow building thunks due to the cmake limitation about
tools binaries also getting cross compiled, we need to switch to meson
if we want that, since they recently started fixing this
https://github.com/mesonbuild/meson/pull/12022
But that's an okay limitation for now.
For the remaining syscalls we need to introduce some new concepts around
checking if pointers and string arrays are readable and I don't want to
do that yet.
No functional change, just moving the code.
This is entirely freestanding from the rest of the signal delegator
handling and mostly gets in the way when I'm working with the rest of
the signal handling. Separate it out to improve readability.
Our JIT will happily consume incorrectly formed 256-bit vector operations in a lot of cases when the host CPU doesn't support 256-bit SVE.
This is what caused the bug in #4006. For every vector operation that
can consume a 256-bit size, add an assert that always checks if 256-bit
SVE is supported in those cases.
This will ensure that #4006 doesn't happen again.
Just like the bug in #4006, we were incorrectly passing a 256-bit
comparison in to the float compare IR operations. Once again it would
"safely" decompose in to 128-bit operations without issue.
PR #4007 added asserts to ensure that 256-bit operations aren't emitted
when the host CPU doesn't support 256-bit SVE and detected this.
This handler was incorrectly using 256-bit IR operation sizes. Due to a
quirk with our IR handling, this would "safely" fall back to a 128-bit
operation and work "correctly".
The problem encountered is that since the IR operation is claiming to be
256-bit, when the value got spilled due to register pressure then a
true 256-bit store and load operation would be generated. This would
then emit an SVE load and store, with the expectation of 256-bit SVE
loadstores. This caused a SIGILL on Oryon since it doesn't support SVE,
but even would generate an invalid predicated loadstore on SVE 128-bit
hardware.
Fixes Aperture Desk Job in FEX.
A source of overhead with multiblock is hitting instructions through a
conditional branch that can never be executed. Usually AVX512
instructions in glibc. This causes us to emit partial blocks for a ton
of targets that will never get executed.
Instead, when we have multiblock enabled, if a block hits an instruction
encoding we don't support, then remove all the decoded instructions from
the block and early terminate it if it isn't the entry block. This
resolves the issue of emitting a bunch of IR and code for blocks never
executed.
If the block of code has an invalid instruction in the entry block for
decoding then it'll still emit code up to the invalid instruction and
raise a SIGILL. This has the potential for generating some additional
blocks of code if a game is abusing SIGILL, but since that's unlikely
it's a good trade-off.
Also removes a few log instructions that don't really provide anything
anymore and just show up as confusing messages when multiblock is
enabled.
For atomics that cross the 16-byte or 64-byte granularity, we need to
lock a mutex to ensure strict emulation of split-locks.
I took another look at these when I found out that Zen3 actually
implements split-locks. Not sure which architecture actually added
support for from them, but I wanted to ensure we have the ability to
handle this.
One thing that we can't handle in user-space is cross-process
split-locks through shared memory. This requires a kernel SIGBUS handler
to ensure a crashing/SIGKILL'd process doesn't lock all FEX processes in
the system.
This fixes a little split-lock abusing test that I have locally. It's a
bit flakey so it isn't viable to run in CI. Considering it is explicitly
testing a race problem.
Required by newer insider preview versions, I noticed many crashes with
this exception number and QueryPeformanceCounter in the backtrace,
testing with XTA found it to not be passed through and instead write
the host CNTVCT (unscaled) into RAX. No other registers seem to be
affected.
The MIDR querying is inherently OS specific and needs a bit of special
casing. Instead let the frontend inform FEXCore how many CPU cores there
are and their MIDRs instead.
This lets us keep the Linux specific code in the frontend.
Fairly trivial because we already supported deferring these signals. We
had just failed the final step of blocking the signal if we can't block
the signal (like with SIGSEGV, SIGBUS, etc).
Once the guest unblocks the signal mask with sigprocmask, the signal
will fire again.
Apparently older glibc relied on this behaviour for signal raising which
this fixes.
There is a common arch bit that is included in each individual mask,
explicitly test for equality against the whole mask to avoid just
testing for this.
When #3899 refactored some of this code, it had split an if-else chain
in to two independent if-else chains. Turns out there was a minor
dependency between them. Add back the few checks necessary to ensure
that FEAT_LRCPC falls through before hitting the FEAT_LSE checks.
This is necessary because their instructions share an encoding so the
masks between the two instruction classes overlap slightly.
useful for checking blow up ratios. I keep calculating this by hand so let's
just add it to the json
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
PR #3980 is adding a feature to merge loadstores in to paired
loadstores, but it was using the incorrect atomic check to determine if
it can safely merge them or not. It was using the GPR atomic check
instead of the vector atomic check. While this would improve performance
on Apple Silicon with its hardware TSO implementation, it would have had
zero impact on Cortex and Oryon.
Instead split out the three config options to live as a boolean check in
the ContextImpl similar to how we disable "AtomicTSOEmulation". Removing
the various configs in the JIT and CPUID so that it queries from the
same context. This makes it clearer that if you are wanting the current
active configuration for memcpy, vector, or general atomic TSO
emulation, you should query one of those three getters.
This also fixes a weird edge case bug in the arm64 JIT where you could
have TSO emulation disable, but still have vector TSO enabled partially.
Just because half a config wasn't checked in {Load,Store}MemTSO for
vectors. If the global "TSOEnabled" option is disabled then TSO should
always be disabled.
Alyssa will be able to pull this in to #3980 once merged and get the
performance uplift on Cortex and Oryon, since our default configuration
is to have vector and memcpy TSO emulation disabled.
Adds support for binfmt_misc through systemd configuration paths. Their
configuration files are basically the raw kernel interface description
in a .conf file, quite a bit more simple than the legacy debian path.
Default enable this path since systemd is the expected default
arrangement these days.
Fixes#2417
rotate right by less than 8:
ror(________________7654321076543120, ...)
rotate left by less than 8:
rol(76543210________________76543210, ...)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
avoid masking. apparently even modern compilers will do cute tricks with 8-bit
math in hot loops ...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
With recent bug fixes, WFE now can sleep for roughly as long as the
programmed architecture timer of 100 microseconds. Still nowhere near as
close as what x86 CPUs can get with waitx and waitpkg, because those
don't get spuriously woken up by an architecture timer.
100 microseconds is a significantly improvement over 52ns although.
not used. we'll probably rip the whole thing out at some point but for now, no
reason to pollute user systems with this.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Instead of clearing a hardcoded 16 bytes, adjust for the actual number
of instructions modified. The implementation will still only clear a
single cacheline so it doesn't change behaviour.
This used to exist in the FEXCore header since the unaligned handler was
done in the frontend. Once it got moved in to FEXCore it had stayed
there. Move it over now.
In the case of a visibility tear when one thread is backpatching while
another is executing. The executing thread can /potentially/ see the
writing of instructions depending on coherency rules or filling of
cachelines.
By ensuring the DMB instructions are backpatched over the NOP
instructions first, this ensures correct atomic visibility even on tear.
When code buffers are shared between threads, FEX needs to be careful
around backpatching its code buffers, since one thread might have
backpatched the code that another thread was also planning on
backpatching.
To handle this case, when the handler fails to find a backpatchable
instruction, check if it was already backpatched. This can be determined
by atomically reading the instructions back and seeing if they have
turned in to the non-atomic variants.
In most cases we can just return saying that it has been handled, in the
case of a store we need to back the PC up 4 bytes to ensure the DMB is
executed before the non-atomic store.
These handlers don't do any code backpatching so locking the spinlock
futex isn't necessary. Move them before the lock to make them a bit more
efficient once code buffers get shared.
If we query the CPU flags ourselves then vixl is no longer a
compile-time or runtime dependency unless the vixl disassembler or
simulator is built.
A bit spicy from all the feature bits we need to load up.
they are both tiny and seemingly unmaintained upstream, let's bring them
under our own umbrella in tree.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Somewhere there was an assumption made that INC and DEC supported the
repeat prefix. This isn't actually the case, while the prefix can be
encoded, it is a nop and should only expect to be used for padding.
Adds a unittest to ensure that behaviour is as expected.
With the previous Copy{To,From}User helpers we need to actually
implement the handlers correctly. We want something that is a bit
lighter so we don't need to implement the faulting path in the syscall
handlers.
Implements a handful of helpers that just check for readable and
writable capability which can be thrown in to an assertion handler that
is zero cost in release mode.
Readable is checked by just attempting to read all bytes.
Writable is checked by attempting to read each byte and writing it back
to the same location.
Uses these helpers in x64/FD.cpp to showcase how they will be used to
detect EFAULT. Tested locally that they work correctly by writing some
small tests for the syscalls that expect EFAULT.
We now have two types of destinations:
* regular destinations. These are SSA. You get exactly 1 per instruction. This
is what almost every instruction should use.
* special destinations, introduced here. These are *not* SSA. They must be
allocated with a special instruction (added later in this PR), and then they
are mutated by the instruction. There are two types, either pure destinations
("out") or read-modify-write source+destinations ("in-out"). The former are
useful for instructions that return multiple destinations, like Memcpy. The
latter are useful for instructions that need a source tied with a special
destination (currently just Pop, introduced later in this series).
Special destinations reuse the mechanism of sources, to get around the
limitations on regular destinations in our current IR. Ops with special
destinations desugar to ops with no destination but extra sources prefixed Out
or Inout.
They further require HasSideEffects so we don't optimize ourselves into corners.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
- We can have the SyscallFunctionDefinitions be the correct size out of
the gate. Both tables are always 512 entries in size.
- In the RegisterSyscall_{32,64} handlers, just get the reference using
operator[]. We always know we will be under the size of the array, add
a an assert to check. Removes a bit of vector range checking overhead.
- Namespace 32-bit syscalls like 64-bit syscalls and include in the
regular header like 64-bit. This was just an oversight
- Use std::fill for the syscall gap for the invalid syscall, just a
minor cleanup.
No functional change.
I don't know what I was thinking when I wrote that code. Drop the silly logic
and let ConstProp inline the immediates. This fixes a lot of silly code
generated for 32-bit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
When a suspend request occurs on a running thread, Windows gives the JIT
~10s to pause and NtContinue with the x86 context. Resumption is then
done through BeginSimulation.
ARM64EC doesn't have an explicit callback for suspend, instead this task
is delegated to the kernel setting a bool at a JIT-defined memory
location, as such the faulting based approach as was done for WOW64
cannot be used.
After thread creation, the WOW64 CPU area context needs to be flushed
into the FEX state before entering the JIT. Wine explicitly calls
BTCpuSetContext to trigger this but Windows doesn't.
Some programs will hook the NTDLL exports that FEX depends on, the
regular ARM64EC call checker will detect such patches and invoke the
JIT to run them, which leads to infinite recursion if those same
exports are used during code compilation. Fix this by resolving all
patchable FFSs to their native ARM implementations for all indirect
calls performed by FEX, skipping any x86 patches.
The class constructor for ContextImpl::CPUID requires HostFeatures to be
available at construction time. Pass the host features struct directly
through during construction time instead, which cleans up the interface
slightly and fixes that issue.
These are only missing if using the hostrunner and the CI machine
doesn't support that particular feature. FEX otherwise always supports
these feature flags so they don't need to exist as options.
Just check the feature bit directly in the HostRunner frontend for these
bits.
These are already linked in by default with clang, sp having these set
here only served to prevent the -nostdlib compiler option from having
any effect, as CMake will always explicitly include the libs in the
compiler cmdline.
It is dangerous for FEX to rely on the system CRT as calls can have side
effects that are also visible to the running application, and if patches
are applied to any CRT exports used during compilation the call checker
would trigger a reentry into the JIT to compile the patch and hence
deadlock. Only functions that FEX actively uses are implemented, with
the rest triggering an abort.
This moves the CPU feature querying to the frontend. The primary purpose
here is for the wow64 frontend to not require linux-isms for querying
these features. This is required since non-Linux environments don't have
the "CPUID" feature for reading EL1 MSRs in EL0.
Wiring up the remaining wow64 registry querying is left for a future
exercise.
This also technically removes an xbyak requirement from FEXCore for when
building the x86 Test harness runner, but that doesn't really matter for
regular use cases.
Started by cherry-picking some cases from the variants that appeared when running
Steam, games, AV1 convolve tests, openssl, ffmpeg, libjpeg-turbo,
openh264, libvpx, gemmlowp, libyuv, and dav1d.
Then turned it around and optimized them all since all variants end up
needing to be split in to two halves, that effectively means we need to
have 16 implementations, plus a couple of special cases for duplicated
results.
Fixes#3795
These two maps used for environment lookup translations were getting
globally initialized and then registers with atexit handlers.
Switch over to a constexpr array and just do linear scans. This plus
short-circuiting the environment loader so it skips all entries that
don't start with `FEX_` has the side benefit of cutting the CPU time to
1/10th the time.
This plus #3917 removes the global static initializers entirely from
this file.
An exception in JIT code acts as a transition to ARM64EC code (in
NTDLL for exception handling etc) as such, much like ExitFunction,
InSimulation must be unset. InSyscallCallback is unset for robustness
against exception in the JIT itself.
Only NZCV and TF are passed through to BeginSimulation as the rest are
lost when converting to a native context and back on the ntdll side. To
prevent thread suspension from wiping out the rest of the flags, only
copy these specific flags into the current JIT EFlags state.
Most syscalls on Windows are done by calling into their NTDLL thunks,
however some DRMs parse out their numbers from NTDLL and directly call
them. Support this by redirecting to their entry thunks in the FEX
syscall handler.
To prevent FEX from redirecting to x86 code when NTDLL exports it calls
into are patched, a custom call checker will be used that checks this LUT
to redirect calls rather than the FFS itself.
- IsImmLogical already existed in our CodeEmitter. We just forgot to
allow nullptr arguments and to use it.
- Adds an equivalent IsImmAddSub helper and uses it
This gets us closer to removing vixl's global initializers from FEXCore.
Saw this vector was getting initialized at runtime, sticking around, and
installing an atexit handler. This is completely unnecessary, just use
the OPT_BASE handler directly to walk the environment variable names.
This is required for handling SMC with the ResetToConsistentState
arguments as used in Windows, as using the NTDLL exported NtContinue
would wipe out any reserved registers in the ARM64EC ABI.
For Windows the syscall numbers are somewhat stable, and the SVC
instruction can be called directly. Since wine doesn't handle that on
ARM64, hardcode the system call number and manually call into wine
dispatcher. Once wine gains proper syscall thunks, those can be
parsed to get the number and the hardcoding dropped.
OutputDebugString etc are exception based and thus don't really work for
FEX's needs as often times logs can happen in places where exceptions
cannot be thrown.
The prior approach using the L2 cache was flawed as it assumed L2
page entries had a 1-1 correspondence with actual pages. While the L2
cache could be extended to handle aliases with EC, this could lead to
thrashing etc. The cost of a lookup in the actual EC code bitmap is
cheap enough to perform every time considering the infrequency of calls
to ARM64EC code when compare to X86 L2 hits.
In the environment section this was causing the next environment
variable line to be merged with the strenum options
Also makes it so strenum options doesn't have a spurious comma at the
end of the list.
Different approach from #3579
Instead of completely ddropping the deprecated path, support the new
path and the old path using python try-except import exceptions.
This allows us to continue using old packages in CI, while supporting
the future API once pkg_resources gets deprecated and removed. Best of
both worlds.
Using a brute force solver to add in more optimized code paths
- Adds 12 single VInsElement implementations
- Adds 4 two IR operation implementations
Not adding any of the two or three IR operation implementations that use
VInsElement because SRA interacts badly and becomes worse than the VTBX
implementation.
This has been leaked state to FEXCore for quite a while. FEXCore never
actually needed this information, moves the bits to the frontend that
are necessary.
Minor behaviour change that `RunUntilExit` now just assumes the primary
thread is using it. This behaviour is on the chopping block to get
removed next anyway.
Instead of passing the TID back to the exit handler, just pass the whole
thread object. This will allow some cleanups with the frontend thread
tracking soon
NFC
Runs through all immediate encodings for vpblendw and crcs the results
to ensure correct behaviour. This was just a concern because of the typo
in documentation. But it is also good to have.
Optimizes the AVX128 blends by reusing the prior SSE4.1 implementation.
Only difference is the destination register isn't reused as a source
register.
One confusing thing is that Felix Cloutier's documentation has a typo on
the 256-bit VPBLENDW instruction where it had the top 128-bit lane
reusing the destination instead of sources. So I wrote a unittest to
ensure correctness.
Fixes#3796
stop prefixing the arguments when we generate allocate ops (in particular), this
is more convenient and simpler. in exchange we need to prefix Op to avoid a
collision on fcmpscalarinsert which has an argument named Op, but that's a local
change at least.
came up when experimenting with new IR, but I think this is probably a win by
itself.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
now that we do everything via NZCV, this is mostly vestigial. DF/x87 flags are
sufficiently rare to be "don't care"s here, and we don't even have multiblock
enabled yet!
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is no longer necessary and it also no longer provides us any useful
information. Since we expose the AVX CPUID flag, basically everything
uses VEX encoding now, so it is basically always set.
Some locations could end up with SRA registers that only spilled one
register.
Allow passing in temporaries from the call site.
Fixes rpid and syscalls asserting.
When AFP is supported then we can actually support DAZ. This might also
fix the audio corruption in Animal Well but I can't test it until Steam
is running on Oryon. Requires a bit of plumbing for MXCSR which we were
hacking around before but now we actually want to store the value.
Fixes#3856
A disowned buffer could be unclaimed or claimed by a different thread in
the time between the !IsFree check and locking the allocation mutex.
Fix this and prevent such errors in the future by always checking
ownership with the allocator locked before attempting to unclaim
buffers.
We support leaf functions now, so add the few that were calling for it.
We will be gaining support for the xsave ones relatively soon, so its
good to have them supported.
Also deletes a couple of cdt/cqm things that aren't exposed and we won't
be supporting.
With a brute force search of methods between 1-3 instructions we cover a
lot more cases more optimally.
There's definitely still more cases (and probably some that can reduce
from 3 instruction to 2), but covering 44 cases is a pretty good margin
already.
VPSHUFD and VPERMILPS are aliases of each other.
Reuses the implementation path from the PSHUFD implementation which has
a few swizzles and then a table lookup.
VPERMILPD is a very simple swizzle per 128-bit lane.
Fixes#3797Fixes#3784
nothing is optimizing around this, it's just adding pointless complexity. if we
want to actually optimize F80Cmp, the right way would be to lift the
implementation into the OpcodeDispatcher or JIT. it wouldn't be terribly
difficult. This kludge doesn't get us closer there.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Disabled_Tests was mostly a copy of Known_Failures. Leave only the race
condition on SIGPROF test (mcount_pic.c).
Also remove pr57275.c from known failures. It passes now that we support
AVX.
Also if a test is disabled, just skip it
We were only running SVE256 and SVE128 with AVX disabled.
Enable asm tests with SVE256, SVE128, and ASIMD, all running with AVX
enabled to hit all the tests.
FEX emulates faulting instructions (e.g. ud2 or int 2d) by jumping to
the dispatcher and filling out a structure with fault details in the
thread context. Parse this out into a windows exception record structure
so the correct fault information can be seen by the guest.
As the exception dispatcher is initially invoked on the emulator stack,
control needs to be transferred to the dispatcher on the guest stack
after recovering the x86 RSP to allow for invoking x86 exception
handlers.
This is used by the kernel (or UNIX side of ntdll in wine) to jump into
x86 code with the given context as is necessary when e.g. returning from
an exception.
FEX is unable to deal with reentrant compilation of any x64 hotpatches
so they need to be ignored by bypassing FFSs and calling directly into
the native target.
ARM64 requires that SP is always 16-byte aligned for memory accesses,
but ARM64EC shares the SP between x64 code and ARM64 code, the former
of which doesn't enforce such a restriction. This causes crashes in
programs such as HITMAN 3 that don't correctly follow the Windows ABI
and call into system library functions with SP only 8-byte-aligned.
Fixup stack alignment in such cases by leaving the 8-byte return
address on the stack and returning to a lone 'ret' instruction instead.
This allows for running x64 applications under wine without having to run all
of wine under FEX. The JIT is invoked when ARM64EC code performs an indirect
branch to x64 code, and left whenever the x64 code calls into ARM64EC
code.
Removes a global initializer and atexit registration
Ownership of this data has always been the frontend and the config
system, we just used these static vectors as a side-channel.
Removes a global static initializer for the vector and its atexit
handler.
This handler array can be consteval similar to the x86 tables so it can
be generated entirely at compile time.
This causes a global initializer that registers an atexit handler.
Be smarter, use an std::array and pass its data around using a span
instead.
Removes the global initializer and removes the atexit installation
We never use more than one logging method at a time so this was
overengineered for what it is doing.
Instead only allow one handler for messages and throw messages each
which just is a pointer.
Removes a global initializer and an atexit handler being installed
This is the initial split to decouple AVX256 composed operations from
their MMX/SSE counterparts. This is to work around the subtle
differences with AVX/SSE zext/insert behaviour.
This was doing a 128-bit load from memory and then a 64-bit zero extend
which looked like a spurious move but it was trying to match the
behaviour of vmovq where it needed the zero extend.
Also adds a unit test to ensure that we aren't loading too much data by
loading right up against a page boundary.
Fixes#3787
Also fixes a bug where it was failing to zero the upper bits of the
destination register in the AVX128 implementation. Which the updated
unit tests now check against.
Fixes a minor precision issue that was reported in #2995. We still don't
return correct values for overflow. x86 always returns maximum negative
int32_t on overflow, ARM will return maximum negative or positive
depending on sign of the double.
The codepath from #3826 was only targeting 256-bit sized operations.
This missed the vpgatherdq/vgatherdpd 128-bit operations. By extending
the codepath to understand 128-bit operations, we now hit these
instruction variants.
With this PR, we now have SVE128 codepaths that handle ALL variants of
x86 gather instructions! There are zero ASIMD fallbacks used in this
case!
Of course depending on the instruction, the performance still leaves a
lot to be desired, and there is no way to emulate x86 TSO behaviour
without an ASIMD fallback, which we will likely need to add as a
fallback at some point.
Based on #3836 until that is merged.
With the introduction of the wide gathers in #3828 this has opened new
avenues for optimizing these cases that would typically fall back to
ASIMD. In the cases that 32-bit SVE scaling doesn't fit, we can instead
sign extend the elements in to double-width address registers.
This then feeds naturally in to the SVE path even though we end up
needing to allocate 512-bits worth of address registers. This ends up
being significantly better than the ASIMD path still.
Relies on #3828 to be merged first
Fixes#3829
SVE has a special version of their gather instruction that gets similar
behaviour to x86's VGATHERQPS/VPGATHERQD instructions.
The quirk of these instructions that the previous SVE implementation
didn't handle and required ASIMD fallback, was that most gather
instructions require the data element size and address element size to
match. This x86 instruction uses a 64-bit address size while loading 32-bit
elements. This matches this specific variant of the SVE instruction, but
the data is zero-extended once loaded, requiring us to shuffle the data
after it is loaded.
This isn't the worst but the implementation is different enough that
stuffing it in to the other gather load will cause headaches.
Basically gets 32 instruction variants to use the SVE version!
Fixes#3827
If the destination isn't any of the incoming sources then we can avoid
one of the moves at the end. This half works around the problem proposed
in #3794, but doesn't solve the entire problem.
To solve the other half of the moving problem means we need to solve the
SRA allocation problem for this temporary register with addsub/subadd, so it gets allocated
for both the FMA operation and the XOR operation.
When loading 256-bits of data with only 128-bits of address indices, we
can sign extend the source indices to be 64-bit. Thus falling down the
ideal path for SVE where each 128-bit lane is loading the data to
addresses in a 1:1 element ratio.
This means we use the SVE path more often because of this.
Based on top of #3825 because the prescaling behaviour was introduced
there. This implements its own prescaling when the sign extension occurs
because ARM's SSHLL{,2} instruction gives us that for free.
This additionally fixes a bug where we were accidentally loading the top
128-bit half of the addresses for gathers when it was unnecessary, and
on the AVX256 side it was duplicating and doing some additional work
when it shouldn't have.
It'll be good to walk the commits when looking at this one, as there are
a couple of incremental changes that are easier to follow that way.
Fixes#3806
Was missing a clamp on the high half when encounting a 128-bit gather
instruction. Was causing us to unconditionally load the top half when it
was unncessary.
FEX allocations can get in the way of allocations that are 4gb-limited
even in 65-bit mode (i.e. those from LuaJIT), so allocate starting from
the top of the AS to prevent conflicts.
If the host supports SVE128, if the address element size and data size is 64-bit, and the scale is not one of the two that is supported by SVE; Then prescale the addresses.
64-bit address overflow masks the top bits so is well defined that we
can scale the vector elements and still execute the SVE code path in
that case. Removing the ASIMD code paths from a lot of gathers.
Fixes#3805
the special case here lines up with the special case of using a larger shift for
a smaller result, so we can just grab CF from the larger result.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The two halves are provided as two uint64_t values that shouldn't be
sign extended between them. Treat them as uint64_t until combined in to
a single int128_t. Fixes long signed divide.
For all the 32-bit rotates (except for RORX) we were failing to zero
extend the 32-bit result to the destination register when the rotate was
masked to zero.
Ensure we do this.
this fails on current main with blocksize=500 due to mentioned RA bug. passes
with blocksize=1.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
At the moment we always run ctest with max number of cpus. If
undefined, it will keep current behaviour, otherwise it will
honour TEST_JOB_COUNT.
Therefore to run ctest one test at a time, use
`cmake ... -DTEST_JOB_COUNT=1`
We had failed to enable these implementations for the
`ExtendedMemOperand` helpers. We had already implemented the non-helper
forms, which are already tested in CI. These helpers just weren't
updated?
Noticed this when running libaom's SSE4.1 tests, where it managed to
execute a pmovzxbq instruction with reg+reg memory source and was
breaking the test results.
There are /very/ few vector register operations that access only 8-bit
or 16-bit in vectors so this flew under the radar for quite a while.
Fixes their unit tests.
Also adds a unittest using sse4.1 pmovzxbq to ensure we support the
reg+reg case, and also a few other instructions to test 8-bit and 16-bit
vector loads and stores.
We didn't have a unit test for this and we weren't implementing it at
all.
We treated it as vmovhps/vmovhpd accidentally. Once again caught by the
libaom Intrinsics unit tests.
This is a very minor performance change. On Cortex CPUs that support
SVE, they do movprfx+<instruction> fusion to remove two cycles and a
dependency from the backend.
This is a minor win to convert from ASIMD mov+bsl to SVE movprfx+bsl
because of this, saving two cycles and a dependency on Cortex A710 and
A715. This is slightly less of a win on Cortex-A720/A725 because it supports
zero-cycle vector register renames, but it is still a win on Cortex-X925
because that is an older core design that doesn't support zero-cycle
vector register renames.
Very silly little thing.
The canonical way to generate a zero register vector in x86 is to xor
itself. Capture this can convert it to canonical zero register instead.
Can get zero-cycle renamed on latest CPUs.
x86 doesn't have a lot of non-temporal vector stores but we do have a
few of them.
- MMX: MOVNTQ
- SSE2: MOVNTDQ, MOVNTPS, MOVNTPD
- AVX: VMOVNTDQ (128-bit & 256-bit), VMOVNTPD
Additionally SSE4a adds 32-bit and 64-bit scalar vector non-temporal
stores, which we keep as regular stores. Since ARM doesn't have matching
semantics for those.
Additionally SSE4.1 adds non-temporal vector LOADS which this doesn't
touch.
- SSE4.1: MOVNTDQA
- AVX: VMOVNTDQA (128-bit)
- AVX2: VMOVNTDQA (256-bit)
Fixes#3364
Arm64's SVE load instruction can be minorly optimized in the case that a
base GPR register isn't provided, as it has a version of the instruction
that doesn't require one.
The limitation of this instruction is that it doesn't support scaling at
all so it only works if the offset scale is 1.
When FEX hits the optimal case that the destination isn't one of the
incoming sources (other than the incomingDest source) then we can
optimize out two moves per 128-bit lane.
Cuts 256-bit non-SVE gather loads from 50 instructions down to 46.
Some applications don't measure rdtsc correctly and instead use cpuinfo
to get the CPU core's base clock speed. Which for most x86 CPUs is the
base clock speed which also matches their cycle counter speed.
Did this as a quick test to see if this would help `Unbound: Worlds
Apart` stuttering while BinaryNinja was disassembling the binary.
Turns out the game doesn't use cpuinfo for its cycle counter speed
determination, but it is good to implement this regardless.
We can support a few combinations of guest and host vector sizes
Host: 128-bit or 256-bit
Guest: 128-bit or 256-bit
The typical case is Host = 128-bit and Guest = 256-bit now that AVX is
implemented.
On 32-bit this changes to Host=128-bit and Guest=128-bit because we
disable AVX.
In the vixl simulator 32-bit turns in to Host=256-bit and Guest=128-bit.
And then in the vixl sim 64-bit turns in to Host=256-bit and
Guest=256-bit.
We cover all four combinations of guest and host vector register sizes!
Fixes a few assumptions that SVE256 = AVX256 basically.
In regular SIB land the index register encoding of 0b100 encodes to "no
register", this feature lets you get SIB encodings without an index
register for flexibility.
In VSIB encoding this isn't expected behaviour and instead there are no
encodings where an index register is missing. Allowing you to encode all
sixteen registers as an index register.
This was causing an abort in `AVX128_LoadVSIB` because the index turned
in to an invalid register.
Working instruction:
`vgatherdps ymm2, dword [eax+ymm5*4], ymm7`
Broken instruction:
`vgatherdps ymm0, dword [eax+ymm4*4], ymm7`
This fixes a crash in libfmod where it is using gathers in the wild.
Fixing a crash in Ender Lilies.
When the source or destination is a register, the address size override
doesn't apply. We were accidentally applying it on all sources
regardless of type which was causing us to zero extend on operations
that aren't affected by address size override.
This fixes the OpenSSL cert error in every application, but most
importantly Steam.
During refactoring this was missed and rerunning unittests locally
caught it. 256-bit operations get their shift only from the lower half
of the vector register.
We can avoid a LOT of pointless work with some dedicated IR ops for specifically
overriding the round mode.
Small behaviour change here: we no longer reset FTZ. I think this is a bug fix?
But if it's not it's not hard to fix.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This does a gather load three ways, SVE256, SVE128, and ASIMD.
This operation is a bit special since it it can't quite handle all
gather loadstores in the 256-bit case and requires the frontend to
decompose the operation in the case that the striding hits a mode that
SVE doesn't support!
The 128-bit case is a lot simpler since both support all the cases where
stride doesn't match. I find this to be a nice compromise while there
aren't any SVE256 products on the market.
In the 128-bit case there is an SVE path which is utilized if the passed
in stride supports what SVE understands, otherwise it falls back to an
ASIMD implementation which manually emulates everything that is
necessary.
This instruction is very explicitly doing basically exactly what AVX
gather instructions want, because it's complex enough that we don't want
to try and make this a generic solution.
A bit confusing because the instruction encoding is the same between
VMOVHLPS and VMOVLPS so this unittest was missed.
Implement the test to ensure it stays working
The TestHarness infrastructure doesn't understand the difference between
converged versus split view.
So fetch the split view immediately and reconverge the view manually
inside of the state object so it continues working with the split ymm
view.
Because we have two views of the YMM registers depending on if the host
supports SVE256 or not, add helper functions to fetch them correctly.
We fetch them in the way that Linux desires them in signal handlers, if
we want to return the converged view directly, that is easy to add
support for. It's unnecessary for now.
Any time we get the value of Literal, we want to assert that it's actually a
literal. We've been open coding this pattern sporadically throughout the
opcodedispatcher. Let's add an ergonomic helper to fetch the value of literal,
asserting that the value is indeed literal.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Previously we could always tell the size of the operation depending on
how this effects the operating size of the instruction. Converting
64-bit down to 32-bit as an example.
AVX gather instructions are the first instruction class that can't infer
this information. The element load size is determined by the W flag but
the operating size of 128-bit or 256-bit is determined by other means.
Expose this flag so we can determine this difference. The FMA
instructions are going to need this flag as well.
This instruction is a little weird.
When accessing memory, the 128-bit operating size of the instruction
only loads 64-bits.
Meanwhile the 256-bit operating size of the instruction fetches a full
256-bits.
Theoretically the hardware could get away with two 64-bit loads or a
wacky 24-byte load, but it looks like to simplify hardware they just
spec'd it that the 256-bit version will always load the full range.
add a bit of coverage for this funny addressing corner. We do handle this
optimally but I had to write this to check ;)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Only installs the tables if SVE256 isn't supported yet AVX is explicitly
enabled with HostFeatures, to protect accidental enablement early.
- Only implements 85 instructions starting out
- Basic vector moves
- Basic vector unary operations
- Basic vector binary operations
- VZeroUpper/VZeroAll
The bulk of the implementation is currently the handling for loading and
storing the halves of the registers from the context or from memory.
This means the load/store helpers must always return a pair unless only
requesting the bottom half of the register, which occurs with 128-bit
AVX operations. The store side then needing to consume the named zero
register if it occurs since those cases will zero the upper bits.
This implementation approach has a few benefits.
- I can pound this out extremely quickly
- SSE implementations are unaffected and don't need to deal with the
insert behaviour of SVE256.
- We still keep the SVE256 implementation for the inevitable future when
hardware vendors actually do implement it (Give it 8 years or
something).
- We can actually unit test this path in CI once it is complete.
- We can partially optimize some paths with SVE128 (Gathers) and support
a full ASIMD path if necessary.
One downside is that I can't enable this in CI yet because it can't pass
all unittests. but that's a non-issue since it is going to be in heavy
flux as I'm hammering out the implementation. It'll get switched on at
the end when it's passing all 1265 AVX unittests. Currently at 1001 on
this.
This is a different feature flag than regular AES as the default AES+AVX
only operates on 128-bit wide vectors.
With the newer `VAES` extension this is expanded to 256-bit.
Fixes#3690
When doing scalar insertions, upper bits come from different arguments
depending on the operation. These are listed in the ARM spec under the
NEP bit documentation.
The Oryon is the first CPU we know of that implemented support for the
RNG extension. It also has an errata where reading the RNDRRS register
never returns success. X86's RDSEED guarantees forward progress with
enough retries.
When an x86 processor messed this up at one point, some Linux systems
would infinite loop (presumably when something in boot was filling an
entropy pool). This required a microcode change to fix that processor.
The rdseed unittest infinite loops on this platform if RNG was exposed.
to be consistent with the scalar _Andn opcode, which is specifically named _Andn
and not _Bic.
noticed while reviewing AVX patches
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixes#3691
We weren't checking if the file was empty before using its `at` function
member. This was causing an early crash if the config file existed but
was empty.
Consolidates the three locations that copy and pasted the json allocator
tools and adds an empty check for all of them.
Also adds two missing checks to the ThunksDB handler that could have
resulted in the same crash if ThunksDB was an empty file.
Needed something inbetween the `InlineJITBlockHeader` and `avx_high` in
order to match alignment requirements of 16-byte for avx_high. Chose the
`DeferredSignalRefCount` because we hit it quite frequently and it is
basically the only 64-bit variable that we end up touching
significantly.
In the future the CPUState object is going to need to change its view of
the object depending on if the device supports SVE256 or not, but we
don't need to frontload the work right now. It'll become significantly
easier to support that path once the RCLSE pass gets deleted.
This is required to be less than the maximum range for LDP and STP in
the Arm64 Dispatcher otherwise it breaks. Necessary to ensure this when
reorganizing the CoreState.
(except for when we explicitly enable AFP).
Since AFP gets saved/restored, we get `msr fpcr` garbage in random instructions
when AFP is enabled. Explicitly disable everywhere since it's not worth our time
to triage which files might hit that path. Fixes instcountci on AFP-supporting
hosts now that we have AFP enabled.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is helpful for devs working on FEXCore, I've been using this locally but it
might make sense to stick it in tree.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In quite a few locations we are mixing the case that SVE256 == AVX or
that AVX means the guest register size is 256-bit.
While this is true today, this is entanglement is going to change very
quickly and cause confusion in follow-up PRs.
Now we have SVE128, SVE256, and SVE2 HostFeatures to disambiguate the
different features which mean different things.
This PR keeps the alias that `SupportsAVX` = `SupportsSVE256 && SupportsSVE2`
but that alias is going to very quickly change its definition.
This currently doesn't do much but soon this will be very important to
ensure the data prefetcher of Cortex keeps the cachelines following this
variable in L1.
Allows a nice way to get information about how TSO emulation is occuring
on the individual's hardware. In particular, the more subtle details
around the implementation rather than just a hard on or off toggle.
Currently our minimum clang version requirement is 12.0 but soon will
require at least 13 or 14. Add a new version check in cmake to ensure
minimum version requirements.
Makes it easier to determine why a build is failing due to old compiler.
Pulled from the seccomp WIP PR where it pulls this object more frequently.
Since it is an opaque frontend pointer it needs to be cast and we
already have a few locations that use it.
No functional change.
No instcountci changes yet, since nothing currently spills in instcountci. This
mitigates spilling later seen with #3703, and should help for certain
pathological blocks even without those changes (maybe we should try to get some
of those blocks in instcountci?).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixes#3675
This was the first time I've ever actually dived in to this code and this
function melted my brain a bit while reading it. It was trying to be too
smart in tracking VMA splits, but if it split right at the end of a VMA
range it would then add a new range at the end where one already
existed. This then caused us to have overlapping VMA ranges and it would
completely break our SMC tracking since all tracking in the map must not
overlap.
Instead of being too smart, just break it down in to 4 merge strategies,
three of which are one shot. This is significantly easier to reason
about and each strategy is mostly self-contained. The fourth strategy in
the list is the most complex since it requires multiple steps since it
needs to walk multiple VMAs.
To help test this I added some sanity checking code that proved
invaluable to ensure everything was correct. That's not getting merged
since the overhead is too much to run, but good to have available. Diff
for this is at
https://gist.github.com/Sonicadvance1/1ee60101ed0742b971a476fadbb51083
FEX had a bug with this instruction where it was incorrectly using both
the address size override and operand size override to truncate the
immediate offset. This isn't how the instruction should behave as it
should actually ignore the address size override.
This now puts it correctly inline with how the jump instruction works
and adds a unit test to ensure it doesn't break again.
This fixes a crash from the Arch rootfs from the glibc dynamic linker
being compiling in a way where a call instruction was getting aligned
using this prefix (Since the compiler knew it does nothing).
The vixl disassembler has gotten more strict about certain instruction types, so these tests
aren't really needed.
Alternatively, we could mark them as unallocated, but we can opt to remove them here.
FEX_ENABLEAVX was removed. Settings for host features should use
FEX_HOSTFEATURES but actually FEX_HOSTFEATURES=enableavx has different
behaviour, so we don't enable it.
Installing PPA with the script now installs a different
URL. This means that GetPPAStatus always returns false.
For the sake of backwards compatibility match the end
of the line to check.
2024-06-14 00:56:42 +00:00
1742 changed files with 327176 additions and 65178 deletions
Loaded 100 of 1742 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ryan Houdek