3027 Commits
Author SHA1 Message Date
Ryan Houdek 1f65bf9e47 Docs: Update for release FEX-2108 FEX-2108 2021-08-02 23:21:04 -07:00
Ryan Houdek 4c431518b0 Merge pull request #1192 from Sonicadvance1/workaround_static_pie
AArch64: Workaround static-pie crashing
2021-08-02 23:19:45 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek d052e87d47 AArch64: Workaround static-pie crashing
Static-pie can only work with GNU ld instead of lld on AArch64 hosts.
This is a known problem on the llvm side: https://bugs.llvm.org/show_bug.cgi?id=49672
2021-08-02 22:02:12 -07:00
Ryan Houdek ff98a43f70 Merge pull request #1191 from Sonicadvance1/improve_fexmountdaemon_stability
FEXMountDaemon: Make squashfs mounting more robust
2021-07-30 16:49:20 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 7e8577a50a Merge pull request #1180 from Sonicadvance1/support_setxid
Linux: Implements support for signals 32 and 33
2021-07-28 00:22:30 -07:00
Ryan Houdek a21cead9dc Fixes memcpy of siginfo_t data
_sifields are offset by a uint32_t between a 32-bit and 64-bit arch.

Fixes thread cancelling on 32-bit processes
2021-07-28 00:10:15 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek de0337db47 Merge pull request #1188 from Sonicadvance1/wine_fixes
Wine fixes
2021-07-27 18:07:53 -07:00
Ryan Houdek a48e41e4a9 Merge pull request #1187 from Sonicadvance1/cleanup_compile_service_on_fork
Cleanup compile service on fork
2021-07-27 18:07:42 -07:00
Ryan Houdek 896aa52851 Merge pull request #1186 from Sonicadvance1/fix_partial_rclse
RCLSE: Fixes an assumption in RCLSE
2021-07-27 18:07:32 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek a5b142cf02 Merge pull request #1178 from Sonicadvance1/implement_vsyscall
Linux: Implements support for vsyscall
2021-07-27 18:06:54 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b0a28e595f Merge pull request #1181 from Sonicadvance1/fix_rapass
Fixes bug in RAPass
2021-07-27 18:06:27 -07:00
Ryan Houdek b8c5cf2736 Adds FXAM unit tests for ABI conformance
Wine was doing a fun x87 stack clean by checking tag bits for empty.
We hadn't previously supported anything but valid. Which cause wine to overflow the stack
trying to save all of the x87 values.

We now support valid and empty well enough that this doesn't break and we can unit test it
2021-07-27 08:25:40 -07:00
Ryan Houdek 2803113e35 Improves 32-bit signal handling
A decent amount of this was broken.
To the point that the ABI was incorrect even, causing 32-bit signals to completely crash in most instances.
This is enough to get my 32-bit signal handler test application to work and basic SIGUSR behaviour
2021-07-27 08:23:25 -07:00
Ryan Houdek 7a84ab8cfc Implements partial support for x87 FTW
This only implements the Empty and Valid tags.
Anything that is not empty is considered valid currently.
2021-07-27 08:22:11 -07:00
Ryan Houdek 77a7ecbae0 Add x87 FTW to CoreState
This will be needed for x87 FTW emulation
2021-07-27 07:57:41 -07:00
Ryan Houdek 2310ea83bb Cleanup compile service on fork
On fork this thread will be gone. Make sure to delete the shared_ptr.
2021-07-27 07:53:02 -07:00
Ryan Houdek 109df27bba Amd64: Very minor optimization in Push/Pop Regs
Just generates a bit nicer code to look at. Doesn't really change much.
2021-07-24 08:45:38 -07:00
Ryan Houdek 2afa14d1f3 ConstProp: Removes VMOV assumption
ConstProp has an assumption that a VMOV of the same size arguments is a no-op.
This isn't quite 100% true as this could be behaving as a zext and the pass can't see through the op correctly.

Something to be said that the vector ops should be looked over to see which op is leaving dirty bits in the upper bits
of the register. But that's a battle for a different time
2021-07-24 08:43:35 -07:00
Ryan Houdek 744b778013 RCLSE: Fixes an assumption in RCLSE
There is an assumption in the RCLSE pass that if a StoreContext + LoadContext pair of the same size
with a vector register that it is safe to remove the LoadContext.
This isn't quite safe since it can't entirely see through the IR to determine if the store op
had already zext the upper bits of the register.

This was causing 8byte loadcontext operations to "load" with garbage data in the upper
bits when it was expecting a zext.

Which granted it is likely a bug that not all vector ops zero their upper bits but that's a battle for a different day.

Additionally:
- Removes a couple dead optimizations in the GPR path that was never hit.
- Cleans up how access types are checked
- Uses ReplaceAllUsesWithRange to reinforce that this is purely a per block optimization still
2021-07-24 08:37:03 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ec6cc5cb4f Merge pull request #1184 from lioncash/vec-alloc
64BitAllocator: Convert std::vector table into std::array
2021-07-23 14:29:03 -07:00
Ryan Houdek 7931dc2c09 Merge pull request #1183 from lioncash/size
FlexBitSet: Use non-template type parameter for index parameters
2021-07-23 14:17:13 -07:00
Lioncash 22590806a0 64BitAllocator: Convert std::vector table into std::array
This is small enough that it can be converted over to a std::array so
the heap use isn't necessary
2021-07-23 17:12:37 -04:00
Lioncash f9313f4000 FlexBitSet: Use non-template type parameter for index parameters
These parameters are used as indexes into the tracked memory, so sizing
the index variable relative to the type being stored is kind of sketchy.

e.g. If the tracked types were uint8_t for example, we'd still want the
indexing parameters to be regularly sized so that we aren't implicitly
truncating values all the time when passing values to a uint8_t
parameter (and while all usages are currently using uint64_t as type
T, we may as well address this).

While we're at it, we can make both Get() and operator[] const member
functions, since they don't directly modify any underlying data, they
only read it.
2021-07-23 17:07:41 -04:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek a9ee4123ef Fixes bug in RAPass
RegisterNode isn't a POD so we can't memset it.
This was overwriting some unique_ptrs which were memory leaking and causing crashes in other PRs
2021-07-22 18:13:44 -07:00
Ryan Houdek 138e88bc80 Linux: Implements support for vsyscall
vsyscall is a legacy interface to expose three syscalls in a faster fashion.
Most applications should have moved to vDSO but this isn't entirely the case.

golang specifically still used vsyscall in the case that vdso isn't available.
Which that isn't completely safe since vsyscall can be completely disabled.
This is fixed in a newer version of their runtime but we must support legacy behaviour.

This makes it so FEX checks if the incoming RIP is a "special" region and adjusts where it
loads instructions from accordingly. Lying to the guest application about the instruction stream
while maintaining the illusion that its RIP is in the correct location.
This was fairly straightforward since we already decoupled the instruction stream from the RIP location.

This fixes golang execution as long as you also disable their preempt signaling.
environment variable to do so is `GODEBUG=asyncpreemptoff=1`
2021-07-22 16:58:17 -07:00
Ryan Houdek 62775161fe Merge pull request #1176 from Sonicadvance1/fix_static_pie_crashing
Works around static-pie crashing
2021-07-20 19:37:56 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek 2fd41de5e0 Merge pull request #1153 from Sonicadvance1/split_opdispatcher
OpcodeDispatcher: Split the opcode handling to multiple files
2021-07-20 04:43:53 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek ebb21e86b0 OpcodeDispatcher: Split the opcode handling to multiple files
These are some fairly large separate files still.
Don't want these to be too terribly small but helps significantly with compile time when working
in the OpcodeDispatcher
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 6ac23003e1 Merge pull request #1168 from Sonicadvance1/fix_spill_slot
Arm64: Fixes fill and spill slot offset calculation
2021-07-19 23:29:01 -07:00
Ryan Houdek 04aacea58b Merge pull request #1173 from Sonicadvance1/fix_binfmt_misc_not_usr
Fixes binfmt_misc install when not installed to /usr
2021-07-19 23:26:33 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 7052d022c1 Merge pull request #1172 from lioncash/jits
Jits: Migrate logs over to fmt where applicable
2021-07-16 11:10:27 -07:00
Lioncash 9c7ebca235 JIT/x86_64: Migrate logs over to fmt 2021-07-16 13:57:15 -04:00