Commit Graph
795 Commits
Author SHA1 Message Date
Ryan Houdek 61260c2a62 FEXConfig: Load application config and advanced tab
Loading a preexisting application config was impossible through the gui.
Adds a way to do so.

Also adds an advanced tab that just displays all the items.
Allows pruning of application config options to be fairly efficient
2021-08-29 12:30:39 -07:00
Ryan Houdek 109c42a629 Merge pull request #1235 from Sonicadvance1/fix_emulated_openat
EmulatedFiles: Fixes openat for emulated files not using FDCWD
2021-08-28 18:09:42 -07:00
Ryan Houdek 9140ba28b7 Merge pull request #1234 from Sonicadvance1/FEXBash_init
FEXBash: Allow creating a bash instance easily
2021-08-28 17:55:33 -07:00
Ryan Houdek 8a331202ee EmulatedFiles: Fixes openat for emulated files not using FDCWD
Fixes lscpu and other applications that open the directory first and
then files inside of that directory.
2021-08-28 17:54:05 -07:00
Ryan Houdek f72ecacd86 FEXBash: Allow creating a bash instance easily
If not passing in any arguments then just immediately create a bash
instance. Incredibly nice little helper
2021-08-28 16:54:25 -07:00
Ryan Houdek 057c1de69d Linux: Fixes accidental execve escape
If the binfmt_misc interpreter was installed then we were running execve
directly.
This allowed shebang programs to escape and see the host architecture
when we weren't planning on it.

Fixes `FEXBash steam` from complaining about missing packages.
2021-08-28 16:37:29 -07:00
Ryan Houdek f161e3bfb0 Massive amount of IWYU cleanup
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
2021-08-28 00:32:15 -07:00
Ryan Houdek de64db5852 FEXLoader: Fixes potential bug in log output to stdout/stderr
Comment in the file for why this can be a bug.

Noticed a game opening a file and our logs were ending up in their
files.
2021-08-27 15:23:17 -07:00
Ryan Houdek b625437071 SignalDelegator: More splitting and cleanup
This is working towards getting the stack frame for guest signals being
pushed over to the Frontend.
Still some more work to do but this is the first step that can be split
up.
2021-08-27 12:46:41 -07:00
Ryan Houdek b15e0c5f6c Fixes jemalloc library ordering
FEXCore relies on jemalloc symbols if compiled with it.
Have FEXCore link to jemalloc instead of the frontend.

Fixes a missing symbol if someone loads libFEXCore

Additionally, stop trying to compile JEMalloc if not enabled
2021-08-26 01:21:56 -07:00
Ryan Houdek f131f07612 Config: Moves non-OS specific configuration loading to FEXCore
Puts the visibility of the main layer, application layers, and
environment in to FEXCore instead of FEX.
These layers aren't specific to FEX/FEXLoader and should live in
FEXCore.

Only the EmptyMapper remains in FEX, which should eventually move over
to FEXConfig since that is the only user.
2021-08-24 23:17:32 -07:00
Ryan Houdek b7af5c641c Linux: Fixes 32-bit syscalls that use 64-bit values
A bunch of these were defined incorrectly. I tested a few of these
locally to ensure they were correct after the fact.

Shows that most 32-bit applications that we've encountered aren't
dealing with files larger than 4GB.
2021-08-23 00:44:04 -07:00
Ryan Houdek 7bfa1c4838 Linux: Fixes 32-bit sigaltstack
Due to an incorrect pointer check, we were never setting the sigaltstack
on 32-bit applications.
Additionally the stack_t type didn't have the members in the correct
order.

This fixes wine 32-bit applications where wine sends an application
SIGUSR1 and expects the altstack to be used. This is because the
altstack has the thread's TEB region at the start of the stack.

Without this when the application was getting sent a SIGUSR1, it would
remain in the application stack, thus getting an invalid TEB and loading
an FS register with zero.
2021-08-20 23:55:56 -07:00
Ryan Houdek 01c49dbb9c FEXLoader: Updates RanAsInterpreter check for FD exec
Only the interpreter can run when executed as FD.
This happens when executed with binfmt_misc and can resolve an issue if
someone sets up the hardlink incorrectly.
2021-08-20 23:48:42 -07:00
Ryan Houdek 6d60689ad4 FileManagement: Stop calling getpid for every file access
We can cache the pid result, we know every case in which the pid is
changed.
This makes watching strace significantly less annoying.
2021-08-20 23:47:48 -07:00
Ryan Houdek c07b5e480b Adds new FEXGetConfig program
This is a simple program to get a few configuration options that are necessary
to expose for pressure-vessel
2021-08-13 20:53:30 -07:00
Ryan Houdek 740270c05f Linux: Implements support for clone with namespaces
This is very tricky to handle and it has a bunch of rough edges.
One of the major problems that we can't workaround is that if we receive a
clone flag that pthreads can't support with THREAD, then we are required to fall down
the pthreads code path.
This is because threads going down the clone path will break TLS and we don't have
a way to work around it currently.

So this adds a clone path, a clone3 path, and keeps the legacy path as well.
Which makes this fairly convoluted but it gets pressure-vessel working on x86-64 host.
It's a bit tricky to setup but it does work.

Still some work necessary to get pressure-vessel working on AArch64 host, but I'm working on that.
2021-08-10 18:31:04 -07:00
Ryan Houdek b120a8ea84 Merge pull request #1196 from Sonicadvance1/offline_telemetry
Implements support for offline *only* telemetry
2021-08-06 23:23:38 -07:00
Ryan Houdek b05adaeba3 Arm64: Reimplements support for binfmt_misc without update-binfmts
Arch doesn't have update-binfmts. Fall back to the classic approach
without it.
2021-08-06 23:14:25 -07:00
Ryan Houdek 8dd41e7fcd Merge pull request #1193 from Sonicadvance1/fix_readlinkat_self
Linux: Fixes readlinkat for self
2021-08-06 22:59:47 -07:00
Ryan Houdek 1c1f40e5af Linux: Fixes readlinkat for self
Similar code to readlinkat. Needs to be correct for self otherwise we
return EINVAL which is unexpected since self should always be a symlink

Fixes a bug in running bwrap
2021-08-06 22:51:31 -07:00
Ryan Houdek c6c94570b4 Implements support for offline *only* telemetry
This information is only ever going to be offline. Will be useful for multiple reasons.

1) Searching for split lock usage in applications, which can be a programming bug.
  a) This isn't visible on AMD systems and on Intel is a fairly new linux feature
2) Having more information about when an application breaks.
3) Useful for some minor profiling for devs looking for statistical data
2021-08-06 22:40:19 -07:00
Ryan Houdek 4641e44276 Linux: Implements pivot_root syscall
Somehow missed this one. Easy enough and matches between architectures.
Used by bubblewrap
2021-08-03 23:30:38 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 4332c22797 Linux: Fixes BRK deallocation
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
2021-07-14 13:23:58 -07:00
Ryan Houdek 5e12c38298 SignalDelegator: Switch Required flag to atomic bool
This is shared between multiple threads but it isn't never changed beyond first set.
Change it to atomic bool with relaxed loads to quiet tsan
2021-07-14 13:22:38 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek 786a341652 BRKHandler: Don't rely on 64Bit Allocator 32-bit hack.
Use the actual 32-bit allocator if we are running a 32-bit process.
This hack in the 64bit allocator is likely to go away in the future
2021-07-12 00:22:59 -07:00
Ryan Houdek faf09fec5f Removes glibc symbol replacement inside FEX
FEX_jemalloc now handles this entirely by itself.
It now replaces glibc during startup rather than when we call for it.
2021-07-06 21:52:57 -07:00
Stefanos Kornilios Mitsis Poiitidis 8f09bd9215 Merge pull request #1150 from Sonicadvance1/fix_old_kernel_defines
Fixes old kernel defines for sockios
2021-07-06 15:39:54 +03:00
Ryan Houdek 9fbeba3b79 FEXLoader: stderr on immediate failure
In the case that there is an immediate configuration failure. Use stderr specifically for outputting.
These errors won't be output typically because silent logging is enabled by default.
In the case of executable missing or rootfs configuration failure, print directly to stderr.
Previously it looked like FEX just exited for no reason.
We had multiple users encounter this and be confused
2021-07-05 18:52:06 -07:00
Ryan Houdek b3bab814de Fixes old kernel defines for sockios
These defines were changed at in kernel v5.2 but the debian defines didn't seem to change?
Redefine them if missing
2021-07-05 18:26:44 -07:00
Ryan Houdek b9efbd6159 Linux: Implements rt_{tg,}sigqueueinfo
We already support receiving user signals in our signal handlers for this.
Just need to push through syscalls.

This fixes Dead Island Definitive Edition so it runs.
2021-07-02 22:53:27 -07:00