Commit Graph
796 Commits
Author SHA1 Message Date
Ryan Houdek 45a2349a0d ELFCodeLoader: Calculate AT_MINSIGSTKSZ
This is the last remaining auxv value that we were missing.

This requires a little bit of setup to match what we are doing in
FEXCore's Dispatcher.

This only tracks how much space is required by the kernel to store its
required state.
2022-11-13 14:09:51 -08:00
Ryan Houdek d7b0e8469e Merge pull request #2148 from Sonicadvance1/fix_null_end
ELFCodeLoader: Fixes AT_PLATFORM null terminator
2022-11-13 13:58:47 -08:00
Mai 8afc3b8e23 Merge pull request #2147 from Sonicadvance1/auxv_secure
ELFCodeLoader: Pass through AT_SECURE
2022-11-13 21:49:27 +00:00
Ryan Houdek 9caa63d5b2 ELFCodeLoader: Fixes AT_PLATFORM null terminator
We were failing to null terminate the platform which is causing strcmp
to fail.
2022-11-13 13:46:36 -08:00
Ryan Houdek fa973f65bf ELFCodeLoader: Pass through AT_SECURE
When we are installed as a binfmt_misc handler the Linux kernel will
assign us AT_SECURE for setuid binaries.

Ensure we are passing this through for any application that will end up
needing it.
2022-11-13 12:50:58 -08:00
Ryan Houdek c027f02e5a ELFCodeLoader: Ensure we set AT_SYSINFO for 32-bit
AT_SYSINFO points to the vsyscall location that the kernel provides.
This AUXV value isn't used on x86-64.
If we have VDSO installed then we can use the one provided from there,
otherwise we need to provide a code page.

It seems like some behaviour changed with glibc provided in Ubuntu 22.10
that it now requires AT_SYSINFO.

Fixes wine 7.0 execution inside the Ubuntu 22.10 rootfs.
2022-11-13 12:39:07 -08:00
Ryan Houdek 0e1a418678 WIP: Segment register index optimization
Segment registers are indexed significantly more than they are changed.
Pay the cost of indexing during the set and store rather than the per
register index.

Should be a fairly significant performance improvement for 32-bit
applications. At least on hardware that doesn't have a data dependent
prefetcher.

Breaks Steam atm and isn't clean.
2022-10-31 19:42:30 -07:00
Ryan Houdek a42b659af9 ELFCodeLoader: Fixes Proton Experimental on 48-bit VA systems
This is a tricky situation that wine-preloader allocates the lower
32MB of stack space through fixed address mmap with MAP_FIXED.

They can't use mmap with an address hint nor MAP_FIXED_NOREPLACE because
it changes behaviour. mmap won't give you the allocation inside the
stack space even if you check `/proc/self/maps` that space isn't yet
allocated. The growable space of the stack blocks those allocations.

So the wine peeps might be SOL if they actually require this allocation
to exist.

To replicate this, allocate the application stack at the same location using an address hint.
This will give us the correct region on a 48-bit VA system, while also
letting it select a different region on a 36-bit VA system.
2022-10-28 02:30:18 -07:00
Ryan Houdek 5ba4bba138 Thunks: Add support for disabling thunks in config
Previously the config options could only have ever enabled thunks rather than
disable them.

Now sort the code so it can enable thunks, then following configs can
redisable them.  Allowing testing with global thunks enabled and
disabling problematic applications.

Also sorts the "ThunkConfigFile" config as lower priority than the
application configs. I wasn't thinking about ordering that hard for
these five configuration paths, but application configs should be higher
priority in this case.
2022-10-26 11:43:16 -07:00
Ryan Houdek 62a24bd38f Merge pull request #2075 from Sonicadvance1/gpuvis_profiler
FEXCore: Adds support for a timeline profiler interface
2022-10-26 08:44:54 -07:00
Ryan Houdek 48e7de9f9e ThunksDB: Fixes Thunks loaded boolean pointer check
Need to dereference the boolean to ensure we only load the thunksDB
files once.
2022-10-24 20:55:58 -07:00
Ryan Houdek 3e48b1a8ac FEXCore: Adds support for a timeline profiler interface
This creates a generic interface that FEXCore can use for timeline
profiling. This allows us to create a generic interface which the
backend details are hidden so we can support multiple timeline profile
APIs.

The only API supported right now is ftrace/gpuvis. Which is extremely
lightweight of an interface with minimal overhead.

We must be careful here since in most cases will will have dozens of
FEX instances running at any given time. So a timeline profiler like
Microprofiler can have major issues since that only ever expects a
single process at a time.

Not enabled by default but just needs the `ENABLE_FEXCORE_PROFILER`
cmake option set to enable.
2022-10-19 19:56:35 -07:00
Mai d1b235dd83 Merge pull request #2080 from Sonicadvance1/fix_64bit_syscall_mman
Syscalls: Fixes 64-bit mmap and munmap
2022-10-19 01:26:15 +00:00
Ryan Houdek d21aa5cac2 ELFCodeLoader: Implement four more auxv values
Implements AT_PLATFORM: Ends up being `i686` or `x86_64` depending on
ELF arch

Implements AT_HWCAP and AT_HWCAP2
AT_HWCAP is just CPUID function 01h EDX result
AT_HWCAP2 only has two defined bits in it, which we don't support
either.

Implements AT_RANDOM
Previously we were just sticking hardcoded values in to this.
Now we pass along the host's AT_RANDOM, or we generate our own if that
doesn't exist

Fixes #788
2022-10-16 19:45:19 -07:00
Ryan Houdek 76c2cc2c3e Syscalls: Fixes 64-bit mmap and munmap
These should be using the real syscalls, not our provided allocators.

While not a problem currently since these redirect to host mmap and
munmap, it will become an issue once we have an allocator that lives
outside of x86-64 space.
2022-10-14 21:43:04 -07:00
wannacu 8f905988e9 Use compatible syscall helpers 2022-10-11 16:35:17 +08:00
wannacu 478c5595ad SMC: Fix possible deadlock 2022-10-11 15:31:27 +08:00
Ryan Houdek c28ca0fac9 ELFCodeLoader: Map primary ELF more like the kernel
The kernel maps the primary ELF with a hint to some place *near* the
middle of the virtual address space. While the interpreter stays at the
top of the address space.

This also adds ASLR to the ELF loading, with a define for debugging and
testing purposes.

Requires both #2013 and #2014 merged first.
2022-10-10 08:38:15 -07:00
Mai d68b84bc27 Merge pull request #2013 from Sonicadvance1/fix_mapper
ELFCodeloader: Map once and then use MAP_FIXED to overwrite
2022-10-10 10:05:53 -04:00
Ryan Houdek af6582ff5b ELFCodeLoader: Fixes dynamic non-interpreter ELFs
Specifically fixes /sbin/ldconfig.
Fixes 8df7c2d84f
Fixes Steam launching

I failed to test ELF files that are dynamic with no interpreter here, so
EntryPoint ended up being set to zero, which results in an instant
crash.

Ensure these values are set correctly after the primary ELF and
interpreter are loaded so starting RIP is correct.
2022-09-28 16:00:02 -07:00
Ryan Houdek 25e5d88ab2 VDSO Emulation: Wires up support for 32-bit VDSO 2022-09-26 14:35:38 -07:00
Ryan Houdek 8df7c2d84f ELFCodeLoader: Map interpreter first
This more closely matches behaviour of the kernel.
Provides an example in source to ensure we don't break it in the future.
2022-09-25 18:07:31 -07:00
Ryan Houdek 46557a7a1f ELFCodeloader: Map once and then use MAP_FIXED to overwrite
Instead of mapping to find a range, unmapping, and then submapping
inside of it.

Map once, then mmap with MAP_FIXED to overwrite the mapping.
This fixes an issue where if you enabled ASAN then it would stick
additional mappings inbetween where we want to map. Thus breaking asan.
2022-09-25 17:50:16 -07:00
Ryan Houdek 26ba8079a3 cmake: Adds uninstall target
Following guidance from cmake's FAQ:
https://gitlab.kitware.com/cmake/community/-/wikis/FAQ#can-i-do-make-uninstall-with-cmake

Due to some of the special handling that we do with installs, we need to
do additional uninstall handling that the install manifest doesn't cover.

Specifically we need to add additional uninstall targets for:
- FEXInterpreter
- binfmt_misc
- guest_thunks (Doing its own uninstall target, so passthrough)

While it isn't generally advised to install and uninstall through source
systems, this is something that users want to do all the time.
This has been asked for a couple of times now.

Fixes #1592
2022-09-15 11:22:24 -07:00
Ryan Houdek c8c0054f67 FEXConfig: Ensure APP_CONFIG_NAME isn't stored in json
Also in FEXLoader make sure to use `EraseSet` for these runtime options.

Fixes a bug where the config was being set to nothing, breaking the
ThunksDB configuration option.
2022-09-13 00:08:20 -07:00
Ryan Houdek 56ae3716bb Syscalls: Use underscored shm syscall names
Removes us needing to carry a patch downstream in the termux package
repo.
2022-09-05 02:39:42 -07:00
Ryan Houdek 2fba3a4a9a LinuxAllocator: Uppercase function names
Not only does this match our internal naming convention. This avoids the
shm symbol namespace collision as well.
2022-09-05 02:39:42 -07:00
Ryan Houdek 8ba0312d40 Syscalls: Prefix _ to shm syscall names to avoid namespace conflict
Termux uses defines for these, so our token pasting fails, but we also
still want to use their define so we can fall down their emulation
library whenever possible.

Prefix an underscore to be able to use both our number definitions and
their defines in the same file.
2022-09-05 02:39:42 -07:00
Ryan Houdek c5a7fc1e2a Resolve most VDSO comments 2022-09-02 15:13:18 -07:00
Ryan Houdek 4e81f847f2 ELFCodeLoader: Load VDSO thunk at application startup
This ensures it will always be available to the application as long as
the library is installed.
2022-09-02 13:31:36 -07:00
lioncash 614d9448f7 HostRunner: Handle upper YMM lanes in sigsegv handler
Now we properly copy out the upper lanes instead of ignoring them.
2022-08-25 14:13:19 -04:00
Stefanos Kornilios Mitsis Poiitidis 097184c3e0 Merge pull request #1926 from Sonicadvance1/no_irloader_on_no_tests
IRLoader/TestHarnessLoader: Don't build if not building tests
2022-08-21 09:50:44 +03:00
Ryan Houdek 35cf7703b1 Thunks: Support direct thunk config in configuration files
Previously in order to enable thunks, we needed an independent
description file of which thunks to be enabled. This is nice for quickly
testing out new games by setting `FEX_THUNKCONFIG` environment variable.

For users that just want to enable thunks this is an unwieldy
indirection that doesn't make much sense at a glance.

Previously this meant you needed two files as an example:
```
  ryanh@ubuntu-linux-20-04-desktop:~/.fex-emu$ cat thunks.json
  {
    "ThunksDB": {
      "GL": 1,
      "Vulkan": 1
    }
  }
  ryanh@ubuntu-linux-20-04-desktop:~/.fex-emu$ cat AppConfig/EnderLiliesSteam-Linux-Shipping.json
  {
    "Config": {
      "ThunkConfig":"~\/.fex-emu\/thunks.json"
    }
  }
```

Instead of this unwieldy redirection just support `ThunksDB` json
directly in the AppConfig.

```
  ryanh@ubuntu-linux-20-04-desktop:~/.fex-emu$ cat AppConfig/EnderLiliesSteam-Linux-Shipping.json
  {
    "Config": {
      <...>
    },
    "ThunksDB": {
      "GL": 1,
      "Vulkan": 1
    }
  }
```

As can be seen this makes this significantly easier for new users
getting in to thunks. Depending on which path to enable thunks the user
is more comfortable with, they can still enable them using the
`ThunkConfig` option or embedding directly in the application
configuration.

Additionally this removes the older non-ThunksDB path to loading thunks.
All users of it have moved on to using ThunksDB.
2022-08-20 10:02:13 -07:00
Ryan Houdek 8c1137543b Config: Adds APP_CONFIG_NAME meta config option
We have separate configurations for the Application path versus the
application name we are using as a configuration choice.

Example 1: FEXBash "wine Crysis64.exe"

Previous APP_FILENAME will contain `/usr/bin/wine`, which is still used
elsewhere.

This new APP_CONFIG_NAME will contain `Crysis64.exe`

Example 2: FEXBash glxgears

Previous APP_FILENAME will contain `/usr/bin/glxgears`
APP_CONFIG_NAME will contain `glxgears`

We didn't have this exposed any other way before.
2022-08-20 09:58:23 -07:00
Stefanos Kornilios Mitsis Poiitidis 1b5aaf1fb8 Allocator: Simplify StealMemory, make it less chatty with kernel space 2022-08-20 13:10:50 +03:00
Ryan Houdek 955595be8a FDUtils: Don't make unknown get_fdpath fatal
Encountered this while running wine things.
Non-fatal so don't explode
2022-08-19 02:28:19 -07:00
Ryan Houdek cd6915917b IRLoader/TestHarnessLoader: Don't build if not building tests
If we're not building tests then just don't build the IRLoader since it
won't get used.
2022-08-17 17:04:13 -07:00
Ryan Houdek 0adbe31112 Merge pull request #1919 from Sonicadvance1/termux_shm_library
Termux: Add android-shmem library
2022-08-15 11:14:27 -07:00
Ryan Houdek edad24479b unittests: Support skipping unit tests based on host feature support
For these unit tests we no longer need to put them in the disabled tests
file. Instead it will be skipped if the host doesn't support the feature
required.
2022-08-14 20:04:26 -07:00
Ryan Houdek 8d1cc6cf51 Termux: Add android-shmem library
Otherwise we fail at linking.
2022-08-14 18:23:53 -07:00
Ryan Houdek a847fac4ca Linux: Emulate classic getdents syscall for x64 and x32
Arm64 doesn't have the classic getdents syscall, only getdents64.
Old glibc versions (like 2.17) don't support getdents64

This fixes 64-bit ls with a centos 7 rootfs. Likely also fixes some other very
old applications.

Packing differences between 32-bit and 64-bit getdents means we need to
template this between the two types, otherwise it's quite similar.

No known applications rely on the 32-bit getdents but worked with test
applications.
2022-08-12 18:21:49 -07:00
Ryan Houdek 0496506fb7 Linux: Define linux_dirent types for x64
These don't match the compatibility defines and they aren't part of the
public interface.
2022-08-12 16:00:06 -07:00
Ryan Houdek 0dde233b1e Disable UnitTestGenerator
This is currently unused and can just cause compilation issues.
Disable it until we start hooking up aggressive fuzzing tests.
2022-08-09 03:13:54 -07:00
Tony Wasserka e8ad1ca0a0 Thunks: Use signature-based thunking of guest function pointers
This changes how host trampolines for guest functions are created. Instead
of doing this purely on the guest-side, it's either the host-side that
creates them in a single step *or* a cooperative two-step initialization
process must be used. In the latter, trampolines are allocated and partially
initialized on the guest and must be finalized on the host before use.
2022-08-08 16:08:46 +02:00
Ryan Houdek 54f62b6701 Merge pull request #1882 from lioncash/fmt
Externals: Update fmt to 9.0.0
2022-08-05 02:30:03 -07:00
Ryan Houdek 0dc33ec893 cmake: Remove the static-pie compilation option
Due to glibc issues around static applications doing dlopen this is a
fundamentally broken option and no longer supported by FEX.

Remove the option entirely as to not be confusing.

We kept this around initially for chroot support, but with our RootFS
mounting AArch64 folders inside the chroot this isn't necessary anymore.
2022-08-04 19:22:11 -07:00
lioncash 9e524a30d6 General: Resolve fmt deprecation warnings
fmt 9.0.0 deprecates implicit conversions of unscoped enum values to
integers to be consistent with scoped enum behavior.

Fairly trivial to resolve
2022-08-04 11:39:51 -04:00
lioncash 51791e9efa HarnessHelpers: Handle SSE register offsets in CompareStates
Last of the things that slipped through while re-adding the ldp/stp
optimization.
2022-08-04 10:56:48 -04:00
Ryan Houdek 59e5da9c7f FEXBash: Changes PS1 to hopefully help users
Currently FEXBash only outputs `FEXBash>` which gives weird docker
vibes. This can confuse users since they no longer see what folder they
are in.

Change this so it still has the user and path exposed like a typical PS1

eg: `FEXBash-ryanh@ryanh-TR2:/mnt/Work/Work/work/FEXNew/Build>`
2022-08-01 15:39:07 -07:00
Tony Wasserka 0caef59e6a Syscalls: Simplify RegisterSyscall implementation 2022-07-28 11:23:23 +02:00