1024 Commits
Author SHA1 Message Date
Ryan Houdek 847b6e5026 LinuxSyscalls: Fixes struct verifier on Ubuntu 20.04
The `linux/types.h` header needs to be included before the rest
otherwise we are missing types.
2022-03-04 17:44:26 -08:00
Mai M 08938ecb11 Merge pull request #1596 from Sonicadvance1/fix_old_kernel_bug
LinuxAllocator: Fixes bug with old kernels and hint allocation
2022-03-02 00:26:51 -05:00
Ryan Houdek 5137af5bae Fixes epoxy include in FEXConfig and FEXLogServer 2022-03-01 03:55:19 -08:00
Ryan Houdek 3418cc8054 LinuxSyscalls: More type fixes 2022-03-01 03:55:15 -08:00
Ryan Houdek 1f835ea1f4 LinuxSyscalls: Fixes semid and ipc types
Newer headers redefine semid_ds and ipc_perm as semid64_ds and
ipc64_perm silently.
Use the new types directly since we are a 64-bit only application.
2022-03-01 03:55:12 -08:00
Ryan Houdek 0a40753624 More missing include fixes 2022-03-01 03:55:10 -08:00
Ryan Houdek 09010e1292 LinuxSyscalls: Remove unused headers now
These don't even exist on termux.
2022-03-01 03:54:01 -08:00
Ryan Houdek bddf3871ba LinuxSyscalls/x32/Types: Fixes stat type definitions
The time argument definitions are defines in termux.
Rename our definition name of these so we don't get caught by define.
2022-03-01 03:50:18 -08:00
Ryan Houdek 1ca9e56502 LinuxSyscalls/x64/Types: Fix guest_stat definition
kernel types don't exist in termux. Use uint64_t and int64_t directly.

Also using reserved `__` causes compile failure.
2022-03-01 03:50:17 -08:00
Ryan Houdek fe58a9ae2c LinuxSyscalls/Thread: Don't use set_robust_list on Termux
Would get caught by seccomp and crash FEX
2022-03-01 03:50:15 -08:00
Ryan Houdek bc7c4dfa74 LinuxSyscalls/x32/Types: Don't redefine SIGEV defines 2022-03-01 03:50:12 -08:00
Ryan Houdek 7af7b1309d Msg: Switch msqd_t to FEX defined type 2022-03-01 03:50:10 -08:00
Ryan Houdek dd4630750f LinuxSyscalls/Types: Adds missing types for Termux 2022-03-01 03:50:07 -08:00
Ryan Houdek 302da029eb Work around Termux not supporting hardlinks
The Android filesystem they are on just doesn't support them
Instead of hardlinking FEXLoader to FEXInterpter, just build the
executable twice and eat the filesystem cost.
2022-03-01 03:50:05 -08:00
Ryan Houdek 3cc59bc68e LinuxSyscalls: Switches to a bunch of raw syscalls
For older and Termux build environments these helper libc functions
don't exist.
2022-03-01 03:49:31 -08:00
Ryan Houdek 30c27851ef FEXRootFSFetcher: Termux build environments 2022-03-01 03:30:47 -08:00
Ryan Houdek 9b29ff61e9 Adds some missing headers 2022-03-01 03:30:45 -08:00
Ryan Houdek 91f780223b Stop self-defining PAGE_SIZE
We only work on targets with 4096 byte page sizes.
Adds a cmake compile test to ensure this is adhered to.
2022-03-01 03:30:43 -08:00
Ryan Houdek a0efd2b01f Resolve comments. 2022-02-28 21:04:03 -08:00
Ryan Houdek 3e9dbda146 Classify syscalls 2022-02-28 21:04:03 -08:00
Ryan Houdek 6501715a3c Allow classifying syscalls with flags
In some cases we can generate more optimal code if we have more
information about a syscall which number gets const-propagated.

In particular optimizing through syscalls, not synchronizing state, and
never returning.

- Noreturn is used by a syscall that never returns, like exit.

This means that it never needs to try and synchronize state coming back

- Not synchronizing state and optimizing through syscalls

Useful for syscalls that don't read the state past arguments and only
returns a value.
2022-02-28 21:03:54 -08:00
Ryan Houdek 3ba2d6cfb4 Merge pull request #1586 from Sonicadvance1/testharness_env
TestHarnessRunner: Wire up environment variable option setting
2022-02-28 07:05:28 -08:00
Ryan Houdek 23daf4cb72 LinuxAllocator: Fixes bug with old kernels and hint allocation
In the face of an application using MAP_FIXED_NOREPLACE AND the host
linux kernel doesn't understand this flag. Then we were falling down the
hint allocation path which would allocate a pointer in 64-bit space,
returning this pointer to a 32-bit userspace and breaking things.

Now when the hint fails with this flag, we know that it intersecting a
range and can early exit.
2022-02-26 21:12:44 -08:00
Ryan Houdek ee165379c5 TestHarnessRunner: Wire up environment variable option setting
Wire up the environment variable option setting so asm files can set
these and it works
2022-02-24 13:37:39 -08:00
Azkali 75988b2ae5 Improve compatibility with older uapi kernel headers
Following up the work previously done in 2079f6b3c7.
Adding more defines for older Linux uapi headers missing some defines.
2022-02-24 09:49:56 +01:00
Ryan Houdek 2b8f60c108 TestHarness: Support for asm files having the option to set config options
Allows some something like the following:
"Env": {
  "FEX_MAXINST": "500"
}

Not that I would recommend overriding MAXINST in the asm tests, as
command line overrides that
2022-02-21 14:53:27 -08:00
Ryan Houdek 944c93c10c FEXLoader: Adds back required MAP_GROWSDOWN
I was overzealous with my removal of MAP_GROWSDOWN.
We still require the primary thread to have this flag set.
2022-02-16 15:41:44 -08:00
Ryan Houdek d95326b23b Linux: Fix missing types for syscall strace 2022-02-10 19:46:43 -08:00
Mai M 43fada7555 Merge pull request #1568 from Sonicadvance1/fix_musl_load
ELFCodeLoader: Fixes typo in AT_BASE calculation
2022-02-10 21:25:48 -05:00
Ryan Houdek b88d8a7cc4 Removes MAP_GROWSDOWN usage
This is just a memory leak waiting to happen.
Only the primary thread in an application really should have this set
since the kernel cleans it up.

We only ever allocate the primary thread of the guest application then
every host thread's stack on top of that. It's up to the guest when it
is cloning to set up new stack pointers, we don't manage that.

We are already allocating the first thread's size at the soft stack
limit with RLIMIT_STACK anyway.

Fixes #1556
2022-02-10 18:03:18 -08:00
Ryan Houdek 6add09b78b ELFCodeLoader: Fixes typo in AT_BASE calculation
Fixes executing musl applications with the dynamic linker.

It was using the main executable's p_offset instead of the
interpreter's.
Wasn't a problem with glibc since it uses a different symbol to find the
base (Don't ask me why it does this).

musl dynamic linker on the other hand just uses AT_BASE directly and
since it was calculated incorrectly it was crashing.

Testing application was `ls` which had a p_offset of 0x40, so it would
try and read some values from AT_BASE, starting at an offset below where
it was mapped.
2022-02-10 17:43:58 -08:00
Ryan Houdek 2a1b27df58 FEXGetConfig: Fix --current-rootfs option
If the configured rootfs wasn't a squashfs then it was failing to return
the directory.

Now it works for both squashfs and directory rootfs again.
2022-02-10 11:45:29 -08:00
Ryan Houdek 8840fc818c FEXLoader: Removes memory mapping check on startup
FEX always builds with PIE and we don't hit this issue anymore anyway.
If some application wants to inject a page in to the lower 32-bits then
we have no reason to complain about it anymore. Just let it go and
hopefully they know what they are doing.

Fixes #1559
2022-02-09 13:52:05 -08:00
Mai M 287c65dc64 Merge pull request #1554 from Sonicadvance1/fix_tricky_stat
Linux: x32: Fixes tricky stat64 defines
2022-02-06 19:52:31 -05:00
Ryan Houdek 0216bcf27f Linux: x32: Fixes tricky stat64 defines
Some build environments use a define to change stat64 and statfs64 to be
the same definition as stat and statfs.

Check if the define exists and if it does then remove the 64bit
constructors.
2022-02-06 16:05:27 -08:00
Ryan Houdek b981fcfe42 Linux: Make sure to use correct accessors for sigevent
Some of these are defined differently depending on environment
2022-02-06 15:33:07 -08:00
Ryan Houdek 9a64e7f100 Linux: Renamed some 64-bit syscall names
Some build environments use defines to rename these. Which breaks our
naming
2022-02-06 14:19:30 -08:00
Ryan Houdek 8e8baec47a Linux: Use raw syscalls for pkey syscalls
For older libc environments
2022-02-06 14:19:30 -08:00
Ryan Houdek 59ca60e39f Fixes a bunch of header includes
Necessary for older build environments
2022-02-06 14:19:30 -08:00
Ryan Houdek 260fc8ba52 Fixes FEXConfig build
Oops. This was added late and didn't test it.
2022-02-01 15:44:06 -08:00
Mai M ade0d0f241 Merge pull request #1543 from Sonicadvance1/fixes_for_1423
Linux: Fixes for older build environments
2022-02-01 16:52:40 -05:00
Ryan Houdek 10ad5db686 Adds an option to disable the IR interpreter
By default we won't build with the interpeter to reduce user confusion.
The interpreter isn't really useful to end users so remove it.

Completely removes it from building except for the fallback operations.

This also removes the selection from FEXConfig to remove selection
confusion there.

File Stats:
FEXLoader Size with Interpreter:    3422768 bytes
FEXLoader Size without Interpreter: 3301944 bytes
Size difference:                    96.4699915%
Bytes removed:                      120824 bytes
4k pages removed:                   29.498046875 -> 30 rounded up

VM Stats (Reported from bloaty):
Memory Size with Interpreter:    6.50Mi
Memory Size without Interpreter: 6.38Mi
Size difference:                 98.1538462%
2022-02-01 13:00:29 -08:00
Ryan Houdek 68c441575d Linux: Fixes for older build environments
Should resolve the new building issues from #1423
2022-02-01 12:17:09 -08:00
Ryan Houdek dc7ec6377b Linux: Safely handle Filemanagement mutex on fork
If an application is forking heavily with threaded file accesses
happening then the mutex can end up in an unknown state.

On fork make sure to lock the mutex then immediately unlock after fork
occurs.

This final step resolves hanging that pressure-vessel hits on startup.
Since it is doing a ton of file opening and forking during
initialization.
2022-01-30 18:15:57 -08:00
Ryan Houdek ce6f4edaaa FileManagement: Use ScopedSignalMaskWithMutex
When using mutexes in syscall helpers we need to be extra careful around
signals.
2022-01-30 18:15:57 -08:00
Ryan Houdek 13f35f7b79 Merge pull request #1530 from Sonicadvance1/rootfs_fetcher_fixes
FEXRootFSFetcher: Fixes some edge case behaviours
2022-01-25 10:29:55 -08:00
Ryan Houdek 4823933f79 Linux: Fixes 32-bit getsockopt and setsockopt
On Set, we have four options that need to be converted.
On Get, we have two options that need to be converted.

This fixes a crash that Tomb Raider 2013 was having on launch.
2022-01-24 17:14:50 -08:00
Ryan Houdek ee04067424 FEXRootFSFetcher: Fixes some edge case behaviours
Makes curl do its continue feature to give the users the best chance of
downloading a rootfs. We don't need to restart the full file transfer on
failure. Helps people with slower connections.

On failure to download, asks the user if they want to retry the download
rather than just exiting with a weird error about hash failure.

Once the image is downloaded, now changes options depending on if
squashfuse or unsquashfs works.

Prevents the user from selecting a bad option and getting unexpected
behaviour. Ideally we would do a squashfs mount test as well for
platforms that don't have working FUSE, like termux. This is harder to
get right and its for an unsupported platform, so I'm not going to
invest more time with it.

Fixes #1525
Fixes #1526
Fixes #1527
2022-01-23 22:54:05 -08:00
Ryan Houdek e4aef26ef5 FEXRootFSFetcher: Adds helper namespace for tool checking
Location to check if curl, squashfuse, and unsquashfs are working.

unsquashfs is a bit more complex where it needs to parse the help output
to see if zstd is supported
2022-01-23 22:44:31 -08:00
Ryan Houdek a41dc8eafa FEXRootFSFetcher: Fix pipe redirecting
In the case of launching without stdout/stderr then redirection could
have these constants be a redirected FD that sits in the same fd number.

Use -2 to indicate no redirection.
Use -1 to indicate closing traditional stderr/stdout
The rest will indicate if stdout and stderr should be replaced as
normal.
Making sure not to close the incoming fds if they matched the
stdout/stderr FD numbers.
2022-01-23 22:41:39 -08:00