Commit Graph
46 Commits
Author SHA1 Message Date
Stefanos Kornilios Mitsis Poiitidis 779aca7e95 Merge pull request #1085 from Sonicadvance1/transparent_huge_pages
Enables transparent huge pages in our 64-bit VA allocator
2021-06-14 17:04:40 +03:00
Stefanos Kornilios Mitsis Poiitidis 3252f793df Merge pull request #1084 from Sonicadvance1/cleanup_stacks_after_fork
Core: After fork make sure to cleanup stacks
2021-06-14 17:03:49 +03:00
Stefanos Kornilios Mitsis Poiitidis 50b5ad5762 Merge pull request #1070 from Sonicadvance1/remove_warnings
Remove most warnings in FEX again
2021-06-14 10:12:03 +03:00
Ryan Houdek 10ac1518e1 Enables transparent huge pages in our 64-bit VA allocator
Transparent huge pages is a feature that the linux kernel opportunistically uses.
Depending on kernel configuration this feature is either enabled always, or when you madvise the region.

To ensure we hit both cases, madvise the regions we allocate in the 64-bit VMA allocator always

Can reduce kernel bookkeeping memory usage for our abusive allocator
2021-06-12 18:52:51 -07:00
Ryan Houdek 085fca01bc Core: After fork make sure to cleanup stacks
After FEX has forked, there aren't any other threads in the process but their stacks remain.
We need to have some book keeping in place to have the stack ranges available to clean up
after fork.

We now keep both live stacks and dead stacks in a dequeue and on fork we will walk both to
clean up all stack objects that aren't our current thread.
2021-06-12 18:45:13 -07:00
Lioncash 6fc117a67c HostAllocator: Make Create64BitAllocator() return a unique_ptr
Communicates the ownership intent in the interface.

Also removes the prototype for CreateBasicAllocator, as it isn't
implemented anywhere.
2021-06-11 19:11:46 -04:00
Lioncash 1b72f73224 GlobalAllocator: Add virtual destructor to interface
Prevents any potential destruction issues from occurring in the
interface.
2021-06-11 19:04:25 -04:00
Ryan Houdek ba18b5dca9 ELFSymbolDatabase: Remove warning 2021-06-10 14:56:51 -07:00
Ryan Houdek b6417153d7 Hook more glibc hooks and allow clearing hooks in Allocator
Due to a disjoint mechanism inside of glibc we need to override both the glibc
publicly visible allocation functions AND the hooks.

We had broken the overriding when we changed the default visibility. So first fix that.
Then only override to FEX's allocators once we are ready for it.
Then also replace the hooks.

Additionally have a way to clear the hooks back to default.
2021-06-09 18:03:27 -07:00
Lioncash 2b9029623e General: Abstract trapping behind a define
Provides a layer of separation from direct usages of compiler builtins.
2021-06-07 06:18:44 -04:00
Lioncash ca2b04b309 LogManager: Add fmt-capable logging functions
Addresses #146 a little more by providing an interface to perform
fmt-compatible logging.

No more, will people on the project be tormented by classic printf
features like:

- Accidentally passing in a non-trivial type
- PRI macros
- Not being able to add support for custom types
- Mixing up signed/unsigned printf formatting specifiers accidentally

fmt-capable versions of the logging functions are named the same as the
existing functions, just with a "Fmt" or _FMT suffix (depending on
whether or not it's a function being used or a macro, respectively).
2021-06-05 08:29:48 -04:00
Ryan Houdek 1e40148f60 Fixes host thread stacks ending up in lower 32-bit VA
Overriding glibc's mmap and munmap functions don't encapsulate that functions they use
for allocating stack data so it was falling down the system mmap/munmap path.
This was causing host side stacks to end up in the 32-bit VA space in 32-bit applications.
2021-05-04 22:15:46 -07:00
Ryan Houdek fd6d33d197 Merge pull request #996 from FEX-Emu/skmp/aotirgen
AOTIR: Offline Generation
2021-05-03 15:58:57 -07:00
Stefanos Kornilios Mitsis Poiitidis d6b7f32d33 AOTGen: Cleanup unwind symbol logic, add to x86-32 binaries 2021-05-02 11:08:12 +03:00
Ryan Houdek 94017d2e65 On old kernels, steal the 32-bit address space before the 64-bit
Increases speed of startup
2021-05-01 09:06:39 -07:00
Ryan Houdek 13b93ef6b5 Allow memory allocation to go through if the base was above the lower bound 2021-05-01 09:06:39 -07:00
Ryan Houdek c5bda47b2a Adds more allocator hooking 2021-05-01 07:39:14 -07:00
Ryan Houdek 5067737052 Adds 64-bit allocator 2021-05-01 07:39:14 -07:00
Stefanos Kornilios Mitsis Poiitidis ab44f800f7 AOTgen: Cleaner interfaces, use LogMan 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis ce9324f50f Kinda-parse unwind table 2021-05-01 09:40:49 +03:00
Ryan Houdek d256bc59fd Override glibc allocator functions 2021-04-30 18:28:45 -07:00
Ryan Houdek 04e0baadc6 Changes LogMan assert to defines
C++ no-op functions can't optimize out the predicate arguments in all cases.
This was causing a problem where zero cost assertions weren't actually zero cost.

The only way to resolve this is to actually use macros sadly enough.

This will give a fairly hefty performance uplift with anything operating on IR.
2021-04-26 13:49:35 -07:00
Ryan Houdek fa546fb492 Replaces naked mman usage in ELFSymbolDatabase 2021-04-16 20:50:09 -07:00
Ryan Houdek ca45485d48 Adds new Allocator namespace for replace mmap and munmap
This is necessary for overriding these functions
2021-04-16 20:49:35 -07:00
Stefanos Kornilios Mitsis Poiitidis 80493487db ELF Loading: Cleanups all around 2021-04-14 10:59:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 59ce750af6 Elf: Add ELFCodeLoader2, with simplified loading logic 2021-04-13 15:59:36 +03:00
Stefanos Kornilios Mitsis Poiitidis a0a382cc5c Clear up file naming a big 2021-04-13 15:59:36 +03:00
Ryan Houdek 21a364c2ad Switches FEXCore over to pthreads implementation
This defaults to a pthread implementation but it can be switched over to a custom thread
handler if the frontend desires.
2021-04-01 00:31:45 -07:00
Ryan Houdek b2825bd848 Adds handler in ELFLoader to determine ELF type. 2021-03-31 23:37:56 -07:00
Ryan Houdek 61cd3eb3ce Fix crash in ELFDB if it tried loading something that wasn't an ELF 2021-03-31 23:37:56 -07:00
Stefanos Kornilios Mitsis Poiitidis 942b8549c6 Docs: Add tags to the source code 2021-03-30 12:21:18 +03:00
Ryan Houdek 5b8bf8dd24 Adds some additional logic for finding absolute soft linked linker
Ubuntu's x86_64 rootfs does a soft link from `/lib64/ld-linux-x86-64.so.2` to an absolute path of `/lib/x86_64-linux-gnu/ld-2.32.so`
Our frontend ELF loader logic would resolve this symlink to be just `/lib/x86_64-linux-gnu/ld-2.32.so` which would then fail.
Adds some additional logic to our frontend that if the symlink ends up being a softlink to an absolute address then it'll first resolve that symlink.
Then combines rootfs + absolute path. If that still fails then it'll fall down the regular linker path, which could still find the linker on the host side.

This is all a bit of a kludge to just work around Ubuntu doing an absolute address rather than a relative one.

Fixes #869
2021-03-21 01:11:47 -07:00
Ryan Houdek 8acb18b2b9 Merge pull request #837 from Sonicadvance1/fix_logmanager
Fixes crash with large strings through LogManager
2021-03-10 21:34:17 -08:00
Ryan Houdek 4e55f29589 Fixes crash with large strings through LogManager
va_list types have an edge where after operating on it, then it is undefined behaviour to do anything other than va_end on the object.
So in order to call vsnprintf on it multiple times we must create copies with va_copy.
Easy enough to work around by doing a few more copies of the va_list
2021-03-10 16:36:33 -08:00
Ryan Houdek e2ec645855 Extends error message about not being able to find guest interpreter
Passes the result back up to the frontend as well which allows us to early exit correctly.
Also ensures that we return ENOEXEC on these error cases so if someone is waiting on a return value, they don't just get zero
Fixes #757
2021-03-10 11:11:36 -08:00
Scott Mansell c3ab5bca5e Fix <elf.h> incompatibilities
ELF32_ST_VISIBILITY is a pretty simple macro, just reimplement it.
2021-02-02 16:47:44 +13:00
Ryan Houdek 436e6e96ce Removes warnings from ELFLoader 2021-01-31 08:01:25 -08:00
Ryan Houdek a5d9e62cc6 More improvements to BRK handling
Base size is now only one page in size. We will then increment that BRK
size by 8MB alignments. 256MB for 32bit applications was causing some
applications on the edge to run out of virtual memory
I was hitting some 32bit applications that were being fairly mean with
BRK. They were allocating all of BRK space then running out of virtual
memory space with its mmap handler fallback after freeing BRK space.

This means we now munmap BRK pages on release for the guest, similar to
behaviour that Linux does.

Additionally I had an application that was getting very upset that BRK
wasn't actually at the end of program space. So allocate at the end of
program space like expected.

brk test now passes from gvisor
2021-01-27 10:12:25 -08:00
Stefanos Kornilios Mitsis Poiitidis 271d5898fb Merge pull request #642 from Sonicadvance1/fix_llvm_asan
Fixes 64bit ELF loading when LLVM ASAN is used
2021-01-13 12:55:58 +02:00
Stefanos Kornilios Mitsis Poiitidis 0c1510e763 Merge pull request #641 from Sonicadvance1/uninstall_handlers
Uninstalls both the signal delegator and logmanager handlers on shutdown
2021-01-13 12:15:10 +02:00
Ryan Houdek 43af4e26d1 Fixes 64bit ELF loading when LLVM ASAN is used
Instead of hardcoding 64bit dynamic ELF files to 0x1'0000'0000 we
instead calculate the size of the ELF in memory. Then allocate the
ELF base up front.

LLVM ASAN steals a large amount of virtual memory space that intersected
with the original hardcoded range which caused memory allocations to
fail.
2021-01-11 09:07:16 -08:00
Ryan Houdek aa3b77e4c8 Uninstalls LogManager handlers on shutdown
In case something manages to have stayed alive and tried throwing a
message, this will no longer crash.

Noticed this with the previous commit
2021-01-11 09:03:56 -08:00
Ryan Houdek a442489001 Makes sure to not calculate zero sized program headers on 32bit
This is the same thing as the 64bit side
2021-01-11 08:51:13 -08:00
Ryan Houdek 36b24ad141 Removes all uses of non-unified memory and SHM regions
Let's just delete the whole thing!
2020-12-10 02:09:52 -08:00
Ryan Houdek a23c045c23 Adds a ELFLoader helper to determine supported ELF file
We will need to check if an application can be executed through the
interpreter by checking program getting executed
2020-12-08 07:28:50 -08:00
Ryan Houdek a28df9f5c1 Moves SonicUtils in to FEXCore
Removes the extraneous and dead VM CPU backend
2020-11-29 00:30:28 -08:00