Compare commits

...
Author SHA1 Message Date
spoopyghosty0 1f8c86b291 Unreal Binned3 reservations with MAP_NORESERVE (frame.unreal_noreserve, GitHub #349)
Transformers Beyond Reality (UE5, FMallocBinned3) reserves 76 GiB with one
mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS) while libUnreal.so's
static constructors run. Quest grants it; the Frame's kernel (overcommit mode 0) refuses a
writable private mapping larger than RAM + swap, and Unreal's out-of-memory handler then
dereferences its not yet created log: SIGSEGV fault addr 0x10 inside System.loadLibrary.

The patch adds MAP_NORESERVE to every anonymous read/write mmap call site of the engine
library (mov w3, #0x22 -> #0x4022 before a bl to mmap's PLT stub, with x0 = 0 and w2 = 3
in the window), suggested from analysis unreal_binned3 (ANALYSIS_VERSION 20). Triage
unreal-reserve-refused, PLAYBOOK row, tests. Headless on the dev Frame the game now gets
past loading to its OpenXR session and first frames.

ELF helpers find .dynsym/.dynstr/.dynamic by section type and link instead of by name: this
game's libOVRPlugin.so has a scrambled section name table, so add_needed raised
"produced NEEDED=[...]" and the build failed in frame.unreal_ovrp_entrypoints, and
dyn_symbols returned nothing (every ovrp_* lookup then counted as missing).
2026-10-16 11:34:07 -04:00
15 changed files with 388 additions and 27 deletions

No files matched your search

+10
View File
@@ -243,6 +243,16 @@ signatures:
severity: error
diagnosis: The screen stays black because the game submits a layer the VrApi bridge can't show. Whole frames with that VrApi layer type are dropped.
suggest: []
- id: unreal-reserve-refused
# GitHub #349: Unreal's out-of-memory handler dereferences its not yet created log while libUnreal.so's static
# constructors run (System.loadLibrary), after the Binned3 allocator's up-front reservation failed
pattern: '(?s)fault addr 0x10\b(?:(?!Fatal signal).){0,8000}?#00 pc [0-9a-f]+\s+\S*lib(?:Unreal|UE4)\.so(?:(?!Fatal signal).){0,8000}?soinfo17call_constructors'
severity: fatal
diagnosis: The game crashed while Unreal was still loading, before its first frame. Its memory allocator (Binned3)
reserves tens of gigabytes of address space at once, which Quest allows and the Frame's kernel refuses in one
piece (for example Transformers Beyond Reality). Rebuild with the reservation marked as not committed.
suggest: [frame.unreal_noreserve]
supersedes: [native-crash]
- id: graphics-requirements-missing
pattern: '(XR_ERROR_GRAPHICS_REQUIREMENTS_CALL_MISSING|Failed to create XR session: -50\b|xrCreateSession[^\n]*-50\b)'
severity: fatal
+16 -2
View File
@@ -2,7 +2,7 @@
# Patches
Every patch and game setting FramePort knows: 169 entries in 5 groups. A game's recipe is its chosen patches and settings; tested games get theirs from the catalog ([GAMES.md](GAMES.md)), and other games get the patches FramePort suggests from its analysis. You change them on the game page under Customize.
Every patch and game setting FramePort knows: 170 entries in 5 groups. A game's recipe is its chosen patches and settings; tested games get theirs from the catalog ([GAMES.md](GAMES.md)), and other games get the patches FramePort suggests from its analysis. You change them on the game page under Customize.
Each entry says what the patch does, whether it changes the APK or only the install, when FramePort suggests it, its options, the catalog games whose recipe names it and the launch-test signatures that suggest it. Patches that are on by default are on for every game, so the catalog rarely names them.
@@ -11,7 +11,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
## Groups
- [Patches from OVRPort](#group-overport) (25)
- [Steam Frame patches](#group-frame) (55)
- [Steam Frame patches](#group-frame) (56)
- [Game settings](#group-adapter) (68)
- [Files and settings on the Frame](#group-device) (8)
- [PC VR (Proton and Revive)](#group-pcvr) (13)
@@ -87,6 +87,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Unity: headset always counts as worn](#frame-unity_user_presence) | `frame.unity_user_presence` | Steam Frame patches | 1 |
| [GL shim for Unreal (no multisampled render-to-texture)](#frame-unreal_gl_shim) | `frame.unreal_gl_shim` | Steam Frame patches | 1 |
| [Unreal: match multiview depth to the eye size](#frame-unreal_multiview_depth) | `frame.unreal_multiview_depth` | Steam Frame patches | 1 |
| [Unreal: reserve memory like on Quest](#frame-unreal_noreserve) | `frame.unreal_noreserve` | Steam Frame patches | |
| [Unreal: stand-ins for OVRPlugin functions OVRPort lacks](#frame-unreal_ovrp_entrypoints) | `frame.unreal_ovrp_entrypoints` | Steam Frame patches | 2 |
| [Unreal: Quest controller bindings (RPOC key map)](#frame-unreal_quest_keymap) | `frame.unreal_quest_keymap` | Steam Frame patches | 3 |
| [Unreal: don't wait for the Quest shader precompile](#frame-unreal_quest_precompile) | `frame.unreal_quest_precompile` | Steam Frame patches | 3 |
@@ -1049,6 +1050,19 @@ Shows the full environment when its upper part is clipped but menu panels look c
> Some UE4 games keep the initial Android window size for their stereo depth buffer, clipping the upper part of the environment while menu panels look correct. Refreshes the cached depth dimensions from the current eye render target before allocation. Preserves multiview and multisampling. Verified in Tales from the Galaxy's Edge 1.1.9; changes one branch only when the function size and surrounding code match.
### <a id="frame-unreal_noreserve"></a>Unreal: reserve memory like on Quest
Lets the game load: it reserves more memory at start than the Frame grants in one piece and crashed before its first frame (for example Transformers Beyond Reality).
- `frame.unreal_noreserve` · also for 2D apps
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Suggested (on):** Unreal's Binned3 allocator reserves more memory at start than the Frame's kernel allows in one piece; without this the game crashes while it loads.
- **Suggested by launch-test signatures:**
- `unreal-reserve-refused` (fatal): The game crashed while Unreal was still loading, before its first frame.
- **Source:** [patches/frame/unreal_noreserve.py](../src/frameport/patches/frame/unreal_noreserve.py)
> Unreal reserves its memory ranges on Android with ordinary read/write mappings. Games built with Unreal's Binned3 allocator reserve tens of gigabytes at once while the engine loads (76 GiB in Transformers Beyond Reality); Quest allows that, the Frame's kernel refuses mappings larger than its memory, and the game crashes before its first frame (SIGSEGV at fault addr 0x10 in libUnreal.so, inside System.loadLibrary). This marks those mappings MAP_NORESERVE, as Quest effectively treats them: memory is still only used when the game touches it.
### <a id="frame-unreal_ovrp_entrypoints"></a>Unreal: stand-ins for OVRPlugin functions OVRPort lacks
Lets older Unreal games start VR on the Frame (for example Star Wars Pinball VR).
+1
View File
@@ -187,6 +187,7 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| Unity game crashes or hangs right as Vulkan starts, after `SLZ Graphics plugin loading!` and OVRPlugin's pre-init `xrDestroyInstance`: SIGSEGV with pc 0 (or pc == fault address in an unloaded library), x16/x23 in `libSLZQuestNative.so` (BONELAB 1.2974) | Stress Level Zero's graphics plugin hooks Unity's Vulkan start-up (IUnityGraphicsVulkanV2 interception: vkCreateInstance/vkCreateDevice) and vkCreateSampler; its vkCreateInstance wrapper calls an invalid pointer on the Frame | `frame.slz_vulkan_hooks` (its two registrations become no-ops; Unity starts Vulkan itself; ~35 s of shader prewarming without its pipeline cache) |
| Unity game shows a picture but the player body is frozen: no head tracking, the controllers stay on the model, no buttons, although OpenXR input works (BONELAB 1.2068, Oculus XR Plugin) | OVRPlugin reports the headset as not worn (`ovrp_GetUserPresent2` → 0) a few seconds after start, and the game's rig only follows a present user (Marrow `XRHMD.IsUserPresent` = Unity's HMD `UserPresence`) | `frame.unity_user_presence` (the Oculus XR Plugin's lookup → `libfp_ovrp.so`, reports present; log `user presence: OVRPlugin 0 …`) |
| Unreal 5 game crashes at its first render pass, backtrace `vulkan.freedreno.so` ← `libVkLayer_fossilize.so` ← `libfp_vk.so (create_render_pass2)` (Metro Awakening) | a subpass without a depth attachment names a depth/stencil resolve attachment; the Frame's Turnip reads the missing depth attachment (NULL) | the Vulkan shim drops that resolve (always, FramePort after 0.11.0; log `dropped the depth resolve of subpass N (no depth attachment)`); triage `unreal-vulkan-driver-crash` |
| Unreal game crashes before its first frame: SIGSEGV `fault addr 0x10` on the main thread, backtrace all `libUnreal.so` above the linker's `soinfo::call_constructors` (inside `System.loadLibrary`), right after `Parsing commandline` (GitHub #349, Transformers Beyond Reality) | The game is built with Unreal's Binned3 allocator, which reserves its whole range while the library loads: one `mmap(NULL, 76 GiB, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS)`. Quest grants it; the Frame's kernel (heuristic overcommit, mode 0) refuses a writable private mapping larger than RAM + swap, and Unreal's out-of-memory handler ("Ran out of memory allocating ...") dereferences its not-yet-created log | `frame.unreal_noreserve` (suggested from analysis `unreal_binned3`): every such anonymous read/write mmap in the engine library gets `MAP_NORESERVE` (one instruction each: `mov w3, #0x22` → `#0x4022`); triage `unreal-reserve-refused`. Found by disassembling the crash pc (the OOM handler; its UTF-16 format string) and the caller's `mov x0, #0x1300000000` |
| Unreal 5 game crashes in `vulkan.freedreno.so` called straight from `libUnreal.so` right after its swapchains, or a few seconds into the first frames (Metro Awakening) | Unreal turns on fragment-density-map foveation because Turnip offers VK_EXT_fragment_density_map, then records barriers (to `FRAGMENT_DENSITY_MAP_OPTIMAL`) and image views (2D array R8G8) for a density map the Frame never provides (VK_NULL_HANDLE); Turnip dereferences it | adapter `vk_hide_fdm=1` (the game doesn't see the extension) + `vk_spec_fixes=1` (leaves out null-image barriers, refuses null-image views); triage `unreal-fdm-missing`. Found by disassembling the driver at the crash pc: the struct offsets (0x58 = VkSubpassDescription2, 0x48/0x60 = image barriers, viewType/format in registers) name the call |
| Own-engine game crashes in `je_free` from its own library on the first frames (Roblox) | the game frees memory it doesn't own; not the Vulkan shim or the format fallback | unsupported |
+5
View File
@@ -589,6 +589,11 @@
"title": "Unreal: match multiview depth to the eye size",
"group": "Steam Frame patches"
},
"frame.unreal_noreserve": {
"anchor": "frame-unreal_noreserve",
"title": "Unreal: reserve memory like on Quest",
"group": "Steam Frame patches"
},
"frame.unreal_ovrp_entrypoints": {
"anchor": "frame-unreal_ovrp_entrypoints",
"title": "Unreal: stand-ins for OVRPlugin functions OVRPort lacks",
+15 -1
View File
@@ -43,7 +43,10 @@ IL2CPP_METADATA = "assets/bin/Data/Managed/Metadata/global-metadata.dat"
# 17: headset (the Meta headset a VR app was made for: Gear VR / Go games run through frame.a32_bridge) (2026-10)
# 18: headset "quest" also for 32-bit apps that require 6DoF head tracking (Santa Cruz builds, e.g. Sports
# Scramble: Touch controllers on the a32 bridge) (2026-10)
ANALYSIS_VERSION = 19
# 19: unreal_obb_delivery also recognises obfuscated DownloaderActivity predicates (GitHub #159, Medal of Honor)
# 20: unreal_binned3 (Unreal built with FMallocBinned3 reserves tens of GiB at load: frame.unreal_noreserve,
# GitHub #349) (2026-10)
ANALYSIS_VERSION = 20
# Android versions by API level (for messages); the Frame's Lepton container runs Android 11 (API 30)
@@ -87,6 +90,13 @@ def cryengine_project(data: bytes) -> bool:
UE_VERIFY_OBB_KEY = ".GameActivity.bVerifyOBBOnStartUp"
def unreal_binned3(engine: bytes) -> bool:
"""Unreal built with FMallocBinned3 as its allocator (its own messages and source path are compiled in; other
builds only name its console variables). It reserves its whole range while the library loads (GitHub #349)."""
return (b"/HAL/MallocBinned3.cpp\0" in engine
or "FMallocBinned3 Attempt to free".encode("utf-16-le") in engine)
def unreal_verify_obb(meta: dict) -> bool:
"""Unreal's DownloaderActivity checks every OBB entry's CRC at the first start (a full read; on a mismatch it waits
on an error screen). Only a real boolean counts: GameActivity reads it with Bundle.getBoolean."""
@@ -410,6 +420,10 @@ def analyze(path: Path, deep: bool = True, data_bytes: int | None = None) -> Ana
# Unreal CRC-checks the OBB before the game starts (frame.unreal_skip_obb_check, GitHub #159)
"unreal_verify_obb": unreal_verify_obb(meta),
"unreal_obb_delivery": unreal_obb_delivery,
# Unreal's Binned3 allocator reserves more address space than the Frame's kernel grants in one writable
# mapping (frame.unreal_noreserve, GitHub #349)
"unreal_binned3": (unreal_binned3(lib_bytes.get(engine_lib.rsplit("/", 1)[1], b""))
if engine_lib else False),
# Unreal's Oculus module needs every OVRPlugin function it looks up (frame.unreal_ovrp_entrypoints)
"unreal_ovrp_lookups": (ovrp_lookups(lib_bytes.get(engine_lib.rsplit("/", 1)[1], b""))
if engine_lib and "libOVRPlugin.so" in libset else []),
+39 -21
View File
@@ -20,6 +20,23 @@ def _elf(data: bytes) -> ELFFile:
return ELFFile(io.BytesIO(data))
def dynsym_section(elf: ELFFile):
"""The dynamic symbol table, found by section type: some shipped libraries have a scrambled section name table
(GitHub #349: Transformers Beyond Reality's libOVRPlugin.so calls .dynsym "text"), types and links are intact."""
sec = elf.get_section_by_name(".dynsym")
if isinstance(sec, SymbolTableSection) and sec["sh_type"] == "SHT_DYNSYM":
return sec
return next((s for s in elf.iter_sections() if s["sh_type"] == "SHT_DYNSYM"), None)
def dynstr_section(elf: ELFFile, dynsym=None):
"""The string table of the dynamic symbols (dynsym's sh_link; by name as a fallback)."""
dynsym = dynsym if dynsym is not None else dynsym_section(elf)
if dynsym is not None and 0 < dynsym["sh_link"] < elf.num_sections():
return elf.get_section(dynsym["sh_link"])
return elf.get_section_by_name(".dynstr")
def needed(data: bytes) -> list[str]:
elf = _elf(data)
for sec in elf.iter_sections():
@@ -48,7 +65,7 @@ def dyn_symbols(data: bytes, defined: bool) -> set[str]:
@lru_cache(maxsize=256)
def _dyn_symbols_cached(data: bytes, defined: bool) -> frozenset[str]:
elf = _elf(data)
sec = elf.get_section_by_name(".dynsym")
sec = dynsym_section(elf)
out = set()
if isinstance(sec, SymbolTableSection):
for sym in sec.iter_symbols():
@@ -69,7 +86,8 @@ def find_symbols(data: bytes, names) -> dict[str, tuple[int, int]]:
if not is_elf(data) or data[4] != 2 or data[5] != 1:
return {}
elf = _elf(data)
dynsym, dynstr = elf.get_section_by_name(".dynsym"), elf.get_section_by_name(".dynstr")
dynsym = dynsym_section(elf)
dynstr = dynstr_section(elf, dynsym)
if dynsym is None or dynstr is None:
return {}
sym_off, sym_size = dynsym["sh_offset"], dynsym["sh_size"]
@@ -147,7 +165,7 @@ def hide_exports(data: bytes, names) -> tuple[bytes, list[str]]:
(native/langpack) still finds the original address. Returns (data, names actually hidden); ELF64 only."""
if not is_elf(data) or not is_64bit(data) or data[5] != 1:
return data, []
sec = _elf(data).get_section_by_name(".dynsym")
sec = dynsym_section(_elf(data))
if not isinstance(sec, SymbolTableSection):
return data, []
wanted = set(names)
@@ -171,7 +189,7 @@ def redirect_exports(data: bytes, names) -> tuple[bytes, list[str]]:
"""
if not is_elf(data) or not is_64bit(data) or data[5] != 1:
return data, []
sec = _elf(data).get_section_by_name(".dynsym")
sec = dynsym_section(_elf(data))
if not isinstance(sec, SymbolTableSection):
return data, []
wanted = set(names)
@@ -201,10 +219,11 @@ def rename_imports(data: bytes, renames: dict[str, str]) -> tuple[bytes, list[st
if len(new) > len(old):
raise ValueError(f"{new} is longer than {old}")
elf = _elf(data)
dynsym, dynstr = elf.get_section_by_name(".dynsym"), elf.get_section_by_name(".dynstr")
dynsym = dynsym_section(elf)
dynstr = dynstr_section(elf, dynsym)
if not isinstance(dynsym, SymbolTableSection) or dynstr is None:
return data, []
versym = elf.get_section_by_name(".gnu.version")
versym = next((s for s in elf.iter_sections() if s["sh_type"] == "SHT_GNU_versym"), None)
str_off = dynstr["sh_offset"]
targets, refs = {}, {} # dynsym index -> st_name of an import to rename; st_name -> referencing dynsym indices
for i, sym in enumerate(dynsym.iter_symbols()):
@@ -376,24 +395,23 @@ def add_needed(data: bytes, library: str) -> bytes:
write_ph(i, p)
# section headers: point .dynamic / .dynstr at the new copies (for tools; the loader uses program headers)
# Found by type and link, not by name: some shipped libraries have a scrambled section name table (GitHub #349,
# Transformers Beyond Reality's libOVRPlugin.so names .dynamic "dding"), whose types and links are still right.
if shoff and shnum:
shstrndx = eh[13]
SHT_DYNAMIC = 6
def sh(i):
return list(struct.unpack_from(sh_fmt, buf, shoff + i * shentsize))
names_hdr = sh(shstrndx)
names_off = names_hdr[4]
def sec_name(h):
end = buf.index(b"\0", names_off + h[0])
return bytes(buf[names_off + h[0]:end]).decode()
for i in range(shnum):
h = sh(i)
n = sec_name(h)
if n == ".dynstr":
h[3], h[4], h[5] = va_start + str_off, file_end + str_off, len(new_str)
elif n == ".dynamic":
h[3], h[4], h[5] = va_start + dyn_off, file_end + dyn_off, len(dyn_bytes)
else:
continue
headers = [sh(i) for i in range(shnum)] # sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link
dyn_secs = [i for i, h in enumerate(headers) if h[1] == SHT_DYNAMIC]
str_secs = {headers[i][6] for i in dyn_secs if 0 < headers[i][6] < shnum}
str_secs |= {i for i, h in enumerate(headers) if h[3] == strtab_va and h[3]} # .dynstr by address
for i in dyn_secs:
h = headers[i]
h[3], h[4], h[5] = va_start + dyn_off, file_end + dyn_off, len(dyn_bytes)
struct.pack_into(sh_fmt, buf, shoff + i * shentsize, *h)
for i in str_secs - set(dyn_secs):
h = headers[i]
h[3], h[4], h[5] = va_start + str_off, file_end + str_off, len(new_str)
struct.pack_into(sh_fmt, buf, shoff + i * shentsize, *h)
result = bytes(buf)
got = needed(result)
+3
View File
@@ -840,6 +840,7 @@
"Lets the game find extra content it keeps next to its data (for example Star Wars Tales' seasons).": "",
"Lets the game finish loading when its separate loading splash stalls the render thread.": "",
"Lets the game leave its loading screen: it waited for a Quest-only step that never starts on the Frame (for example Vader Immortal).": "",
"Lets the game load: it reserves more memory at start than the Frame grants in one piece and crashed before its first frame (for example Transformers Beyond Reality).": "",
"Lets the game use language files already in its data instead of waiting for a download that never comes.": "",
"Lets the game use your controllers as a gamepad instead of a pointer, for 2D games with touch controls (for example SDL games).": "",
"Lets typing reach this app from Steam's on-screen keyboard or your PC's keyboard. VR keeps working.": "",
@@ -1692,12 +1693,14 @@
"Unreal builds of Meta XR Audio abort ('ClassNotFoundException com.oculus.os.AnalyticsEvent') on non-Quest devices. Patches that one JNI FindClass call so the library continues (for example NOPE Challenge).": "",
"Unreal games can check their whole data file (.obb) before starting (bVerifyOBBOnStartUp). On the Frame that check's progress and error screens are invisible (Lepton runs VR games without an Android window), so the game seems to hang on its DownloaderActivity, for example Contractors. This turns the check off in the manifest; Unreal still checks that the OBB is there with the right size, so a missing or wrong-version OBB still stops the game.": "",
"Unreal games start CrashReportClient when they crash, which leaves a crash dialog instead of simply closing. This passes -nocrashreports to the game and, on the Frame, renames the game's copy of CrashReportClient.exe so it can't start (your game files on this PC aren't changed).": "",
"Unreal reserves its memory ranges on Android with ordinary read/write mappings. Games built with Unreal's Binned3 allocator reserve tens of gigabytes at once while the engine loads (76 GiB in Transformers Beyond Reality); Quest allows that, the Frame's kernel refuses mappings larger than its memory, and the game crashes before its first frame (SIGSEGV at fault addr 0x10 in libUnreal.so, inside System.loadLibrary). This marks those mappings MAP_NORESERVE, as Quest effectively treats them: memory is still only used when the game touches it.": "",
"Unreal's Java OBB delivery check can leave a game on an invisible DownloaderActivity screen even with the manifest CRC check off (e.g. Into The Radius). Makes known Unreal delivery predicates return true, including recognized obfuscated versions, bypassing the Java name and size gate. Experimental: the native engine still needs complete, matching, readable OBB data.": "",
"Unreal's Oculus module looks up every OVRPlugin function it was built against and starts no VR when one is missing; OVRPort's OpenXR OVRPlugin lacks some old ones (for example ovrp_GetPTWNear, which Unreal 4.25 builds like Star Wars Pinball VR look up). The game then never starts an OpenXR session and crashes a few seconds in. Adds a generated library to libOVRPlugin.so with a stand-in for each missing function that reports failure (ovrpFailure), so Unreal's Oculus support starts. The functions OVRPort has are untouched.": "",
"Unreal: bypass the stalled loading splash": "",
"Unreal: don't wait for the Quest shader precompile": "",
"Unreal: match multiview depth to the eye size": "",
"Unreal: Quest controller bindings (RPOC key map)": "",
"Unreal: reserve memory like on Quest": "",
"Unreal: skip the OBB delivery check": "",
"Unreal: stand-ins for OVRPlugin functions OVRPort lacks": "",
"Unreal: start without the OBB check": "",
@@ -38,7 +38,7 @@ def _words(data: bytes | bytearray, off: int, n: int) -> list[int]:
def _function_offset(data: bytes, size: int) -> int | None:
"""File offset of FUNC in the loader (None: not an ELF, no such export, or shorter than `size` bytes)."""
try:
dynsym = elf._elf(data).get_section_by_name(".dynsym")
dynsym = elf.dynsym_section(elf._elf(data))
except Exception: # noqa: BLE001
return None
found = dynsym.get_symbol_by_name(FUNC) if dynsym is not None else None
@@ -51,7 +51,7 @@ def _x0_address(data: bytes, segs, at: int, back: int = 8) -> int | None:
def _function(data: bytes, name: str) -> tuple[int, int] | None:
sec = elf._elf(data).get_section_by_name(".dynsym")
sec = elf.dynsym_section(elf._elf(data))
for sym in (sec.get_symbol_by_name(name) or []) if sec else []:
if sym["st_value"] and sym["st_size"]:
return sym["st_value"], sym["st_size"]
@@ -15,7 +15,7 @@ VIDEO_LIBS = ("libavcodec", "libffmpeg", "libvlc", "libmpv", "libijkffmpeg")
def raise_swapchain_limit(data: bytes) -> tuple[bytes | None, int]:
"""Rewrite the `cmp wN, #4096` size guards at the start of OVRPort's swapchain entry points."""
try:
dynsym = elf._elf(data).get_section_by_name(".dynsym")
dynsym = elf.dynsym_section(elf._elf(data))
except Exception: # noqa: BLE001
return None, 0
if dynsym is None:
@@ -0,0 +1,159 @@
"""Unreal's Android memory reservations without commit-charge accounting (GitHub #349, Transformers Beyond Reality).
Unreal reserves virtual memory on Android with `mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0)` (FPlatformVirtualMemoryBlock::AllocateVirtual: Android has no separate commit step). Games built with
FMallocBinned3 reserve the whole allocator range up front while libUnreal.so loads, 76 GiB in Transformers Beyond
Reality. Quest's kernel lets that through; the Frame runs with the kernel's heuristic overcommit (mode 0), which
refuses any single writable private mapping larger than RAM + swap (16 + 8 GiB), so the reservation fails, Unreal's
out-of-memory handler runs before its log exists and dereferences a null pointer (SIGSEGV, fault addr 0x10, from a
static constructor inside System.loadLibrary). Adding MAP_NORESERVE to those mmap calls gives them the Quest
behaviour: pages are still only backed when touched, the mapping just isn't charged up front.
Each site is `mov w3, #0x22` (MAP_PRIVATE|MAP_ANONYMOUS) a few instructions before a `bl` to mmap's PLT stub, with
`mov x0, xzr` and `mov w2, #3` in the same window; the one instruction becomes `mov w3, #0x4022`. Nothing else changes
and nothing moves; only arm64 engine libraries."""
from __future__ import annotations
import io
import struct
from ...analysis import elf
from ..base import ApkContext, Patch, Suggestion, register
ENGINE_LIBS = ("libUE4.so", "libUnreal.so")
MOV_W3_FLAGS = 0x52800443 # mov w3, #0x22 MAP_PRIVATE|MAP_ANONYMOUS
MOV_W3_NORESERVE = 0x52880443 # mov w3, #0x4022 MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE
MOV_W2_RW = 0x52800062 # mov w2, #3 PROT_READ|PROT_WRITE
MOV_X0_NULL = 0xAA1F03E0 # mov x0, xzr
WINDOW_BEFORE, WINDOW_AFTER = 8, 9
def _mmap_slots(data: bytes) -> set[int]:
"""GOT slots of mmap/mmap64 (R_AARCH64_JUMP_SLOT relocations)."""
from elftools.elf.elffile import ELFFile
from elftools.elf.relocation import RelocationSection
e = ELFFile(io.BytesIO(data))
dynsym = elf.dynsym_section(e)
if dynsym is None:
return set()
slots = set()
for sec in e.iter_sections():
if not isinstance(sec, RelocationSection) or sec["sh_type"] != "SHT_RELA":
continue
for r in sec.iter_relocations():
sym = r["r_info_sym"]
if sym and dynsym.get_symbol(sym).name in ("mmap", "mmap64"):
slots.add(r["r_offset"])
return slots
def _plt_slot(data: bytes, segs, va: int) -> int | None:
"""The GOT slot a PLT stub at `va` jumps through: adrp x16, page; ldr x17, [x16, #off]."""
off = elf.vaddr_to_offset(segs, va)
if off is None or off + 8 > len(data):
return None
adrp, ldr = struct.unpack_from("<II", data, off)
if adrp & 0x9F00001F != 0x90000010 or ldr & 0xFFC003FF != 0xF9400211:
return None
imm = ((adrp >> 29) & 3) | ((adrp >> 5) & 0x7FFFF) << 2
if imm & 1 << 20:
imm -= 1 << 21
return (va & ~0xFFF) + (imm << 12) + ((ldr >> 10) & 0xFFF) * 8
def mmap_sites(data: bytes) -> tuple[list[int], list[int]]:
"""File offsets of the flag instructions of anonymous read/write mmap calls: (still MAP_PRIVATE|MAP_ANONYMOUS,
already MAP_NORESERVE)."""
if not elf.is_elf(data) or not elf.is_64bit(data) or data[5] != 1:
return [], []
slots = _mmap_slots(data)
if not slots:
return [], []
segs = elf.load_segments(data)
stubs: dict[int, bool] = {}
todo, done = [], []
for word, out in ((MOV_W3_FLAGS, todo), (MOV_W3_NORESERVE, done)):
needle = struct.pack("<I", word)
at = data.find(needle)
while at >= 0:
va = next((at - o + v for v, o, s in segs if o <= at < o + s), None) if at % 4 == 0 else None
if va is not None:
for k in range(1, WINDOW_AFTER + 1):
if at + 4 * k + 4 > len(data):
break
w = struct.unpack_from("<I", data, at + 4 * k)[0]
if w >> 26 != 0b100101: # bl
continue
imm = w & 0x3FFFFFF
target = va + 4 * k + ((imm - (1 << 26)) if imm & 1 << 25 else imm) * 4
if target not in stubs:
stubs[target] = _plt_slot(data, segs, target) in slots
if stubs[target]:
window = {struct.unpack_from("<I", data, at + 4 * j)[0]
for j in range(-WINDOW_BEFORE, k) if at + 4 * j >= 0}
if MOV_W2_RW in window and MOV_X0_NULL in window:
out.append(at)
break
at = data.find(needle, at + 1)
return todo, done
def patch_lib(data: bytes) -> tuple[bytes, int]:
todo, _done = mmap_sites(data)
if not todo:
return data, 0
buf = bytearray(data)
for at in todo:
struct.pack_into("<I", buf, at, MOV_W3_NORESERVE)
return bytes(buf), len(todo)
class UnrealNoReserve(Patch):
id = "frame.unreal_noreserve"
title = "Unreal: reserve memory like on Quest"
description = ("Unreal reserves its memory ranges on Android with ordinary read/write mappings. Games built with "
"Unreal's Binned3 allocator reserve tens of gigabytes at once while the engine loads (76 GiB in "
"Transformers Beyond Reality); Quest allows that, the Frame's kernel refuses mappings larger "
"than its memory, and the game crashes before its first frame (SIGSEGV at fault addr 0x10 in "
"libUnreal.so, inside System.loadLibrary). This marks those mappings MAP_NORESERVE, as Quest "
"effectively treats them: memory is still only used when the game touches it.")
order = 49
needs_vr = False
def applies(self, a):
return a.engine == "Unreal" and "arm64-v8a" in a.abis
def detect(self, a):
if self.applies(a) and (a.extra or {}).get("unreal_binned3"):
return Suggestion(True, "Unreal's Binned3 allocator reserves more memory at start than the Frame's "
"kernel allows in one piece; without this the game crashes while it loads.")
return None
def apply(self, ctx: ApkContext) -> bool:
ws = ctx.ws
if ws.abi != "arm64-v8a":
return False
lib = next((n for n in ENGINE_LIBS if ws.has(ws.lib(n))), None)
if not lib:
return False
data, count = patch_lib(ws.read_lib(lib))
if not count:
if not mmap_sites(data)[1]:
ctx.notes.append(f"{self.id}: no anonymous read/write mmap calls found in {lib}, left unchanged")
return False
ws.put(ws.lib(lib), data)
ctx.notes.append(f"{count} mmap call(s) in {lib} reserve with MAP_NORESERVE")
return True
def validate(self, ctx: ApkContext):
ws = ctx.ws
lib = next((n for n in ENGINE_LIBS if ws.has(ws.lib(n))), None)
if not lib or ws.abi != "arm64-v8a":
return []
todo, done = mmap_sites(ws.read_lib(lib))
return [("Unreal memory reservations use MAP_NORESERVE", bool(done) and not todo,
f"{len(done)} rewritten, {len(todo)} left")]
register(UnrealNoReserve)
+2
View File
@@ -118,6 +118,8 @@ SUMMARIES = {
"empty Gear VR controls on the Frame, for example Vader Immortal).",
"frame.unreal_splash_bypass": "Lets the game finish loading when its separate loading splash stalls the "
"render thread.",
"frame.unreal_noreserve": "Lets the game load: it reserves more memory at start than the Frame grants in one "
"piece and crashed before its first frame (for example Transformers Beyond Reality).",
"frame.unreal_multiview_depth": "Shows the full environment when its upper part is clipped but menu panels "
"look correct.",
"frame.unreal_thumb_touch": "Makes the hands' thumbs follow what your thumb touches (they always pointed up: the "
+23
View File
@@ -47,3 +47,26 @@ def test_add_needed(variant):
assert all(off % 16384 == va % 16384 for va, off, _ in loads)
# idempotent
assert elf.add_needed(out, "libglshim.so") == out
def _scramble_section_names(lib: bytes) -> bytes:
"""Rotate every section's sh_name by one, like a shipped libOVRPlugin.so whose names are shifted (GitHub #349:
.dynamic named "dding", .dynsym "text"); types and links stay right."""
buf = bytearray(lib)
shoff = struct.unpack_from("<Q", buf, 40)[0]
shentsize, shnum = struct.unpack_from("<H", buf, 58)[0], struct.unpack_from("<H", buf, 60)[0]
names = [struct.unpack_from("<I", buf, shoff + i * shentsize)[0] for i in range(1, shnum)]
for i, name in enumerate(names[1:] + names[:1], start=1):
struct.pack_into("<I", buf, shoff + i * shentsize, name)
return bytes(buf)
def test_scrambled_section_names():
base = elf.add_needed(build_stub_library(["ovr_X", "ovr_Y"], soname="libgame.so"), "liblog.so")
lib = _scramble_section_names(base)
assert elf.dynsym_section(elf._elf(lib)) is not None
assert elf.dyn_symbols(lib, True) == {"ovr_X", "ovr_Y"}
assert elf.find_symbols(lib, ["ovr_Y"]).keys() == {"ovr_Y"}
out = elf.add_needed(lib, "libfp_x.so") # raised "produced NEEDED=[...]" when sections were found by name
assert elf.needed(out) == ["libfp_x.so", "liblog.so"]
assert elf.dyn_symbols(out, True) == {"ovr_X", "ovr_Y"}
+21
View File
@@ -17,6 +17,27 @@ LOG_BAD = ("""
""")
def test_unreal_reservation_refused_while_loading():
"""GitHub #349: Unreal's out-of-memory handler crashes inside libUnreal.so's static constructors."""
lib = "/data/app/~~x==/com.Meta4.TFBeyondReality-y==/lib/arm64/libUnreal.so (BuildId: 57d064dc)"
crash = (
"10-16 14:41:01.814 1185 1185 F libc : Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x10"
" in tid 1185 (TFBeyondReality), pid 1185 (TFBeyondReality)\n"
"10-16 14:41:01.858 1219 1219 F DEBUG : Cause: null pointer dereference\n"
"10-16 14:41:01.955 1219 1219 F DEBUG : backtrace:\n"
f"10-16 14:41:01.955 1219 1219 F DEBUG : #00 pc 00000000018bbe3c {lib}\n"
f"10-16 14:41:01.955 1219 1219 F DEBUG : #01 pc 00000000017ceccc {lib}\n"
"10-16 14:41:01.957 1219 1219 F DEBUG : #15 pc 000000000004b048 /apex/com.android.runtime/bin/"
"linker64 (__dl__ZN6soinfo17call_constructorsEv+380)\n")
r = triage("", "EXITED", crash=crash)
ids = {f.id for f in r.findings}
assert "unreal-reserve-refused" in ids and "native-crash" not in ids
assert next(f for f in r.findings if f.id == "unreal-reserve-refused").suggest == ["frame.unreal_noreserve"]
# the same fault address in a later render-thread crash isn't this
later = crash.replace("__dl__ZN6soinfo17call_constructorsEv+380", "__pthread_start(void*)+64")
assert "unreal-reserve-refused" not in {f.id for f in triage("", "EXITED", crash=later).findings}
def test_lua_pointer_error_and_cleanup_stack_are_distinguished():
pointer = triage("PANIC: bad light userdata pointer", "EXITED")
finding = next(f for f in pointer.findings if f.id == "lua-lightuserdata-pointer")
+91
View File
@@ -0,0 +1,91 @@
import struct
import pytest
from test_unreal_splash import context, engine
from frameport.analysis import detect, elf
from frameport.apk.workspace import ApkWorkspace
from frameport.patches.frame import unreal_noreserve as nr
SLOT = 0x123458 # mmap's GOT slot (any address: _mmap_slots is replaced below)
def adrp(rd: int, pc: int, target: int) -> int:
pages = (target >> 12) - (pc >> 12)
return 0x90000000 | (pages & 3) << 29 | ((pages >> 2) & 0x7FFFF) << 5 | rd
def bl(pc: int, target: int) -> int:
return 0x94000000 | ((target - pc) >> 2) & 0x3FFFFFF
def mmap_engine(prot=nr.MOV_W2_RW, flags=nr.MOV_W3_FLAGS):
"""An engine library whose code calls mmap(NULL, size, prot, flags, -1, 0) through a PLT-style stub."""
data, off = engine(symbol="FAndroidVirtualReserve", size=0x80, prologue=b"")
va = next(v for v, o, s in elf.load_segments(data) if o <= off < o + s) + off - next(
o for v, o, s in elf.load_segments(data) if o <= off < o + s)
stub = va + 0x40
call = [nr.MOV_X0_NULL, prot, flags, 0x12800004, 0xAA1F03E5] # mov w4, #-1; mov x5, xzr
code = call + [bl(va + 4 * len(call), stub)]
words = code + [0xD503201F] * (16 - len(code)) # nop up to the stub
words += [adrp(16, stub, SLOT), 0xF9400211 | ((SLOT & 0xFFF) // 8) << 10, 0x91000210, 0xD61F0220]
buf = bytearray(data)
struct.pack_into(f"<{len(words)}I", buf, off, *words)
return bytes(buf), off + 8 # file offset of the flags instruction
@pytest.fixture(autouse=True)
def fake_got(monkeypatch):
monkeypatch.setattr(nr, "_mmap_slots", lambda data: {SLOT})
def test_reservation_gets_map_noreserve_and_survives_rebuild(tmp_path):
data, flags_at = mmap_engine()
assert nr.mmap_sites(data) == ([flags_at], [])
patch = nr.UnrealNoReserve()
ctx = context(tmp_path, data)
with ctx.ws:
assert patch.apply(ctx)
fixed = ctx.ws.read_lib("libUE4.so")
assert fixed == data[:flags_at] + struct.pack("<I", nr.MOV_W3_NORESERVE) + data[flags_at + 4:]
assert patch.validate(ctx)[0][1]
assert not patch.apply(ctx)
ctx.ws.write(tmp_path / "out.apk")
with ApkWorkspace(tmp_path / "out.apk") as ws:
ctx.ws = ws
assert not patch.apply(ctx)
assert nr.mmap_sites(ws.read_lib("libUE4.so")) == ([], [flags_at])
@pytest.mark.parametrize("kwargs", [{"prot": 0x52800022}, {"flags": 0x52800043}]) # PROT_READ only; MAP_SHARED
def test_other_mmap_calls_are_left_alone(tmp_path, kwargs):
data, _ = mmap_engine(**kwargs)
assert nr.mmap_sites(data) == ([], [])
ctx = context(tmp_path, data)
with ctx.ws:
assert not nr.UnrealNoReserve().apply(ctx)
def test_call_to_another_function_is_left_alone(monkeypatch):
data, _ = mmap_engine()
monkeypatch.setattr(nr, "_mmap_slots", lambda data: {SLOT + 8})
assert nr.mmap_sites(data) == ([], [])
def test_binned3_detection():
assert detect.unreal_binned3(b"x\0./Runtime/Core/Private/HAL/MallocBinned3.cpp\0y")
assert detect.unreal_binned3("FMallocBinned3 Attempt to free an unrecognized pointer %p".encode("utf-16-le"))
# other UE5 builds only name Binned3's console variables
assert not detect.unreal_binned3("MallocBinned3.FlushThreadCacheMaxWaitTime".encode("utf-16-le"))
def test_suggested_only_for_binned3_unreal():
patch = nr.UnrealNoReserve()
class A:
engine, abis, extra = "Unreal", ["arm64-v8a"], {"unreal_binned3": True}
assert patch.detect(A()).recommended
A.extra = {}
assert patch.detect(A()) is None
A.engine = "Unity"
assert not patch.applies(A())