Compare commits

...
100 Commits
Author SHA1 Message Date
spoopyghosty0 0df0bd16b5 hevc: tone map HDR (PQ/HLG) Main 10 to SDR BT.709 instead of flat colours
The software Main 10 path narrowed HDR10 pictures to 8 bits as they were, so
PQ shown as SDR looked grey and BT.2020 colours dull. hdr_tonemap.h maps
brightness with BT.2390's EETF (203 cd/m2 reference white = SDR white,
highlights rolled off up to MaxCLL / the mastering peak / 1000 cd/m2) and
converts colour per chroma sample through linear light into BT.709, moving
out-of-gamut colours toward grey of the same luminance. The component then
prefers its bitstream colour aspects, so surfaces and its RGBA conversion see
BT.709. Bands run in the colour helper threads (about 10 ms per 1080p picture
on one big core). Decoder revision 10; artifacts rebuilt with build.py.
2026-10-19 12:41:20 -04:00
Veticia Misumena 7128432e39 hevc: decode Main 10 in software instead of hanging on Iris
Iris on the Frame's kernel (6.18) offers HEVC Main 10 only into its
compressed Q10C capture format. The component always asks for NV12, and a
Main 10 stream in such a session returned no pictures: playback stayed at
0:00, the flush on leaving timed out (ACodec -110) and the next video hit
MediaCodec::reclaim, so every later decoder start in the app failed until it
restarted (seen in Bigscreen Beta with local files over DLNA).

- hevc_profile.h reads general_profile_idc from the parameter sets
  (CODECCONFIG or the first access unit, Annex B or hvcC); streams beyond
  Main never open Iris and decode in FFmpeg's software HEVC decoder, which
  the component already contains.
- Its 10-bit planar pictures are rounded to 8-bit planar before the existing
  outputs (no HDR tone mapping).
- The HEVC component lists Main 10 (level 5.1), so format-aware players
  select it instead of reporting NoSupport and using it anyway.
- Decoder revision 9; artifacts rebuilt with native/hevc/build.py (the same
  toolchain rebuilds revision 8 byte for byte).

Tested on a Steam Frame in Bigscreen Beta: 1080p and 4K Main 10 clips and a
real 1080p Main 10 episode play; 8-bit HEVC and H.264 afterwards play too.
FFmpeg's software HEVC decoder on the Frame's CPU, 4 threads as the
component uses: 1080p Main 10 episode 232 fps, synthetic 4K Main 10 72 fps.
2026-10-19 12:22:36 -04:00
spoopyghosty0 3268c3c412 Site: a Room tab in the clickable app (turnable 3D view of the showcase's made-up bedroom, pretend scan, raw scan vs clean-up, labels with the door suggestion); room data from scripts/showcase/site_room.py 2026-10-19 11:12:48 -04:00
spoopyghosty0 11dd195688 Site: release data refreshed (1.3.0) 2026-10-19 01:12:55 -04:00
spoopyghosty0 5b86843430 FramePort 1.3.0 2026-10-19 00:59:27 -04:00
spoopyghosty0 3e5044ce4a Site: contributor data and games list refreshed (1.3.0) 2026-10-19 00:58:53 -04:00
spoopyghosty0 5345434ba7 Lint: wrap long lines, raise ... from None in the room-scan lock, zip(strict=True) in the scene tests (CI's ruff check failed on the MR merge) 2026-10-19 00:57:08 -04:00
spoopyghosty0 0ca7bc45f0 Room tab: the right column sizes to its content (the Test Quest MR line was clipped when Delete room wrapped or the furniture chips took two rows); the top row stays level by measuring both sides (a short column's last card grows, a tall one makes the 3D view taller); box sizes under 10 cm show two decimals (a 3 cm rug was 0.0 m); room shots re-rendered 2026-10-19 00:23:58 -04:00
spoopyghosty0 a4702d2a13 Docs screenshots: re-rendered with the Room tab in the sidebar 2026-10-19 00:23:57 -04:00
spoopyghosty0 455bc987b2 Docs: the Room tab (experimental) in the README features, INSTALL.md and the site's app tour; showcase shots room + room-editing from a made-up bedroom (bed under a wall TV, desk with two monitors under a window, closet, door, lamp, plant; noisy scan mesh, clean-up report and labels to match); steps open_room / room_editing 2026-10-19 00:23:57 -04:00
spoopyghosty0 4b0b0daa6d Room tab layout: the top row ends level (legend on the 3D view, the right column as tall as it), Test Quest MR as one line in In this room, equal cards and gaps, full-width sliders and segmented tracks, one slider per row in the box editor, dropdown and Delete the same height, even suggestion buttons, aligned Labels columns and counts; clean-up report follows the scanner's scan.json keys (walked_skipped as a note, truncated as a warning) 2026-10-19 00:23:57 -04:00
spoopyghosty0 91f73fbd03 ui_smoke: Room tab editing steps (clean-up, more settings, box selected, doors, a suggestion taken, applying, applied); the pretend Frame cleans up and saves rooms; docs 2026-10-19 00:23:56 -04:00
spoopyghosty0 77b527f129 Room tab: Clean-up card (Recommended or your settings, the key ones in sight, More settings, Apply as a job, the last clean-up's counts), Labels card (rule decisions, Add as door / Add as closet), Furniture & screens editor (select in the list or by clicking the 3D view, type/move/size/turn, add/delete, doors and windows on a wall, Save) 2026-10-19 00:23:23 -04:00
spoopyghosty0 6ab6c11f06 Rooms: REFINE_DEFAULTS (the owner's clean-up defaults + slider metadata), rooms.refine / put_room, doors and windows on a wall and suggestion helpers in mr/room, view3d furniture overlay + picking, CLI frameport frame room refine [--set key=value] [--reset] 2026-10-19 00:23:23 -04:00
spoopyghosty0 92e84a1a4b Agent v107: room scan clean-up settings (rooms/refine.json, passed to every build), room_refine (fp_roomscan --refine-only on a copy of the room folder), raw files + scan.json kept with the room, room_status returns refine/report/labels/suggestions, edited rooms keep their boxes over a new clean-up 2026-10-19 00:23:22 -04:00
spoopyghosty0 a0b71fdfb7 Docs: room refinement (scan.md options + passes + parity, ROOM_FORMAT raw files and scan.json, CLAUDE.md MR line) 2026-10-19 00:23:22 -04:00
spoopyghosty0 3e8f0ce369 Tests: room refinement (floating pole + walked-through column removed, panels wind inward, --refine-only reproduces, --refine 0 = raw output, label rules on an older room folder) 2026-10-19 00:23:22 -04:00
spoopyghosty0 b65abb4b95 Room scanner: refinement + label rules after the mesh (raw files, scan.json, --refine-only)
The owner's tuned room-scan cleanup (2026-10-18) ported to C, pass for pass: plane offsets re-estimated from the
scan, one-sided structural bands, flat wall/floor/ceiling panels (or snap), near clearing, erase spheres, the walked
volume from the recording's head path, furniture tops, clipping with the unseen-edge allowance, islands, thin
floating pieces, Taubin smoothing; label rules on the scanner's volumes (merged/nested/outside boxes removed, shape
relabelling, SCREEN flatness from the raw mesh, door/closet suggestions in scan.json).

room.txt + room-mesh.bin are now refined (ceiling re-estimated); the unrefined output stays as room-raw.txt +
room-mesh-raw.bin with the same UUIDs; scan.json holds heads, settings, report, labels, suggestions. The scan path
refines its raw files as read back, so --refine-only on them reproduces the scan byte for byte; --refine 0 writes
the old output. The fusion grid is freed before the refinement (peak memory unchanged).
2026-10-19 00:23:22 -04:00
spoopyghosty0 0478588886 Tests: map the floor plan back with forward (-Z) at the top, as preview_svg now draws it (4f871b20) 2026-10-19 00:23:21 -04:00
spoopyghosty0 1a5892ee40 RoomCanvas: a room full of furniture draws only its outlines while the view moves (smaller patches) 2026-10-19 00:23:21 -04:00
spoopyghosty0 e3788c0b2b Room tab: fit the view to the scan without its stray far-off bits (2nd-98th percentile), rounded icons, US spelling 2026-10-19 00:23:20 -04:00
spoopyghosty0 feda5f4a03 Docs: the Room tab (INSTALL.md, CLAUDE.md UI notes) 2026-10-19 00:23:20 -04:00
spoopyghosty0 6d9fb0b1e6 ui_smoke: Room tab steps; the pretend Frame has a made-up scanned room with a mesh, a scan that steps forward and can be cancelled 2026-10-19 00:23:20 -04:00
spoopyghosty0 b78160ca6f Room tab: scan in the headset with progress and Cancel, play area, delete, the room in 3D (window + full browser view), what's in it, an experimental pill and a Test Quest MR tip; the room cache moves to ui/room_state.py (mesh cached per Frame in rooms-cache), the Frame page keeps a short room card with Open Room 2026-10-19 00:23:20 -04:00
spoopyghosty0 20c36b112e RoomCanvas: the room in 3D inside the window (Flet canvas, one redraw worker at <=18 fps through a LoopUpdater, fewer mesh edges while moving; drag turns, scroll/pinch zooms, right-drag moves, double-click fits) 2026-10-19 00:23:19 -04:00
spoopyghosty0 fcfba77f31 Room 3D models: mr/view3d (walls, furniture boxes, mesh reduced to a few thousand edges, orbit camera, dollhouse projection) and a token-protected 127.0.0.1 server for a self-contained WebGL room page (FPRM parsed with DataView) 2026-10-19 00:23:19 -04:00
spoopyghosty0 8870afe18a Room scans can be cancelled: agent room_capture_stop (agent 106) stops the scan job and marks it cancelled; rooms.cancel_scan, rooms.scan raises ScanCancelled 2026-10-19 00:23:19 -04:00
spoopyghosty0 4be04b8a34 Room floor plan: seen from above with forward (-Z) at the top; it had +Z at the top with +X still on the right, i.e. the room mirrored 2026-10-19 00:23:18 -04:00
spoopyghosty0 e231c5a6fe scene: drop PR #246's per-call semantic-label log line (it logged every call; mr-scene logs each entity's labels once), as in the PR's merge 2026-10-19 00:23:18 -04:00
spoopyghosty0 7585acd0a8 Tests: the quest_buttons harness declares scene_app_mesh (mr-scene's xrCreateInstance sets it; the native test didn't compile) 2026-10-19 00:23:18 -04:00
spoopyghosty0 b9fb5791b0 Regenerated docs/PATCHES.md, site patches.json and the locale template for the two new patches 2026-10-19 00:23:17 -04:00
spoopyghosty0 a5199495fd Artifacts: FrameBridge (arm64-v8a + armeabi-v7a) rebuilt from the merged source (PR #246's layer limit and quad filter on mr-scene's scene emulation); xrshim rebuilt, unchanged bytes; SHA256SUMS regenerated (now also lists the roomscan finish_*.png prompts, which it had missed). None of PR #246's binaries are kept. 2026-10-19 00:23:17 -04:00
spoopyghosty0 28a5795323 Docs: PLAYBOOK rows for depth occlusion, the player-in-room check and the layer limit; DEVICE_NOTES note on what PR #246 brought into mr-scene 2026-10-19 00:23:16 -04:00
spoopyghosty0 d6cf9a3503 Tests: restore the scan test's APK inside a PC program folder (PR #246 had made it a .dll, which no longer tests anything); ruff fixes in the PR's test 2026-10-19 00:23:16 -04:00
spoopyghosty0 2a769dddba Catalog (PR #246): min_app 1.2.2 on the MR recipes (they need this branch's layer limit and room emulation); drop the com.mrf.* copies of games the catalog already has (com.kluge.SynthRiders, com.PotamWorks.SmashDrums, jp.co.hit_point.vneko: same titles, other package ids, they listed every game twice); LEGO Bricktales back to 'use VR mode' (MR not verified, its recipe turns no MR setting on); honest notes where no tested version was recorded 2026-10-19 00:23:15 -04:00
spoopyghosty0 7a65e86cdc FrameBridge layer limit: FrameBridge's own fade quad is always kept with the projection layers, the app's other layers fill the rest in submission order; one log line per trim (at most 5), trimmed layers no longer counted as unusable 2026-10-19 00:23:15 -04:00
spoopyghosty0 2be8873fec frame.meta_depth_occlusion: offered but not turned on by detection, only for 64-bit IL2CPP Unity games; asset files are streamed and only those with the keywords rewritten (max 512 MiB); Laser Dance's asset fixes only for the build whose original values all match; revision 1
- detect: many Meta XR SDK games carry the Depth API without using it; until a headset check shows more games
  with the symptom (or OVRPort is found advertising XR_META_environment_depth, which a FrameBridge hide setting
  would fix generally), the patch comes from Laser Dance's catalog recipe or the user's choice.
- ApkWorkspace.size/contains: entries are searched in 4 MiB chunks instead of read whole.
- Laser Dance: the fixes' original values at all seven places together identify the build (no versionName was
  recorded with them); another build gets none of them.
- tests: lz4 is skipped when missing, the known/unknown Laser Dance build, chunked search.
2026-10-19 00:23:14 -04:00
spoopyghosty0 7d5ed42d18 frame.unity_scene_active_rooms: never suggested (catalog recipe or the user's choice, Laser Dance), shown only for Unity scene games; plain summaries for both new Unity patches, stray blank lines removed
The PR turned it on for every Unity game with USE_SCENE or MRUK's library: a game-logic change keyed to one
game's symptom (MRUK games mostly don't use OVRSceneManager at all). A scanned room around the player usually
passes IsUserInRoom by itself.
2026-10-19 00:23:14 -04:00
spoopyghosty0 90bf8bcd67 adapter: scene_emul needs the xrshim through XRSHIM_SETTINGS already (mr-scene); drop PR #246's duplicate 2026-10-19 00:23:14 -04:00
spoopyghosty0 d028390da5 Library: drop PR #246's meta_spatial_permissions and meta_depth_occlusion migrations and the re-analysis hook that forced frame.meta_depth_occlusion into user recipes (mr-scene's scene_permissions covers the room games; non-user recipes follow detection and the catalog after the analysis bump; user recipes stay as edited) 2026-10-19 00:23:14 -04:00
spoopyghosty0 0a542d849e xrshim: emulated functions again only for a real instance (OpenXR: XR_NULL_HANDLE serves only the pre-instance functions; with mr-scene the shim would otherwise hand out the routed xrPollEvent/xrLocateSpace) 2026-10-19 00:23:13 -04:00
alicecatpaw 09358fb65e Address review comments on meta_depth_occlusion
- Limit _patch_shader_proximity_fade to known builds (Laser Dance) and known shader signatures/keywords to avoid modifying unrelated shaders
- Rename HARD_OCCLUSION and SOFT_OCCLUSION keywords in .assets files only, leaving global-metadata.dat intact so managed code cannot enable renamed occlusion variants
- Defer meta_depth_occlusion library migration for entries analysed before the field existed until re-analysis, and update user recipes on re-analysis
- Bump ANALYSIS_VERSION to 23 so older library entries are re-analysed for meta_depth_occlusion
2026-10-19 00:23:13 -04:00
alicecatpaw ae49d886da Address review: bump ANALYSIS_VERSION to 14, restore hand-tracking heuristic, and fix library migration 2026-10-19 00:23:13 -04:00
alicecatpaw 7afaa2629b Add catalog recipes for Neko Atsume, Punch Fit, Smash Drums, and Hitstream 2026-10-19 00:23:12 -04:00
alicecatpaw 5e27333f74 Add catalog recipes for Laser Dance and Synth Riders MR modes 2026-10-19 00:23:12 -04:00
alicecatpaw 68f278a744 Add Meta depth occlusion and active room patches for Unity MR games
- Implement frame.meta_depth_occlusion to zero out Meta EnvironmentDepthManager methods in libil2cpp.so and neutralize close-proximity depth shader occlusion in Unity assets

- Implement frame.unity_scene_active_rooms to bypass Meta OVRSceneManager active room position evaluations in libil2cpp.so

- Recommend frame.meta_permissions when spatial data or scene permissions are declared

- Remove redundant hand-tracking heuristic forcing controller_fix=0

- Add unit tests for depth occlusion, active room bypass, and manifest heuristics
2026-10-19 00:23:12 -04:00
alicecatpaw a0c26c9646 Emulate Meta spatial entity discovery and enforce runtime layer limits
- Emulate XR_META_spatial_entity_discovery (xrDiscoverSpacesMETA, xrRetrieveSpaceDiscoveryResultsMETA) and XR_META_boundary_visibility
- Add system properties for space discovery and boundary visibility
- Fix semantic labels length and null termination
- Allow xrGetInstanceProcAddr with null handle for emulated procs in xrshim
- Trim composition layers exceeding runtime maxLayerCount in xrEndFrame and validate quad extents
- Hook xrshim when scene_emul is enabled in adapter settings
- Update prebuilt native artifacts and checksums
2026-10-19 00:23:12 -04:00
spoopyghosty0 45900ef294 Installing a room-emulation game syncs the room scanner and its headset prompts to the Frame (in-game scan requests used whatever copy a PC-started scan had left there, e.g. none on a fresh Frame or without the new "Almost done" prompts) 2026-10-19 00:23:11 -04:00
spoopyghosty0 95eed63460 MR scan: once every direction, the ceiling and the floor are covered before the minimum scan time, the prompt says "Almost done" with the time left instead of a ring stuck at 12/12 (owner turned past 360 degrees waiting) 2026-10-19 00:23:11 -04:00
spoopyghosty0 04d03b8f87 MR scan preview: one calm colour (FramePort's secondary blue, opacity by floor/furniture/wall), filled 10 cm squares instead of 3x3 blotches, fainter, sort order below the scan prompts (owner: colours and layers clashed) 2026-10-19 00:23:11 -04:00
spoopyghosty0 6617941d47 MR scan preview (agent 104): a world-locked floor map overlay shows what the headset scan has covered (10 cm cells: floor grey, furniture orange, walls blue), redrawn at most once a second; also in game-requested scans; --no-preview / preview:false; disables itself on overlay errors or slow uploads 2026-10-19 00:23:10 -04:00
spoopyghosty0 05344906ef MR diagnostics: log room-entity locates (base space, flags; every one without both valid bits): MRUK keeps an anchor pending forever otherwise (Pinball FX's load never succeeds) 2026-10-19 00:23:10 -04:00
spoopyghosty0 94ece1a232 MR scan on request (FrameBridge): a capture request asks FramePort to scan (request/ack/result files), holds the completion, reloads the scanned room on ok, fails with XR_ERROR_RUNTIME_FAILURE otherwise, falls back to the stored room without a scanner (setting scene_scan_request, default on) 2026-10-19 00:23:10 -04:00
spoopyghosty0 38bd178aa3 MR scan on request (Frame side, agent 103): launch.sh watcher _scan_watch runs the headset room scan when FrameBridge requests one (request/ack/result files in the game's external files dir), saves it as the current room and copies it into the game folder 2026-10-19 00:23:09 -04:00
spoopyghosty0 56c821cce3 Triage: no scene-missing / mr-discovery-unsupported warning when FrameBridge serves the room (they matched OVRPort's extension list and the shim's own routing line; TestQuestMR saw the full room) 2026-10-19 00:23:09 -04:00
spoopyghosty0 3298c5f64d MR: scene capture completes after 1.5 s like a real scan (Pinball FX re-asked at once); log each room query per entity and the labels an app gets (with its support info) 2026-10-19 00:19:18 -04:00
spoopyghosty0 087ad75f47 MR: shim-routed generic calls handle only fake room spaces and room events; real ones go to OVRPort's dispatcher (its headset identity, events and controller spaces were skipped for every scene_emul game: Vermillion's controllers, NOPE) 2026-10-19 00:19:18 -04:00
spoopyghosty0 460c503e43 MR diagnostics: log the head against the floor outline when a game looks for the room (MRUK's current room; Pinball FX asks for a scan again and again) 2026-10-19 00:19:17 -04:00
spoopyghosty0 3687a6eb0d MR: Meta scene/anchor permissions with every scene_emul recipe (migration scene_permissions; NOPE/Pinball never asked for the room), emulate XR_META_boundary_visibility, log the app's passthrough calls 2026-10-19 00:19:17 -04:00
spoopyghosty0 e07441fb09 MR: hiding the STAGE bounds becomes per-game setting scene_hide_bounds (Demeter on; NOPE stalled on a black screen without a play area) 2026-10-19 00:19:16 -04:00
spoopyghosty0 212801ead2 Cpp2IL: run with invariant globalization (.NET aborts without libicu on fresh WSL/Linux); don't cache .NET start failures as unreadable games, retry old cached ones; CLAUDE.md/PLAYBOOK: STAGE bounds note 2026-10-19 00:19:16 -04:00
spoopyghosty0 7971354675 MR: hide the STAGE bounds from scene-emulation apps (OVRPlugin can't outline the Frame's 1x1 m rect: Demeter's guardian check threw), box walls in loop order 2026-10-19 00:19:15 -04:00
spoopyghosty0 c3144f65dd MR: room entities offer SHARABLE like Meta's runtime; log refused scene components (Demeter rejected the room) 2026-10-19 00:19:15 -04:00
spoopyghosty0 210d6e80c5 MR: rooms older Meta SDKs accept (labels as Meta's runtime maps them, GLOBAL_MESH only with the mesh extension), rectangular rooms from perpendicular walls, ceilings found after a short look up, 60 s minimum scan 2026-10-19 00:19:15 -04:00
spoopyghosty0 f8ee87145f MR scan: record only while passthrough streams meshes (a leftover block started a scan), pause + prompt when it stops, refuse scans under 30 frames 2026-10-19 00:19:14 -04:00
spoopyghosty0 52df211266 MR scan prompts: starts by itself (no passthrough wait), ceiling then 'look back down', floor step, finish early once everything was seen 2026-10-19 00:19:14 -04:00
spoopyghosty0 976260ed14 MR: agent and PC room parsers accept the optional wall label (scanner output was refused) 2026-10-19 00:19:13 -04:00
spoopyghosty0 893db92985 MR: scene emulation always brings the xrshim (OVRPort's dispatcher refuses scene calls; Pinball FX VR) 2026-10-19 00:19:11 -04:00
spoopyghosty0 a690d0ee3f Ship robust room scanner and document capture performance 2026-10-19 00:18:59 -04:00
spoopyghosty0 d48dcaba4e Fit stable room surfaces from smoothed depth fans 2026-10-19 00:18:58 -04:00
spoopyghosty0 f68c7f4232 Skip unusable tracking frames and reject inpainted depth 2026-10-19 00:18:58 -04:00
spoopyghosty0 fc5933bb2a Ship deterministic room scanner artifact and device validation notes 2026-10-19 00:18:58 -04:00
spoopyghosty0 0e41e1fcca Validate room scanning geometry and harden bounded reconstruction 2026-10-19 00:18:56 -04:00
spoopyghosty0 bbb05cfc0b Add offline room reconstruction and synthetic depth renderer 2026-10-19 00:18:44 -04:00
spoopyghosty0 71a427ac0f CLAUDE.md: mixed-reality room scanning notes 2026-10-19 00:18:44 -04:00
spoopyghosty0 e8eea8548c MR: scan the room in the headset - RoomView 3D capture with headset prompts, wall labels, agent 92
The agent records Valve's passthrough depth mesh (IVRBlockQueue + IVRPaths, Background/Overlay OpenVR app), converts
poses with the final raw->standing transform, shows pre-rendered prompts as a SteamVR overlay, then builds the room
with fp_roomscan (room_capture / room_capture_status). PC: rooms.scan, Frame page progress, frameport frame room scan.
FrameBridge reads the optional wall label (INVISIBLE_WALL_FACE). ROOM_FORMAT: outline orientation stated exactly.
2026-10-19 00:18:42 -04:00
spoopyghosty0 ab4bb5b68d Finish room launcher integration and document device validation 2026-10-19 00:18:40 -04:00
spoopyghosty0 ee902e4c40 Diagnose missing MR rooms and unsupported scene discovery 2026-10-19 00:18:16 -04:00
spoopyghosty0 e1fd4fd244 Enable scene emulation for scene and spatial anchor games 2026-10-19 00:18:15 -04:00
spoopyghosty0 a5172c1e2f Show room readiness and floor plans in the Frame UI 2026-10-19 00:18:12 -04:00
spoopyghosty0 5c9d2e5c62 Add room CLI and reuse host helper synchronization 2026-10-19 00:18:11 -04:00
spoopyghosty0 79c5a210f8 Store rooms on the Frame and synchronize Quest launchers 2026-10-19 00:18:10 -04:00
spoopyghosty0 4864703e11 Add portable room format validation and floor plan previews 2026-10-19 00:17:51 -04:00
spoopyghosty0 b3e5b2401e Document scene implementation validation and exact device handoff 2026-10-19 00:17:51 -04:00
spoopyghosty0 301f734dbf Rebuild FrameBridge scene emulation artifacts deterministically 2026-10-19 00:17:49 -04:00
spoopyghosty0 577b72969a Verify shim routing and protect erased or stale scene handles 2026-10-19 00:17:34 -04:00
spoopyghosty0 11ccdb7fac Test room geometry discovery persistence and malformed data on the host 2026-10-19 00:17:33 -04:00
spoopyghosty0 7fe604dbec Report scanned room or guardian box on scene capture requests 2026-10-19 00:17:33 -04:00
spoopyghosty0 2306b2e788 Route all emulated scene APIs and fake space consumers through the shim 2026-10-19 00:17:33 -04:00
spoopyghosty0 65b39a51ca Persist random UUID application anchors across sessions 2026-10-19 00:17:32 -04:00
spoopyghosty0 8473fe429e Add META discovery and persistence entry points and events 2026-10-19 00:17:32 -04:00
spoopyghosty0 3586d520d3 Expose room polygons volume bounds and triangle meshes 2026-10-19 00:17:32 -04:00
spoopyghosty0 79f60057fa Load scanned STAGE room files and bounded mesh sidecars 2026-10-19 00:17:31 -04:00
spoopyghosty0 77b01cbadb MR: optional wall label in room.txt; in-headset scan prompts (render_prompts.py) 2026-10-19 00:17:31 -04:00
spoopyghosty0 b9cb73a6c7 MR: Phase 0 findings - Valve's RoomView 3D mesh is readable through IVRBlockQueue/IVRPaths 2026-10-19 00:17:31 -04:00
spoopyghosty0 11a677ed1c MR: room file contract (room.txt, room-mesh.bin, scan dumps) 2026-10-19 00:17:30 -04:00
spoopyghosty0 d3a0846775 Release prep: the #406, #254 and #387 fixes together on main
Adapter rebuilt from the combined source (fixed_resolution + swapchain_pool), libfp_xrruntime.so back in SHA256SUMS, ANALYSIS_VERSION 24 for static OpenXR loaders (23 is frame.main_process), agent v108 for the static-loader launcher line (above the MR branches' 104-107 so a Frame with one of those still updates), patch docs and translations template regenerated, Ruff line length.
2026-10-19 00:10:31 -04:00
Codex 25db57f468 Fix #387: pool GLES panel swapchains before the runtime count limit
Add opt-in swapchain_pool with private GLES images and up to two packed
runtime atlases. Start at eight live app runtime swapchains to reserve
headroom before the session cap. Preserve image ownership, GL state,
source fences, crops and the existing cylinder/equirect layer ordering.

Enable for 4XVR, add count-limit triage without unknown suggestions for
older apps, and document the pending headset menu check. Leave default
recipes and adapter revisions unchanged. Rebuild arm64/arm32 artifacts.

Validation: 132 targeted tests passed (2 skipped), including the native
runtime/GL harness; Ruff and generated docs pass. Full host suite:
2095 passed, 137 skipped, 12 failures and 16 setup errors, all denied
socket creation in this sandbox. No headset validation or push.
2026-10-19 00:04:16 -04:00
Codex bd2a742356 Route embedded Khronos OpenXR loaders through FrameBridge
Detect and redirect static loader manifests and Android brokers, ship a runtime negotiation shim and the inner loader chain, and maintain per-game manifests on install and launch. Add the PrimedGun recipe, triage and host coverage.
2026-10-19 00:03:49 -04:00
spoopyghosty0 d0a2d89c92 Keep opt-in OpenXR layer recommendations at full swapchain resolution (#406)
Route OVRPlugin recommendations through the existing extension shim, retain native loader hooks, and log projection rect changes once per second per eye. Enable the fix for Batman, document verification, and rebuild both adapter architectures.
2026-10-19 00:03:20 -04:00
193 changed files with 19339 additions and 416 deletions

No files matched your search

+19 -2
View File
@@ -59,7 +59,21 @@ Detailed device findings per game/issue: docs/DEVICE_NOTES.md (read before debug
agent ends at EOF); sources and costs in docs/FRAME_RUNTIME.md "Monitoring sources"; game card (fps from
FrameBridge pacing; End game = Steam's Exit game, then cmd_stop), tiles with `C.Sparkline` (Flet canvas, no charts
extension), details, a pooled process table (Game / Steam & SteamVR / All; right-click: end / force kill / end
game; MON_CRITICAL needs force, MON_NEVER is refused)); settings; welcome; activity panel). Files, Screenshots and the Library share right-click menus
game; MON_CRITICAL needs force, MON_NEVER is refused)); room (`views/room.py`, persistent, "experimental" pill:
scan with progress + Cancel = agent v109 `room_capture_stop`, play area, delete; cache `ui/room_state.py` shared
with the Frame page's short card, mesh cached in `<data>/rooms-cache/<frame>/`; in-window 3D `ui/room3d.py`
RoomCanvas over `mr/view3d.py` (pure-Python projection, mesh reduced to ≤4k/≤1.5k edges at rest/moving, ≤18 fps
worker; each redraw sends NEW shape controls: changing a Points list made Flet diff it item by item, 0.3-1 s);
"Open full 3D view" = `mr/room_viewer.py` token server on 127.0.0.1 + hand-written WebGL `mr/viewer_page.py`,
stopped on disconnect/close; Test Quest MR tip = `mr.TESTER_*`; editing cards `views/room_edit.py` (agent v109):
Clean-up = fp_roomscan's refinement settings, `install/rooms.REFINE_DEFAULTS` = the owner's defaults + slider
metadata (agent `ROOM_REFINE_DEFAULTS` must match, a test checks), saved non-defaults in the Frame's
`rooms/refine.json`, used by every build; Apply = job → agent `room_refine` (`--refine-only` on a copy of the room
folder from `room-raw.txt`/`room-mesh-raw.bin`, scan.json → status `report`/`labels`/`suggestions`); Labels =
the label rules' decisions + Add as door / Add as closet; Furniture & screens = box editor (click a box in the
canvas: `view3d.pick`) + doors/windows on a wall (`mr.room.plane_on_wall`), Save = `rooms.put_room` → `room_put
edited` (keeps raw files, `edits.txt` survives a new clean-up; "Back to the scan's boxes" = keep_edits false); no
eraser in the GUI (`rooms.refine(erase=)` only)); settings; welcome; activity panel). Files, Screenshots and the Library share right-click menus
(one `ft.ContextMenu` per view, filled on right-click; on one of several selected items they act on the whole
selection, `C.menu_targets`) and click-and-drag multi-select (`C.DragSelect`: pan start/end on the area + item
hover events, which Flutter also sends with the button held; Flet can't report item positions, so no rubber band).
@@ -455,7 +469,9 @@ Repo is on an NTFS drive (`core.fileMode=false`); line endings are LF (`.gitattr
- No XR_FB_display_refresh_rate (`xrGetDisplayRefreshRateFB` -7), yet OVRPort's dispatcher offers it and answered 0 Hz
→ FrameBridge always answers with the measured rate (1/predictedDisplayPeriod, else 72). PreyVR (#306, idTech 4:
`USERCMD_MSEC = 1000 / GetRefresh()`) had 0 ms tics: async thread spinning, `com_frameTime` stuck at 0, menu black.
- Guardian STAGE bounds report 1×1 m (scene emulation uses ≥1.5 m).
- Guardian STAGE bounds report 1×1 m (scene emulation uses ≥1.5 m). OVRPlugin treats that rect as a configured
boundary but can't outline it ("ovrpBoundary_Outer is not supported"): Demeter's guardian check indexed an empty
array → with `scene_emul` the app's STAGE `xrGetReferenceSpaceBoundsRect` returns XR_SPACE_BOUNDS_UNAVAILABLE.
- GL goes through **Zink** (Mesa GL on Vulkan). Mesa GLSL is strict: `#pragma` before `#extension` fails, implicit
int/float conversions fail (enable `GL_EXT_shader_implicit_conversions`), num_views=2 shaders on single-view FBOs
give GL_INVALID_OPERATION. Some GLES games hit `zink: DEVICE LOST` (Unity MSAA RTT; Sniper Elite VR even without).
@@ -590,6 +606,7 @@ Toy Master doesn't request a model after that (it uses its own), so the glb load
**Vader Immortal (GitHub #49):** stuck on its loading card and leaked GPU mappings (solved below: `gl_error_clear`); diagnostics FrameBridge `layer_debug`, `strip_color_bias`, `frame_balance`, `snapshot=N` (`snapshot_gl.c`: saves the submitted left-eye image as `files/fb_snap_*.ppm`). → docs/DEVICE_NOTES.md#vader-immortal-headless-investigation
**SteamVR overlay apps:** host OpenVR overlays composite over Lepton games; detection `analysis/vroverlay.py`, patch `pcvr.vr_overlay`, Linux fields `vr_overlay`/`vr_overlay_autostart`, agent `register_vr_overlay`/`unregister_vr_overlay`/`launch_vr_overlay`; SteamVR on the Frame reads only `binary_path_linux_arm`; FramePort's own autostart (`ensure_overlay_service`, `frameport-vr-overlays.service`). See also docs/FRAME_RUNTIME.md "Overlay apps". → docs/DEVICE_NOTES.md#steamvr-overlay-apps
**Vader's leak solved / AC Nexus flips:** stale GL errors broke SteamVR's GL interop → adapter `gl_error_clear` (GLES, catalog for Vader ep1-3); read `xrclient_<pkg>.txt` in Steam's logs for runtime errors; the runtime caps swapchains per session (~15) → flip emulation in place; adapter `flip_dump`. → docs/DEVICE_NOTES.md#vaders-leak-solved--ac-nexus-flips
**Mixed reality: scanned rooms (branch mr-scene):** agent `room_capture` records Valve's RoomView 3D depth mesh -> `fp_roomscan` -> `room.txt` + `room-mesh.bin` (docs/mr/ROOM_FORMAT.md; refined by default: the owner's cleanup passes + label rules in `native/roomscan/refine.c`/`rules.c`, unrefined output kept as `room-raw.txt` + `room-mesh-raw.bin`, head path/settings/report/labels/door suggestions in `scan.json`, `--refine-only <dir>` re-runs it, `--refine 0` = old output; docs/mr/scan.md "Refinement") in `~/.local/share/frameport/rooms/<universe>/`, copied next to settings.conf (`_room_sync`); FrameBridge (`scene_room.c/scene_meta.c/scene_anchors.c`, all through xrshim) serves walls, volumes, planes, the room mesh, META discovery/persistence, saved anchors and `XR_META_boundary_visibility`; `scene_emul` + `frame.meta_permissions` for every USE_SCENE/USE_ANCHOR_API game (migrations `scene_emul_all`, `scene_permissions`); per-game `scene_hide_bounds` (Demeter). → docs/DEVICE_NOTES.md#mixed-reality-scanned-rooms
**Unreal's self-restart:** UE 4.21+ restarts itself after the first shader precompile and Lepton ends the container → launch.sh RESTART_BLOCK starts Lepton once more (`upgrade_launchers`; `FRAMEPORT_NO_RESTART=1` off). → docs/DEVICE_NOTES.md#unreals-self-restart
**xrWaitFrame runs ahead:** the runtime sometimes returns a display time <1 refresh after the last submit and rejects the second submission (`VRCompositorError_AlreadySubmitted`) → the VrApi bridge re-waits (`pacing 2s:` counters). → docs/DEVICE_NOTES.md#xrwaitframe-runs-ahead-journey-of-the-gods
**Lepton's logcat mirror dies** (EOF, read failure, or silent) → launch.sh `_logcat_keeper` reads `podman exec <container> logcat` itself into launch.log (log `<base>/logcat-keeper.log`). → docs/DEVICE_NOTES.md#leptons-logcat-mirror-dies
+5
View File
@@ -36,6 +36,9 @@ Frame, patches each game so it runs there, uploads it and adds it to your Steam
in FramePort.
- **Your Frame from your PC:** Type on Frame (your keyboard on the Frame), Files, Screenshots, Live view (what the
headset shows, in your browser) and Monitor (frame rate, temperatures, battery, processes).
- **Your room for mixed reality (experimental):** scan the room in the headset, see it in 3D, clean up the scan and
fix furniture, screen and door labels, so mixed-reality games get your room. Try it with
[Test Quest MR](https://github.com/spoopyghosty0/TestQuestMR).
- **Updates itself**, and reports problems without personal data.
Details for each: [Install and first steps](docs/INSTALL.md).
@@ -44,6 +47,8 @@ Details for each: [Install and first steps](docs/INSTALL.md).
![Screenshots](docs/images/screenshots.png)
![Room (experimental): the scanned room in 3D, clean-up and labels](docs/images/room.png)
## Quick start
1. [Download](https://github.com/spoopyghosty0/frameport/releases/latest) the build for Windows, macOS (Apple
+1409 -3
View File
File diff suppressed because it is too large. Load diff
+37 -4
View File
@@ -9,24 +9,57 @@ d6337034fb0bdcca7ef9b7063d85b7586b8508b63b435959e8fffaba0691091c ./arm64-v8a/li
40defdaddcda53bd2649eb48076fae1622bfc2bfc88e9e03c94fc8b45af2a4f2 ./arm64-v8a/libfp_ovrtrace.so
0167c2fde5dfde62897f9d85afc4fc8e97784e988b5b8df6cea8c407240b3424 ./arm64-v8a/libfp_tkeys.so
e32f9b3222bfe2e58c193db809ac2cf54086df36fd2ec6651aa2e58308f6b39d ./arm64-v8a/libfp_vk.so
3c548b567c6984c91bf3f4a2b0bae1336f569f3e9fa9c3204872fd9876f88284 ./arm64-v8a/libfp_xrruntime.so
383054f8b3b41dde76d062c71856cd3163655e50009c08454bc1793f393ead65 ./arm64-v8a/libfpg.so
4a13a33e112c9e4e3c5ff4337ed6e988a9e5b3602f48d654df148ef5fafbf426 ./arm64-v8a/libfpglmv.so
36400eae4ba3807bcfcd54463daa217347032b07218792faa448d8cbbd086a0e ./arm64-v8a/libframe_xrshim.so
19ba5ed3c4b780f5ae46251f6958f552acfc5f13c8bb39f995579593fa8990c6 ./arm64-v8a/libglshim.so
f0894d9cc08aed4c0f6468134e76909d583a208c3f5e528bc3ee795f51b19fcb ./arm64-v8a/libopenxr_loader_generic.so
7c33d7fb9fd176ca222efb2d87cf525e2f0604457c16896027a5881594495f76 ./arm64-v8a/libopenxr_loader_generic.so
1feaeafad467c4cafdf2b018a4d84b0bee200c3f711697b4ce97e66a3ba256ca ./arm64-v8a/libovrplatformcompat.so
a4ed80ae45ca90758239f152ea5a2e8c127d491e1e697ea5680db6fd7f72db07 ./arm64-v8a/libvrapi.so
a4ed80ae45ca90758239f152ea5a2e8c127d491e1e697ea5680db6fd7f72db07 ./arm64-v8a/libvrapi_foveation.so
4e03a8389981ed5a27b4c159cc83168c4f93d0d75479a445b02e5669aa26a71b ./armeabi-v7a/libopenxr_loader_generic.so
19ae0a45e80d8031300eea0d54810c14e4711fa8cd383c29ca65d85a74f577c4 ./armeabi-v7a/libopenxr_loader_generic.so
1871eae093432d277da4bc751bf5f3269dfcf11f9168260b0c3caadb0dedb19d ./dex/oculusos-stubs.dex
b634ab5640e258563c536e658cad87080553df6f34f62269a21d554844e58bfe ./hevc/COPYING.FFmpeg
141de01f0d40b97d4f9a720db5ab8ea442f6aa35db5272048899292884994a1c ./hevc/libstagefrighthw.so
4516971bbb2b635e19b14d2d37b9353962876f7de02043e7457327052c1fcbc0 ./hevc/manifest.json
82b11650e8eaab014aaab7e67ec781499df670c2519484f4ec9f2316db17c03c ./hevc/libstagefrighthw.so
96c7c947963dce72c3c2633c005f2801de54d16896d14d7afd91304ed1399d22 ./hevc/manifest.json
46c3dcad2c43cc30bea3b3680b362ed84a99c15d60714bbbbb422c80ecf90405 ./hevc/media_codecs_frameport.xml
529ea35d5d82474afe9e09ad1216d269317f56dd24a05fb100fced52b7b4dbca ./hevc/podman.py.txt
6cda3076a8c09b31bb1f90da614ea89e7ee194589626eeac613e522849604e47 ./licenses/LuaJIT.txt
1aa733117cf57ccff7cf23f0425dbfd73b0332a3ef1c905ceeca0ccd0449c4e0 ./linux-arm64/XR_APILAYER_FRAMEPORT_timefix.json
28c2430a02bbd8902c5bfb9562c6fd0e318654f9e05c095bfeb94b0450ab1b07 ./linux-arm64/libxr_frameport_timefix.so
2cc16b90abcd3d359f1658c581172bd650017bf2725e24268915f50904c000ec ./linux-arm64-bin/fp_roomscan
b10b3a5c10c3339fc63fd9f6cb4d01ea29be11a863d0ebad1389ea69fd471940 ./linux-arm64-bin/fp_venc
00c2dead0fd7497929833467f0e55699d57d26be1b85dce35402993852f7b111 ./roomscan/ceiling.png
c1962de31ed2810d31c760cf9ee63316ac2b8c9528ccbcd9299ebf5ba39b2995 ./roomscan/done.png
0dfaea4c68979b0a45a9fef06fdf1a92468ffc8f46f80a3a014fce1d805972d5 ./roomscan/failed.png
f2bae84f1d1512f87b25e688acdddb29b3f360f3f0d0cd4fdeeb741a79e7b19a ./roomscan/finish_00.png
9be7720c45f162e99813cb9a7a03e026fed53b41f06637b30ded309eeac91759 ./roomscan/finish_01.png
220d699f879b3a8669cfaf4dc714496e8c1db7e7878e7cd12b1a41aec9b62faa ./roomscan/finish_02.png
42871b440b77449d4fe41be1c809a930148a874caa8bc92b13b5d25bb6effa83 ./roomscan/finish_03.png
a470f62624c94ed6791cc7ee03066e1ffb0a676d520658a0a8b02a6af1605b3b ./roomscan/finish_04.png
195ff1c649fd9b42bfa6eea29c43a8b67116fd6ffe9b0fe53dcbdb1074bd34fa ./roomscan/finish_05.png
a45accab86b02d7b78327a17b84a6fea543ecc549282055dd3182c686fc2b1b8 ./roomscan/finish_06.png
b88d6c590ea839a9d11b3a4de8c3effb39752f7114318966fddae09995e0fd37 ./roomscan/finish_07.png
4a3615db26cf765b855a997be3e067aed58464894dab4a6700fdec5c2b854501 ./roomscan/finish_08.png
f532a83044e707c66dbc74787ce7b844890bc05829d3e9c8f7e33dae9db246a5 ./roomscan/finish_09.png
ffce61a9f13bbb250fad3582396a5bec82a1bad1d585b7ef9b8f65d343806b26 ./roomscan/finish_10.png
4a63772adacb31004749ab04b592076bd36cda61eef52f59037d5abdc9e527db ./roomscan/finish_11.png
f373e7021cfce0d1078227f473425a10035ee426102b78f13e0fea4833b736d5 ./roomscan/finish_12.png
1147391822be3f1aa5351374b8fd9ec85f8b5ce40e3cb9f3146d778e6fbc652a ./roomscan/floor.png
eb3c8b55a9dbafa030797ca9f06a22c3ee3bbef96cb7adea26d433ebad43d43f ./roomscan/look_00.png
6ba965ff6d22a469c558f2a68229bd2ff48fc28badb95889ca245310d1dd3e96 ./roomscan/look_01.png
0a11cec25ebc76fe11d505d1f2ec162c9b4a6754393065800e73b1f81e2d94c8 ./roomscan/look_02.png
fa5a4c94d20af56797b9fcc111148767847a6633ade9217cab373de232efad90 ./roomscan/look_03.png
e80fa493e380fba39084be2ee62859adcff4e39aa66566abee10a4d6339c52e9 ./roomscan/look_04.png
133617f0d8d1e69c37408aac80033f71856b592b62e3b780cc12b446bd974680 ./roomscan/look_05.png
d5dd296ff23641c6973c57ac139147e99c7b783316ab60537adb2155dd196880 ./roomscan/look_06.png
c08570aa78f966ae937b2e21d0bad887c5352fbca56aa579a2a7ef8541fb96ad ./roomscan/look_07.png
3767e7db5fb6babc872fa62c42604f749c65936717d332101745b201cd450cd5 ./roomscan/look_08.png
2924dc65e5c497545340209edc16060e12e1834d11af6871d1c1baa50b7b2562 ./roomscan/look_09.png
cd0150dfc10b2aea39403c74eff72fd05fb0111178ef39b4c98657a6d16d105e ./roomscan/look_10.png
011355e412f73ade3896e9037490e607c4365d0cf8ceb807d236bf68b414d3ed ./roomscan/look_11.png
1b8b47ffb47568d540921f065b5958eeb44a6489498d58e0f33f5d1f535154e2 ./roomscan/look_12.png
926c7bc6a103adee5b9bf938f5d4e39095ab77fd6270b5301be3b0428edd9ae6 ./roomscan/start.png
c2c70f4af1a5c3e6089dff1130161dd417f8044f99ec639eeecf1083930aab16 ./win-x64/fp_oculushmd.exe
60ebc5fea05b8945082e2ca8b9f2eb74616012a26eb50133af273dff0fcddc42 ./win-x64/fp_vrsettings.exe
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
+2 -2
View File
@@ -1,8 +1,8 @@
{
"revision": 8,
"revision": 10,
"runtime_sha256": "456e912c75cd389abcf6a63bc80e2a53bdc334371d00b200c93680388ae955e2",
"files": {
"libstagefrighthw.so": "141de01f0d40b97d4f9a720db5ab8ea442f6aa35db5272048899292884994a1c",
"libstagefrighthw.so": "82b11650e8eaab014aaab7e67ec781499df670c2519484f4ec9f2316db17c03c",
"podman.py": "529ea35d5d82474afe9e09ad1216d269317f56dd24a05fb100fced52b7b4dbca",
"media_codecs_frameport.xml": "46c3dcad2c43cc30bea3b3680b362ed84a99c15d60714bbbbb422c80ecf90405",
"COPYING.FFmpeg": "b634ab5640e258563c536e658cad87080553df6f34f62269a21d554844e58bfe"
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

@@ -8,19 +8,21 @@ notes: 'Native OpenXR GLES video player. 360 theatres/videos (equirect2 layers,
details: 'Settings: equirect_emul (GLES worker draws 360 layers into a projection layer), stable_local, focus_hold,
scale 0.75; frame.swapchain_limit for its 7680x3840 theatre swapchains. Its hardware decoding mode (Java MediaCodec,
Vr4pMediaPlayer; its own FFmpeg is the software path) gets the Frame''s video hardware with frame.hw_video_decode
(added 2026-10-09, not yet checked in the headset).'
(added 2026-10-09, not yet checked in the headset). swapchain_pool shares extra GLES panel images to avoid the
runtime swapchain count cap (#387); host-tested only, menu interaction needs a headset check.'
tested_version: 2.0.22
engine: Other
xr: OpenXR
frame:
- frame.hw_video_decode
adapter:
swapchain_pool: 1
equirect_emul: 1
stable_local: 1
focus_hold: 1
scale: 0.75
verified:
date: '2026-10-01'
updated: '2026-10-10'
updated: '2026-10-18'
min_app: 0.12.1
source_hint: 4XVR Video Player
@@ -0,0 +1,16 @@
package: com.And7Limited.Hitstream
title: Hitstream
status: works
tested_version: 2.06bQuest
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
adapter:
controller_models: 1
haptic_fix: 1
notes: Fitness rhythm game; uses controller models and haptics fix.
verified:
date: '2026-10-12'
known_good_sha256: ecc9e2eee79073ae97ad5d3f70c21e8f049f899f21b51dfd541430bb7dd04cac
source_hint: Hitstream
+2 -1
View File
@@ -12,6 +12,7 @@ overport_extra:
- patch_force_passthrough
adapter:
scene_emul: 1
scene_hide_bounds: 1
device:
- device.text_input_window
verified:
@@ -22,4 +23,4 @@ verified:
frame:
- frame.meta_permissions
- frame.unity_text_input
updated: '2026-10-15'
updated: '2026-10-17'
+3 -2
View File
@@ -16,6 +16,7 @@ frame:
- frame.hw_video_decode
- frame.unity_text_input
adapter:
fixed_resolution: 1
sync_guard: 1
surface_native: 1
fade_quad: 1
@@ -26,6 +27,6 @@ verified:
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: 796d1cab08f614d72c50d9702974c719c0908f5de91183cfe02d1fd5eb09551a
updated: '2026-10-15'
min_app: 1.1.0
updated: '2026-10-18'
min_app: 1.2.2
source_hint: Batman- Arkham Shadow
+4 -2
View File
@@ -5,8 +5,10 @@ source_hint: NOPE CHALLENGE
engine: Unreal
xr: OpenXR
status: works
notes: Meta XR Audio telemetry patch + nodebug. MR "Reality Breach" mode unavailable.
details: The MR 'Reality Breach' mode needs passthrough; standard VR mode used.
notes: "Meta XR Audio telemetry patch + nodebug. MR mode: needs a scanned room."
details: The MR 'Reality Breach' mode needs passthrough and a room; standard VR mode was verified.
adapter:
scene_emul: 1
alt_overport:
- patch_remove_unreal_force_quit
frame:
@@ -0,0 +1,19 @@
package: com.mrf.tvb.laser_dance
title: Laser Dance
engine: Unity
xr: OpenXR
status: works
notes: Mixed reality laser obstacle course; requires spatial data and room layout emulation (scene_emul=1). frame.meta_depth_occlusion keeps close lasers visible (its asset fixes only for the build they were made for); tested version not recorded (PR #246).
details: Mixed reality game that projects lasers across your room; needs Meta scene emulation and spatial data permission.
adapter:
controller_fix: 1
haptic_fix: 1
scene_emul: 1
passthrough_emul: 1
frame:
- frame.meta_permissions
- frame.meta_depth_occlusion
- frame.unity_scene_active_rooms
- frame.unity_text_input
min_app: 1.2.2
updated: '2026-10-18'
+2 -1
View File
@@ -1,13 +1,14 @@
package: com.zenstudios.PFXVRQuest
title: Pinball FX VR
status: issues
notes: Plays; mixed reality mode not working yet. Space warp off (fewer stutters).
notes: "Plays; space warp off (fewer stutters). MR mode: needs a scanned room."
tested_version: '1.9'
engine: Unreal
xr: OpenXR
alt_overport:
- patch_remove_unreal_force_quit
adapter:
scene_emul: 1
equirect_emul: 1
hide_space_warp: 1
verified:
+21
View File
@@ -0,0 +1,21 @@
package: jp.strippers.punchfit
title: Punch Fit
status: works
tested_version: 1.7.2
engine: Unity
xr: OpenXR
frame:
- frame.meta_permissions
- frame.unity_text_input
adapter:
haptic_fix: 1
pose_time_fix: 1
scene_emul: 1
passthrough_emul: 1
notes: Mixed reality workout arena; composition layer trimming prevents 21+ layer overflow.
verified:
date: '2026-10-12'
known_good_sha256: 1a0050272526360542d62e7e6d1fc2daecb765c34378a6fe4ad51568ec550c0d
source_hint: Punch Fit
min_app: 1.2.2
updated: '2026-10-18'
+5 -3
View File
@@ -1,14 +1,16 @@
package: org.primedgun.v2
title: PrimedGun v2
status: unknown
notes: 'Without the Vulkan library rename: the game builds the library name from two pieces, so the renamed name didn''t
exist ("libfp_vkn.so"), its renderer fell back to OpenGL and VR couldn''t start (GitHub #254). Not yet confirmed in
the headset.'
notes: 'Vulkan via a statically linked Khronos OpenXR loader bypassed FrameBridge (GitHub #254).
The embedded-loader patch routes it through FrameBridge; device validation is still pending.
Keep the Vulkan library rename disabled: this game assembles its library name from two pieces,
so renaming it prevented Vulkan from loading and VR could not start. Not yet confirmed in the headset.'
tested_version: 2.0.0-alpha.4
engine: Other
xr: OpenXR
frame:
- frame.sdl_clipboard
- frame.static_openxr_loader
frame_remove:
- frame.vk_sanitize
updated: '2026-10-18'
+32
View File
@@ -6,6 +6,36 @@
# with diagnostics. Launch tests and real play sessions (agent session_log, `frameport session`) are both matched.
# Sources: every failure we hit porting 34 games to the Steam Frame (see docs/PLAYBOOK.md).
signatures:
- id: mr-no-room
pattern: 'FrameBridge[^\r\n]*scene: (?:capture requested -> (?:guardian box|using guardian room)|room from guardian|room [^\r\n]*(?:from guardian|configured/default size|box))'
severity: info
diagnosis: 'The game is using a simple fallback room. Scan your room for mixed reality, or use Frame > Room > Use my play area for walls without furniture. A recorded scan can be imported until headset capture is available.'
suggest: []
- id: mr-discovery-unsupported
pattern: '(?:Unknown proc addr:\s*xrDiscoverSpacesMETA|(?:MRUK[^\r\n]*)?NoRoomsFound)'
# discovery reached FrameBridge and found the room (OVRPort's "Unknown proc addr" comes before the shim takes over)
unless: 'FrameBridge[^\r\n]*scene: discovery request=\d+ results=[1-9]'
severity: warning
diagnosis: 'The game cannot discover a Meta room. Enable room layout (adapter.scene_emul), rebuild and reinstall for the updated extension shim, then save a room under Frame > Room.'
suggest: [adapter.scene_emul]
- id: dynamic-resolution-foveation
pattern: 'projection view [01](?: rect change)?:[^\r\n]*rect=(?:[1-9]\d*,\d+|0,[1-9]\d*) '
unless: 'fixed_resolution: recommended'
severity: info
diagnosis: The game submits an offset projection viewport. With Valve foveation, dynamic resolution can move the sharp centre or expose a white edge. Keeping full resolution may help; it increases GPU load and needs the updated adapter and OVRPlugin shim in a rebuilt APK.
question: With foveated rendering enabled, was the sharp centre misplaced or was there a white line at the picture edge?
suggest: [adapter.fixed_resolution]
- id: static-openxr-loader-bypass
pattern: 'RuntimeManifestFile::FindManifestFiles - using global runtime file /[^\r\n]*etc/openxr/1/active_runtime(?:\.[A-Za-z0-9_-]+)?\.json'
unless: 'FrameBridge[: ]|fp_xrruntime[: ]'
severity: warning
diagnosis: A Khronos loader loaded the system runtime without FrameBridge. A statically linked loader may bypass the bundled loader chain; analyze again, enable the embedded OpenXR loader patch when detected, then rebuild and reinstall to write its per-game manifest.
suggest: [frame.static_openxr_loader]
- id: swapchain-limit-reached
pattern: 'FrameBridge:\s+xrCreateSwapchain [^\r\n]*\bresult=-10\b'
severity: error
diagnosis: The Frame runtime reached its swapchain count limit (about 15 live swapchains per session), not necessarily a memory limit. Extra menus or video panels may close or stay gray. Updated FramePort builds provide Share panel images (swapchain limit) for GLES apps; Vulkan apps are unaffected.
suggest: []
- id: quest2-headset-identity
pattern: 'venderId[^\r\n]*systemName Oculus Quest2'
severity: info
@@ -387,6 +417,8 @@ signatures:
# "Name=XR_FB_scene SpecVersion=1", OVRPort's "overportOXR: - XR_FB_scene 1", Unity's 'Meta XR Feature: ...
# Extensions="..."' (GitHub #404); XR_FB_scene_capture alone doesn't count either
pattern: '(?m)^(?!.*(?:Unavailable OpenXR extension: |Name=|overportOXR: - |Extensions=")).*XR_FB_scene\b|XR_FB_spatial_entity.*(not supported|UNSUPPORTED)|xrQuerySpacesFB'
# FrameBridge serves the room (OVRPort still lists XR_FB_scene and the shim names xrQuerySpacesFB): no finding
unless: 'FrameBridge[^\r\n]*scene: emulation active'
severity: warning
diagnosis: The game wants Meta's room data (scene API), which the Frame lacks. The adapter can emulate a room from the guardian bounds.
suggest: [adapter.scene_emul]
+176 -1
View File
@@ -146,11 +146,23 @@ pose (images drawn only after head turns, or ahead from xrWaitFrame and sometime
lists its own `/sdcard/4XPlayer`, not Movies: agent v25 `link_media` hard-links sent files into an app's own top-level
folder (`app_media_dirs`), `frameport frame send --app <pkg>`, GUI game page "Add videos" (players) / "Add videos &
files…". A 4XVR webm stereo swapchain was refused with XR_ERROR_LIMIT_REACHED (-10) → right eye grey; our projection
swapchain was halved (1536/eye) in case memory is the limit (unverified). Frame data snapshot
swapchain was originally halved (1536/eye) while investigating memory. Issue #387 below confirms a swapchain
count cap; reducing image sizes does not avoid it. Frame data snapshot
(SteamVR runtimes, Lepton scripts, logs; never commit Valve binaries): `~/frameport-research/frame-data-2026-09-30/`.
Not yet verified in the headset.
**Issue #387 (2026-10-18, host-tested only):** 4XVR's extra sub-panels hit the session cap at roughly 15–16 live
swapchains: the same 256×128 create succeeds then fails in the same millisecond, so shrinking images is not a fix.
Default-off GLES `swapchain_pool` (catalog enabled for 4XVR) starts at eight live app runtime swapchains, reserving runtime
headroom for up to two 4096² SRGB atlases. Single-face/layer/sample/mip colour images use private app-context textures;
submitted panels are blitted into packed atlas rectangles after cylinder strips and rect_clamp. Destroy frees rectangles.
Vulkan is untouched. No adapter revision bump: this opt-in fix should not mark every installed game for update.
The new catalog key already makes older apps reject the recipe via `unusable_reason`; triage suggests no unknown setting.
Headset check: repeatedly open/close all 4XVR submenus while playing stereo and 360 video, verify both eyes, panel crops,
curved strips, alpha and frame pacing; confirm no new -10 errors or menu closures and reuse after closing panels.
## Companion apps / intents (Stremio + 4XVR)
**Companion apps / intents (2026-10-03, tested with Stremio + 4XVR):** a second APK can be installed into a running
@@ -741,6 +753,63 @@ can be enabled and persist after restart. Check actual runtime refresh/pacing se
identity alone does not implement refresh-rate support. Rebuild with Quest 2 for the control: original `Oculus Quest2`,
original Quest 2 UUID, type 9 if logged, and previous option visibility. Also smoke-test a Unity OVRPlugin game
with the default Quest 3 choice for regressions. No device access was used during implementation.
## Mixed reality: scanned rooms
Live floor coverage (agent v104) is enabled by default for manual and game-requested scans; the capture
payload accepts `preview: false`, and `frameport frame room scan --no-preview` disables it. A world-locked,
upward-facing 6 m square at floor height follows the current raw-to-STAGE calibration and stays centred on the
scan's first recorded camera position. Every 16th depth vertex contributes to raw-space 10 cm voxels; at most once
per second, newly occupied cells redraw a 256×256 RGBA map in one colour (FramePort's secondary blue), each cell a
filled 10 cm square, only the opacity telling floor (<0.3 m, faint), raised surfaces (0.3–1.5 m) and tall surfaces
(>1.5 m, strongest, the room's outline) apart; unseen transparent. Overlays have no depth (they draw over hands and
passthrough), so the map stays faint; three colours in 3x3 blotches clashed in the owner's first run. It shows
coverage, not a validated room boundary. IVROverlay_028 slots 1/3/20/22/33/43/62 (OpenVR 2.15.6 header)
create/destroy/sort order (1, below the scan prompts' 10)/set width/set absolute STAGE transform/show/upload raw
pixels. Each upload is 256 KiB (at most 256 KiB/s); timing for the first
20 uploads and every tenth thereafter, plus failures, goes to `rooms/scans/scan-*.fprs.log` and the built room’s `scan.log`. Overlay errors or an
upload taking over 50 ms disable the preview while capture continues; headset appearance and upload cost still
need device validation.
When an MR game asks for a room scan, FramePort scans in the headset and FrameBridge loads the new room before
answering the game. If no scanner answers within five seconds, the game gets the stored room. Set
`scene_scan_request=0` to always use the stored room. A scan that starts but fails keeps the old room and reports
a capture failure to the game.
**Mixed reality: scanned rooms (agent v92, 2026-10-16, branch mr-scene; docs/mr/):** Quest MR games read the room
through Meta's scene APIs; the Frame offers none, and Valve's chaperone is no room (2D outline raised to 2.43 m, often
the default 1 m circle -> the Android runtime's STAGE rect falls back to 1x1 m; XRService's plane detection is
horizontal-only, its persistent map sparse ORB, its XRIPC single-client; cameras are XRService's alone,
IVRTrackedCamera gives no frames). **The passthrough's RoomView 3D depth mesh is readable** (docs/mr/PHASE0.md:
IVRBlockQueue_005 `/xrservice/roomview3d/meshData|meshLayout`, metadata through the C++ `IVRPaths_002` on the block
handle, `meshIsUpToDate` on the queue handle; 132x128 fan, ~30/s; raw->standing moves during a session -> convert
with the final one), so FramePort scans rooms itself: agent `room_capture` (Background/Overlay OpenVR app, prompts =
artifacts/roomscan/*.png via IVROverlay_028 slots 1/3/20/22/35/43/44/63, `scripts/mr/render_prompts.py`) -> `.fprs`
dump -> `fp_roomscan` (native/roomscan, freestanding, offline) -> `room.txt` + `room-mesh.bin`
(docs/mr/ROOM_FORMAT.md; outline = positive shoelace in x-z, wall inward normal (-dz, dx)) in
`~/.local/share/frameport/rooms/<universe>/`; launch.sh copies it next to settings.conf (`_room_sync`, games with
`scene_emul`); FrameBridge (`scene_room.c/scene_meta.c/scene_anchors.c`) serves N walls, volumes (BOUNDED_3D),
planes, GLOBAL_MESH (XR_META_spatial_entity_mesh), XR_META discovery/persistence (newer Meta SDKs and MRUK), anchors
saved in `files/framebridge-anchors.txt`; all through xrshim (OVRPort's dispatcher drops unknown META calls).
`scene_emul` is now suggested for every USE_SCENE/USE_ANCHOR_API game (migration `scene_emul_all`; NOPE and Pinball
FX had none). PC: Frame page "Room" card, `frameport frame room scan|status|from-boundary|import|export|build|show`.
Real data (owner's room, 2026-10-16): floor at 0, table surfaces, orthogonal walls found; the ceiling needs "look up".
Scan on request (agent v103): scene-emulation launchers run `_scan_watch` alongside the game. FrameBridge's
`framebridge-scan-request` in the game's external files directory gets an ack, then the existing prompted headset
capture/build flow saves and copies the new room before an `ok` result; errors, cancellation and timeout return
`failed`. Captures share a global lock, stale protocol files are cleared on launch, and the watcher stops with the
launcher. Logs: `<base>/scan-watch.log`. Device validation is still required.
PR #246 (@alicecatpaw, merged into mr-scene 2026-10-18): FrameBridge trims frames to maxLayerCount (Punch Fit sent
21+ layers; projection layers and FrameBridge's fade quad always stay), drops quads with zero size or an empty image
rect, and logs up to 10 quads with pose and swapchain. Two Unity patches, both only from a catalog recipe or the
user's choice: `frame.meta_depth_occlusion` (Depth API occlusion off; Laser Dance's lasers vanished up close) and
`frame.unity_scene_active_rooms` (OVRSceneManager's player-in-room check). The PR's own META discovery and boundary
visibility code was replaced by this branch's (`scene_meta.c`). The PR's recipes were checked on the author's Frame
only; open question: does OVRPort's dispatcher advertise XR_META_environment_depth (then a FrameBridge setting that
hides it would fix depth occlusion for every engine).
## Vulkan pipelines refused: Fossilize and Turnip (Transformers, #349)
Transformers Beyond Reality: Redux (Unreal, Vulkan) ended its first frames with Unreal's "Shader compilation failures
@@ -785,3 +854,109 @@ SteamVR's own Touch remap (`/opt/steamvr/drivers/frame_controller/resources/inpu
layout `oculus_touch`) is the diagram's "SteamVR default": X -> d-pad down, Y -> d-pad left/right/up, A -> A,
B -> B/X/Y, Menu -> View, plus trigger click -> bumper. Games with their own Valve binding file in that folder
(Beat Saber, Batman, Accounting+, In Death, ...) use that file instead.
## Fixed layer resolution / Batman (#406, #394)
Batman: Arkham Shadow (`com.camouflaj.manta`, Unity + OVRPlugin, OpenXR/Vulkan) submits
`2768x2768`, then corner-anchored `rect=0,32 2736x2736`, `rect=0,64 2704x2704` inside the
same swapchain. On SteamOS 0.5.6 Valve's FDM injection covers the full swapchain: shrinking
viewports can move the sharp centre towards the top right and expose a coarse white right edge.
The runtime recommendation driving OVRPlugin's dynamic resolution remains unverified.
`adapter.fixed_resolution=1` (default off, Batman recipe on) overrides
`xrGetRecommendedLayerResolutionMETA`: tracked full swapchain dimensions, `isValid=XR_TRUE`.
The existing rect_clamp size table already records successful xrCreateSwapchain calls and
retires entries at destruction. Projection views must have matching sizes; quads are supported;
unknown swapchains, mixed sizes and other layer types retain the runtime behavior. Submitted
rectangles are never enlarged after rendering. Full resolution may increase GPU load.
The cached OVRPort 3.4.3-aa54c3f dispatcher contains both the extension and function name
(and a runtime-missing diagnostic). Its fixed routing table is not a guarantee that every
version forwards the call unchanged. Enable the existing arm64 xrshim for this setting:
OVRPlugin's loader lookup reaches `framebridge_extension_proc` directly, while native OpenXR
loaders use the adapter's xrGetInstanceProcAddr hook. No extension hiding: keeping the
recommendation API available avoids relying on an unverified OVRPlugin fallback. The adapter
revision stays unchanged because the new setting is opt-in; the catalog requires 1.2.2 so older
remote clients keep their bundled recipe (`recommend.catalog.unusable_reason`). Rebuild and
reinstall Batman to receive the binary and shim routing; this is not a settings-only fix for
an old APK. Device/headset verification is pending.
Headless check: expect `extension shim: xrGetRecommendedLayerResolutionMETA -> FrameBridge`
for OVRPlugin, then up to five `fixed_resolution: recommended WxH -> 2768x2768 (runtime=...)`
lines. Compare the size to the actual eye swapchain creation log; scaling can change it.
`projection view 0 rect change: ... rect=0,0 2768x2768` (and view 1) should remain full size
through gameplay/loading transitions. Rect changes continue to log at most once per second
per eye, comparing against the last logged state so a throttled final change is still shown.
No recommendation logs means this API path was not exercised: do not claim the fix worked.
Headset check: sharp region centred in both eyes, no white right edge, correct menus/loading
fades, and acceptable pacing with full-resolution GPU load.
This is separate from #303's `foveation_storage_fix` (clearing SUBSAMPLED allocation flags;
see PLAYBOOK and FRAME_RUNTIME), which fixes black/coarse storage corruption. It also leaves
#394's `FOVE_LEVEL=lo/med/hi` presets intact (commits 01ad5e6b, d7f0d514): those select the
full-swapchain density pattern, not the game's dynamic viewport.
## Embedded Khronos OpenXR loaders (GitHub #254)
PrimedGun v2 (`org.primedgun.v2`, 2.0.0-alpha.4) embeds the Khronos loader in
`libmetroid_prime_port.so` (no loader DT_NEEDED). Its diagnostics show both Android
broker queries failing, then `/vendor/etc/openxr/1/active_runtime.json`, runtime
`SteamVR/OpenXR`, and no FrameBridge messages. The supplied `openxr_android.cpp`
initializes the loader with the VM and SDL activity. This is Vulkan OpenXR; retain
the catalog's removal of `frame.vk_sanitize` (the game assembles the Vulkan soname).
`frame.static_openxr_loader` is general: analysis v23 detects arm64 game libraries
with Khronos manifest strings and negotiation/manifest markers. Complete manifest
paths (including ABI variants) are replaced in `.rodata`, NUL padded, with
`/sdcard/fp_openxr_runtime.json`. Khronos also builds paths from fragments: the
verified `platform_utils.hpp` layout uses five partition prefixes, `/etc/openxr/`,
the major API version and `/active_runtime[.aarch64].json`. For this complete table,
prefixes become `/sd` and the subdir becomes `card/fp/`; all candidates then resolve
to `/sdcard/fp/1/active_runtime[.aarch64].json`. The undecorated manifest is written;
a missing ABI-specific file falls back to it. No offsets or sizes change. **Pooled
partition strings can also be used by game code**; this cannot be ruled out from
synthetic tests. Only the complete known fragment table is accepted. Other layouts
need analysis/rewrite support before they can be patched.
Both broker authorities become `org.fp_null.openxr.*`, causing provider lookup to
fail and the loader to fall back to files even if a broker is later installed.
Only the game's copy is changed. Agent v108 writes both manifest spellings into
`<base>/lepton-data/external` (the game's private `/sdcard`), records
`static_openxr_loader` in deployment.json, and repairs them before every launch.
Launcher upgrades preserve that repair. A configuration-only finalize also writes
them. The payload is `{"file_format_version":"1.0.0","runtime":{"library_path":"libfp_xrruntime.so","name":"FramePort"}}`.
The shim exports negotiation and initialization, loads FrameBridge by soname,
forwards Android loader initialization through its GIPA, and returns that GIPA to
the outer loader. FrameBridge then loads `libopenxr_loader_original.so`, an
unmodified Khronos loader reused from OVRPort's downloaded generic runtime.
This works for recipes with `overport: False`: the patch supplies the original
loader before `frame.adapter` installs FrameBridge, its settings payload and
foveation support. Settings still use `settings.conf` through the normal launcher.
No Meta dispatcher is needed. The loader's unmodified system paths keep it from
recursing through the game manifest. Android linker namespaces, symbol binding
between the two loaders, runtime handles across the chain, and FrameBridge's
behavior without the OVRPort dispatcher still require a device check.
Source verification: Khronos OpenXR-SDK-Source
[`release-1.0.34/src/common/platform_utils.hpp`](https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/release-1.0.34/src/common/platform_utils.hpp)
and the same fragment layout in release-1.1.53;
[`manifest_file.cpp`](https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/release-1.0.34/src/loader/manifest_file.cpp)
leaves a bare library filename unchanged for dlopen and tries brokers before files;
[`runtime_interface.cpp`](https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/release-1.0.34/src/loader/runtime_interface.cpp)
forwards loader initialization to the runtime export before negotiation (or through
GIPA afterwards). The reporter's diagnostic ELF summary contains no embedded
loader version or raw string table; the exact game's loader revision and binary
rewrite remain unverified. Host tests cover complete and assembled synthetic ELF
layouts, idempotence, unchanged other bytes, APK apply/validate with no OVRPort,
negotiation rejection, Android init forwarding and missing-chain failure. No Frame
connection or device test was performed.
Headless acceptance after rebuilding and installing with this development build:
`uv run frameport test org.primedgun.v2`. Expect `fp_xrruntime: chain resolved:
FrameBridge`, `fp_xrruntime: xrInitializeLoaderKHR forwarded to FrameBridge: 0`,
`fp_xrruntime: negotiated runtime interface 1: FrameBridge`, then FrameBridge's
`scale=... loader=...libopenxr_loader_original.so`, `xrCreateInstance` and pacing /
`new layer:` lines. The outer loader should log the private sdcard manifest, the
inner loader the real vendor manifest. Require RUNNING and submitted frames;
headset rendering and controls need separate confirmation.
+47
View File
@@ -66,6 +66,53 @@
`launch.sh` at the new folder and only then deletes the old copy; on the same filesystem it renames instead.
Moving a game whose Lepton data holds absolute paths, or a PC VR prefix, is not yet verified on the device.
## Room data
Live floor coverage (agent v104) is enabled by default for manual and game-requested scans; the capture
payload accepts `preview: false`, and `frameport frame room scan --no-preview` disables it. A world-locked,
upward-facing 6 m square at floor height follows the current raw-to-STAGE calibration and stays centred on the
scan's first recorded camera position. Every 16th depth vertex contributes to raw-space 10 cm voxels; at most once
per second, newly occupied cells redraw a 256×256 RGBA map in one colour (FramePort's secondary blue), each cell a
filled 10 cm square, only the opacity telling floor (<0.3 m, faint), raised surfaces (0.3–1.5 m) and tall surfaces
(>1.5 m, strongest, the room's outline) apart; unseen transparent. Overlays have no depth (they draw over hands and
passthrough), so the map stays faint; three colours in 3x3 blotches clashed in the owner's first run. It shows
coverage, not a validated room boundary. IVROverlay_028 slots 1/3/20/22/33/43/62 (OpenVR 2.15.6 header)
create/destroy/sort order (1, below the scan prompts' 10)/set width/set absolute STAGE transform/show/upload raw
pixels. Each upload is 256 KiB (at most 256 KiB/s); timing for the first
20 uploads and every tenth thereafter, plus failures, goes to `rooms/scans/scan-*.fprs.log` and the built room’s `scan.log`. Overlay errors or an
upload taking over 50 ms disable the preview while capture continues; headset appearance and upload cost still
need device validation.
- SteamVR on the Frame has no Meta scene/anchor extensions. Its chaperone supplies only a 2D outline raised to
2.43 m, without furniture, ceiling or a room mesh. Valve's default boundary is a 20-point circle of roughly 1 m
radius; FramePort refuses to treat it as a room.
- FrameBridge's `scene_emul=1` serves saved room data, with a guardian box when no room is available. OVRPort's
loader only dispatches functions in its own table: newer Meta entry points go through FramePort's extension shim.
MRUK uses discovery (`xrDiscoverSpacesMETA`, older versions `xrQuerySpacesFB`) and requests Space Setup when
discovery returns no rooms. Volumes hang below their top-face anchor in Meta's coordinate convention.
- Room files use STAGE coordinates (metres, +Y up, floor y=0, -Z forward). The format contract is
[mr/ROOM_FORMAT.md](mr/ROOM_FORMAT.md). On the Frame: `~/.local/share/frameport/rooms/<universe>/room.txt`,
optional `room-mesh.bin` and `scan.log`; `rooms/current` points to the most recently saved room.
- A room belongs to its chaperone universe. The agent reads the last universe switch from `vrserver.txt`, falling
back to the newest universe in `~/.config/openvr/config/chaperone_info.vrchap`; dummy 424242 means none.
A matching room is selected first, else `current`; the app reports `aligned: false` after a universe change.
- Quest launchers copy that room beside `settings.conf` before Lepton starts; FrameBridge reads
`dirname($FRAMEBRIDGE_CONFIG)/room.txt`. Stale copies are removed if no room exists or emulation is disabled.
Each game's persistent anchors stay in `/sdcard/Android/data/<pkg>/files/framebridge-anchors.txt`.
- With `scene_emul`, launch.sh starts `_scan_watch <pkg>` after Lepton, closing fd 9; its exit trap stops the
watcher, which also exits if the launcher disappears. It polls once a second and logs to `<base>/scan-watch.log`.
In `<base>/lepton-data/external/Android/data/<pkg>/files/`, FrameBridge writes `framebridge-scan-request`
(`request <id> <unix_seconds>`); the watcher clears stale request/ack/result files at startup, writes
`framebridge-scan-ack` (`ack <id>`), reuses the headset capture/build worker, stores the current room and copies
it beside settings.conf before atomically writing `framebridge-scan-result` (`ok <id>` or `failed <id> <reason>`).
A global `rooms/.scan.lock` excludes concurrent captures, including manual scans; `rooms/.lock` still protects
room publication. Launcher exit cancels the scan; a 350-second deadline leaves time for the six-minute protocol timeout.
- Frame → Room and `frameport frame room` manage rooms. `from-boundary` creates walls without furniture;
`import` / `export` transfer room files; `show` writes an SVG preview; `build <dump.fprs>` syncs the bundled
`fp_roomscan` by SHA-256 and runs it on the Frame with `--offline`. `scan` = agent `room_capture`: records the
RoomView meshes (`/xrservice/roomview3d/meshData` + `/meshLayout`, `IVRBlockQueue_005`) while in-headset prompts
guide the player, then builds the room from that recording.
## Save backups (Quest games, agent v101, discussion #352)
- A Lepton game's saves are two folders of `<base>/lepton-data`: `internal/<pkg>` (= `/data/data/<pkg>`:
`shared_prefs/`, `files/`, `databases/`; Unreal's `files/UE4Game|UnrealGame/.../Saved` often lives here) and
+5 -1
View File
@@ -67,6 +67,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Gravity Lab | Quest | ✅ Works | |
| Gunjack | Quest | ✅ Works | |
| H.U.N.T | Quest | ✅ Works | |
| Hitstream | Quest | ✅ Works | |
| I Am Cat | Quest | ✅ Works | |
| I Am Monkey | Quest | ✅ Works | |
| I Expect You To Die 2 | Quest | ✅ Works | |
@@ -81,6 +82,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| KillerFrequency | Quest | ✅ Works | |
| Ladder | Quest | ✅ Works | |
| Lambda1VR | Quest | ✅ Works | |
| Laser Dance | Quest | ✅ Works | |
| LEGO® Bricktales | Quest | ✅ Works | |
| Lost Recipes | Quest | ✅ Works | |
| Lucky's Tale | Quest | ✅ Works | |
@@ -109,6 +111,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Pistol Whip | PC VR | ✅ Works | |
| Please Don't Touch Anything | Quest | ✅ Works | |
| PowerWash Simulator VR | Quest | ✅ Works | |
| Punch Fit | Quest | ✅ Works | |
| QuestCraft | Quest | ✅ Works | |
| Racket: Nx | Quest | ✅ Works | |
| RacketPinball | Quest | ✅ Works | |
@@ -146,6 +149,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Taskmaster | Quest | ✅ Works | |
| Tea For God | Quest | ✅ Works | |
| Test Quest Controllers | Quest | ✅ Works | |
| Test Quest MR | Quest | ✅ Works | |
| TetrisEffect | Quest | ✅ Works | |
| The Boys VR | Quest | ✅ Works | |
| The Climb | Quest | ✅ Works | |
@@ -194,7 +198,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Majora's Mask VR | Quest | ⚠️ Works with issues | Audio on the file select screen sounds crunchy. |
| Myst | Quest | ⚠️ Works with issues | Minor graphical glitches on some objects. |
| Phantom: Covert Ops | Quest | ⚠️ Works with issues | DLC/store button crashes (no Meta store). |
| Pinball FX VR | Quest | ⚠️ Works with issues | Plays; mixed reality mode not working yet. |
| Pinball FX VR | Quest | ⚠️ Works with issues | Plays; space warp off (fewer stutters). |
| Plezy | Quest | ⚠️ Works with issues | Some videos play with strong artifacts and wrong colours; the cause is unknown, possibly hardware video decoding. |
| Puzzling Places | Quest | ⚠️ Works with issues | Works fine, but pointers dont show well on main menu, easy to work around. |
| Quantum Void | Quest | ⚠️ Works with issues | no hands |
+23
View File
@@ -183,6 +183,29 @@ Website owners: see [Install button](INSTALL_BUTTON.md).
so stop it when you're done.
- **Screenshots** shows the screenshots you took in the headset, sorted by game and day, to view or download.
## Your room (mixed reality, experimental)
Mixed-reality games ask for your room: walls, furniture and its surfaces. **Room** (in the sidebar) keeps it:
- **Scan room in the headset**, then put the headset on with passthrough and look around for about a minute; the tab
shows how far the scan is. **Cancel scan** stops it and keeps your saved room. **Use my play area** makes walls from
the SteamVR play area instead (no furniture).
- The room shows in 3D: drag to turn it, scroll to zoom, right-drag to move it, double-click to see all of it.
**Open full 3D view** shows every detail of the scan in your browser (a page on this PC only).
- **Clean-up** tidies every scan (flat walls, no scan behind walls or where you walked, no loose bits). The
recommended settings suit most rooms; change them (**More settings** has all of them) and press **Apply** to clean
up your saved scan again. Your settings are also used for your next scans.
- **Labels** shows what the scanner's boxes became and why. A tall, thin box by a wall can be **Add as door** or
**Add as closet**.
- **Furniture & screens**: pick a box in the list or click it in the 3D view to change its type, place or size, add
or delete boxes, and add doors and windows on a wall. **Save** sends them to the Frame; they stay when you clean up
the scan again (**Back to the scan's boxes** under Labels drops them).
- The free **Test Quest MR** app (Get Test Quest MR on the tab) shows what a game receives from your scan.
![Room](images/room.png)
![Room: a box picked to edit, and the labels of the clean-up](images/room-editing.png)
## Updating
FramePort checks for a new version at start and every 6 hours. When there is one, the Library shows **Update now**:
+133 -24
View File
@@ -2,7 +2,7 @@
# Patches
Every patch and game setting FramePort knows: 187 entries in 5 groups. A game's recipe is its chosen patches and settings; tested games get theirs from the catalog ([GAMES.md](GAMES.md)), and other games get the patches FramePort suggests from its analysis. You change them on the game page under Customize.
Every patch and game setting FramePort knows: 194 entries in 5 groups. A game's recipe is its chosen patches and settings; tested games get theirs from the catalog ([GAMES.md](GAMES.md)), and other games get the patches FramePort suggests from its analysis. You change them on the game page under Customize.
Each entry says what the patch does, whether it changes the APK or only the install, when FramePort suggests it, its options, the catalog games whose recipe names it and the launch-test signatures that suggest it. Patches that are on by default are on for every game, so the catalog rarely names them.
@@ -11,8 +11,8 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
## Groups
- [Patches from OVRPort](#group-overport) (25)
- [Steam Frame patches](#group-frame) (62)
- [Game settings](#group-adapter) (77)
- [Steam Frame patches](#group-frame) (65)
- [Game settings](#group-adapter) (81)
- [Files and settings on the Frame](#group-device) (10)
- [PC VR (Proton and Revive)](#group-pcvr) (13)
@@ -64,7 +64,8 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [GL wrapper: OpenGL ES depth formats (Minecraft)](#frame-ltw_depth) | `frame.ltw_depth` | Steam Frame patches | default |
| [Fix Lua startup crashes on Frame](#frame-luajit_pointers) | `frame.luajit_pointers` | Steam Frame patches | 1 |
| [Start the game in the app's main process](#frame-main_process) | `frame.main_process` | Steam Frame patches | default |
| [Declare Meta-only permissions](#frame-meta_permissions) | `frame.meta_permissions` | Steam Frame patches | 5 |
| [Unity: disable Meta depth occlusion](#frame-meta_depth_occlusion) | `frame.meta_depth_occlusion` | Steam Frame patches | 1 |
| [Declare Meta-only permissions](#frame-meta_permissions) | `frame.meta_permissions` | Steam Frame patches | 7 |
| [Meta XR Audio: skip Quest telemetry class](#frame-metaxr_telemetry) | `frame.metaxr_telemetry` | Steam Frame patches | default |
| [Clear android:debuggable](#frame-nodebug) | `frame.nodebug` | Steam Frame patches | 3 |
| [Oculus OS telemetry class stubs](#frame-oculusos) | `frame.oculusos` | Steam Frame patches | 17 |
@@ -78,6 +79,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [SDL apps: start without a clipboard service](#frame-sdl_clipboard) | `frame.sdl_clipboard` | Steam Frame patches | 6 |
| [Stress Level Zero: let Unity start Vulkan itself](#frame-slz_vulkan_hooks) | `frame.slz_vulkan_hooks` | Steam Frame patches | 1 |
| [Start straight in VR (skip the 2D launcher)](#frame-start_activity) | `frame.start_activity` | Steam Frame patches | |
| [Embedded OpenXR loader: use FrameBridge](#frame-static_openxr_loader) | `frame.static_openxr_loader` | Steam Frame patches | 1 |
| [Allow swapchains larger than 4096 px](#frame-swapchain_limit) | `frame.swapchain_limit` | Steam Frame patches | default |
| [Team Beef ports: treat the Frame as a Meta headset](#frame-tbxr_vendor) | `frame.tbxr_vendor` | Steam Frame patches | 1 |
| [Unity App UI: no Android 12 vibrator](#frame-unity_appui_vibrator) | `frame.unity_appui_vibrator` | Steam Frame patches | |
@@ -87,7 +89,8 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Unity: skip OVROverlay layers (fades, splash screens)](#frame-unity_no_overlay_copy) | `frame.unity_no_overlay_copy` | Steam Frame patches | 2 |
| [Unity: start VR without Meta's system apps](#frame-unity_oculus_check) | `frame.unity_oculus_check` | Steam Frame patches | 14 |
| [Unity: keep MSAA off at runtime (OVRManager)](#frame-unity_runtime_msaa_off) | `frame.unity_runtime_msaa_off` | Steam Frame patches | 33 |
| [Make Unity text fields work](#frame-unity_text_input) | `frame.unity_text_input` | Steam Frame patches | 119 |
| [Unity: skip Meta's player-in-room check](#frame-unity_scene_active_rooms) | `frame.unity_scene_active_rooms` | Steam Frame patches | 1 |
| [Make Unity text fields work](#frame-unity_text_input) | `frame.unity_text_input` | Steam Frame patches | 122 |
| [Unity: share thread keys (games that run out of them)](#frame-unity_tls_key_pool) | `frame.unity_tls_key_pool` | Steam Frame patches | 1 |
| [Unity: headset always counts as worn](#frame-unity_user_presence) | `frame.unity_user_presence` | Steam Frame patches | 1 |
| [GL shim for Unreal (no multisampled render-to-texture)](#frame-unreal_gl_shim) | `frame.unreal_gl_shim` | Steam Frame patches | 1 |
@@ -112,8 +115,8 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Pointer angle](#adapter-aim_pitch) | `adapter.aim_pitch` | Game settings | |
| [Pointer turn](#adapter-aim_yaw) | `adapter.aim_yaw` | Game settings | |
| [Fix inside-out mixed reality](#adapter-alpha_blend_emul) | `adapter.alpha_blend_emul` | Game settings | |
| [Use controllers](#adapter-controller_fix) | `adapter.controller_fix` | Game settings | 1 |
| [Show Steam Frame controllers](#adapter-controller_models) | `adapter.controller_models` | Game settings | 6 |
| [Use controllers](#adapter-controller_fix) | `adapter.controller_fix` | Game settings | 2 |
| [Show Steam Frame controllers](#adapter-controller_models) | `adapter.controller_models` | Game settings | 7 |
| [Show cube loading screens](#adapter-cube_quads) | `adapter.cube_quads` | Game settings | |
| [Stand in for cube layers](#adapter-cube_standin) | `adapter.cube_standin` | Game settings | |
| [Show curved menus and screens](#adapter-cylinder_strips) | `adapter.cylinder_strips` | Game settings | |
@@ -126,6 +129,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Per-eye diagnostics (log)](#adapter-eye_debug) | `adapter.eye_debug` | Game settings | |
| [Draw fades over the picture](#adapter-fade_quad) | `adapter.fade_quad` | Game settings | 2 |
| [Colour space support](#adapter-fb_color_space) | `adapter.fb_color_space` | Game settings | |
| [Keep full resolution (no dynamic resolution)](#adapter-fixed_resolution) | `adapter.fixed_resolution` | Game settings | 1 |
| [Upside-down panel images (diagnostics)](#adapter-flip_dump) | `adapter.flip_dump` | Game settings | |
| [Fix upside-down menus](#adapter-flip_emul) | `adapter.flip_emul` | Game settings | |
| [Rotate panels 180°](#adapter-flip_quads) | `adapter.flip_quads` | Game settings | |
@@ -138,7 +142,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [GL shim: high-precision vertex shaders](#adapter-gl_highp_markers) | `adapter.gl_highp_markers` | Game settings | 1 |
| [GL shim: integer vertex inputs](#adapter-gl_int_attribs) | `adapter.gl_int_attribs` | Game settings | 1 |
| [Flat panel diagnostics (log)](#adapter-gl_mv_debug) | `adapter.gl_mv_debug` | Game settings | |
| [Fix freezes when controllers vibrate](#adapter-haptic_fix) | `adapter.haptic_fix` | Game settings | 104 |
| [Fix freezes when controllers vibrate](#adapter-haptic_fix) | `adapter.haptic_fix` | Game settings | 107 |
| [Vibration strength](#adapter-haptic_scale) | `adapter.haptic_scale` | Game settings | |
| [Turn off Quest Vulkan foveation](#adapter-hide_fb_foveation) | `adapter.hide_fb_foveation` | Game settings | |
| [Turn off space warp](#adapter-hide_space_warp) | `adapter.hide_space_warp` | Game settings | 5 |
@@ -148,10 +152,10 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Vulkan format workaround](#adapter-mutable_fix) | `adapter.mutable_fix` | Game settings | |
| [Unity frame loop: wait for skipped frames](#adapter-ovrp_begin_gate) | `adapter.ovrp_begin_gate` | Game settings | 1 |
| [Steadier hands (older Unity)](#adapter-ovrp_hold_physics) | `adapter.ovrp_hold_physics` | Game settings | 1 |
| [See your room](#adapter-passthrough_emul) | `adapter.passthrough_emul` | Game settings | |
| [See your room](#adapter-passthrough_emul) | `adapter.passthrough_emul` | Game settings | 2 |
| [Steadier head tracking](#adapter-pose_consistency) | `adapter.pose_consistency` | Game settings | 2 |
| [Pose timing diagnostics (log)](#adapter-pose_debug) | `adapter.pose_debug` | Game settings | |
| [Fix hands lagging behind controllers](#adapter-pose_time_fix) | `adapter.pose_time_fix` | Game settings | 16 |
| [Fix hands lagging behind controllers](#adapter-pose_time_fix) | `adapter.pose_time_fix` | Game settings | 17 |
| [Treat newer Quest controllers as Touch](#adapter-profile_remap) | `adapter.profile_remap` | Game settings | |
| [Animate fingers from touch](#adapter-proximity_emul) | `adapter.proximity_emul` | Game settings | |
| [Controller buttons](#adapter-quest_buttons) | `adapter.quest_buttons` | Game settings | |
@@ -161,8 +165,10 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Re-create play space](#adapter-respace_kick) | `adapter.respace_kick` | Game settings | |
| [Render resolution (supersampling)](#adapter-scale) | `adapter.scale` | Game settings | 15 |
| [Room depth](#adapter-scene_depth) | `adapter.scene_depth` | Game settings | |
| [Pretend room layout](#adapter-scene_emul) | `adapter.scene_emul` | Game settings | 14 |
| [Use room layout](#adapter-scene_emul) | `adapter.scene_emul` | Game settings | 18 |
| [Room height](#adapter-scene_height) | `adapter.scene_height` | Game settings | |
| [Hide play-area size](#adapter-scene_hide_bounds) | `adapter.scene_hide_bounds` | Game settings | 1 |
| [Scan when requested](#adapter-scene_scan_request) | `adapter.scene_scan_request` | Game settings | |
| [Room width](#adapter-scene_width) | `adapter.scene_width` | Game settings | |
| [Eye-image snapshots (diagnostics)](#adapter-snapshot) | `adapter.snapshot` | Game settings | |
| [Keep menus in place](#adapter-stable_local) | `adapter.stable_local` | Game settings | 1 |
@@ -172,6 +178,7 @@ Link to an entry with its id, dots replaced by dashes: `PATCHES.md#frame-unity_t
| [Native video surfaces (Batman renderer)](#adapter-surface_native) | `adapter.surface_native` | Game settings | 1 |
| [Swap eyes](#adapter-swap_eyes) | `adapter.swap_eyes` | Game settings | |
| [Retry refused picture formats](#adapter-swapchain_fix) | `adapter.swapchain_fix` | Game settings | |
| [Share panel images (swapchain limit)](#adapter-swapchain_pool) | `adapter.swapchain_pool` | Game settings | 1 |
| [Steady controller input after focus](#adapter-sync_guard) | `adapter.sync_guard` | Game settings | 3 |
| [Hide Vulkan foveation](#adapter-vk_hide_fdm) | `adapter.vk_hide_fdm` | Game settings | 1 |
| [Without Steam's shader cache layer](#adapter-vk_no_fossilize) | `adapter.vk_no_fossilize` | Game settings | 1 |
@@ -760,6 +767,18 @@ Keeps the game from being closed about a minute after it starts (it runs in a se
> Some apps run their game activity in a process of its own (android:process, e.g. Halo CE VR's ":halo_game"), started from a 2D launcher activity. Lepton starts the game activity directly and watches only the app's main process, which then never exists: about a minute in, Lepton decides the app "did not start" and closes it, at whatever point the game has reached. This runs those activities in the main process (only their android:process value changes).
### <a id="frame-meta_depth_occlusion"></a>Unity: disable Meta depth occlusion
Keeps objects visible up close in mixed-reality games that would hide them behind your room using Meta's depth sensor, which the Frame lacks (for example Laser Dance).
- `frame.meta_depth_occlusion`
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Offered (off):** Uses Meta's Depth API occlusion (HARD_OCCLUSION / SOFT_OCCLUSION): turn on if objects vanish when you come close to them.
- **Catalog games (1):** Laser Dance.
- **Source:** [patches/frame/meta_depth_occlusion.py](../src/frameport/patches/frame/meta_depth_occlusion.py)
> The Frame has no depth sensor (XR_META_environment_depth), so the environment depth Meta's Depth API reads is never valid. Shaders using it then hide virtual objects near the player behind a room that isn't there (for example Laser Dance's lasers vanish when approached). Rewrites EnvironmentDepthManager's support/availability checks and its occlusion-keyword setter in libil2cpp.so (found with Cpp2IL), and renames the HARD_OCCLUSION / SOFT_OCCLUSION shader keywords in the game's asset files that contain them, so occlusion is never turned on. For Laser Dance's known build it also keeps the laser beams' materials and close-range LODs visible. Only for games with this symptom.
### <a id="frame-meta_permissions"></a>Declare Meta-only permissions
Declares Meta's permissions some mixed-reality games ask for, so they don't stop.
@@ -767,7 +786,8 @@ Declares Meta's permissions some mixed-reality games ask for, so they don't stop
- `frame.meta_permissions`
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Suggested (on):** Mixed-reality game that needs the room model: its 'use spatial data' permission must be granted (for example Demeter).
- **Catalog games (5):** Arkio, BattleTalent, Demeter, Real VR Fishing, Test Quest MR.
- **Suggested (on):** Uses Meta's room or anchor APIs: their permissions must be granted, or the game never asks for the room (for example NOPE's MR mode).
- **Catalog games (7):** Arkio, BattleTalent, Demeter, Laser Dance, Punch Fit, Real VR Fishing, Test Quest MR.
- **Source:** [patches/frame/manifest_fixes.py](../src/frameport/patches/frame/manifest_fixes.py)
> Declares com.oculus.permission.* / horizonos.permission.* that the app uses, so Android grants them at install (for example the 'use spatial data' permission of mixed-reality games like Demeter).
@@ -947,6 +967,21 @@ Opens the app straight in VR, skipping its flat launcher screen. Turn off to use
> Some apps open a flat Android launcher that then starts a separate VR activity (for example WiiCompiled). Shown with Lepton's flat window, that window stays in view after the VR part starts. This makes the VR activity the one Lepton starts (its LAUNCHER category), so the app opens in VR with no flat window. Everything only the launcher does (choosing or importing content) is skipped: turn it off for a run when you need the launcher, and turn off "Show the app's Android window".
### <a id="frame-static_openxr_loader"></a>Embedded OpenXR loader: use FrameBridge
Makes an embedded OpenXR loader use FrameBridge and your game settings.
- `frame.static_openxr_loader`
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Suggested (on):** The embedded Khronos OpenXR loader bypasses FrameBridge.
- **Requires:** [`frame.adapter`](#frame-adapter)
- **Catalog games (1):** PrimedGun v2.
- **Suggested by launch-test signatures:**
- `static-openxr-loader-bypass` (warning): A Khronos loader loaded the system runtime without FrameBridge.
- **Source:** [patches/frame/static_openxr_loader.py](../src/frameport/patches/frame/static_openxr_loader.py)
> Routes a game's statically linked Khronos OpenXR loader through FrameBridge by rewriting its runtime manifest paths in place into its own /sdcard (including assembled paths). Disables its Android runtime broker queries so they fall back to that file. Ships a runtime shim and an unmodified inner Khronos loader, which reads the system runtime manifest. Reinstall to write the per-game manifest.
### <a id="frame-swapchain_limit"></a>Allow swapchains larger than 4096 px
Allows very large pictures (8K video, theaters) instead of quitting.
@@ -1071,6 +1106,18 @@ Keeps a smoothing setting off that the game turns on itself and that can freeze
> Meta's OVRManager switches Unity to the headset's recommended MSAA level (4x) while the game runs, whatever its quality settings say (log: "Switching to the recommended level"). Multisampled render-to-texture on GLES can hang the Frame's GPU, up to a restart of the whole headset (for example Lucky's Tale in a menu) or a crash of its GL driver (for example The Room VR, whose own code sets 4x). Rewrites OVRDisplay.recommendedMSAALevel -&gt; 0 and makes QualitySettings.antiAliasing's setter do nothing in libil2cpp.so (found with Cpp2IL), so MSAA stays off.
### <a id="frame-unity_scene_active_rooms"></a>Unity: skip Meta's player-in-room check
Lets a mixed-reality game load the room even if it thinks you stand outside it (for example Laser Dance asking again and again to scan your room).
- `frame.unity_scene_active_rooms`
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Suggested:** not automatically; set on the game page (Customize), by a recipe or after a launch test.
- **Catalog games (1):** Laser Dance.
- **Source:** [patches/frame/unity_runtime.py](../src/frameport/patches/frame/unity_runtime.py)
> Meta's OVRSceneManager (ActiveRoomsOnly) loads only the room whose floor outline contains the player (IsUserInRoom). If the player is outside the emulated room's floor, loading stops with 'No scene model found' (for example Laser Dance with the play-area room). Rewrites OVRSceneManager.IsUserInRoom and PointInPolygon2D -&gt; true in libil2cpp.so (found with Cpp2IL), so the room loads wherever the player stands. Only for games with this symptom: a scanned room around the player usually passes the check by itself.
### <a id="frame-unity_text_input"></a>Make Unity text fields work
Lets you type into this app's text fields on the Frame (they'd close at once).
@@ -1078,7 +1125,7 @@ Lets you type into this app's text fields on the Frame (they'd close at once).
- `frame.unity_text_input` · also for 2D apps
- **Changes:** Changes the APK (a rebuild and reinstall apply it).
- **Suggested (on):** Unity app with text fields (…): they close at once on the Frame because it has no system keyboard.
- **Catalog games (119):** AgeOfJoy, AllInOneSports, Ancient_Dungeon, Arcsmith, Arkio, Assassin's Creed Nexus, Attack on Titan VR: Unbreakable, Audica and 111 more.
- **Catalog games (122):** AgeOfJoy, AllInOneSports, Ancient_Dungeon, Arcsmith, Arkio, Assassin's Creed Nexus, Attack on Titan VR: Unbreakable, Audica and 114 more.
- **Source:** [patches/frame/unity_text_input.py](../src/frameport/patches/frame/unity_text_input.py)
> Unity's TMP_InputField / InputField wait for Android's on-screen keyboard and close themselves a frame later when there is none (Lepton has no on-screen keyboard; Meta's system keyboard does this on a Quest), so a selected text field only flashes a caret. Rewrites TouchScreenKeyboardShouldBeUsed -&gt; false and isKeyboardUsingEvents (TMP) / InPlaceEditing (uGUI) -&gt; true in libil2cpp.so, found per game with Cpp2IL (downloaded on first use), so fields stay selected and take key presses: Steam's keyboard (with "Show the app window") or Type on Frame from the PC.
@@ -1423,7 +1470,7 @@ Turn off only for games you play with your hands instead of controllers.
- **Options:**
- Value: int, default `1` (always written).
- Game settings → Controllers → "Use controllers".
- **Catalog games (1):** Silhouette.
- **Catalog games (2):** Laser Dance, Silhouette.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Reports Frame controllers as Oculus Touch and hides synthetic hand tracking. Set 0 for games that require hand tracking (for example Silhouette).
@@ -1438,7 +1485,7 @@ For games that show the headset's own controllers. Takes effect after a reinstal
- **Options:**
- Value: int, default `0`.
- Game settings → Controllers → "Show Steam Frame controllers".
- **Catalog games (6):** Neko Atsume Purrfect, Ocean Rift, Real VR Fishing, SynthRiders, VR HOT Quest, Walkabout Mini Golf.
- **Catalog games (7):** Hitstream, Neko Atsume Purrfect, Ocean Rift, Real VR Fishing, SynthRiders, VR HOT Quest, Walkabout Mini Golf.
- **Suggested by launch-test signatures:**
- `render-model-requested` (info): The game shows Quest controllers (or none) because the Frame doesn't provide controller models.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
@@ -1619,6 +1666,23 @@ For games that stop at start because the headset reports no colour space (for ex
> Offers Meta's colour-space extension when the Frame's runtime lacks it: the game is told the headset uses Rec.709 and may choose any colour space (no conversion is done). For games that quit before their VR session with "Rec.709 color space unavailable" (for example OutRun Quest VR, GitHub #336). Only reaches games that use their own OpenXR loader; OVRPort's dispatcher can't forward the new functions.
### <a id="adapter-fixed_resolution"></a>Keep full resolution (no dynamic resolution)
Prevents dynamic resolution from moving Valve foveation off-centre or leaving a white edge line. May increase GPU load. Used at the next launch.
- `adapter.fixed_resolution`
- **Changes:** Game setting: stored in the build and on the Frame (settings.conf); can be changed without a rebuild.
- **Suggested:** not automatically; set on the game page (Customize), by a recipe or after a launch test.
- **Options:**
- Value: int, default `0`.
- Game settings → Picture → "Keep full resolution (no dynamic resolution)" (advanced).
- **Catalog games (1):** Batman: Arkham Shadow.
- **Suggested by launch-test signatures:**
- `dynamic-resolution-foveation` (info): The game submits an offset projection viewport.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Keeps OpenXR layer recommendations at the full swapchain size. Prevents dynamic resolution from moving Valve foveation off-centre or leaving a coarse white edge line. May increase GPU load.
### <a id="adapter-flip_dump"></a>Upside-down panel images (diagnostics)
Saves a few UI panels before and after FrameBridge turns them upright, for testing without the headset.
@@ -1806,7 +1870,7 @@ For games that freeze the Frame when a controller vibrates (for example Lucky's
- **Options:**
- Value: int, default `0`.
- Game settings → Controllers → "Fix freezes when controllers vibrate" (advanced).
- **Catalog games (104):** AgeOfJoy, Ancient_Dungeon, Attack on Titan VR: Unbreakable, Audica, AugmentedEmpire, Battlemarked, BattleSisters, BattleTalent and 96 more.
- **Catalog games (107):** AgeOfJoy, Ancient_Dungeon, Attack on Titan VR: Unbreakable, Audica, AugmentedEmpire, Battlemarked, BattleSisters, BattleTalent and 99 more.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Routes OVRPlugin through FramePort's extension shim, which turns Meta's amplitude-envelope vibrations into plain ones: OVRPort's loader reads their nanosecond duration as seconds and allocates gigabytes, freezing the Frame when the game vibrates a controller (for example Lucky's Tale's save slots, GitHub #9). Needs a rebuild.
@@ -1954,6 +2018,7 @@ Shows the Frame's cameras where the game expects passthrough.
- **Options:**
- Value: int, default `1`.
- Game settings → Mixed reality → "See your room".
- **Catalog games (2):** Laser Dance, Punch Fit.
- **Suggested by launch-test signatures:**
- `passthrough-missing` (error): The game needs Meta passthrough (the camera view), which the Frame lacks; the adapter emulates it.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
@@ -2000,7 +2065,7 @@ For games whose hands trail the controllers when you move them (for example Batt
- **Options:**
- Value: int, default `0`.
- Game settings → Controllers → "Fix hands lagging behind controllers" (advanced).
- **Catalog games (16):** Battlemarked, BattleSisters, Beat Saber, Dance Central, Dungeons of Eternity, Mini Motor Racing X, OrbusVR, Phantom: Covert Ops and 8 more.
- **Catalog games (17):** Battlemarked, BattleSisters, Beat Saber, Dance Central, Dungeons of Eternity, Mini Motor Racing X, OrbusVR, Phantom: Covert Ops and 9 more.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Poses a game asks for at a CLOCK_MONOTONIC time, or more than 0.5 s before the frame's display time, are located at the same moment in the runtime's time (or now). Meta's OVRPlugin passes the monotonic clock on as the OpenXR time, which on the Frame runs seconds behind the runtime's clock, so its "now" poses (Unity's physics step, controller poses) came from the past: hands trailing the controllers (for example BattleSisters, Sniper Elite VR).
@@ -2133,23 +2198,23 @@ Changes the game's render resolution. Higher is sharper but slower; 1.0× is the
> Override the guardian depth (0 = use guardian, min 1.5 m).
### <a id="adapter-scene_emul"></a>Pretend room layout
### <a id="adapter-scene_emul"></a>Use room layout
Mixed-reality games that build their level from your room get a simple room with walls.
Scene games get your saved room. Without one, they get a simple room from the play area.
- `adapter.scene_emul`
- **Changes:** Game setting: stored in the build and on the Frame (settings.conf); can be changed without a rebuild.
- **Suggested (on), value `1`:** Mixed-reality game that builds its level from the room model: emulate a guardian-sized room (for example Demeter).
- **Suggested (on), value `1`:** The game asks Meta's system for your room layout (scene), which the Frame doesn't have: emulate a room from the play area (for example VR HOT's room setup).
- **Suggested (on), value `1`:** The game uses Meta's scene or spatial anchor APIs: serve the saved room on the Frame, or a simple play-area room until one is scanned.
- **Options:**
- Value: int, default `0`.
- Game settings → Mixed reality → "Pretend room layout".
- **Catalog games (14):** Arkio, AugmentedEmpire, Battlemarked, BattleTalent, cubism, Demeo, Demeter, Ghostbusters: Rise of the Ghost Lord and 6 more.
- Game settings → Mixed reality → "Use room layout".
- **Catalog games (18):** Arkio, AugmentedEmpire, Battlemarked, BattleTalent, cubism, Demeo, Demeter, Ghostbusters: Rise of the Ghost Lord and 10 more.
- **Suggested by launch-test signatures:**
- `mr-discovery-unsupported` (warning): The game cannot discover a Meta room.
- `scene-missing` (warning): The game wants Meta's room data (scene API), which the Frame lacks.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Fake XR_FB_scene/spatial entities: a guardian-sized room with floor, ceiling and four walls, for mixed-reality games that build their level from the room (for example Demeter).
> Serve Meta scene and spatial entity APIs from the saved room (walls, furniture and mesh). Without a saved room, use a simple play-area box (for example Demeter).
### <a id="adapter-scene_height"></a>Room height
@@ -2165,6 +2230,35 @@ Ceiling height of the pretend room.
> Ceiling height for scene_emul.
### <a id="adapter-scene_hide_bounds"></a>Hide play-area size
For games that reject the room because of the play area.
- `adapter.scene_hide_bounds`
- **Changes:** Game setting: stored in the build and on the Frame (settings.conf); can be changed without a rebuild.
- **Suggested:** not automatically; set on the game page (Customize), by a recipe or after a launch test.
- **Options:**
- Value: int, default `0`.
- Game settings → Mixed reality → "Hide play-area size" (advanced); shown while `adapter.scene_emul` is on.
- **Catalog games (1):** Demeter.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> With scene_emul, the game's play-area (STAGE bounds) query answers "no play area". The Frame reports a 1x1 m area whose outline OVRPlugin can't give, and some Unity MR games then fail their guardian check (for example Demeter). Off by default: other games stalled without a play area.
### <a id="adapter-scene_scan_request"></a>Scan when requested
Scan your room in the headset when the game asks for a room scan.
- `adapter.scene_scan_request`
- **Changes:** Game setting: stored in the build and on the Frame (settings.conf); can be changed without a rebuild.
- **Suggested:** not automatically; set on the game page (Customize), by a recipe or after a launch test.
- **Options:**
- Value: int, default `1`.
- Game settings → Mixed reality → "Scan when requested" (advanced); shown while `adapter.scene_emul` is on.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> With scene_emul, let FramePort scan your room in the headset when the game requests a scan. If no scanner answers, use the saved room. Turn off to always use the saved room.
### <a id="adapter-scene_width"></a>Room width
0 = the size of your play area.
@@ -2295,6 +2389,21 @@ Lets OpenGL ES games start when the Frame refuses their picture format. Leave it
> Retry rejected GLES formats/MSAA with sRGB, samples=1.
### <a id="adapter-swapchain_pool"></a>Share panel images (swapchain limit)
Keeps extra menus and panels working when the Frame reaches its image count limit. GLES apps only; Vulkan apps are unaffected.
- `adapter.swapchain_pool`
- **Changes:** Game setting: stored in the build and on the Frame (settings.conf); can be changed without a rebuild.
- **Suggested:** not automatically; set on the game page (Customize), by a recipe or after a launch test.
- **Options:**
- Value: int, default `0`.
- Game settings → Troubleshooting → "Share panel images (swapchain limit)" (advanced).
- **Catalog games (1):** 4XVR Video Player.
- **Source:** [patches/settings.py](../src/frameport/patches/settings.py)
> Share GLES panel images in runtime atlases to avoid the Frame's swapchain count limit (about 15 per session). Useful when opening extra menus closes them or leaves panels gray. Vulkan apps are unaffected.
### <a id="adapter-sync_guard"></a>Steady controller input after focus
For games that crash right after you return to them (for example after the Steam menu).
+13 -2
View File
@@ -12,7 +12,9 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
## Startup failures (visible in launch.log)
| Symptom | Cause | Fix |
| `RuntimeManifestFile::FindManifestFiles - using global runtime file /vendor/etc/openxr/1/active_runtime.json`, no FrameBridge lines (#254) | Embedded Khronos loader bypasses the APK loader chain | `frame.static_openxr_loader`: redirect its manifest paths and broker authorities, bundle runtime shim + inner loader + FrameBridge; reinstall to write `/sdcard/fp_openxr_runtime.json`. Device check pending |
|---|---|---|
| Valve foveation sharp centre moves to the top right or a white line appears at the right edge as Batman shrinks its projection rect (#406 / #394) | Dynamic resolution draws a corner-anchored subrect inside the full swapchain used by Valve FDM | Enable `adapter.fixed_resolution` (Game settings → Picture), rebuild/reinstall for the new adapter and OVRPlugin shim; verify `fixed_resolution: recommended ... -> ...` and full-size `projection view ... rect change` logs. Headset verification pending; more GPU load |
| Quest 3-only options missing; OVRPlugin logs `venderId 10291, systemName Oculus Quest2` (#373, Asgard's Wrath 2) | OVRPort's dispatcher overwrites FrameBridge's name and supplies a Quest 2 `XR_META_headset_id`; OVRPlugin prioritizes that UUID | `device.headset_model`: Quest 3 by default, per-game Quest 3S / Quest Pro / Quest 2; rebuild the APK. Byte-checked dispatcher name + UUID edit; Quest 2 leaves the dispatcher unchanged. See DEVICE_NOTES 'Per-game headset identity' |
| Startup exits through `libxlua.so (lj_err_throw)`, then Unity aborts in mutex cleanup; captured Lua error is `bad light userdata pointer` (GitHub #257, BattleTalent 4.7) | This older LuaJIT encodes lightuserdata in 47 bits; Frame supplies valid wider native pointers. Controlled tests with the unchanged library pass a real low-address allocation and fail its own high-address static pointer | `frame.luajit_pointers` automatically selects a full-pointer Lua engine only for the verified library SHA-256, retaining xLua bindings and custom helpers. The scoped patch reaches OpenXR startup for 60 s headless; rendering and gameplay remain unverified. A `lj_err_throw` cleanup stack without the original error message is insufficient to infer this cause |
| A GL4ES Team Beef game draws stretched triangles or spikes across characters and the scene (GitHub #191, Lambda1VR) | The newer GL shim exports GLES draw functions that intercept GL4ES's desktop OpenGL calls, bypassing its client-array and immediate-mode translation | `frame.tbxr_vendor` keeps the shim's five draw symbols private when the APK bundles `libgl4es.so`. GL queries and single-sampled framebuffer fixes remain active; other games retain the draw hooks. A/B eye captures on SteamOS 0.4.5: old shim normal, current shim spikes, current shim with private draw symbols normal at ~72 fps. Worn-headset gameplay still needs confirmation |
@@ -117,7 +119,7 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| Vermillion: the hand and brush disappear over the canvas while painting, but still paint (GitHub #200) | the game shows a canvas being painted as its own compositor (quad) layer and turns the canvas's scene renderer off; the Frame draws quad layers above every projection layer whatever the order, so the canvas covers the hand (Vulkan game: no GL layer emulation) | `frame.vermillion_scene_canvas` (catalog): `PaintableMesh.HasCompositorLayerAvailable` / `ShouldUseCompositorOverlay` -> false in libil2cpp.so (Cpp2IL), so the game's own fallback draws the canvas in the scene (Klownicle's finding) |
| Unity game hangs right after its first frames, no `pacing:` lines; SIGQUIT dump shows UnityMain in a game library's `JNI_OnLoad` → `usleep` (for example SKYBOX: `libskybox.so`, `SignatureChecker`) | the game checks its APK signing certificate; every port is re-signed | can't be fixed without defeating anti-tamper (FramePort doesn't): mark "Can't run". Diagnose hangs: `podman exec <container> kill -3 <pid>` writes all thread stacks to launch.log |
| Video player doesn't list sent videos (for example 4XVR "Internal Storage" only shows some) | the player lists its own /sdcard folder (4XVR: /sdcard/4XPlayer), not /sdcard/Movies; Android's media index doesn't work in Lepton | game page "Add videos" / game menu "Add videos and files" (Videos + the app's own folder, hard links) or `frameport frame send <files> --app <pkg>` |
| One eye gray in a stereo video, log `xrCreateSwapchain … result=-10` | the runtime refused a swapchain (XR_ERROR_LIMIT_REACHED) | fewer/smaller swapchains: lower `equirect_res`; unresolved |
| One eye gray in a stereo video, log `xrCreateSwapchain … result=-10` | the runtime refused a swapchain (XR_ERROR_LIMIT_REACHED) | GLES: `adapter.swapchain_pool=1` shares extra panel images in atlases (#387; headset validation pending). This is a count cap, so lowering resolution does not fix it; Vulkan unaffected |
| VrApi game (32-bit bridge or `frame.vrapi_bridge`): holding the menu button to recenter turns the view ~90-120 degrees to the side | the bridge's app space is LOCAL + the head's yaw at the start; the Frame's recenter moves LOCAL to the head, so the old yaw was added again | fixed in the VrApi bridge (2026-10-13): on XR_TYPE_EVENT_DATA_REFERENCE_SPACE_CHANGE_PENDING (LOCAL) and on `vrapi_RecenterPose` it takes the origin from the head again (log `recentered: head yaw …`); rebuild the game |
| Menu/screen jumps to where you look (for example 4XVR) | the app re-creates LOCAL spaces (4XVR: every 2–4 s) and the Frame places them at the current head yaw, or the app recenters after the Frame's brief focus dips | turn on `adapter.layer_debug`, check `that LOCAL space sits at … deg` vs `focus_hold: hid a … ms focus dip`; patches: `adapter.stable_local` (new LOCAL spaces line up with the first) / `adapter.focus_hold` (on by default: hides dips up to `focus_hold_ms` once the game has been focused for a second; never at start-up: a global debounce broke AC Nexus) |
| Pointer misses menu items | the Frame's aim pose differs from Touch's | `adapter.layer_debug` logs `aim in grip … pitch/yaw`; correct with `aim_pitch`/`aim_yaw`/`aim_forward` (`frameport settings`, no rebuild) |
@@ -200,7 +202,16 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| Mixed reality inside out: the room shows on the game's objects, black where the room should be (Unreal, for example Star Wars: Beyond Victory, GitHub #329) | Unreal keeps 1 - coverage in the scene alpha and OVRPlugin chains XrCompositionLayerAlphaBlendFB (colour factors ONE_MINUS_SRC_ALPHA / SRC_ALPHA, `layer_debug` shows `n=1000041001(3,2/3,2)`) to the eye layer; the Frame's runtime ignores it | adapter `alpha_blend_emul=1` (default, Vulkan): inverts the alpha of the eye images in place while passthrough is emulated; logs `alpha_blend: inverting the alpha of swapchain=…` |
| Quest Touch controllers (or none) shown in a game that uses Meta's runtime controller models | XR_FB_render_model missing on the Frame | adapter `controller_models=1`: the agent converts the Frame's SteamVR controller models to glb at install, the adapter serves them. Games with their own controller meshes aren't affected (none of the 34 catalog games use runtime models) |
| `overportOXR: Unknown proc addr: xr…` for a function the adapter emulates | overport's dispatcher only forwards functions in its own table | route the plugin's lookups through `native/xrshim` (done for XR_FB_render_model when `controller_models=1`) |
| MR game stuck waiting for room data (Demeter) | no Meta scene API | adapter `scene_emul=1` (+ `frame.meta_permissions`) |
| MR game asks for a room scan | The game requested a new room through Meta scene capture | FramePort scans in the headset and loads the new room. `scene_scan_request=0` turns it off and uses the stored room. |
| MR game stuck waiting for room data (e.g. Demeter) | No Meta scene API | `adapter.scene_emul=1` (+ `frame.meta_permissions`); Frame → Room → Use my play area, or import a scanned room |
| Unity MR game rejects every room / its "use guardian" fallback does nothing; `ovrpBoundary_Outer is not supported`, `IndexOutOfRangeException` in a guardian check (e.g. Demeter's `RemoveObjectNotInGuardian`) | OVRPlugin calls the Frame's 1x1 m STAGE rect a configured boundary but can't give its outline | `adapter.scene_hide_bounds=1` (with `scene_emul`): FrameBridge then answers the app's STAGE `xrGetReferenceSpaceBoundsRect` with `XR_SPACE_BOUNDS_UNAVAILABLE` (log `scene: STAGE bounds hidden`). Per game only: NOPE stalled without a play area. Demeter also needs a floor of at least 2.5 x 2.5 m, as on a Quest |
| MR game with `scene_emul` never asks for the room (no `scene: query`/`discovery` lines), black screen or MR mode with no room (e.g. NOPE, Pinball FX); `m_XR_META_boundary_visibility extension is not supported` every frame | Meta's scene/anchor permissions undeclared (Lepton can't grant them); boundary suppression unsupported | `frame.meta_permissions` (now with every `scene_emul` recipe of a USE_SCENE/USE_ANCHOR_API game, migration `scene_permissions`); FrameBridge emulates `XR_META_boundary_visibility` (log `scene: boundary suppressed`). `passthrough:` lines show the app's passthrough calls |
| Build says `Unity text fields: not fixed: Cpp2IL couldn't start on this PC` (older versions: `Cpp2IL couldn't read this game's code: at Cpp2IL.Program.Main`) | .NET aborts without libicu (fresh WSL/Linux) | FramePort runs Cpp2IL with `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1`; such failures aren't cached, so the next build retries |
| MR uses an empty/simple room; FrameBridge `scene: capture requested -> guardian box` / `using guardian room`, `room from guardian`, or fallback box dimensions (`mr-no-room`) | No saved room, so FrameBridge uses its guardian box | Save a room in Frame → Room. A real SteamVR boundary supplies walls; Valve's default circle is refused. Import a scanner-built room for furniture/mesh; headset capture is not implemented yet |
| `Unknown proc addr: xrDiscoverSpacesMETA` or MRUK `NoRoomsFound` (`mr-discovery-unsupported`) | Scene discovery is unavailable or no room was discovered | Enable `adapter.scene_emul`, rebuild/reinstall to get the updated extension shim, then save a room. All games declaring USE_SCENE or USE_ANCHOR_API now get this suggestion |
| MR game: virtual objects (e.g. Laser Dance's lasers) vanish when you come close to them | Meta's Depth API occlusion: the Frame has no depth sensor (XR_META_environment_depth), so the environment depth the occlusion shaders read is never valid and near objects count as hidden behind the room | `frame.meta_depth_occlusion` (Unity IL2CPP; offered, not turned on, when the game has the HARD_OCCLUSION / SOFT_OCCLUSION keywords; analysis `meta_depth_occlusion`): EnvironmentDepthManager's checks -> false and its keyword setter does nothing (Cpp2IL), the keywords renamed in the asset files that contain them; Laser Dance's known build also gets its laser materials/LODs fixed. From PR #246; not checked in the owner's headset yet |
| Unity MR game keeps asking for a room scan / `No scene model found` although FrameBridge served a room (OVRSceneManager with ActiveRoomsOnly) | OVRSceneManager loads only the room whose floor outline contains the player (IsUserInRoom); with the play-area room the player can stand outside it (FrameBridge logs the head against the floor outline: `scene: head in STAGE ... OUTSIDE the floor outline`) | Scan the room first (Frame → Room). Still stuck: `frame.unity_scene_active_rooms` (catalog or by hand, Unity IL2CPP scene games): IsUserInRoom / PointInPolygon2D -> true (Cpp2IL). From PR #246 (Laser Dance) |
| MR game with many quads (21+): frames refused with `XR_ERROR_LAYER_LIMIT_EXCEEDED` (e.g. Punch Fit) | The runtime takes at most maxLayerCount (16) layers | FrameBridge keeps the projection layers and its own fade quad, then the app's other layers in submission order up to the limit (log `xrEndFrame: N layers exceed the runtime limit 16`). From PR #246 |
| Hand-tracking game janky (Silhouette) | Frame synthesizes hands from controllers | `controller_fix=0` passes hands through; not really fixable |
| Eye distortion while moving (Arcsmith, Time Stall) | unknown (not eye swap, tracking, Valve layers, depth or pacing) | unresolved |
| The Climb 666.0: hand outlines ripple in both eyes while menus and gameplay work (GitHub #238) | Unknown. Sessions paced around 72 fps, controller tracking reported valid, and SteamVR motion smoothing was already off; these checks do not establish the cause | Unresolved after wearer comparisons on 2026-10-12: disabling Valve's Vulkan layers, setting `r_AntialiasingMode = 1`, and setting `r_AntialiasingMode = 0` with `r_MotionBlur = 0` each gave the same or worse result. These were config comparisons; runtime cvar values were not verified. Restored the playable configuration (`r_vr_foveated = 0` with the default Vulkan layers), retaining the startup/rendering crash fixes above. Do not keep the unsuccessful experimental settings as a fix |
+4
View File
@@ -117,6 +117,10 @@ Hooks (`steps.HOOKS`) cover what a real Frame would do:
- `pairing_done`: the Frame ran the setup command, so FramePort connects.
- `live_stream`: a test picture through the real relay.
- `monitor_details`, `type_tab`, `select_files` and `settings_section`.
- `open_room` (the Room tab with the pretend Frame's room drawn) and `room_editing` (its editing cards: `cleanup`,
`more`, `box`, `doors`, `suggestion`). The room is a made-up bedroom built in code (`fakes.fake_room`, never a real
scan): a bed under a wall-mounted TV, a desk with two monitors under a window, a closet, a door, a lamp and a plant,
with a noisy scanned-surface mesh and a clean-up report and labels that match it.
In a video, clicking Install runs the scripted install, and the game is then installed on the pretend Frame.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 141 KiB

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 147 KiB

After

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 112 KiB

After

Width:  |  Height:  |  Size: 108 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 421 KiB

After

Width:  |  Height:  |  Size: 425 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 505 KiB

After

Width:  |  Height:  |  Size: 506 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 663 KiB

After

Width:  |  Height:  |  Size: 665 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 216 KiB

After

Width:  |  Height:  |  Size: 217 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 145 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 599 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 594 KiB

After

Width:  |  Height:  |  Size: 596 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

After

Width:  |  Height:  |  Size: 91 KiB

+49
View File
@@ -0,0 +1,49 @@
# Phase 0: reading Valve's RoomView 3D mesh (dev Frame, SteamOS 0.4.5 build 20261007, 2026-10-16)
Result: **a third-party OpenVR client can read the passthrough depth mesh, and it carries enough of the room for
a scan** (floor, table-height surfaces, orthogonal walls; the ceiling only weakly).
## Access
- Background OpenVR app (`VR_InitInternal2(..., VRApplication_Background)`, never starts SteamVR) through
`/opt/steamvr/bin/linuxarm64/libopenvr_api.so`.
- `FnTable:IVRBlockQueue_005`: `Connect("/xrservice/roomview3d/meshData")` and `.../meshLayout` work (created by
Valve's cv driver; no exclusivity seen next to the compositor's own reader). Read with
`WaitAndAcquireReadOnlyBlock(queue, &block, &buffer, EBlockQueueReadType_New, timeout_ms)` + `ReleaseReadOnlyBlock`.
- Metadata through the C++ interface `IVRPaths_002` (vtable slot 0 `ReadPathBatch(root, PathRead_t*, n)`, slot 2
`StringToHandle(&handle, path)`; `FnTable:IVRPaths_001` doesn't exist), root = the **block** handle, except
`/meshData/meshIsUpToDate` which is a property of the **queue** handle:
| path | tag | meaning |
|---|---|---|
| `/meshLayout/layoutId` | 3 (u64) | id of the index layout (static while the mesh resolution stays) |
| `/meshLayout/vertexCount` | 2 (i32) | 16896 = 132 x 128 |
| `/meshLayout/indexCount` | 2 | 99822 (u32 triangle indices into the grid, row width 132) |
| `/meshData/meshId` | 3 | frame counter (~30 per second while passthrough runs) |
| `/meshData/layoutId` | 3 | matches the layout block |
| `/meshData/elementCount` | 2 | floats in the data block (vertexCount x 3) |
| `/meshData/rawFromMeshPose` | 20 (HmdMatrix34) | mesh -> SteamVR raw tracking space |
| `/meshData/posePredictionBaseTime` | 7 (f64) | seconds (system uptime clock) |
| `/meshData/meshIsUpToDate` (queue) | 4 (bool) | false in standby / headset off |
- STAGE (= standing universe) = `IVRSystem_023` vtable slot 13 `GetRawZeroPoseToStandingAbsoluteTrackingPose()` x
rawFromMeshPose. Check: the mesh origin is 6.7 cm from `GetDeviceToAbsoluteTrackingPose(Standing)`'s head
position in every frame (camera vs head centre). **The raw->standing transform changes during a session** (its
height went 1.60 -> 1.07 m while recording: Valve's floor estimate) -> capture raw poses and convert all frames
with the transform at the end of the capture.
## The mesh
- A fixed fan of view directions, +Z forward, rows top->bottom (row 0 = up in the image, -Y), ~126 deg across the
corners (corner directions ~(+-0.67, +-0.66, 0.34)); one depth per grid point. Not head-locked in world terms:
its pose is the passthrough camera pose at capture.
- Range: reliable ~0.3-4 m; far values are inpainted (up to ~10 m) -> drop > 4 m. Strongly smoothed (soft edges,
small objects lost); locally planar: plane-fit residual of 12x12 patches median 5-17 mm.
- First frames after tracking starts can have NaN poses -> skip non-finite frames.
- Fused normals (60 s of looking around, ~3 frames/s used): floor clearly at y ~= 0, a table-height surface at
0.7 m, a lower one at 0.5 m; vertical surfaces in two orientation clusters ~85 deg apart (orthogonal walls), wall
offsets readable; ceiling: few down-facing points (people rarely look straight up; prompt for it).
- Data rate: 16896 x 12 B vertices + 99822 x 4 B indices per frame (the dump repeats the indices per frame).
## Consequences for the design
- Capture = stdlib Python on the Frame (ctypes), as above; the C scanner works offline on the dump.
- Scanner input quality: smooth, noisy at depth discontinuities; walls/floor via normals + RANSAC are robust;
furniture only as coarse boxes; ceiling needs a guided "look up" step, else the default height.
+113
View File
@@ -0,0 +1,113 @@
# FramePort room files (mixed reality)
Quest MR games read a room model (floor, ceiling, walls, furniture, a global mesh) through Meta's scene APIs. The
Steam Frame has no such data for apps (see docs/FRAME_RUNTIME.md "Room data"), so FramePort scans the room itself
(`fp_roomscan`) and FrameBridge (`native/adapter/scene_emu.c`) serves it to games. This file is the contract between
the scanner, the Frame agent, the PC app and FrameBridge.
## Coordinates
- Everything is in **STAGE space** = SteamVR's standing universe = OpenXR `XR_REFERENCE_SPACE_TYPE_STAGE`:
metres, right-handed, +Y up, floor at y = 0, -Z = the universe's forward.
- A room belongs to one chaperone universe (`universe` line). The Frame sometimes starts a new universe without
relocalizing; then the stored room may be misaligned (the agent reports `aligned: false`, nothing breaks).
## room.txt (UTF-8 text, one record per line, `#` starts a comment, fields separated by spaces)
```
frameport-room 1 # magic + format version (required, first line)
universe <decimal u64> # chaperone universe id the room was scanned in (0 = unknown)
source scan|boundary|box|manual # how it was made
time <unix seconds>
room <uuid> # the room anchor
floor <uuid> <y> # floor height (normally 0)
ceiling <uuid> <y> # ceiling height
outline <n> x0 z0 x1 z1 ... x(n-1) z(n-1)
# floor polygon, n >= 3, not closed, POSITIVE signed area:
# sum(x[i]*z[i+1] - x[i+1]*z[i]) > 0 (turning from +X towards +Z; seen from
# above with -Z forward that is clockwise). Readers reverse a negative outline.
wall <uuid> x0 z0 x1 z1 <y0> <y1> [LABEL]
# one wall face from (x0,z0) to (x1,z1), bottom y0, top y1.
# Its normal points INTO the room: walls follow the outline's order, and the
# inward normal is (-dz, dx)/|d| for d = (x1-x0, z1-z0).
# LABEL (optional, default WALL_FACE): INVISIBLE_WALL_FACE for an edge the
# scanner closed without seeing a wall (open side of a room, doorway gap).
volume <uuid> <LABEL> cx cz top_y yaw_deg w d h
# furniture box: top-face centre (cx, top_y, cz), yaw about +Y in degrees
# (0 = local X along world X), width w along local X, depth d along the
# horizontal local axis perpendicular to it, height h downwards from the top
plane <uuid> <LABEL> px py pz qx qy qz qw w h
# extra 2D plane (DOOR_FRAME, WINDOW_FRAME, WALL_ART, ...): pose of its centre,
# Meta convention (bounds in local XY, local +Z = the plane's normal)
mesh <uuid> <file name> <vertex count> <index count>
# global mesh sidecar in the same folder (GLOBAL_MESH entity)
```
- `<uuid>` = 32 lowercase hex digits (no dashes). New ones per scan; FrameBridge uses them as the XrUuidEXT bytes.
- Labels are Meta's semantic label strings: FLOOR, CEILING, WALL_FACE, INVISIBLE_WALL_FACE, TABLE, COUCH, BED,
STORAGE, SCREEN, LAMP, PLANT, DOOR_FRAME, WINDOW_FRAME, WALL_ART, OTHER, GLOBAL_MESH.
- Readers ignore unknown record types (forward compatibility) and reject a file whose first line isn't
`frameport-room <version>` with a version they know. Missing ceiling -> 2.5 m above the floor. Missing walls ->
walls from the outline edges. Missing outline -> the bounding rectangle of the walls.
### How FrameBridge maps records to Meta entities (scene_emu.c)
- ROOM: ROOM_LAYOUT (floor, ceiling, walls in file order) + SPACE_CONTAINER (every other entity) + LOCATABLE/STORABLE.
- floor/ceiling: pose at the outline centroid at height y; local +Z = +Y (floor) / -Y (ceiling);
BOUNDED_2D = the outline's bounding rect in local XY; BOUNDARY_2D = the outline polygon in local XY.
- wall: pose at the segment midpoint, height (y0+y1)/2; local +X along the wall (x0,z0)->(x1,z1), local +Y up,
local +Z = inward normal; BOUNDED_2D = length x (y1-y0) centred.
- volume: pose at the top-face centre, rotation = yaw about +Y then local +Z pointing up (+Y); BOUNDED_3D offset
(-w/2, -d/2, -h), extent (w, d, h) (Meta: volumes hang below their anchor); BOUNDED_2D = the top face (w x d).
- plane: pose as given; BOUNDED_2D = w x h centred.
- mesh: entity at the identity pose labelled GLOBAL_MESH with TRIANGLE_MESH (XR_META_spatial_entity_mesh).
## room-mesh.bin (little-endian)
```
char[4] "FPRM", u32 version = 1, u32 vertex_count, u32 index_count,
float32 xyz[vertex_count], u32 indices[index_count] (triangles; counter-clockwise seen from the room's inside)
```
Coordinates in STAGE space. Kept below 200k triangles (decimated).
## Raw files and scan.json (scanner sidecars)
`fp_roomscan` refines its output by default (docs/mr/scan.md "Refinement"). `room.txt` + `room-mesh.bin` are always
the refined room (what games and the GUI read; the contract above is unchanged). Next to them:
- `room-raw.txt` + `room-mesh-raw.bin`: the scanner's unrefined output in the same formats (its `mesh` record names
`room-mesh-raw.bin`). Same UUIDs as the refined room.txt, so anchors stay the same when the room is refined again.
The refined room differs in: the ceiling height (re-estimated from the scan, wall tops follow it), the volumes
(after the label rules) and the mesh. Records the scanner doesn't model (e.g. `plane`) are copied verbatim.
- `scan.json` (UTF-8 JSON, `version` 1):
```
{"version": 1,
"heads": [[x, y, z], ...], # head path of the recording (STAGE), <= 2000 points, decimated evenly
"refine": {"refine": 1, "walls": "panels", "band": 0.35, ..., "erase": [[x, y, z, r], ...]},
# every refinement setting used (keys = the CLI options, docs/mr/scan.md)
"report": {"triangles", "kept", "structural", "near_walls", "outside", "small_pieces", "thin_pieces",
"erased", "walked", "tops", "panel_triangles", "truncated", "walked_skipped", "floor_y",
"ceiling_y", "boxes", "relabelled", "removed_boxes", "suggestions",
"plane_shifts": [{"kind": "wall"|"floor"|"ceiling", "wall": i, "shift": m}, ...]},
"labels": [{"from": "COUCH", "to": "TABLE"|null, "reason": "top 0.69 m", "suggest": "door"|"closet" (optional),
"id": "<volume uuid>", "box": {"cx", "cz", "top_y", "yaw_deg", "w", "d", "h"}}, ...],
# one per scanner volume, in room-raw.txt order; to = null: removed
"suggestions": [{"kind": "door"|"closet", "box": {...}, "wall": i, "along": t, "reason": "..."}, ...]}
# tall thin slabs; never applied by the scanner. wall = index of the nearest
# wall record in room-raw.txt, along = 0..1 position on it (from x0,z0)
```
`report` counts triangles of the raw mesh per pass (`panel_triangles` are added ones), `wall` in `plane_shifts`
indexes the wall records. Readers ignore unknown keys.
- `fp_roomscan --refine-only <dir>` re-runs only the refinement from these files (an older folder without them:
its room.txt + mesh are copied to the raw names first). Publishing order: [raw mesh, raw text,] `scan.json`,
`room-mesh.bin`, `room.txt`.
## Where the files live
- Frame: `~/.local/share/frameport/rooms/<universe>/room.txt` + `room-mesh.bin` (+ `scan.log`, the raw files and
`scan.json` above).
`~/.local/share/frameport/rooms/current` = symlink to the newest room's folder.
- Per game: launch.sh copies the current universe's room (else `current`) to `<app_dir>/room.txt` (+ mesh) before
Lepton starts, next to `settings.conf`; FrameBridge reads `dirname($FRAMEBRIDGE_CONFIG)/room.txt`. No room ->
the old guardian box.
- A game's own saved anchors: `/sdcard/Android/data/<pkg>/files/framebridge-anchors.txt`
(`anchor <uuid> px py pz qx qy qz qw` per line, STAGE space), written by FrameBridge on save/erase.
## Scan dumps (*.fprs) — recorded RoomView 3D meshes, input of `fp_roomscan --offline`
```
char[4] "FPRS", u32 version = 1, then frames until EOF:
f64 time_s, float32 pose[7] (px py pz qx qy qz qw: mesh -> STAGE), u32 vertex_count, u32 index_count,
float32 xyz[vertex_count] (mesh space), u32 indices[index_count]
```
+216
View File
@@ -0,0 +1,216 @@
# Scene emulation stream
Implemented against `ROOM_FORMAT.md` without changing the contract, patch revisions, agent version or analysis version.
No Frame access or external downloads were used. Headset behavior still needs maintainer verification.
## Implementation
- `scene_emu.c` owns entities, component APIs, pose transforms, bounded query snapshots and the event queue.
`scene_room.c` parses room files and meshes; `scene_meta.c` implements META discovery and batch persistence;
`scene_anchors.c` implements per-game saved anchors. All are included by the adapter, with no new dependencies.
- At session creation, read `dirname(FRAMEBRIDGE_CONFIG)/room.txt`. Invalid magic/version, unusable geometry or
absent files retain the original guardian box, settings and seven deterministic UUIDs. Unknown records are
ignored; malformed records are logged and skipped. Missing ceiling, outline and walls follow the contract.
- Floor/ceiling use the polygon area centroid and real local-XY polygons. Walls retain file order, with local +X
along their segment, +Y up and +Z inward. Furniture poses use yaw then +Z up; volume bounds extend downwards.
The room container includes every other room-file entity, excluding independently created application anchors.
- Geometry exposes BOUNDED_2D, BOUNDED_3D, semantic labels and TRIANGLE_MESH as appropriate. OpenXR has no separate
`BOUNDARY_2D` component enum: `xrGetSpaceBoundary2DFB` belongs to BOUNDED_2D. Labels preserve comma-separated strings.
All variable-sized outputs support count-only and insufficient-capacity calls, including independent mesh arrays.
- Limits: 2,048 entities including application anchors, 1,024 outline points, 255 label bytes, 64 KiB record lines,
one global mesh, 600,000 vertices/indices (200,000 triangles) and 16 MiB mesh payload. Sidecar filenames cannot
escape the config directory. Magic, version, counts, exact byte size, finite coordinates and indices are checked.
- FB queries and META discovery snapshot results under the mutation lock, before queuing available/complete events.
META component/UUID filters intersect; unrecognized filters impose no restriction. FB max-result limits are honored.
There are 64 retained query snapshots and 63 queued events; calls that cannot fit their events return
`XR_ERROR_LIMIT_REACHED`. Geometry readers share the mutation lock with persistence.
- Application anchors use kernel randomness (`getrandom`, then `/dev/urandom`), never index-derived UUIDs.
Saved poses are in STAGE space. FB/META saves write a private temporary file, flush/fsync, then rename to
`/sdcard/Android/data/<package>/files/framebridge-anchors.txt`. A process suffix such as `:game` is stripped.
Erase removes application anchors from disk and the discoverable in-memory set. Failed writes report failure
through completion events and restore memory. Room-file entities remain owned by the scan: save/erase are no-ops
for those entities. Storage is local; there is no cloud synchronization.
- A new session reloads the room and saved anchors. Fake handles fit both adapter ABIs and include a session
generation; erased/stale fake spaces are never passed to SteamVR. Session generations wrap after 256 sessions.
- Capture completes immediately and logs exactly `scene: capture requested -> scanned room` for a usable room file,
or `scene: capture requested -> guardian box` for the fallback. The former covers all file sources, including manual
and boundary files. The load summary preserves the actual `source` and `universe`.
The cached `openxr-f2448a8797` headers already contain discovery, persistence and mesh declarations and enum values.
No local structure/enum substitutes were necessary. Both META extensions are added to the existing conditional
extension advertisement/instance filtering. Their system capability structures report support when emulation is active.
Passthrough and unrelated adapter settings are unchanged.
## Routing and integration
All of the following use `scene_emulation` through **both** the adapter's normal `xrGetInstanceProcAddr` and
`framebridge_extension_proc`, reached through xrshim's existing name-agnostic lookup:
```
xrQuerySpacesFB xrRetrieveSpaceQueryResultsFB
xrGetSpaceUuidFB xrEnumerateSpaceSupportedComponentsFB
xrSetSpaceComponentStatusFB xrGetSpaceComponentStatusFB
xrGetSpaceContainerFB xrGetSpaceRoomLayoutFB
xrGetSpaceBoundingBox2DFB xrGetSpaceBoundingBox3DFB
xrGetSpaceBoundary2DFB xrGetSpaceSemanticLabelsFB
xrRequestSceneCaptureFB xrCreateSpatialAnchorFB
xrSaveSpaceFB xrEraseSpaceFB
xrGetSpaceTriangleMeshMETA xrDiscoverSpacesMETA
xrRetrieveSpaceDiscoveryResultsMETA xrSaveSpacesMETA
xrEraseSpacesMETA
```
The shim additionally routes `xrPollEvent`, `xrGetSystemProperties`, `xrLocateSpace`, `xrLocateSpaces`,
`xrLocateSpacesKHR` and `xrDestroySpace` to the adapter while scene emulation is active. Other shim names continue to
OVRPort, preserving existing haptics/render-model handling. The real shim is host-tested against a dispatcher that
refuses every scene function; every emulated scene lookup succeeds and an actual discovery/retrieve round trip runs.
**Plumb-stream requirement:** ensure `scene_emul=1` also installs xrshim and rewrites OVRPlugin's loader string to it.
The pre-existing adapter patch only did this for other shim settings. These files are outside this stream's ownership.
Delivery of room files before Lepton starts and detection of universe misalignment also belong to plumb. FrameBridge
serves the supplied coordinates; it does not silently transform a room belonging to another universe.
No contract changes are needed. Whether each MRUK version accepts a room without further permissions, how it chooses
volume versus mesh geometry, and whether its visible placement matches the scan need headset tests. The existing
single-active-session architecture is retained; simultaneous independent sessions are not implemented.
## Offline verification
- 25 host tests passed, compiling the actual emulation and shim. Coverage includes malformed records, unknown
records, mesh corruption/path/cap checks, derived geometry, 300 walls, poses/normals, all buffer sizing idioms,
discovery event order/filters/backpressure, UUID query/discovery of saved anchors, new sessions/processes,
FB/META save/erase and atomic-write failure rollback.
- ASan/UBSan: scanned room, generated walls and box cases passed. LeakSanitizer was disabled because this sandbox
prohibits ptrace; its default exit failed with that environmental limitation.
- `python native/build.py --only adapter,xrshim` succeeded from cached tools. Two final builds were byte-identical,
including `SHA256SUMS`. The shim was rebuilt and is byte-identical to its previous artifact (the routing lives in
the adapter). Artifact content changes are the two adapter binaries and two checksum lines.
- Full pytest was attempted: **1,771 passed, 40 skipped, 11 failed, 16 errors**. All 27 failing/erroring tests create
local HTTP sockets, which the sandbox prohibits (`PermissionError: [Errno 1] Operation not permitted`). They are
in `test_deeplink`, `test_devkit`, `test_install_links_extra`, `test_livestream`, `test_setup_url` and `test_ui_state`.
With exactly those 27 tests deselected, and the additional real-shim test included: **1,772 passed, 40 skipped,
27 deselected**. An unrestricted maintainer run of the whole suite is still required.
- The worktree had no usable venv. Verification reused the existing local dependencies through an ignored venv
`.pth`, with `PYTHONPATH="$PWD/src"`, `UV_CACHE_DIR=/tmp/frameport-uv-cache`, `UV_OFFLINE=true` and
`uv run --no-sync pytest`. No dependencies were downloaded. The build cache was made a local directory with
symlinks to cached tools/headers so generated shader files stayed inside this writable worktree.
Adapter SHA-256:
```
arm64-v8a: a90c8d58904857bc198a48fd7a411b7807f68f14fa58fd268c57e2c2ca87e389
armeabi-v7a: 92daecaf5b86e1857ee8a61b76598e5645f8b427e5f373067c1f87bf8ff99d07
```
## Maintainer device tests
These commands are a handoff, not commands run by this agent. Use an isolated populated `FRAMEPORT_HOME`, cached
build tools, a paired Frame and legally obtained game sources. Stop games before changing their files. Back up the
existing recipe, APK, room files and saved anchors first. Integrate plumb before testing normal room delivery.
On the PC, from the integrated checkout (replace `frame.local` if necessary):
```sh
export FRAMEPORT_HOME="$PWD/.home"
export FRAMEPORT_NO_UPDATE_CHECK=1 FRAMEPORT_NO_CATALOG_UPDATE=1
FRAME=steamos@frame.local
PKG=com.Novelab.Demeter
uv run frameport recipe "$PKG" --set scene_emul=1
uv run frameport install "$PKG" --frame "$FRAME" --force
uv run frameport test "$PKG" --frame "$FRAME" --seconds 45
ssh "$FRAME"
```
In that Frame shell, find the deployment's actual base (also supports a game moved to another drive):
```sh
PKG=com.Novelab.Demeter
BASE=$(python3 -c 'import json,os,sys; p=os.path.expanduser("~/Applications/quest-frame/"+sys.argv[1]+"/deployment.json"); print(json.load(open(p)).get("base",os.path.dirname(p)))' "$PKG")
grep -E 'scene:|extension shim:.*(Spaces|TriangleMesh|BoundingBox3D|PollEvent)' "$BASE/launch.log"
```
With a current scan delivered beside `settings.conf`, expect:
```
scene: room from scan file (N walls, M volumes, P planes, K mesh triangles, universe U)
scene: capture requested -> scanned room
```
The capture line appears only if the game requests capture. A source made from a boundary/manual file names that
source in the summary. The shim logs each requested function, for example
`extension shim: xrDiscoverSpacesMETA -> FrameBridge`. There must be no `Unknown proc addr` for a scene function
that the game obtains through the shim. Repeat the rebuild/install/headset check for `com.gritGames.Nope` and
`com.zenstudios.PFXVRQuest` with `scene_emul=1`; these exercise newer MRUK discovery/persistence paths.
Wear the headset and launch from Steam. Enter each game's MR mode and request room setup if offered. Verify the
floor polygon, every wall, ceiling and furniture match the scan; tables sit below their top-face anchors; doors and
windows have the correct orientation. Headless startup alone cannot prove tracking or visual placement.
Fallback/undo test, while stopped: preserve `$BASE/room.txt` and `$BASE/room-mesh.bin` under a separate backup
folder, then disable plumb's room delivery for this run or move the source room out of `rooms/current` as well.
Without that precaution launch.sh will restore the file. Launch again; expect the original box summary and
`scene: capture requested -> guardian box`. Test `scene_width=3 scene_depth=4 scene_height=2.7` with the existing
`frameport settings` command and expect `scene: room 3.00 x 4.00 x 2.70 m`. Restore the saved files, recipe/settings
and normal room delivery afterward. Do not delete a real game's saved anchors to test the fallback.
### Android persistence probe (independent of a game's UI)
This probe uses the **real scene code with the fake runtime from the host harness**. It checks Android storage,
process-name handling and both persistence APIs; the headset tests above check the real SteamVR integration.
The following cross-compilation command was verified offline:
```sh
native/.cache/ndk-27.2.12479018/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang \
-std=gnu11 -O2 -Wall -Wextra -Werror -Wno-unused-function \
-I native/.cache/openxr-f2448a8797 -I native/adapter tests/native/scene_emu_test.c \
-lm -o /tmp/frameport-scene-test-android
scp /tmp/frameport-scene-test-android "$FRAME":/tmp/
```
Keep a freshly installed game running so its Lepton container exists, then run in the Frame shell:
```sh
APPID=$(python3 -c 'import json,os,sys; print(json.load(open(os.path.expanduser("~/Applications/quest-frame/"+sys.argv[1]+"/deployment.json")))["appid"])' "$PKG")
CONTAINER=lepton-steamlaunch-$APPID
podman cp /tmp/frameport-scene-test-android "$CONTAINER":/data/local/tmp/frameport-scene-test
podman exec "$CONTAINER" test ! -e /sdcard/Android/data/org.frameport.scenetest
# Continue only if that test succeeds: use a fresh test-only package directory.
podman exec "$CONTAINER" mkdir -p /sdcard/Android/data/org.frameport.scenetest/files /data/local/tmp/frameport-scene-probe
for mode in save reload erase-failure reload erase empty; do
podman exec "$CONTAINER" sh -c "FRAMEBRIDGE_CONFIG=/data/local/tmp/frameport-scene-probe/settings.conf exec -a org.frameport.scenetest /data/local/tmp/frameport-scene-test $mode" || break
done
podman exec "$CONTAINER" wc -l /sdcard/Android/data/org.frameport.scenetest/files/framebridge-anchors.txt
```
Every invocation must exit 0 without `FAIL`; logs show `anchors saved`, `loaded 2 saved anchors`, a deliberate
`anchor write failed` during `erase-failure` (then the following reload still finds two), and `anchors erased`.
The final file has zero lines. `save` also recreates a session internally, while separate invocations validate
new-process reload. Undo, before stopping this container:
```sh
podman exec "$CONTAINER" rm -rf /sdcard/Android/data/org.frameport.scenetest /data/local/tmp/frameport-scene-probe
podman exec "$CONTAINER" rm -f /data/local/tmp/frameport-scene-test
rm -f /tmp/frameport-scene-test-android
```
Remove the PC's temporary probe binary too. Restore backed-up game files/settings and install the previous APK if
rolling the feature back. No probe command changes real application anchors.
## Commit handoff under the sandbox
This worktree's real Git metadata is on a read-only mount. Commits therefore live on `mr/emul` in
`/tmp/frameport-mr-emul.git`, with a portable bundle at `/tmp/frameport-mr-emul.bundle`. The actual worktree contains
all final source/artifact edits but its original branch metadata cannot be advanced here. No local instruction,
app-data or credential files were staged. The base is `4bf48d3`.
After reviewing the worktree and verifying it contains exactly these changes, the maintainer can import the commits
and align the existing worktree index without discarding its files:
```sh
git bundle verify /tmp/frameport-mr-emul.bundle
git fetch /tmp/frameport-mr-emul.bundle mr/emul
git log --oneline 4bf48d3..FETCH_HEAD
git reset --mixed FETCH_HEAD
```
Run the unrestricted whole suite before merging. The bundle preserves the ordered implementation/test/build commits.
+171
View File
@@ -0,0 +1,171 @@
# Room plumbing stream
Implemented on `mr/plumb`; no Frame access or device verification. The shared `ROOM_FORMAT.md`, agent version,
analysis version and patch revisions were left unchanged. No native binaries belong to this stream.
## What changed
- `frameport.mr.room`: room/entity dataclasses, v1 text and binary-mesh validation, canonical serialization,
missing ceiling/wall/outline defaults, clockwise outline/wall repair, summaries and top-down SVG previews.
Mesh counts, indices, finite coordinates, UUIDs and sidecar basenames are checked before import.
- Standalone stdlib agent room commands live in one marked block. Rooms are staged under
`~/.local/share/frameport/rooms/<universe>/`; a `current` symlink selects the latest saved room. Uploads validate
before replacing a room. Deleting the current room selects the newest remaining room.
- `room_status` returns `rooms`, current `universe`, selected `room`, `room_text`, `aligned`, `default_circle`
(null when no usable boundary exists) and `scanner_installed`. Summaries include wall/plane counts, furniture
counts by label, floor area and ceiling height. Status text lets the UI preview a room without another transfer.
- Universe detection takes the last matching vrserver log line, then the newest chaperone record if no log match
exists. Dummy universe 424242 means unknown. Boundary conversion chains collision quads' bottom edges and refuses
the default 20-point, roughly 1 m circle. Boundary ceilings default to 2.5 m; SteamVR's 2.43 m wall extrusion
does not measure the real ceiling.
- Finalize records `scene_room` from adapter settings. A marked `_room_sync` launcher line follows the existing
config-sync pattern, before Lepton starts. Launcher upgrades also handle old recipes listing
`adapter.scene_emul`. Matching universe wins over `current`; absent rooms remove stale sidecars. Live setting
changes update the launcher and clear copies when disabled. Non-scene launchers keep their existing layout.
- PC room helpers implement status/import/export/delete/boundary/build/show/capture. Scanner upload reuses the
SHA-256 helper extracted from live view; `fp_venc` retains its soft-failure behavior. Import uploads canonical
text. Export checks mesh counts; build uploads FPRS dumps, runs the scanner and cleans its staging directory.
- CLI: `frameport frame room status|from-boundary|import|export|delete|build|show`, with `--frame` on each command.
`show --output <path.svg>` writes a preview (default below `FRAMEPORT_HOME/rooms`). Build accepts `--universe`.
- Frame page Room card: checklist, asset-URL preview, boundary/delete/capture actions, refresh and recoverable errors.
SSH and preview writes run in a background worker; state is cached per connection. Scene-game pages link to Room
setup when the connected Frame has no room. `ui_smoke.py --fake-frame` supplies local room-command answers.
- Every USE_SCENE/USE_ANCHOR_API game gets the scene-emulation suggestion. Migration `scene_emul_all` enables it once
in automatic recipes, preserving user recipes. NOPE and Pinball FX keep their other recipe fields and now request
scene emulation. Triage explains guardian fallback and missing discovery; patch docs and translation template
were regenerated.
## Integration questions and limits
1. Headset capture is intentionally unfinished: `room_capture` returns `{"error":"not implemented yet"}`. The
`capture_roomview(seconds, out_path)` hook documents the pending RoomView queue work. The GUI explains this and
offers the boundary path. Do not present the capture button as a completed scan workflow yet.
2. This branch has no `fp_roomscan` artifact; merge the scan stream before using `frame room build`. The exact scanner
invocation is `fp_roomscan --offline <dump> --out <temporary directory> --universe U --json`. Stdout must contain
one JSON value, stderr is saved as `scan.log`, and output must include a valid `room.txt` with the requested U.
3. Merge the emulation/shim stream before expecting games to use saved rooms, meshes or new Meta entry points.
Runtime log spelling is owned there; triage accepts both existing `using guardian room`/dimension logs and the
brief's `guardian box`/`room from guardian` messages. Reconcile new success logs at integration.
4. Confirm the chaperone JSON path, exact universe-switch logs and whether collision quads are already in STAGE on
the device. This implementation follows the brief's path and coordinate contract; it does not apply a second
standing-origin transform. A universe match alone cannot detect Valve changing an origin without changing its ID.
5. Agent and desktop perform independent validation because the deployed agent cannot import the desktop package.
The desktop canonicalizes outlines and fills defaults; the agent preserves scanner output and validates it.
The format contract needed no changes.
6. Rooms are shared across games; per-game saved anchors are not deleted or rewritten by these commands. Replacing
a room creates new entity UUIDs, so a game may need its own room-setup reset. No automatic game reset was added.
## Offline validation
All commands used `FRAMEPORT_HOME` in this worktree (tests use temporary isolated homes), an existing dependency
venv and offline/no-sync uv settings. No dependencies or native toolchains were downloaded.
- Room, PC/CLI, UI, heuristics, migration, MR triage and translation checks passed: 63 tests. The related
agent/launcher/screenshots/codec/gamepad regression group passed 170 tests. The final deployment/configuration
regression check passed 48 tests, including all four recipe/settings precedence cases.
- Ruff over changed Python files and `scripts/patch_docs.py --check` passed. The UI detector reported no findings.
`scripts/i18n_extract.py` regenerated `src/frameport/locales/template.json`.
- `pytest -m games`: 3 skipped (no game dumps). Dev/holdout evaluation used only the checked-in demo analysis
snapshot, 21 catalog matches: dev dated 9/17 exact, F1 81.1%; holdout dated 1/4 exact, F1 73.3%. These are limited
fixture results, not a new score for the owner's complete collection; the evaluator exits 1 for recipe differences.
- Final full suite: **1790 passed, 40 skipped, 11 failed, 16 errors** in 87.35 s. All 27 failures/errors are
`PermissionError: [Errno 1] Operation not permitted` when existing tests create sockets. The whole-suite green
requirement cannot be met in this sandbox; rerun `uv run pytest -q` on a host permitting loopback sockets.
Browser UI smoke also requires a loopback server, so it
was not run. New UI tests construct the real Flet controls and exercise deferred preview/error/cache states.
## Exact device checks (maintainer only)
Run these after integrating emul + scan. Substitute the paired Frame address below; keep the current worktree's
`FRAMEPORT_HOME=.home` and use explicit `--frame` so the test does not open another app data directory. The source
connection syncs the agent by digest even without a version bump, unless the Frame has a newer agent version.
The maintainer must perform integration version bumps before testing against a newer installed agent.
```sh
export FRAME='steamos@<frame>'
export PKG='com.gritGames.Nope' # repeat with com.zenstudios.PFXVRQuest and a known scene game
uv run frameport frame info --frame "$FRAME"
uv run frameport frame room status --frame "$FRAME" > .home/mr-status.json
# Back up ALL rooms on the Frame before destructive checks; use an unused backup name.
ssh "$FRAME" 'test ! -e ~/.local/share/frameport/rooms.before-mr-test && { test ! -d ~/.local/share/frameport/rooms || cp -a ~/.local/share/frameport/rooms ~/.local/share/frameport/rooms.before-mr-test; }'
```
1. Inspect status: no room gives `room: null`, `aligned: false`. With Valve's default circle, `default_circle: true`;
`uv run frameport frame room from-boundary --frame "$FRAME"` must fail with the default-circle explanation.
Set a real play area in SteamVR, repeat `from-boundary`: expect source `boundary`, one wall per boundary edge,
correct floor area and `aligned: true`. Open Frame → Room: visible preview, warning for the unscanned room,
aligned row ready; the capture button should show the friendly unavailable message without crashing.
2. Exercise import/export and preview with a manual fixture in the current universe:
```sh
uv run frameport frame room status --frame "$FRAME" > .home/mr-status.json
uv run python - <<'PY'
import json
from pathlib import Path
from frameport.mr.room import Mesh, Volume, box_room, dumps, write_mesh
p = Path('.home/mr-check'); p.mkdir(parents=True, exist_ok=True)
r = box_room(4, 3, 2.5)
r.source = 'manual'
r.universe = json.loads(Path('.home/mr-status.json').read_text())['universe']
r.volumes.append(Volume('a'*32, 'TABLE', .5, 0, .8, 20, 1.2, .7, .8))
r.mesh = Mesh(vertices=[(-2, 0, -1.5), (2, 0, -1.5), (2, 0, 1.5)], indices=[0, 1, 2])
(p/'room.txt').write_text(dumps(r))
(p/'room-mesh.bin').write_bytes(write_mesh(r.mesh))
PY
uv run frameport frame room import .home/mr-check/room.txt --frame "$FRAME"
uv run frameport frame room export .home/mr-export --frame "$FRAME"
uv run frameport frame room show --output .home/mr-check/preview.svg --frame "$FRAME"
cmp .home/mr-check/room-mesh.bin .home/mr-export/room-mesh.bin
```
Expect four walls, one TABLE, area 12 m² and height 2.5 m. The preview must label TABLE and the game must show it
hanging below its top-face anchor. Check the corresponding universe folder and `rooms/current` on the Frame.
3. Use a recorded dump from the same standing universe (do not invent one). Copy it to `.home/mr-check/scan.fprs`,
then run the command below. Expect the scanner JSON under `report`, source `scan`, aligned room data, `scan.log`
on the Frame and an installed executable. Run twice: an unchanged scanner must not upload again.
```sh
uv run frameport frame room build .home/mr-check/scan.fprs --frame "$FRAME"
uv run frameport frame room status --frame "$FRAME"
```
If the recording is from another universe, pass its original `--universe <id>`; do not relabel its coordinates.
4. Prepare the game's source and its existing backed-up signing key in the isolated test library, then force a
rebuild/reinstall after integration: `uv run frameport install "$PKG" --frame "$FRAME" --force --apk-only`.
Repeat for both catalog games. In each deployment, expect `scene_room: true`. Run `frame info` twice to upgrade
old launchers and check one marker only:
```sh
uv run frameport frame info --frame "$FRAME"
uv run frameport frame info --frame "$FRAME"
ssh "$FRAME" "grep -c '# frameport-room-sync' ~/Applications/quest-frame/$PKG/launch.sh"
uv run frameport settings "$PKG" scene_emul=1 --frame "$FRAME"
```
Expected marker count: 1. Start the game with Steam's Play button in the headset, enter its MR mode, then inspect
`<deployment.base>/room-sync.log`: JSON has the selected universe and alignment. The two room files must be
beside `<deployment.base>/settings.conf`. FrameBridge must load that room; `xrDiscoverSpacesMETA` must resolve
through the extension shim, with no `Unknown proc addr`/MRUK `NoRoomsFound`. Exact new native success log strings
should be taken from the emul stream. Verify walls/table placement physically; never infer alignment from FPS.
5. Exit the game. Switch `scene_emul=0` using Game settings (or the settings command above with 0): marker and game
sidecars disappear. Switch it back to 1: one marker returns. Delete the room and launch again: stale copies are
removed and existing FrameBridge guardian fallback logs trigger informational `mr-no-room` triage.
6. To check a universe mismatch, back up/export the room, change its `universe` record to another valid ID and import
it, then delete the original universe's room with `frame room delete --universe <original-id>`. Status must select
`current` but report `aligned: false`; the Room card warns. Restore the original file before playing physically.
The dummy universe 424242 should report current universe 0 rather than claim alignment.
## Undo device checks
Close all games. Restore the backed-up store (or remove only the fixture universes if there was no previous store):
```sh
ssh "$FRAME" 'if test -d ~/.local/share/frameport/rooms.before-mr-test; then rm -rf ~/.local/share/frameport/rooms; mv ~/.local/share/frameport/rooms.before-mr-test ~/.local/share/frameport/rooms; fi'
```
Set each tested game's `scene_emul` back to its recorded original value. To remove a test room without a backup:
`uv run frameport frame room delete --universe <fixture-universe> --frame "$FRAME"`. Start a scene game once to
refresh/remove its copies. The scanner can be removed with
`ssh "$FRAME" 'rm -f ~/.local/share/frameport/bin/fp_roomscan'`; the next build reinstalls it. Game saves and
`framebridge-anchors.txt` are untouched. Restore the previous app/agent/native artifacts through the maintainer's
normal integration rollback if testing the combined native implementation fails.
+452
View File
@@ -0,0 +1,452 @@
# Offline room scanner (scan stream)
`fp_roomscan` converts recorded FPRS v1 meshes to the [room contract](ROOM_FORMAT.md), including the optional
wall label adopted from upstream commit `8cb2783`:
`room.txt` and `room-mesh.bin`, refined by default (see "Refinement" below), plus the unrefined `room-raw.txt` +
`room-mesh-raw.bin` and `scan.json` (head path, settings, per-pass report, label decisions, door/closet
suggestions; ROOM_FORMAT.md "Raw files and scan.json"). It does not capture RoomView queues, contact a headset, install rooms, or modify
the current-room symlink. Scanner work changes `native/roomscan/`, its build target and artifact/checksum,
`scripts/mr/synth_scan.py`, `tests/test_roomscan.py`, and this note. The upstream contract commit also brings its
existing prompt assets and renderer unchanged. No runtime, agent, patch, or analysis versions were bumped.
## Build and run
Use the cached NDK; no downloads are needed for this target:
```sh
python native/build.py --only roomscan
```
The result is `artifacts/linux-arm64-bin/fp_roomscan`, a static aarch64 Linux ELF with its own `_start`, raw Linux
syscalls, and no interpreter, dynamic section, libc, libm, UAPI headers, generated tables, or new dependencies.
`platform.h` is the reconstruction code's entire I/O/allocation boundary. The same eight C sources compile on the
host:
```sh
cc -std=c11 -O2 -Wall -Wextra -Werror -ffp-contract=off -fno-math-errno -DFP_HOSTED native/roomscan/*.c -o /tmp/fp_roomscan
python scripts/mr/synth_scan.py /tmp/room.fprs
/tmp/fp_roomscan --offline /tmp/room.fprs --out /tmp/scanned-room --universe 0 --json
```
The output directory's parent must exist; the scanner creates the final directory if needed. It always prints
exactly one JSON summary, whether `--json` is supplied or omitted. Success exits 0, errors exit 2 with
`{"ok":false,"error":"..."}`. `--help` prints usage and exits 0. Successful summaries contain:
```json
{"ok":true,"frames":30,"skipped_frames":0,"far_vertices":0,"discontinuity_triangles":0,
"walls":4,"detected_walls":4,"volumes":2,"floor_y":-0.0023,
"ceiling_y":2.5992,"coverage_percent":100.0,"hint_used":false,"triangles":70997,
"accepted_triangles":118804,"rejected_triangles":704,"heads":28,
"refine":{"triangles":70997,"kept":7159,"structural":46121,"near_walls":148,"outside":0,"small_pieces":38,
"thin_pieces":0,"erased":0,"walked":203,"tops":1896,"panel_triangles":12,"truncated":0,"walked_skipped":false,
"floor_y":-0.0051,"ceiling_y":2.6028,"boxes":2,"relabelled":1,"removed_boxes":0,"suggestions":0,
"plane_shifts":[{"kind":"wall","wall":0,"shift":-0.0004},...,{"kind":"ceiling","shift":-0.0021}],
"output_triangles":7171,"output_volumes":2},"warnings":[],
"timings_ms":{"read":0,"fusion":0,"model":0,"mesh":0,"refine":0,"total":0}}
```
The example above illustrates the schema; actual counts and timings depend on the input. Heights are absolute
STAGE coordinates. `frames` includes skipped frames; `skipped_frames`, `far_vertices`, and
`discontinuity_triangles` count input exclusions once, not once per pass. `walls` counts output polygon
edges including synthesized edges; `detected_walls` counts fitted physical planes. Coverage means the percentage
of the output perimeter supported by observed wall planes, **not camera coverage of all room surfaces**. Timings
include parsing, fusion, structural/furniture modelling, mesh generation, refinement, and total time including
output. The scanner fields describe the unrefined scan (`triangles`, `volumes`, `ceiling_y` are the raw files');
`refine` is the refinement report (same keys as scan.json's `report`) plus the refined `output_triangles` and
`output_volumes`. `heads` counts the head-path points kept for the walked-volume pass.
All frame headers, timestamps/vertices/poses, approximately unit quaternions, counts, indices and EOF boundaries
are checked before publication. A non-finite timestamp/pose/vertex or invalid tracking quaternion skips the entire
frame after consuming and structurally validating its record, with a count and warning. A bad frame never erases
usable frames elsewhere in the dump. Vertices outside the camera-relative range (default 4 m) and every triangle
touching them are discarded. Triangles with a camera-depth jump over `--depth-jump + --depth-ratio * nearest_depth`
are rejected before fusion; this complements long-edge and grazing rejection. Frame counts are capped at
1 million vertices / 3 million indices;
oversized records fail rather than allocating from untrusted counts. A header-only dump or one without usable
triangles fails. Empty frames may accompany useful frames. Structural corruption (truncation, unsupported header,
bad counts or indices) remains fatal, preserves any existing output byte for byte and creates no output directory.
UUIDs come from `getrandom`, fresh per scan; mesh geometry is repeatable.
All files are computed first, then each is written to an exclusive, randomly named temporary file and renamed, in
the order `room-mesh-raw.bin`, `room-raw.txt`, `scan.json`, `room-mesh.bin`, `room.txt` (meshes before their room
text, `room.txt` last). A write failure removes the temporary files. These are atomic **file** replacements, not a
multi-file filesystem transaction: the installer should scan into a fresh staging directory
and publish that directory/symlink after success. Do not run concurrent writers into the same output directory.
There is no fsync-based power-loss durability promise. Neither scan nor build writes application data outside
the worktree's `FRAMEPORT_HOME`; fixtures and test output use temporary directories.
## Reconstruction
1. The first streaming pass validates every frame, rotates its mesh by the supplied quaternion, and translates
it into STAGE. It rejects degenerate triangles, excessively long edges, out-of-range points, and grazing
surfaces. Winding is corrected toward the observing camera. A deterministic bounded reservoir of 32,768
accepted triangle centroids determines median position and padded bounds. No frames are retained across reads.
2. The second pass rasterizes accepted triangles at 0.75 voxel spacing. Per-cell surfels average positions and
inward normals, weighted by area and squared incidence. A capped accumulated weight lets later views correct
earlier ones. The grid is sized from the sample bounds and capped at 8 x 8 x 4 metres about the data median.
Unobserved cells have zero weight. This implementation uses a bounded dense backing array for the sparse
observations, rather than a hash table; the 4.2-million-cell hard cap bounds all later work. Complete regular
fans are recognized from their index topology, without hardcoding a camera resolution. Their normals come from
12 x 12 neighbourhood covariance plane fits, rejecting patches above `--patch-residual` RMS; irregular meshes
retain triangle normals. Each cell counts contributing frames once and needs `--min-views` evidence, capped at
ten percent of usable frames (rounded down, at least one) for brief captures. This suppresses transient stereo
geometry without requiring a short scan to see each surface five times.
3. Horizontal normal/height histograms select the dominant floor within the STAGE floor band and a ceiling above
the minimum ceiling height. Missing floor uses STAGE y=0 with a warning; missing ceiling uses floor plus
`--ceiling-default`, warning exactly `ceiling not seen`. A ceiling must also have at least 15% as many supported
cells as all upward-facing horizontal surfaces, so a few isolated down-facing patches do not invent a ceiling.
4. RANSAC fits near-vertical planes using distance and normal agreement, then refines them by position covariance.
Minimum height, span, and area reject furniture faces. Wall directions within the snap tolerance of the
dominant wall's orthogonal directions are snapped, with offsets refitted. Directed wall segments are connected
by nearest endpoints and adjacent line intersections. This preserves the L-shaped fixture's re-entrant corner.
Parallel same-facing planes within 25 cm / approximately 14° merge, suppressing duplicate softened faces.
Planes below 20% of the strongest original plane's area are discarded. When at least 65% of candidate wall area
supports an orthogonal family, directions outside 12° of that family are discarded (`--snap-degrees 0` disables
this orthogonal consensus filter). Large gaps close along observed endpoints. Ambiguous endpoints with three
or four planes can use inward half-plane intersections; concave rooms retain segment ordering. Otherwise a
warned observed-bounds rectangle is used. Polygon checks reject crossing edges. The output follows the contract's positive
shoelace ordering in (x,z), with left-facing inward normals.
5. Furniture tops are clustered separately from vertical faces, within supported horizontal height bands. A
softened edge connecting a tabletop to a wall can no longer absorb the whole room into one box. Top boxes use
a minimum-area horizontal orientation search and the median surface height; vertical screen candidates use
face-direction harmonics / PCA. Bounds trim the outer 2% to suppress stereo halos. Top boxes extend down to the
floor (the footprint and lower extent remain geometric estimates). Weak/sparse top patches are discarded.
Labels follow height, area, long-side (with two voxels of measurement tolerance),
thinness and near-wall rules, in order BED, COUCH, TABLE, SCREEN, STORAGE, OTHER. Only volumes are emitted;
optional table plane records are not required. Floor/ceiling/wall surfaces are excluded before clustering.
6. Surfel splats form an incidence/area-weighted signed distance field within three voxels of each surface. The
sign is positive in observed free room space. Marching cubes traces zero crossings on cube faces, resolving
ambiguous faces from their centre sign. It fans closed contours and orients triangles toward the positive
gradient. Unknown corners suppress extraction rather than bridging unobserved space. If necessary, extraction
is repeated on a coarser lattice (strides 2–4) until at most 200,000 triangles remain, with a decimation warning;
excess beyond that fails. Vertices are unshared triangle vertices, so at most 600,000 vertices. Structural
planes are semantic anchors, not substituted for the observed global mesh.
## Refinement
After the mesh, the scanner cleans the room the way the owner tuned it in the room-scan cleanup page (2026-10-18);
`native/roomscan/refine.c` and `rules.c` port that page's `refine()` and label rules pass for pass (local parity
check on the owner's recording: identical counts and bit-identical refined meshes, see below). `--refine 0`
switches it off: room.txt and room-mesh.bin are then byte-for-byte what the scanner wrote before refinement existed
(the raw files and scan.json are still written, so refinement can be switched on later without a new scan).
The input is exactly what the raw files hold: the raw room is written as text and read back (doubles parsed from the
4-decimal text, like the page reads it), the head path is rounded the same way. Therefore `--refine-only <dir>`
on the raw files reproduces the scan's refined files byte for byte. The mesh is welded at 1 mm (quantized, as the
page does) for the passes that need topology; the output is again unshared triangle vertices.
1. **Label rules** (`rules.c`, `--label-rules 0` off) on the scanner's volumes, using the raw room (walls, outline,
floor): a box that holds >= 60% of two boxes whose centres are >= 1 m apart covers several things (removed); a
box >= 75% inside a bigger box with a top no higher (+5 cm) is part of it (removed); a centre > 10 cm outside the
outline is behind a wall (removed). A tall thin slab from the floor (thinner than 22 cm, >= 1.8 m high, bottom
<= 0.3 m, at most 0.5 m behind the wall line) becomes a door (<= 1.1 m wide) or closet **suggestion** in
scan.json (box removed, nothing added: the GUI decides). Thin (< 22 cm) upright boxes are SCREEN only when
raised >= 0.3 m, top <= 2.2 m, width/height >= 1.2, within 15 degrees of the nearest wall (when closer than
0.35 m) and flat: the raw mesh's depth spread inside the box <= 3 cm (with >= 30 triangles; fewer = unknown,
accepted); otherwise removed. Then BED (top 0.25-0.65 m, area >= 2 m^2, long side 1.8-3 m), COUCH (top
0.35-0.6 m, long side >= 1.4 m), TABLE (top 0.6-1.1 m), STORAGE (>= 1.1 m high, >= 25 cm deep, within 0.35 m of a
wall); anything taller than 1.1 m otherwise is removed, the rest is OTHER. Every box gets a reason in scan.json.
These thresholds are constants in `rules.c` (the page's `RULES0`).
2. **Furniture**: triangles within 10 cm of a ruled box (after the rules) are furniture; the passes below never
classify, clear or walk them away, and the tubes pass keeps pieces that are mostly furniture.
3. **Planes**: seen walls (`WALL_FACE` records), floor and ceiling. Each plane's offset moves to the median signed
distance of the non-furniture triangles facing it (normal within `--facing-deg`) within `--band` of it (walls:
within their length + band), when there are more than 50. An unseen ceiling (floor + 2.5 m) moves to the real
one; the re-estimated ceiling is written to the refined room.txt (ceiling record and wall tops).
4. **Structural triangles**: facing a plane, not furniture, behind it by less than `--band`, in front of it by less
than `--band-in` (walls), `--floor-in`, `--ceil-in`; in a wall's top `--top-strip` below the ceiling the full
band applies in front too (the depth bends where wall meets ceiling). The nearest qualifying plane owns it.
5. **Erase spheres** (`--erase "x,y,z,r;..."`, from the GUI's eraser) drop every triangle whose centroid is inside.
6. **Walked volume** (`--walked` = radius R, 0 off): where the user stood, nothing solid can be. Non-furniture
triangles from 0.2 m above the floor up to 0.25 m above a head position, within R horizontally of it, go.
Needs the head path (scan.json); without it the pass is skipped (`walked_skipped`).
7. **Walls** (`--walls`): `panels` (default) removes the structural triangles, then non-furniture triangles closer
than `--near` to any plane (within the wall's extent), and adds flat panels: one quad per seen wall (moved by its
offset shift) from floor to ceiling, and the outline ear-clipped at the floor and the ceiling heights, all wound
counter-clockwise seen from inside. `snap` instead projects every vertex of a structural triangle onto its plane
(the last owner wins). `off` keeps them.
8. **Furniture tops** (`--tops`, 0 off): upward-facing (normal y > 0.8) triangles within `--tops` of a box top and
inside its footprint are flattened onto it.
9. **Clip**: centroids further than `--clip` outside the outline (`--clip` + `--clip-open` past an unseen wall), or
further than `--clip` above the ceiling / below the floor plane go. `--clip 1 --clip-open 2` switches it off.
10. **Islands**: connected pieces smaller than `--island` m^2 go (`--protect 1` keeps furniture triangles).
11. **Thin floating pieces** (`--tubes`, 0 off): pieces of >= 10 triangles, < 1 m^2, at most half furniture, whose
longest principal axis (vertex PCA, std) is >= `--tubes` times the second and the second <= `--tube-thick`.
12. **Smoothing**: `--smooth` Taubin iterations (lambda 0.5, mu -0.53) over the kept triangles.
Output: the kept scan triangles, then the panels (`refine.panel_triangles`), at most 200,000 triangles (more kept
triangles would be cut, `truncated`, with a warning). The refinement adds about 20 ms on the host for the owner's
64k-triangle scan; its buffers (welding hash, double positions, union-find, adjacency) stay below ~170 MiB for a
200k-triangle mesh and the fusion grid is freed before it runs, so peak memory is still the fusion phase's.
`--refine-only <room dir> [refinement options] [--json]` runs only steps 1-12 from `room-raw.txt` +
`room-mesh-raw.bin` (+ the head path from `scan.json`) and rewrites scan.json, room-mesh.bin and room.txt. A folder
from an older scanner without raw files: its room.txt + mesh become the raw files first (the room text is rewritten
with the mesh record pointing at `room-mesh-raw.bin`), a warning says so, and there is no head path. Its JSON:
`{"ok":true,"refine_only":true,"raw_triangles":..,"volumes":..,"floor_y":..,"ceiling_y":..,"heads":..,
"refine":{...},"warnings":[...],"timings_ms":{"refine":..,"total":..}}`.
Local parity (2026-10-18, private recording `room1.fprs`, not in the repository): the reference page's functions run
under node on this scanner's raw output give the same per-pass counts, the same label decisions and reasons, and
bit-identical refined meshes for the defaults, snap, off + protect, erase spheres + tubes 2, and a wide-band setting
(one triangle differs there: a facing test exactly at the threshold), and the same on an older 128k-triangle room
folder. Distance ties at outline corners decide which wall's clip allowance applies, so the corner distance uses
V8's Math.hypot algorithm.
Peak allocation stays below 260 MiB, including maximum frame buffers, the grid, model worklists and mesh buffers
(the latter phases do not retain frame buffers). Default synthetic scenes use a much smaller grid. Thresholds
are CLI options below or named constants next to their algorithms. No room-specific coordinates or dimensions are
used; on-device quality and speed still need validation.
## Tuning
Distances are metres, areas m², volumes m³. `--universe` accepts the full unsigned 64-bit range; 0 means unknown.
| Option | Default | Accepted range / meaning |
| --- | ---: | --- |
| `--voxel` | 0.05 | 0.04–0.20, spatial resolution; lower values cost more memory/work |
| `--ceiling-default` | 2.5 | 1.8–3.8, height above floor when ceiling is unseen |
| `--truncation` | 3 | 1–5, TSDF band in voxel units |
| `--max-edge` | 0.6 | 0.05–1.5, reject longer triangle edges |
| `--grazing` | 0.12 | 0.01–0.9, minimum absolute camera/normal cosine |
| `--range` | 4 | 1–10, maximum camera-relative vertex distance; >4 m opts back into inpainted depth |
| `--depth-jump` | 0.25 | 0.02–1, absolute part of a triangle's maximum depth discontinuity |
| `--depth-ratio` | 0.12 | 0–0.5, additional fraction of nearest vertex depth |
| `--min-views` | 5 | 1–16, temporal support target; brief scans cap it as described above |
| `--patch-residual` | 0.03 | 0.005–0.1, maximum RMS residual of a regular-fan normal patch |
| `--plane-distance` | 0.045 | 0.01–0.15, point-to-plane inlier tolerance |
| `--min-plane-area` | 0.45 | 0.1–4, minimum occupied plane area estimate |
| `--wall-height` | 1.5 | 0.5–3, minimum observed vertical wall span |
| `--snap-degrees` | 5 | 0–15, orthogonal snap tolerance |
| `--floor-band` | 0.35 | 0.05–1, floor search distance from STAGE y=0 |
| `--ceiling-min` | 1.8 | 1.5–3, minimum absolute ceiling y |
| `--min-volume` | 0.05 | 0.005–1, reject smaller furniture boxes |
| `--cluster-gap` | 0.10 | 0.04–0.20, voxel neighbourhood gap (rounded up to cells) |
| `--gap` | 0.65 | 0.1–1.5, maximum adjacent-wall endpoint extrapolation |
| `--horizontal` | 0.85 | 0.6–0.99, minimum floor/ceiling/top normal cosine |
Refinement (all distances metres; defaults = the owner's settings):
| Option | Default | Accepted range / meaning |
| --- | ---: | --- |
| `--refine` | 1 | 0/1; 0 = the unrefined output in room.txt/room-mesh.bin |
| `--walls` | panels | `off`, `snap` (project onto the planes) or `panels` (flat wall/floor/ceiling panels) |
| `--band` | 0.35 | 0.02–1, structural band behind a surface; plane re-estimation window |
| `--facing-deg` | 60 | 5–89, maximum angle between a triangle's normal and the surface's |
| `--band-in` | 0.12 | 0–1, band in front of a wall (room side) |
| `--floor-in` | 0.08 | 0–1, band above the floor |
| `--ceil-in` | 0.25 | 0–1, band below the ceiling |
| `--top-strip` | 0.5 | 0–2, top part of a wall where the full band applies in front too |
| `--near` | 0.03 | 0–0.3, panels: also clear non-furniture scan this close to a plane |
| `--clip` | 0.05 | 0–1, clip distance outside the outline / beyond floor and ceiling |
| `--clip-open` | 0.10 | 0–2, extra allowance past unseen walls (`--clip 1 --clip-open 2` = no clipping) |
| `--island` | 0.20 | 0–5 m^2, minimum connected piece area (0 off) |
| `--smooth` | 8 | 0–50 integer, Taubin iterations |
| `--tops` | 0.10 | 0–0.5, flatten upward scan this close to a box top (0 off) |
| `--protect` | 0 | 0/1, islands never remove furniture triangles |
| `--tubes` | 3 | 0–20, elongation (PCA axis ratio) of a thin floating piece (0 off) |
| `--tube-thick` | 0.15 | 0.01–1, maximum second PCA axis (std) of a thin floating piece |
| `--walked` | 0.40 | 0 or 0.05–1.5, walked-volume radius around the head path (0 off) |
| `--label-rules` | 1 | 0/1, label rules on the scanner's volumes |
| `--erase` | none | `"x,y,z,r;..."`, up to 64 spheres (r 0–5 m) whose scan is removed |
Additional constants in `model.c` include RANSAC trials, normal agreement, wall span, furniture margins,
label height/area thresholds, box trimming and circle recognition. Fusion sampling/weight limits are in
`fusion.c`; mesh splat radii/decimation limits in `mesh.c`; hard resource limits in `scan.h`.
`--hint outline.txt` accepts one `x z` point per line, decimal/scientific notation, optional `#` comments, either
winding, at most 96 points / 8 KiB. At least eight points with radial coefficient of variation below 5% count as
the default circle and are ignored. A non-circular hint is used only when walls are incomplete **and every
observed wall has a compatible hint edge** within `--gap`. Complete scans are never constrained to chaperone.
Incompatible inset play boundaries are ignored with a warning. Hint-completed edges retain synthesized status.
## Limits for integration
- Synthesized wall records now append `INVISIBLE_WALL_FACE` as their optional eighth field. Observed walls omit
the label and default to `WALL_FACE`; comments are no longer used to carry semantics. This matches upstream's
authorized contract change exactly; adapters need that same contract revision.
- A private RoomView recording was used for offline robustness work. It, its derived geometry, and its results
stay outside committed files. Synthetic fixtures independently exercise the input characteristics. They are
regressions, not proof
of headset accuracy. Furniture labels are geometric heuristics; adjacent/touching objects can merge, occluded
objects may split, top-only clusters have inferred depth, and thin fragments can be screen candidates. Nearby
parallel walls and small wall jogs can merge under coarse depth uncertainty. Large disconnected coplanar wall segments are
currently fitted together; complicated rooms can fall back to a warned rectangle.
- Cropped or sparse captures can produce an incomplete mesh and synthesized outline. Surfaces are not filled
through unseen regions. Furniture is not an accessibility/safety boundary. Rooms beyond the grid cap are
cropped with a warning. Default-circle recognition may also ignore an intentionally circular boundary.
- Output paths are fixed by the contract. The plumbing stream should stage a new directory and switch `current`
after scanner success, preserving the previous room for rollback. Assign the recording's actual universe id;
this standalone scanner cannot query SteamVR or validate universe alignment.
## Offline validation
`tests/test_roomscan.py` builds with the host compiler (skips only if none exists). Its tests cover the noisy
4 x 3 x 2.6 m room with table/couch, six-wall L, moving and rotated/translated mesh poses, ceiling fallback, a single
frame, missing walls, circular/matching hints, correct UUIDs/wire layout, interior mesh winding, triangle budget,
repeatable mesh bytes with fresh UUIDs, truncated/corrupt dumps, invalid counts/indices/poses and invalid CLI values.
Refinement tests: a floating pole and a column in the walked path (`synth_scan.py` `TUBE`/`GHOST`, `extra=`)
are meshed by the scanner and removed by the tubes / walked passes (each kept with its pass off, the table top kept);
panels wind into the room on the re-estimated planes; `--refine-only` reproduces a scan's files byte for byte;
`--refine 0` equals the raw output (and refining it later equals the default); scan.json settings/report/heads; head
path decimation to <= 2000 points; erase spheres; the label rules on a hand-written older room folder (merged,
nested, behind a wall, SCREEN from a flat panel, a door suggestion with wall and position, unknown records kept,
idempotent, `--label-rules 0`); invalid refinement options.
The test module takes about fourteen seconds including synthetic rendering and compilation (under the 60 s budget).
New cases cover non-finite frames before/after usable frames, per-vertex far rejection, depth jumps below the edge
limit, weak ceiling evidence, wall labels and a complete camera-fan scan. The deterministic stdlib renderer now
defaults to a 132 x 128 full index fan (99,822 indices), +Z forward / -Y image up, with poses mapping to STAGE.
It adds separable depth smoothing, spatially correlated noise, far inpainting, two initial NaN poses, 3 Hz timestamps
and head-height drift from 0.9 to 1.7 m. `--profile legacy` retains the smaller 110° x 76° test cone. New CLI controls
are `--profile`, `--invalid-start`, `--smoothing` and optional resolution overrides. Use `--invalid-start 0` for a
usable one-frame fixture; default RoomView warm-up frames otherwise have no valid tracking.
```sh
UV_OFFLINE=1 UV_CACHE_DIR=/tmp/frameport-uv-cache uv run pytest -q tests/test_roomscan.py
UV_OFFLINE=1 UV_CACHE_DIR=/tmp/frameport-uv-cache uv run ruff check scripts/mr/synth_scan.py tests/test_roomscan.py
```
The synthetic camera fan passes AddressSanitizer + UndefinedBehaviorSanitizer. Two consecutive final native builds
produce identical SHA-256 `2cc16b90abcd3d359f1658c581172bd650017bf2725e24268915f50904c000ec` (97,808 bytes, with refinement),
matching `artifacts/SHA256SUMS`. ELF inspection confirms aarch64, little-endian, no PT_INTERP / PT_DYNAMIC, and no
embedded local build paths.
The required full `uv run pytest -q` run completed with **1,788 passed, 40 skipped, 11 failed, 16 errors** in 79 s.
All 27 failures/errors are the sandbox's `PermissionError: [Errno 1] Operation not permitted` on socket creation
in existing HTTP/pairing/live-stream tests (`test_deeplink`, `test_devkit`, `test_install_links_extra`,
`test_livestream`, `test_setup_url`, `test_ui_state`). This environment forbids sockets, including loopback.
The unchanged baseline had the identical 27 failures/errors before the scanner tests were added (1,747 passed).
Consequently the whole-suite-pass requirement cannot be satisfied here; rerun it outside that restriction at
integration. No tests were changed/skipped to hide the restriction and no network/device access was attempted.
No patch/settings/catalog/triage files were touched, so the conditional patch documentation regeneration is
unnecessary.
## Ten-minute capture budget (synthetic benchmark)
The scanner is streaming and single-threaded. On an Intel Core i7-14700K host, a **synthetic-only** 1,800-frame
132 x 128 dump (1,083,751,208 bytes, equivalent to ten minutes at 3 Hz) took **7.41 s elapsed / 7.08 s CPU**, with
**27,208 KiB maximum RSS** at the default voxel size. The file repeats a 30-view synthetic sweep, including its
warm-up frames; it tests parsing/fusion throughput rather than 1,800 independent viewpoints. Geometry-dependent
model/mesh work and storage caches affect timing. No private recording or derived room data is in that benchmark.
Reproduce with bounded copies rather than loading a gigabyte into Python:
```sh
python scripts/mr/synth_scan.py /tmp/roomview.fprs
python - <<'PY'
import shutil
with open('/tmp/roomview.fprs', 'rb') as source, open('/tmp/roomscan-1800.fprs', 'wb') as out:
out.write(source.read(8))
for repeat in range(60):
source.seek(8)
shutil.copyfileobj(source, out, 1024 * 1024)
PY
/usr/bin/time -v /tmp/fp_roomscan --offline /tmp/roomscan-1800.fprs --out /tmp/roomscan-benchmark --json
```
For the Frame's Cortex-A720/X4-class cores, budgeting **3–5 times the host CPU time plus storage** gives roughly
**23–38 s** for this workload, leaving headroom under the requested ~60 s. This is an estimate, not an ARM
measurement or an 8-core speedup claim. Worst-case allocations are bounded below 260 MiB independently of frame
count (48-byte cells, 4.2-million-cell cap, bounded frame buffers/worklists), below both the original 300 MB and
follow-up 600 MB limits. A maximum-sized complex room can add model/mesh work beyond this small synthetic sweep.
The maintainer must time a ten-minute real capture on the Frame and check CPU time, wall time and RSS before
claiming the target is achieved on-device. Hardware square root instructions replace iterative division; neither
build links libm. The build and host command both require `-fno-math-errno` for that intrinsic.
## Exact device checks (maintainer only; none executed by this stream)
1. On the development PC, build, then render the canonical fixtures:
```sh
python native/build.py --only roomscan
python scripts/mr/synth_scan.py /tmp/box.fprs
python scripts/mr/synth_scan.py /tmp/l.fprs --shape l
python scripts/mr/synth_scan.py /tmp/down.fprs --no-ceiling
python scripts/mr/synth_scan.py /tmp/one.fprs --frames 1 --invalid-start 0
scp artifacts/linux-arm64-bin/fp_roomscan /tmp/box.fprs /tmp/l.fprs /tmp/down.fprs /tmp/one.fprs steamos@<FRAME>:~/Downloads/
```
2. In a terminal **on the Frame**, use an isolated directory; no game or current room is changed:
```sh
mkdir -p "$HOME/.local/share/frameport/roomscan-test"
cd "$HOME/.local/share/frameport/roomscan-test"
cp "$HOME/Downloads/fp_roomscan" ./fp_roomscan
chmod 700 ./fp_roomscan
./fp_roomscan --help
./fp_roomscan --offline "$HOME/Downloads/box.fprs" --out box --universe 0 --json > box.json
cat box.json
./fp_roomscan --offline "$HOME/Downloads/l.fprs" --out l --json > l.json
./fp_roomscan --offline "$HOME/Downloads/down.fprs" --out down --ceiling-default 2.7 --json > down.json
./fp_roomscan --offline "$HOME/Downloads/one.fprs" --out one --json > one.json
```
Expected JSON: box has `ok:true`, `walls:4`, `detected_walls:4`, `volumes:2`, floor within 5 cm of 0,
ceiling within 5 cm of 2.6, coverage 100; the only expected box warnings concern skipped warm-up frames and
out-of-range vertices. `box/room.txt` includes TABLE and COUCH volume records.
L has six walls and the concave corner near (0,0), rather than a rectangle. Down includes exactly the warning
string `ceiling not seen` (other warnings are expected), with ceiling minus floor 2.7. One includes
`few frames; scan more viewpoints`. Every mesh should have at most 200,000 triangles. JSON is the scanner log;
there are no expected stderr status lines.
3. Verify sidecars, wall labels and the corrupt-input exit status on the Frame:
```sh
python3 - <<'PY'
import json, pathlib, struct
for name in ('box', 'l', 'down', 'one'):
report = json.loads(pathlib.Path(name + '.json').read_text())
data = pathlib.Path(name, 'room-mesh.bin').read_bytes()
magic, version, nv, ni = struct.unpack_from('<4sIII', data)
assert report['ok'] and magic == b'FPRM' and version == 1
assert ni % 3 == 0 and 0 < ni // 3 <= 200000
assert len(data) == 16 + nv * 12 + ni * 4
for line in pathlib.Path(name, 'room.txt').read_text().splitlines():
if line.startswith('wall '):
fields = line.split()
assert len(fields) in (8, 9)
if len(fields) == 9:
assert fields[-1] == 'INVISIBLE_WALL_FACE'
print(name, nv, ni // 3, report['warnings'])
pathlib.Path('broken.fprs').write_bytes(b'FPRS\x01\0\0\0\0')
PY
./fp_roomscan --offline broken.fprs --out broken --json
echo "$?"
test ! -e broken/room.txt
test ! -e broken/room-mesh.bin
```
Expected: `{"ok":false,"error":"truncated frame header"}`, exit status 2, no `broken` outputs.
4. Record actual RoomView dumps with the maintainer's recorder (FPRS v1, mesh-to-STAGE poses, stable universe).
Copy the recording to `~/Downloads/capture.fprs`. Look around slowly for about a minute, including floor,
ceiling, every wall, and furniture from several directions. With the **recorded universe id** assigned below:
```sh
scan_universe=0 # replace 0 with the recorder's actual decimal universe id
/usr/bin/time -v ./fp_roomscan --offline "$HOME/Downloads/capture.fprs" --out real --universe "$scan_universe" --json > real.json 2> real-timing.log
cat real.json real-timing.log
cat real/room.txt
```
Measure wall offsets/yaws, ceiling and table top physically and compare to the records (target 5 cm / 2°).
Check resident memory below 300 MB, sensible coverage/warnings, tabletop labels and no false wall/furniture
blobs from smeared edges. Repeat with `--max-edge 0.35 --grazing 0.2` into `real-stricter` if edges smear;
try `--voxel 0.08 --plane-distance 0.07` into `real-coarser` if depth is noisy. Keep both JSON logs and raw
dumps **outside git** for threshold calibration. These are proposed tuning probes, not established device fixes.
5. Repeat the synthetic 1,800-frame benchmark from the preceding section on the Frame, using the same command
and `/usr/bin/time -v`. Then test a ten-minute independent real capture. Target: about 60 s elapsed or less,
less than 600 MB RSS, successful JSON and bounded mesh size. Record CPU time as well as wall time so storage
overhead is visible. With difficult stereo edges compare `--min-views 8 --patch-residual 0.02` in a fresh output
directory; short captures cap view support automatically. Include the JSON's exclusion counters in triage;
skipped frames may occur anywhere in the capture. Keep all personal results outside git.
6. After merging plumbing/emulation, publish this staged room through that stream's room-install path and try
Demeter, then an MRUK discovery/mesh game. Check room loading, furniture alignment, and warning/fallback
behaviour in-headset; the scanner alone cannot verify the Meta API path. Do not replace `rooms/current` by hand
during these standalone checks.
Undo standalone checks: remove only the isolated `~/.local/share/frameport/roomscan-test` directory and the five
test files copied to `~/Downloads` (plus `capture.fprs` if no longer needed). No Steam settings, games, agent,
room universe folder, or current-room symlink were changed by these steps.
+18
View File
@@ -108,3 +108,21 @@ shots:
steps:
- go: settings
- wait_for: "Portal (OLED)"
- name: room # experimental: the pretend Frame's made-up bedroom scan (fakes.fake_room), no real scan
docs: [README.md, docs/INSTALL.md]
viewport: [1280, 1690]
steps:
- hook: open_room
- wait_for: "Furniture & screens"
- park: true
- name: room-editing # a box picked in the list (its type, place and size) next to the clean-up's labels
docs: [docs/INSTALL.md]
viewport: [1280, 2000] # (tall enough that nothing scrolls)
steps:
- hook: open_room
- wait_for: "Furniture & screens"
- hook: {name: room_editing, args: [box]}
- park: true
crop: {anchor: "Furniture & screens", left: -26, top: -28, width: 670, height: 1262}
+15
View File
@@ -4,6 +4,14 @@ FrameBridge includes automatic metadata retirement for the verified Meta XR Audi
Wwise build; see [AUDIO_METADATA.md](../docs/AUDIO_METADATA.md) for its lifetime
invariant and regression coverage. It uses the existing adapter installation.
`adapter/swapchain_pool.c` implements default-off GLES `swapchain_pool` (#387): at eight live app runtime
swapchains, eligible single-layer colour images use private GL textures packed into up to two lazy 4096² sRGB
runtime atlases. EndFrame copies each referenced source once and rewrites the final quads (including cylinder
strips), cylinders and stereo projection views after existing layer fixes. OpenXR image ownership, GL bindings,
scissor state, source fences, rectangle reuse and session cleanup are covered by `tests/test_swapchain_pool.py`
(`FRAMEPORT_NATIVE_TESTS=1`, host compiler + cached OpenXR and GLES/EGL headers). Vulkan is unaffected. Host-tested;
4XVR submenu interaction, visual correctness and copy cost still need a headset check.
Sources of the prebuilt binaries in `../artifacts/` (committed, with `SHA256SUMS`). End users never compile anything;
developers rebuild with `python native/build.py` (downloads NDK r27c, OpenXR headers at pinned commits, d8 into
`native/.cache/`, git-ignored). The adapter, GL shim and platform compat rebuild byte-identically; the VrApi bridge
@@ -22,6 +30,7 @@ embeds debug paths (`-g`) and the dex depends on the d8 version, so those differ
| `zinkfix/` | `arm64-v8a/libVkLayer_fp_shaderfix.so` | Shader-fix Vulkan layer `VK_LAYER_FP_shader_fix` for OpenGL ES games (patch `frame.zink_shader_fix`): sits between Zink (Mesa's GL on Vulkan) and the driver and inserts the words of `zink_shader_fix` (same format as the Vulkan shim's `vk_shader_fix`: size + SHA-256 match) into SPIR-V modules; `zink_shader_dump=1` writes every distinct module to `files/fp_spirv/`. The engine library loads it first (DT_NEEDED); its constructor puts the layer in front of Android's GraphicsEnv debug layer list (private Android 11 `getDebugLayers`/`setDebugLayers`, found through libvulkan.so's dependencies) before EGL starts, so the loader picks it up from the app's lib dir. Exports only the enumeration functions and `VK_LAYER_FP_shader_fixGet*ProcAddr` (never plain `vk*` entry points). Vader Immortal's lightspeed shaders (Klownicle, GitHub #49). |
| `xrlayer/` | `linux-arm64/libxr_frameport_timefix.so` + `XR_APILAYER_FRAMEPORT_timefix.json` | OpenXR API layer for Windows PC VR games under Proton on the Frame (glibc aarch64, built freestanding with the NDK's clang), on by default (patch `pcvr.xr_timefix`). Retries `xrCreateInstance` as an OpenXR 1.0 app when the runtime rejects 1.1 (the Frame's SteamVR runtime does, and Proton 11's VR helper asks for 1.1). Also emulates `xrConvertTimespecTimeToTimeKHR`/`xrConvertTimeToTimespecTimeKHR` if the runtime refuses them (the Frame's Android runtime does; its Linux runtime, used by Proton, supports them as of 2026-09-29 — so this part is only a fallback; Proton's wineopenxr needs them for `XR_KHR_win32_convert_performance_counter_time`, which Revive requires) with the adapter's xrWaitFrame-calibrated offset, and drops the extension from xrCreateInstance if the runtime rejects it. Enabled per game by the Proton launch.sh (`XR_API_LAYER_PATH`, `XR_ENABLE_API_LAYERS`). `tests/test_xrlayer.py` builds it for x86_64 and drives it via ctypes. |
| `oculushmd/` | `win-x64/fp_oculushmd.exe` | PC VR counterpart of overport's `patch_oculus_unreal` (patch `pcvr.oculus_unreal`, default for Unreal Rift games on the Frame). Unreal's OculusHMD and LibOVR's `ovr_Detect` only start when the Windows named event `OculusHMDConnected` exists and is signalled (the Oculus service creates it). `fp_oculushmd.exe <command line>` creates it (manual reset, signalled), runs the command (Revive's injector) in a job object and keeps the event until every process in the job — the game and its children — has exited (fallback without job support: 3 minutes); returns the command's exit code and logs `FramePort oculushmd: …` to stderr (launch.log). Freestanding Windows x64 console exe (no CRT, no Windows SDK): the NDK's clang + `lld-link /Brepro` with a kernel32 import library generated by `llvm-dlltool`, so it's byte-reproducible. Tested on Windows from WSL with a build whose event name is `-DEVENT_NAME=L"…"` (the Oculus service owns the real event on a PC with the Oculus app). |
| `xrruntime/` | `arm64-v8a/libfp_xrruntime.so` | Runtime negotiation endpoint for embedded Khronos loaders (`frame.static_openxr_loader`). Loads FrameBridge by soname, forwards Android loader initialization through its GIPA, and returns FrameBridge GIPA. The unmodified inner Khronos loader is reused from OVRPort’s generic loader (also when `overport: False`); its paths remain system paths. Built with `--only xrruntime`. |
| `xrshim/` | `arm64-v8a/libframe_xrshim.so` | FrameBridge extension shim, only in builds with `controller_models=1`. overport's dispatcher (`libopenxr_loader.so`) answers `xrGetInstanceProcAddr` from a fixed table and refuses everything else (`overportOXR: Unknown proc addr: xrLoadRenderModelFB`), so the adapter never sees XR_FB_render_model lookups. Meta's OVRPlugin gets the function with `dlopen("libopenxr_loader.so")` + `dlsym`, so FramePort rewrites that `.rodata` string in `libOVRPlugin.so` (same length, in place) to `libframe_xrshim.so`. The shim returns the adapter's emulated functions (`framebridge_extension_proc`) and forwards every other name to overport. A DT_NEEDED interposer does not work here (dlsym on a handle searches that library first); tried and dropped. |
| `venc/` | `linux-arm64-bin/fp_venc` | Hardware H.264 encoder for the live view, run on the Frame (spec: `venc/SPEC.md`). Reads SteamVR's "headset view" webcam (v4l2loopback `/dev/video99`, RGB24), box-scales and converts it to NV12 (BT.709 limited range; `convert.c`, NEON with a bit-identical scalar reference, also built for x86_64 by the host test), and feeds the SoC's V4L2 stateful encoder (`dev-encoder.rst`, Qualcomm iris `/dev/video-enc0`): CBR, no B-frames, SPS/PPS on every IDR, one frame per slot of a fixed grid (fps = a divisor of the panel refresh read through DRM KMS, default cap 45), Annex B on stdout. stdin `k` = keyframe, `q`/EOF = drain and stop; `--probe` prints the setup as JSON, `--selftest` compares NEON and scalar output. Fully static and freestanding (raw syscalls, own `_start`; only the NDK's kernel UAPI headers, staged in `.cache/uapi-arm64`), byte-reproducible. Lives in `linux-arm64-bin/` because `linux-arm64/` is uploaded whole as the OpenXR layer. |
| `java-stubs/` | `dex/oculusos-stubs.dex` | No-op `com.oculus.os.AnalyticsEvent` / `UnifiedTelemetryLogger` for native code that looks them up. |
@@ -31,3 +40,9 @@ missing symbols (no compiler needed).
After changing anything here: `python native/build.py`, commit artifacts + SHA256SUMS, then run `frameport parity`
(the report marks updated binaries as "expected") and a device launch test.
`adapter.fixed_resolution` (off by default, #406/#394) keeps META recommended layer
resolution at tracked full swapchain size. Both adapter GIPA and xrshim's existing
extension entry point serve it; the setting adds OVRPlugin shim routing. The extension
stays visible, avoiding dependence on a plugin fallback. No adapter revision bump for
this opt-in fix. See DEVICE_NOTES "Fixed layer resolution / Batman" for verification.
+2 -1
View File
@@ -42,7 +42,8 @@ static XRAPI_ATTR XrResult XRAPI_CALL eye_hook_xrReleaseSwapchainImage(XrSwapcha
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
if (release_wait_active()) eye_wait_for_gpu();
XrResult result = fn(swapchain, info);
int pooled = pool_find(swapchain);
XrResult result = pooled >= 0 ? pool_cycle(pooled, 2, NULL) : fn(swapchain, info);
if (eye_debug) {
int64_t now = monotonic_ns();
pthread_mutex_lock(&eye_lock);
+22
View File
@@ -0,0 +1,22 @@
// SPDX-License-Identifier: GPL-3.0-only
// swapchain_size is the adapter's mutex-protected table, shared with rect_clamp.
static int fixed_layer_size(const XrCompositionLayerBaseHeader *layer, XrExtent2Di *full) {
if (!layer) return 0;
uint32_t w = 0, h = 0;
if (layer->type == XR_TYPE_COMPOSITION_LAYER_PROJECTION) {
const XrCompositionLayerProjection *p = (const XrCompositionLayerProjection *)layer;
if (!p->viewCount || !p->views) return 0;
for (uint32_t v = 0; v < p->viewCount; ++v) {
uint32_t vw, vh;
if (!swapchain_size(p->views[v].subImage.swapchain, &vw, &vh)) return 0;
if (v && (vw != w || vh != h)) return 0; // no single full size for this layer
w = vw; h = vh;
}
} else if (layer->type == XR_TYPE_COMPOSITION_LAYER_QUAD) {
const XrCompositionLayerQuad *q = (const XrCompositionLayerQuad *)layer;
if (!swapchain_size(q->subImage.swapchain, &w, &h)) return 0;
} else return 0; // preserve runtime behavior for other layer types
if (!w || !h || w > INT32_MAX || h > INT32_MAX) return 0;
full->width = (int32_t)w; full->height = (int32_t)h;
return 1;
}
+304 -35
View File
@@ -131,6 +131,8 @@ static int swapchain_fix = 1;
static int cube_quads = 1;
static int cube_standin = 1; // cube swapchains the runtime refuses: served by the adapter (cube_standin.c)
static int rect_clamp = 1;
static int fixed_resolution;
static int swapchain_pool = 0;
static int fb_color_space; // fb_color_space: offer XR_FB_color_space when the runtime lacks it (#336)
static int action_dedup = 1; // rename duplicate action names in a set (session_fixes.c; GitHub #323)
static int pose_consistency = 0; // repeat xrLocateViews queries for one display time get the same poses (GitHub #8) // clamp submitted image rects to their swapchain (SteamVR rejects a 1 px overrun)
@@ -147,6 +149,9 @@ static int flip_quads = 0;
static int flip_emul_setting = 1;
static int cylinder_strips_on = 1; // show cylinder layers as flat quad strips (cylinder_strips)
static int scene_emul; // scene emulation setting (per game)
static int scene_scan_request = 1;
static int scene_hide_bounds; // per game: the app's STAGE bounds query answers "no play area" (Demeter)
static int scene_events_via_shim; // the shim delivers room events: the layer's xrPollEvent leaves them alone
static float scene_height = 2.5f; // scene emulation ceiling height (m)
static float scene_width, scene_depth; // optional room size override (m)
static int64_t xr_time_offset; // XrTime minus CLOCK_MONOTONIC ns, measured at xrWaitFrame
@@ -207,7 +212,9 @@ static void read_settings(const char *path) {
if (sscanf(line, "swapchain_fix=%f", &value) == 1) swapchain_fix = value != 0;
if (sscanf(line, "cube_quads=%f", &value) == 1) cube_quads = value != 0;
if (sscanf(line, "cube_standin=%f", &value) == 1) cube_standin = value != 0;
if (sscanf(line, "fixed_resolution=%f", &value) == 1) fixed_resolution = value != 0;
if (sscanf(line, "rect_clamp=%f", &value) == 1) rect_clamp = value != 0;
if (sscanf(line, "swapchain_pool=%f", &value) == 1) swapchain_pool = value != 0;
if (sscanf(line, "fb_color_space=%f", &value) == 1) fb_color_space = value != 0;
if (sscanf(line, "action_dedup=%f", &value) == 1) action_dedup = value != 0;
if (sscanf(line, "pose_consistency=%f", &value) == 1) pose_consistency = value != 0;
@@ -224,6 +231,8 @@ static void read_settings(const char *path) {
if (sscanf(line, "flip_emul=%f", &value) == 1) flip_emul_setting = value != 0;
if (sscanf(line, "cylinder_strips=%f", &value) == 1) cylinder_strips_on = value != 0;
if (sscanf(line, "scene_emul=%f", &value) == 1) scene_emul = value != 0;
if (sscanf(line, "scene_scan_request=%f", &value) == 1) scene_scan_request = value != 0;
if (sscanf(line, "scene_hide_bounds=%f", &value) == 1) scene_hide_bounds = value != 0;
if (sscanf(line, "scene_height=%f", &value) == 1 && value > 1.5f && value < 5.0f) scene_height = value;
if (sscanf(line, "scene_width=%f", &value) == 1 && value > 0.5f && value < 20.0f) scene_width = value;
if (sscanf(line, "scene_depth=%f", &value) == 1 && value > 0.5f && value < 20.0f) scene_depth = value;
@@ -298,6 +307,7 @@ static void initialize(void) {
if (input_diag) LOG("per-game: input_diag=1 (controller-input diagnostics)");
if (proximity_emul) LOG("per-game: proximity_emul=%d (finger proximity from capacitive touch: %s)", proximity_emul,
proximity_emul > 1 ? "thumb + index" : "thumb");
if (fixed_resolution) LOG("per-game: fixed_resolution=1 (full swapchain recommendations)");
if (hide_fb_foveation) LOG("per-game: hide_fb_foveation=1 (Quest Vulkan foveation suppressed)");
if (hide_space_warp) LOG("per-game: hide_space_warp=1 (XR_FB_space_warp hidden, space warp info removed)");
if (strip_color_bias) LOG("per-game: strip_color_bias=%d (layer color scale/bias removed)", strip_color_bias);
@@ -388,8 +398,9 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateInstance(const XrInstanceCreateInfo *info
else if (!strcmp(ext, XR_FB_COLOR_SPACE_EXTENSION_NAME)) {
requested_color_space = 1;
} else if (scene_emul && is_scene_extension(ext)) {
if (!emulate_scene) LOG("emulating Meta scene/spatial-entity extensions from the guardian bounds");
if (!emulate_scene) LOG("emulating Meta scene/spatial-entity extensions from room file or guardian bounds");
emulate_scene = 1;
if (!strcmp(ext, "XR_META_spatial_entity_mesh")) scene_app_mesh = 1;
} else if (controller_models && render_models_available && !strcmp(ext, XR_FB_RENDER_MODEL_EXTENSION_NAME)) {
emulate_render_model = 1;
LOG("emulating XR_FB_render_model with Steam Frame controller models");
@@ -554,7 +565,7 @@ static int64_t pick_format(XrSession session, int64_t wanted) {
return chosen;
}
// Swapchains the runtime actually created; layers referencing anything else are dropped.
// Known app swapchains, including pooled images rewritten into runtime atlases before submission.
static pthread_mutex_t swapchains_lock = PTHREAD_MUTEX_INITIALIZER;
static XrSwapchain *swapchains;
static size_t swapchain_count, swapchain_capacity;
@@ -595,6 +606,29 @@ static int swapchain_size(XrSwapchain handle, uint32_t *w, uint32_t *h) {
return found;
}
#include "fixed_resolution.h"
static XRAPI_ATTR XrResult XRAPI_CALL fixed_recommended_resolution(XrSession session,
const XrRecommendedLayerResolutionGetInfoMETA *info, XrRecommendedLayerResolutionMETA *out) {
PFN_xrGetRecommendedLayerResolutionMETA fn = (PFN_xrGetRecommendedLayerResolutionMETA)
lookup(active_instance, "xrGetRecommendedLayerResolutionMETA");
if (!info || !out) return XR_ERROR_VALIDATION_FAILURE;
XrExtent2Di full;
if (!fixed_layer_size(info->layer, &full))
return fn ? fn(session, info, out) : XR_ERROR_FUNCTION_UNSUPPORTED;
// Read the runtime recommendation only for diagnostics. The override also works
// when the runtime lacks this function (OVRPort advertises it itself).
XrRecommendedLayerResolutionMETA old = {XR_TYPE_RECOMMENDED_LAYER_RESOLUTION_META};
XrResult result = fn ? fn(session, info, &old) : XR_ERROR_FUNCTION_UNSUPPORTED;
static unsigned logged;
if (logged++ < 5) LOG("fixed_resolution: recommended %dx%d -> %dx%d (runtime=%d)",
old.recommendedImageDimensions.width, old.recommendedImageDimensions.height,
full.width, full.height, result);
out->recommendedImageDimensions = full;
out->isValid = XR_TRUE;
return XR_SUCCESS;
}
// Clamp `r` to a w x h image; returns whether it changed (2 = it was empty and now covers the whole image).
static int clamp_rect(XrRect2Di *r, uint32_t w, uint32_t h) {
XrRect2Di c = *r;
@@ -619,6 +653,10 @@ static int clamp_rect(XrRect2Di *r, uint32_t w, uint32_t h) {
}
static void forget_swapchain(XrSwapchain handle) {
pthread_mutex_lock(&sizes_lock);
for (int i = 0; i < MAX_SIZES; ++i)
if (sizes[i].handle == handle) memset(&sizes[i], 0, sizeof sizes[i]);
pthread_mutex_unlock(&sizes_lock);
pthread_mutex_lock(&swapchains_lock);
for (size_t i = 0; i < swapchain_count; ++i)
if (swapchains[i] == handle) { swapchains[i] = swapchains[--swapchain_count]; break; }
@@ -644,6 +682,16 @@ static void cube_outputs_destroy(void);
static void cube_session_destroy(XrSession session);
static XrResult standin_enumerate(XrSwapchain handle, uint32_t capacity, uint32_t *count, XrSwapchainImageBaseHeader *images);
static int pool_find(XrSwapchain h);
static int pool_should_create(XrSession session);
static void pool_note_runtime(XrSession session, XrSwapchain handle);
static void pool_forget_runtime(XrSwapchain handle);
static int pool_create(XrSession session, const XrSwapchainCreateInfo *ci, XrSwapchain *out, int limited);
static void pool_destroy(int i);
static void pool_cleanup(XrSession session);
static XrResult pool_cycle(int i, int op, uint32_t *index);
static XrResult pool_enumerate(int i, uint32_t cap, uint32_t *count, XrSwapchainImageBaseHeader *images);
static int known_swapchain(XrSwapchain handle) {
int found = 0;
pthread_mutex_lock(&swapchains_lock);
@@ -731,6 +779,8 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
snap_on_create(*swapchain, &fixed);
return XR_SUCCESS;
}
// Start before the session's ~15-slot cap; leave headroom for 360/video outputs and a second atlas.
if (pool_should_create(session) && pool_create(session, &fixed, swapchain, 0)) return XR_SUCCESS;
clear_gl_errors("xrCreateSwapchain");
XrResult result = fn(session, &fixed, swapchain);
LOG("xrCreateSwapchain %ux%u format=%lld samples=%u array=%u faces=%u usage=0x%llx flags=0x%llx result=%d handle=%p",
@@ -738,6 +788,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
(unsigned long long)fixed.usageFlags, (unsigned long long)fixed.createFlags, result,
(void *)(uintptr_t)(XR_SUCCEEDED(result) && swapchain ? *swapchain : 0));
if (XR_SUCCEEDED(result)) {
pool_note_runtime(session, *swapchain);
remember_swapchain(*swapchain);
note_swapchain_size(*swapchain, fixed.width, fixed.height);
flip_on_create_swapchain(*swapchain, &fixed);
@@ -745,6 +796,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
snap_on_create(*swapchain, &fixed);
return result;
}
if (result == XR_ERROR_LIMIT_REACHED && pool_create(session, &fixed, swapchain, 1)) return XR_SUCCESS;
if (!swapchain_fix) return standin_after_failure(session, &fixed, swapchain, result);
// Frame's runtime rejects some GLES formats (GL_RGBA8) and MSAA swapchains (overport #71).
@@ -762,12 +814,14 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
}
}
if (XR_SUCCEEDED(result)) {
pool_note_runtime(session, *swapchain);
remember_swapchain(*swapchain);
note_swapchain_size(*swapchain, fixed.width, fixed.height);
flip_on_create_swapchain(*swapchain, &fixed);
emul_on_create_swapchain(*swapchain, &fixed);
snap_on_create(*swapchain, &fixed);
}
if (result == XR_ERROR_LIMIT_REACHED && pool_create(session, &fixed, swapchain, 1)) return XR_SUCCESS;
return standin_after_failure(session, &fixed, swapchain, result);
}
@@ -778,6 +832,9 @@ XRAPI_ATTR XrResult XRAPI_CALL xrDestroySwapchain(XrSwapchain swapchain) {
static int logged;
if (logged++ < 200) LOG("layer_debug: xrDestroySwapchain %p", (void *)swapchain);
}
pool_forget_runtime(swapchain);
int pooled = pool_find(swapchain);
if (pooled >= 0) { pool_destroy(pooled); return XR_SUCCESS; }
if (is_standin(swapchain)) { standin_destroy(swapchain); return XR_SUCCESS; }
surf_scene_destroy(swapchain);
forget_swapchain(swapchain);
@@ -834,7 +891,7 @@ static int cylinder_strips(const XrCompositionLayerCylinderKHR *c, XrComposition
}
// A layer is usable if every swapchain it references was created successfully.
// Swapchains the runtime can show: created by it (adapter-served 360 pictures only feed the adapter's own drawing).
// Runtime images and pooled images (rewritten after layer fixes). Adapter-served 360 pictures only feed its drawing.
static int runtime_swapchain(XrSwapchain handle) {
return known_swapchain(handle) && !(equirect_emul && emul_is_virtual(handle));
}
@@ -847,8 +904,12 @@ static int layer_usable(const XrCompositionLayerBaseHeader *layer) {
if (!runtime_swapchain(p->views[v].subImage.swapchain)) return 0;
return 1;
}
case XR_TYPE_COMPOSITION_LAYER_QUAD:
return runtime_swapchain(((const XrCompositionLayerQuad *)layer)->subImage.swapchain);
case XR_TYPE_COMPOSITION_LAYER_QUAD: {
const XrCompositionLayerQuad *q = (const XrCompositionLayerQuad *)layer;
if (q->size.width <= 0.0f || q->size.height <= 0.0f) return 0;
if (q->subImage.imageRect.extent.width <= 0 || q->subImage.imageRect.extent.height <= 0) return 0;
return runtime_swapchain(q->subImage.swapchain);
}
case XR_TYPE_COMPOSITION_LAYER_CYLINDER_KHR:
if (no_cylinder) return 0;
return runtime_swapchain(((const XrCompositionLayerCylinderKHR *)layer)->subImage.swapchain);
@@ -870,15 +931,44 @@ static int layer_usable(const XrCompositionLayerBaseHeader *layer) {
static uint64_t next_fake_handle = 0x7f000001;
#define FAKE_HANDLE(type) ((type)(uintptr_t)__atomic_fetch_add(&next_fake_handle, 1, __ATOMIC_RELAXED))
// The app's passthrough calls, logged (each kind at most 8 times) to see where an MR game stops (NOPE: black).
static void passthrough_log(const char *what, uint64_t handle, uint32_t flags, int purpose) {
static int counts[8];
static const char *const kinds[8] = {"create", "start", "pause", "destroy", "layer create", "layer resume",
"layer pause", "layer destroy"};
for (int k = 0; k < 8; ++k)
if (!strcmp(what, kinds[k]) && counts[k]++ < 8)
LOG("passthrough: %s 0x%llx flags=0x%x purpose=%d", what, (unsigned long long)handle, flags, purpose);
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_create_passthrough(XrSession s, const XrPassthroughCreateInfoFB *i, XrPassthroughFB *out) {
(void)s; (void)i; if (!out) return XR_ERROR_VALIDATION_FAILURE; *out = FAKE_HANDLE(XrPassthroughFB); return XR_SUCCESS;
(void)s; if (!out) return XR_ERROR_VALIDATION_FAILURE; *out = FAKE_HANDLE(XrPassthroughFB);
passthrough_log("create", (uint64_t)(uintptr_t)*out, i ? (uint32_t)i->flags : 0, -1);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_start(XrPassthroughFB p) {
passthrough_log("start", (uint64_t)(uintptr_t)p, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_pause(XrPassthroughFB p) {
passthrough_log("pause", (uint64_t)(uintptr_t)p, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_handle(XrPassthroughFB p) {
passthrough_log("destroy", (uint64_t)(uintptr_t)p, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_handle(XrPassthroughFB p) { (void)p; return XR_SUCCESS; }
static XRAPI_ATTR XrResult XRAPI_CALL emu_create_passthrough_layer(XrSession s, const XrPassthroughLayerCreateInfoFB *i,
XrPassthroughLayerFB *out) {
(void)s; (void)i; if (!out) return XR_ERROR_VALIDATION_FAILURE; *out = FAKE_HANDLE(XrPassthroughLayerFB); return XR_SUCCESS;
(void)s; if (!out) return XR_ERROR_VALIDATION_FAILURE; *out = FAKE_HANDLE(XrPassthroughLayerFB);
passthrough_log("layer create", (uint64_t)(uintptr_t)*out, i ? (uint32_t)i->flags : 0, i ? (int)i->purpose : -1);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_layer_resume(XrPassthroughLayerFB l) {
passthrough_log("layer resume", (uint64_t)(uintptr_t)l, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_layer_pause(XrPassthroughLayerFB l) {
passthrough_log("layer pause", (uint64_t)(uintptr_t)l, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_layer_handle(XrPassthroughLayerFB l) {
passthrough_log("layer destroy", (uint64_t)(uintptr_t)l, 0, -1); return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_layer_handle(XrPassthroughLayerFB l) { (void)l; return XR_SUCCESS; }
static XRAPI_ATTR XrResult XRAPI_CALL emu_passthrough_layer_style(XrPassthroughLayerFB l, const XrPassthroughStyleFB *st) {
(void)l; (void)st; return XR_SUCCESS;
}
@@ -914,12 +1004,12 @@ static PFN_xrVoidFunction passthrough_emulation(const char *name) {
static const struct { const char *name; PFN_xrVoidFunction fn; } table[] = {
{"xrCreatePassthroughFB", (PFN_xrVoidFunction)emu_create_passthrough},
{"xrDestroyPassthroughFB", (PFN_xrVoidFunction)emu_passthrough_handle},
{"xrPassthroughStartFB", (PFN_xrVoidFunction)emu_passthrough_handle},
{"xrPassthroughPauseFB", (PFN_xrVoidFunction)emu_passthrough_handle},
{"xrPassthroughStartFB", (PFN_xrVoidFunction)emu_passthrough_start},
{"xrPassthroughPauseFB", (PFN_xrVoidFunction)emu_passthrough_pause},
{"xrCreatePassthroughLayerFB", (PFN_xrVoidFunction)emu_create_passthrough_layer},
{"xrDestroyPassthroughLayerFB", (PFN_xrVoidFunction)emu_passthrough_layer_handle},
{"xrPassthroughLayerPauseFB", (PFN_xrVoidFunction)emu_passthrough_layer_handle},
{"xrPassthroughLayerResumeFB", (PFN_xrVoidFunction)emu_passthrough_layer_handle},
{"xrPassthroughLayerPauseFB", (PFN_xrVoidFunction)emu_passthrough_layer_pause},
{"xrPassthroughLayerResumeFB", (PFN_xrVoidFunction)emu_passthrough_layer_resume},
{"xrPassthroughLayerSetStyleFB", (PFN_xrVoidFunction)emu_passthrough_layer_style},
{"xrCreateGeometryInstanceFB", (PFN_xrVoidFunction)emu_create_geometry},
{"xrDestroyGeometryInstanceFB", (PFN_xrVoidFunction)emu_geometry_handle},
@@ -957,6 +1047,13 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetSystemProperties(XrInstance instance, XrSyst
((XrSystemPassthroughPropertiesFB *)p)->supportsPassthrough = XR_TRUE;
if (p->type == XR_TYPE_SYSTEM_SPATIAL_ENTITY_PROPERTIES_FB && emulate_scene)
((XrSystemSpatialEntityPropertiesFB *)p)->supportsSpatialEntity = XR_TRUE;
// META scene capability chains used by current MRUK.
if (p->type == XR_TYPE_SYSTEM_SPACE_DISCOVERY_PROPERTIES_META && emulate_scene)
((XrSystemSpaceDiscoveryPropertiesMETA *)p)->supportsSpaceDiscovery = XR_TRUE;
if (p->type == XR_TYPE_SYSTEM_SPACE_PERSISTENCE_PROPERTIES_META && emulate_scene)
((XrSystemSpacePersistencePropertiesMETA *)p)->supportsSpacePersistence = XR_TRUE;
if (p->type == XR_TYPE_SYSTEM_BOUNDARY_VISIBILITY_PROPERTIES_META && emulate_scene)
((XrSystemBoundaryVisibilityPropertiesMETA *)p)->supportsBoundaryVisibility = XR_TRUE;
if (p->type == XR_TYPE_SYSTEM_PASSTHROUGH_PROPERTIES2_FB)
((XrSystemPassthroughProperties2FB *)p)->capabilities = XR_PASSTHROUGH_CAPABILITY_BIT_FB;
if (p->type == XR_TYPE_SYSTEM_RENDER_MODEL_PROPERTIES_FB && emulate_render_model)
@@ -1004,7 +1101,22 @@ XRAPI_ATTR XrResult XRAPI_CALL xrLocateSpace(XrSpace space, XrSpace baseSpace, X
pose_time_note("xrLocateSpace", (uint64_t)(uintptr_t)space, (uint64_t)(uintptr_t)baseSpace, time);
if (emulate_scene && location) {
XrPosef pose; XrSpaceLocationFlags flags;
if (locate_fake(space, baseSpace, time, &pose, &flags)) {
pthread_mutex_lock(&scene_lock);
int fake = locate_fake(space, baseSpace, time, &pose, &flags);
int entity = fake ? fake_space_index(space) : -1, base_entity = fake ? fake_space_index(baseSpace) : -1;
pthread_mutex_unlock(&scene_lock);
if (fake) {
// MRUK keeps an anchor pending (no load Success, ever) unless both valid bits are set (Pinball FX)
static int logged, logged_invalid;
const XrSpaceLocationFlags need = XR_SPACE_LOCATION_ORIENTATION_VALID_BIT | XR_SPACE_LOCATION_POSITION_VALID_BIT;
int invalid = (flags & need) != need;
if (logged < 30 || (invalid && logged_invalid < 60)) {
LOG("scene: locate entity %d in %s %d (space 0x%llx) at %lld: flags 0x%llx%s", entity,
base_entity >= 0 ? "entity" : "real space", base_entity, (unsigned long long)(uintptr_t)baseSpace,
(long long)time, (unsigned long long)flags, invalid ? " NOT VALID" : "");
++logged;
if (invalid) ++logged_invalid;
}
location->pose = pose;
location->locationFlags = flags;
return XR_SUCCESS;
@@ -1026,8 +1138,8 @@ static XrResult locate_spaces_common(const char *name, XrSession session, const
if (info) pose_time_note(name, info->spaceCount ? (uint64_t)(uintptr_t)info->spaces[0] : 0,
(uint64_t)(uintptr_t)info->baseSpace, info->time);
if (emulate_scene && info && locations) {
any_fake = fake_space_index(info->baseSpace) >= 0;
for (uint32_t i = 0; i < info->spaceCount && !any_fake; ++i) any_fake = fake_space_index(info->spaces[i]) >= 0;
any_fake = is_fake_space(info->baseSpace);
for (uint32_t i = 0; i < info->spaceCount && !any_fake; ++i) any_fake = is_fake_space(info->spaces[i]);
}
if (!any_fake) {
PFN_xrLocateSpaces fn = (PFN_xrLocateSpaces)lookup(active_instance, name);
@@ -1050,7 +1162,12 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_locate_spaces_khr(XrSession session, c
}
XRAPI_ATTR XrResult XRAPI_CALL xrDestroySpace(XrSpace space) {
if (fake_space_index(space) >= 0) return XR_SUCCESS; // entity spaces live for the whole session
if (is_fake_space(space)) {
pthread_mutex_lock(&scene_lock);
int valid = fake_space_index(space) >= 0;
pthread_mutex_unlock(&scene_lock);
return valid ? XR_SUCCESS : XR_ERROR_HANDLE_INVALID; // entity spaces live for the whole session
}
PFN_xrDestroySpace fn = (PFN_xrDestroySpace)lookup(active_instance, "xrDestroySpace");
return fn ? fn(space) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
@@ -1058,6 +1175,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrDestroySpace(XrSpace space) {
#include "input_diag.c"
#include "session_fixes.c"
#include "layer_emul_gl.c"
#include "swapchain_pool.c"
#include "cube_standin.c"
#include "snapshot_gl.c"
#include "surface_swapchain.c"
@@ -1244,6 +1362,10 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSession(XrInstance instance, const XrSess
scene_on_create_session(*session);
session_fixes_on_create_session(*session);
emul_on_create_session(*session, info);
if (swapchain_pool) {
for (const XrBaseInStructure *p = info->next; p; p = p->next)
if (p->type == XR_TYPE_GRAPHICS_BINDING_OPENGL_ES_ANDROID_KHR) pool_session = *session;
}
fade_on_create_session(*session, info);
}
return result;
@@ -1312,6 +1434,12 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateSwapchainImages(XrSwapchain swapchain,
XrSwapchainImageBaseHeader *images) {
PFN_xrEnumerateSwapchainImages fn = (PFN_xrEnumerateSwapchainImages)lookup(active_instance, "xrEnumerateSwapchainImages");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (pooled >= 0) {
XrResult r = pool_enumerate(pooled, capacity, count, images);
if (XR_SUCCEEDED(r) && capacity && images) emul_on_enumerate(swapchain, 1, images);
return r;
}
if (is_standin(swapchain)) return standin_enumerate(swapchain, capacity, count, images);
XrResult result = equirect_emul && emul_is_virtual(swapchain) ? emul_virtual_enumerate(swapchain, capacity, count, images)
: hide_fb_foveation_vulkan ? fb_hide_foveation_images(fn, swapchain, capacity, count, images)
@@ -1328,6 +1456,8 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateSwapchainImages(XrSwapchain swapchain,
XRAPI_ATTR XrResult XRAPI_CALL xrWaitSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageWaitInfo *info) {
PFN_xrWaitSwapchainImage fn = (PFN_xrWaitSwapchainImage)lookup(active_instance, "xrWaitSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (pooled >= 0) return pool_cycle(pooled, 1, NULL);
if (is_standin(swapchain)) return XR_SUCCESS;
XrResult result = fn(swapchain, info);
__atomic_add_fetch(&sc_waits, 1, __ATOMIC_RELAXED);
@@ -1338,6 +1468,8 @@ XRAPI_ATTR XrResult XRAPI_CALL xrWaitSwapchainImage(XrSwapchain swapchain, const
XRAPI_ATTR XrResult XRAPI_CALL xrReleaseSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageReleaseInfo *info) {
PFN_xrReleaseSwapchainImage fn = (PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (pooled >= 0) return pool_cycle(pooled, 2, NULL);
if (is_standin(swapchain)) return XR_SUCCESS;
__atomic_add_fetch(&sc_releases, 1, __ATOMIC_RELAXED);
return fn(swapchain, info);
@@ -1347,6 +1479,12 @@ XRAPI_ATTR XrResult XRAPI_CALL xrAcquireSwapchainImage(XrSwapchain swapchain, co
uint32_t *index) {
PFN_xrAcquireSwapchainImage fn = (PFN_xrAcquireSwapchainImage)lookup(active_instance, "xrAcquireSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (pooled >= 0) {
XrResult r = pool_cycle(pooled, 0, index);
if (XR_SUCCEEDED(r)) emul_on_acquire(swapchain, *index);
return r;
}
if (is_standin(swapchain)) { // adapter-served cube stand-in: one image
if (!index) return XR_ERROR_VALIDATION_FAILURE;
*index = 0;
@@ -1375,7 +1513,8 @@ static XRAPI_ATTR XrResult XRAPI_CALL surf_composite_release(XrSwapchain handle,
tracked_swapchain *t=find_tracked(handle,0);
if(t && t->acquired_count && t->waited_count)surf_scene_capture(t,t->acquired[0]);
pthread_mutex_unlock(&flip_lock);
XrResult result=equirect_emul?hook_xrReleaseSwapchainImage(handle,info):
XrResult result=pool_find(handle)>=0 && !equirect_emul ? xrReleaseSwapchainImage(handle,info) :
equirect_emul?hook_xrReleaseSwapchainImage(handle,info):
((PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage"))(handle,info);
if(XR_SUCCEEDED(result)) {
surf_scene_release_commit(handle);
@@ -1389,7 +1528,8 @@ static XRAPI_ATTR XrResult XRAPI_CALL surf_composite_release(XrSwapchain handle,
return result;
}
static XRAPI_ATTR XrResult XRAPI_CALL surf_composite_wait(XrSwapchain handle,const XrSwapchainImageWaitInfo *info) {
XrResult result=equirect_emul?hook_xrWaitSwapchainImage(handle,info):
XrResult result=pool_find(handle)>=0 && !equirect_emul ? xrWaitSwapchainImage(handle,info) :
equirect_emul?hook_xrWaitSwapchainImage(handle,info):
((PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage"))(handle,info);
if(result==XR_SUCCESS) {
pthread_mutex_lock(&flip_lock);tracked_swapchain *t=find_tracked(handle,0);
@@ -1412,7 +1552,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrPollEvent(XrInstance instance, XrEventDataBuffe
mx_audio_initialize(); mx_batman_geometry_initialize();
PFN_xrPollEvent fn = (PFN_xrPollEvent)lookup(instance, "xrPollEvent");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (emulate_scene && event) {
if (emulate_scene && event && !scene_events_via_shim) {
pthread_mutex_lock(&scene_lock);
int got = pop_event(event);
pthread_mutex_unlock(&scene_lock);
@@ -1703,7 +1843,8 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
frames = 0; drift_sum = drift_max = 0;
}
}
if ((!cube_standin && !layer_fix && !swap_eyes && !emulate_passthrough && !flip_quads && !flip_emul && !strip_depth && !fade_quad) || !info || !info->layerCount ||
if (swapchain_pool && info && info->layerCount > 64) return XR_ERROR_LAYER_LIMIT_EXCEEDED;
if ((!swapchain_pool && !cube_standin && !layer_fix && !swap_eyes && !emulate_passthrough && !flip_quads && !flip_emul && !strip_depth && !fade_quad) || !info || !info->layerCount ||
info->layerCount > 64)
return fn(session, info);
if (layer_debug) { // the submitted layer list (order, flags, eyes, chained structs) whenever its shape changes
@@ -1750,6 +1891,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
XrCompositionLayerQuad quads[64];
XrCompositionLayerQuad fade_out; // fade_quad: at most one fade layer per frame
int faded = 0;
const XrCompositionLayerBaseHeader *own_fade = NULL; // fade_quad's black quad (FrameBridge's own)
cube_outputs_begin();
uint32_t count = 0, dropped = 0, swapped = 0, passthrough = 0;
const XrCompositionLayerBaseHeader *surface_projections[SURF_MAX];
@@ -2055,7 +2197,37 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
// the app's own layer: fade_quad=2 removed it from the copy submitted)
int level = fade_quad && !faded && count < 64 ? fade_level(info->layers[i]) : 0;
const XrCompositionLayerBaseHeader *fade_l = level ? fade_layer(session, level, count, &fade_out) : NULL;
if (fade_l) { kept[count++] = fade_l; faded = 1; ++swapped; }
if (fade_l) { kept[count++] = fade_l; own_fade = fade_l; faded = 1; ++swapped; }
}
// The runtime refuses frames with more layers than maxLayerCount (XR_ERROR_LAYER_LIMIT_EXCEEDED: Punch Fit sends
// 21+). Keep every projection layer and FrameBridge's own fade quad, then the app's other layers in submission
// order until the limit (from PR #246; the earliest app layers were what Punch Fit needed).
uint32_t max_layers = runtime_max_layers ? runtime_max_layers : 16, trimmed_layers = 0;
if (count > max_layers) {
const XrCompositionLayerBaseHeader *trimmed[64];
uint32_t tcount = 0, reserved = 0, res_kept = 0, other_kept = 0;
for (uint32_t i = 0; i < count; ++i)
if (kept[i]->type == XR_TYPE_COMPOSITION_LAYER_PROJECTION || kept[i] == own_fade) ++reserved;
if (reserved > max_layers) reserved = max_layers;
uint32_t budget = max_layers - reserved;
for (uint32_t i = 0; i < count; ++i) {
int keep_always = kept[i]->type == XR_TYPE_COMPOSITION_LAYER_PROJECTION || kept[i] == own_fade;
uint32_t *n = keep_always ? &res_kept : &other_kept;
if (*n < (keep_always ? reserved : budget)) { trimmed[tcount++] = kept[i]; ++*n; }
else ++trimmed_layers;
}
static int layer_limit_logged;
if (layer_limit_logged++ < 5)
LOG("xrEndFrame: %u layers exceed the runtime limit %u: left out %u of the app's other layers "
"(kept %u projection/fade layers and %u others)", count, max_layers, trimmed_layers, res_kept,
other_kept);
for (uint32_t i = 0; i < tcount; ++i) kept[i] = trimmed[i];
count = tcount;
}
PoolFrame pool_frame;
if (swapchain_pool && session == pool_session) {
count = pool_rewrite_layers(kept, count, &pool_frame);
++swapped;
}
XrFrameEndInfo fixed = *info;
fixed.layers = kept;
@@ -2064,11 +2236,35 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
static int warned, failures, passthrough_logged;
if (dropped && warned++ < 5) LOG("xrEndFrame: dropped %u unusable layer(s)", dropped);
if (passthrough && !passthrough_logged++) LOG("xrEndFrame: passthrough layer -> ALPHA_BLEND environment");
static int views_logged;
if (views_logged < 3) { // describe the stereo layer layout a few times to debug eye/stereo problems
for (uint32_t i = 0; i < info->layerCount; ++i) {
const XrCompositionLayerBaseHeader *l = info->layers[i];
if (l && l->type == XR_TYPE_COMPOSITION_LAYER_QUAD) {
static struct { XrSwapchain swapchain; XrRect2Di rect; uint32_t array; } last_rect[2];
static double last_rect_log[2];
struct timespec rect_now;
clock_gettime(CLOCK_MONOTONIC, &rect_now);
double rect_seconds = rect_now.tv_sec + rect_now.tv_nsec / 1e9;
for (uint32_t i = 0; i < info->layerCount; ++i) {
const XrCompositionLayerBaseHeader *l = info->layers[i];
if (!l || l->type != XR_TYPE_COMPOSITION_LAYER_PROJECTION) continue;
const XrCompositionLayerProjection *p = (const XrCompositionLayerProjection *)l;
for (uint32_t v = 0; v < p->viewCount && v < 2; ++v) {
const XrSwapchainSubImage *sub = &p->views[v].subImage;
if ((last_rect[v].swapchain != sub->swapchain || last_rect[v].array != sub->imageArrayIndex ||
memcmp(&last_rect[v].rect, &sub->imageRect, sizeof(sub->imageRect))) &&
rect_seconds - last_rect_log[v] >= 1.0) {
LOG("projection view %u rect change: swapchain=%p rect=%d,%d %dx%d array=%u", v,
(void *)(uintptr_t)sub->swapchain, sub->imageRect.offset.x, sub->imageRect.offset.y,
sub->imageRect.extent.width, sub->imageRect.extent.height, sub->imageArrayIndex);
last_rect[v].swapchain = sub->swapchain;
last_rect[v].rect = sub->imageRect;
last_rect[v].array = sub->imageArrayIndex;
last_rect_log[v] = rect_seconds;
}
}
}
static int views_logged, quads_logged;
if (views_logged < 3 || quads_logged < 10) { // describe the stereo layer layout and quads
for (uint32_t i = 0; i < fixed.layerCount; ++i) {
const XrCompositionLayerBaseHeader *l = fixed.layers[i];
if (l && l->type == XR_TYPE_COMPOSITION_LAYER_QUAD && quads_logged < 10) {
const XrCompositionLayerQuad *qd = (const XrCompositionLayerQuad *)l;
char chain[160] = "";
size_t used = 0;
@@ -2079,10 +2275,12 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
(unsigned long long)((const XrCompositionLayerImageLayoutFB *)n)->flags);
used += w;
}
LOG("quad layer: flags=0x%llx size=%.2fx%.2f next:%s", (unsigned long long)qd->layerFlags,
qd->size.width, qd->size.height, chain);
LOG("quad layer %u: swapchain=%p flags=0x%llx size=%.2fx%.2f pos=%.2f,%.2f,%.2f next:%s", i,
(void *)(uintptr_t)qd->subImage.swapchain, (unsigned long long)qd->layerFlags,
qd->size.width, qd->size.height, qd->pose.position.x, qd->pose.position.y, qd->pose.position.z, chain);
++quads_logged;
}
if (!l || l->type != XR_TYPE_COMPOSITION_LAYER_PROJECTION) continue;
if (!l || l->type != XR_TYPE_COMPOSITION_LAYER_PROJECTION || views_logged >= 3) continue;
const XrCompositionLayerProjection *p = (const XrCompositionLayerProjection *)l;
for (uint32_t v = 0; v < p->viewCount; ++v) {
const XrCompositionLayerProjectionView *pv = &p->views[v];
@@ -2247,28 +2445,94 @@ static XRAPI_ATTR XrResult XRAPI_CALL fb_hide_update_swapchain(XrSwapchain swapc
return fn ? fn(swapchain, state) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
// Generic calls the shim routes here for the emulated room (OVRPort's dispatcher refuses fake handles' functions):
// only fake room spaces and FramePort's own room events are handled locally; every real call goes on to OVRPort's
// dispatcher, whose own work (headset identity, events, controller spaces) was skipped for every scene game before.
static PFN_xrVoidFunction dispatcher_proc(const char *name) {
static PFN_xrGetInstanceProcAddr gipa;
static int tried;
if (!tried) {
tried = 1;
void *h = dlopen("libopenxr_loader.so", RTLD_NOW | RTLD_NOLOAD); // OVRPort's dispatcher (xrshim's next)
if (h) gipa = (PFN_xrGetInstanceProcAddr)dlsym(h, "xrGetInstanceProcAddr");
LOG("scene: shim calls for real handles go to OVRPort's dispatcher (%s)", gipa ? "found" : "MISSING");
}
PFN_xrVoidFunction fn = NULL;
if (gipa && active_instance) gipa(active_instance, name, &fn);
return fn;
}
static XRAPI_ATTR XrResult XRAPI_CALL shim_locate_space(XrSpace space, XrSpace base, XrTime time, XrSpaceLocation *loc) {
PFN_xrLocateSpace next = is_fake_space(space) || is_fake_space(base) ? NULL
: (PFN_xrLocateSpace)dispatcher_proc("xrLocateSpace");
return next ? next(space, base, time, loc) : xrLocateSpace(space, base, time, loc);
}
static XrResult shim_locate_spaces_common(const char *name, XrSession session, const XrSpacesLocateInfo *info,
XrSpaceLocations *locations) {
int any_fake = info && is_fake_space(info->baseSpace);
for (uint32_t i = 0; info && i < info->spaceCount && !any_fake; ++i) any_fake = is_fake_space(info->spaces[i]);
PFN_xrLocateSpaces next = any_fake ? NULL : (PFN_xrLocateSpaces)dispatcher_proc(name);
return next ? next(session, info, locations) : locate_spaces_common(name, session, info, locations);
}
static XRAPI_ATTR XrResult XRAPI_CALL shim_locate_spaces(XrSession s, const XrSpacesLocateInfo *i, XrSpaceLocations *l) {
return shim_locate_spaces_common("xrLocateSpaces", s, i, l);
}
static XRAPI_ATTR XrResult XRAPI_CALL shim_locate_spaces_khr(XrSession s, const XrSpacesLocateInfo *i, XrSpaceLocations *l) {
return shim_locate_spaces_common("xrLocateSpacesKHR", s, i, l);
}
static XRAPI_ATTR XrResult XRAPI_CALL shim_destroy_space(XrSpace space) {
PFN_xrDestroySpace next = is_fake_space(space) ? NULL : (PFN_xrDestroySpace)dispatcher_proc("xrDestroySpace");
return next ? next(space) : xrDestroySpace(space);
}
static XRAPI_ATTR XrResult XRAPI_CALL shim_poll_event(XrInstance instance, XrEventDataBuffer *event) {
scene_events_via_shim = 1;
if (event) { // room events first: OVRPort's dispatcher might drop event types it doesn't know
pthread_mutex_lock(&scene_lock);
int got = pop_event(event);
pthread_mutex_unlock(&scene_lock);
if (got) return XR_SUCCESS;
}
PFN_xrPollEvent next = (PFN_xrPollEvent)dispatcher_proc("xrPollEvent");
return next ? next(instance, event) : xrPollEvent(instance, event);
}
// For native/xrshim: emulated functions that overport's dispatcher doesn't know (it never asks us for them).
__attribute__((visibility("default"))) PFN_xrVoidFunction framebridge_extension_proc(const char *name) {
pthread_once(&init_once, initialize);
if (name && !strcmp(name, "framebridge_hide_fb_foveation"))
if (!name) return NULL;
if (fixed_resolution && !strcmp(name, "xrGetRecommendedLayerResolutionMETA"))
return (PFN_xrVoidFunction)fixed_recommended_resolution;
if (!strcmp(name, "framebridge_hide_fb_foveation"))
return (PFN_xrVoidFunction)framebridge_hide_fb_foveation;
if (hide_fb_foveation_vulkan && name) {
if (hide_fb_foveation_vulkan) {
if (!strcmp(name, "xrCreateFoveationProfileFB")) return (PFN_xrVoidFunction)xrCreateFoveationProfileFB;
if (!strcmp(name, "xrUpdateSwapchainFB")) return (PFN_xrVoidFunction)fb_hide_update_swapchain;
}
if (name && !foveation_fix && (foveation_storage_fix || foveation_eye)) {
if (!foveation_fix && (foveation_storage_fix || foveation_eye)) {
if (!strcmp(name, "xrCreateFoveationProfileFB")) return (PFN_xrVoidFunction)xrCreateFoveationProfileFB;
if (!strcmp(name, "xrGetFoveationEyeTrackedStateMETA")) return (PFN_xrVoidFunction)xrGetFoveationEyeTrackedStateMETA;
if (!strcmp(name, "xrDestroyFoveationProfileFB") || !strcmp(name, "xrUpdateSwapchainFB") ||
!strcmp(name, "xrGetSwapchainStateFB")) return lookup(active_instance, name);
}
if(surface_native && name){
if (surface_native) {
#define NATIVE_PROC(fn) if(!strcmp(name,#fn))return (PFN_xrVoidFunction)fn;
NATIVE_PROC(xrGetVulkanGraphicsDeviceKHR)
NATIVE_PROC(xrGetVulkanDeviceExtensionsKHR)
NATIVE_PROC(xrCreateVulkanDeviceKHR)
NATIVE_PROC(xrCreateVulkanInstanceKHR)
#undef NATIVE_PROC
}
// The shim bypasses OVRPort's fixed dispatch table for all scene calls, including consumers of fake spaces.
if (name && emulate_scene) {
PFN_xrVoidFunction scene = scene_emulation(name);
if (scene) return scene;
// xrGetSystemProperties stays with OVRPort (its headset identity; it calls down to our hook for the scene flags)
#define SCENE_PROC(fn, impl) if (!strcmp(name, #fn)) return (PFN_xrVoidFunction)impl;
SCENE_PROC(xrPollEvent, shim_poll_event)
SCENE_PROC(xrLocateSpace, shim_locate_space)
SCENE_PROC(xrLocateSpaces, shim_locate_spaces)
SCENE_PROC(xrLocateSpacesKHR, shim_locate_spaces_khr)
SCENE_PROC(xrDestroySpace, shim_destroy_space)
#undef SCENE_PROC
}
PFN_xrVoidFunction color = name ? color_space_emulation(name) : NULL;
return color ? color : (name ? render_model_emulation(name) : NULL);
@@ -2302,6 +2566,10 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance, const
*function = (PFN_xrVoidFunction)fb_hide_update_swapchain;
return XR_SUCCESS;
}
if (fixed_resolution && !strcmp(name, "xrGetRecommendedLayerResolutionMETA")) {
*function = (PFN_xrVoidFunction)fixed_recommended_resolution;
return XR_SUCCESS;
}
HOOK(xrGetSystemProperties)
HOOK(xrApplyHapticFeedback)
HOOK(xrCreateReferenceSpace)
@@ -2335,6 +2603,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance, const
HOOK_AS(xrReleaseSwapchainImage, surf_composite_release)
HOOK_AS(xrWaitSwapchainImage, surf_composite_wait)
}
if (swapchain_pool) { HOOK_AS(xrDestroySession, hook_xrDestroySession) }
if (cube_standin) {
HOOK_AS(xrEndSession, cube_end_session)
HOOK_AS(xrDestroySession, hook_xrDestroySession)
@@ -2353,7 +2622,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance, const
if (instance != XR_NULL_HANDLE) HOOK_AS(xrGetDisplayRefreshRateFB, hook_get_refresh_rate)
if (surface_emul) HOOK_AS(xrCreateSwapchainAndroidSurfaceKHR, hook_xrCreateSwapchainAndroidSurfaceKHR)
if (eye_debug || release_wait) HOOK_AS(xrReleaseSwapchainImage, eye_hook_xrReleaseSwapchainImage)
if (layer_debug || cube_standin) { // counters + cube stand-ins; every other wait/release hook above takes precedence
if (swapchain_pool || layer_debug || cube_standin) { // pooled images, counters + cube stand-ins; every other wait/release hook above takes precedence
HOOK(xrWaitSwapchainImage)
HOOK(xrReleaseSwapchainImage)
}
+7 -2
View File
@@ -304,6 +304,7 @@ static void emul_on_destroy_swapchain(XrSwapchain handle) {
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrReleaseSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageReleaseInfo *info) {
PFN_xrReleaseSwapchainImage fn = (PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (is_standin(swapchain)) return XR_SUCCESS;
pthread_mutex_lock(&emul_lock);
emul_source *s = emul_active() ? emul_find_source(swapchain, 0) : NULL;
@@ -314,7 +315,8 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrReleaseSwapchainImage(XrSwapchain s
if (fence) p_glFlush();
}
pthread_mutex_unlock(&emul_lock);
XrResult result = emul_is_virtual(swapchain) ? XR_SUCCESS : fn(swapchain, info);
XrResult result = pooled >= 0 ? pool_cycle(pooled, 2, NULL) :
emul_is_virtual(swapchain) ? XR_SUCCESS : fn(swapchain, info);
pthread_mutex_lock(&emul_lock);
s = emul_active() ? emul_find_source(swapchain, 0) : NULL;
if (s && XR_SUCCEEDED(result) && s->acquired_count) {
@@ -336,8 +338,10 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrReleaseSwapchainImage(XrSwapchain s
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrWaitSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageWaitInfo *info) {
PFN_xrWaitSwapchainImage fn = (PFN_xrWaitSwapchainImage)lookup(active_instance, "xrWaitSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
int pooled = pool_find(swapchain);
if (is_standin(swapchain)) return XR_SUCCESS;
XrResult result = emul_is_virtual(swapchain) ? XR_SUCCESS : fn(swapchain, info);
XrResult result = pooled >= 0 ? pool_cycle(pooled, 1, NULL) :
emul_is_virtual(swapchain) ? XR_SUCCESS : fn(swapchain, info);
pthread_mutex_lock(&emul_lock);
emul_source *s = emul_active() ? emul_find_source(swapchain, 0) : NULL;
if (s && s->reading >= 0 && s->reading == s->last_acquired) {
@@ -1005,6 +1009,7 @@ static void emul_on_destroy_session(XrSession session) {
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrDestroySession(XrSession session) {
cube_session_destroy(session);
emul_on_destroy_session(session);
pool_cleanup(session);
PFN_xrDestroySession fn = (PFN_xrDestroySession)lookup(active_instance, "xrDestroySession");
return fn ? fn(session) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
+26
View File
@@ -0,0 +1,26 @@
// First-fit edge packer. Removing a live rectangle immediately frees its space.
#pragma once
#include <stdint.h>
typedef struct {
int x, y, w, h;
} PoolRect;
static int pool_place(const PoolRect *used, int n, int side, int w, int h, PoolRect *out) {
if (w <= 0 || h <= 0 || w > side || h > side) return 0;
for (int yi = -1; yi < n; ++yi)
for (int xi = -1; xi < n; ++xi) {
int x = xi < 0 ? 0 : used[xi].x + used[xi].w;
int y = yi < 0 ? 0 : used[yi].y + used[yi].h;
if (x > side - w || y > side - h) continue;
int fits = 1;
for (int i = 0; i < n; ++i)
if (x < used[i].x + used[i].w && x + w > used[i].x && y < used[i].y + used[i].h && y + h > used[i].y) {
fits = 0;
break;
}
if (fits) {
*out = (PoolRect){x, y, w, h};
return 1;
}
}
return 0;
}
+115
View File
@@ -0,0 +1,115 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by scene_emu.c. Persist only application anchors; scanned entities remain owned by the room file.
#include <sys/random.h>
#include <sys/stat.h>
#include <unistd.h>
#ifndef SCENE_EXTERNAL_ROOT
#define SCENE_EXTERNAL_ROOT "/sdcard/Android/data"
#endif
static int anchor_path(char *path, size_t size) {
char process[256] = {0};
FILE *f = fopen("/proc/self/cmdline", "r");
if (!f) return 0;
size_t n = fread(process, 1, sizeof(process)-1, f);
fclose(f);
if (!n) return 0;
char *colon = strchr(process, ':');
if (colon) *colon = 0;
// Only package-like process names: never let a command line become a path traversal.
if (!strchr(process, '.') || !strcmp(process, ".") || !strcmp(process, "..")) return 0;
for (const char *p = process; *p; ++p)
if (!((*p >= 'a' && *p <= 'z') || (*p >= 'A' && *p <= 'Z') || (*p >= '0' && *p <= '9') || *p == '.' || *p == '_')) return 0;
int len = snprintf(path, size, "%s/%s/files/framebridge-anchors.txt", SCENE_EXTERNAL_ROOT, process);
return len > 0 && (size_t)len < size;
}
static int anchor_uuid(XrUuidEXT *u) {
size_t have = 0;
while (have < sizeof(u->data)) {
ssize_t n = getrandom(u->data + have, sizeof(u->data) - have, 0);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) break;
have += (size_t)n;
}
if (have < sizeof(u->data)) {
FILE *f = fopen("/dev/urandom", "rb");
if (!f) return 0;
have = fread(u->data, 1, sizeof(u->data), f);
fclose(f);
if (have != sizeof(u->data)) return 0;
}
u->data[6] = (uint8_t)((u->data[6] & 15) | 0x40);
u->data[8] = (uint8_t)((u->data[8] & 63) | 0x80);
for (int i = 0; i < entity_count; ++i) if (uuid_equal(u, &entities[i].uuid)) return 0;
return 1;
}
static int write_anchors(void) {
char path[4096], tmp[4128];
if (!anchor_path(path, sizeof(path))) return 0;
snprintf(tmp, sizeof(tmp), "%s.tmp-XXXXXX", path);
int fd = mkstemp(tmp);
if (fd < 0) return 0;
FILE *f = fdopen(fd, "w");
if (!f) { close(fd); unlink(tmp); return 0; }
int ok = 1;
for (int i = 0; i < entity_count; ++i) {
const scene_entity *e = &entities[i];
if (!e->is_anchor || !e->saved || e->erased) continue;
char uuid[33];
for (int k = 0; k < 16; ++k) snprintf(uuid + k*2, 3, "%02x", e->uuid.data[k]);
const XrPosef p = e->pose_in_stage;
if (fprintf(f, "anchor %s %.9g %.9g %.9g %.9g %.9g %.9g %.9g\n", uuid,
p.position.x, p.position.y, p.position.z, p.orientation.x, p.orientation.y, p.orientation.z, p.orientation.w) < 0) ok = 0;
}
if (fflush(f) || fsync(fd)) ok = 0;
if (fclose(f)) ok = 0;
if (ok && rename(tmp, path)) ok = 0;
if (!ok) { LOG("scene: anchor write failed (%d)", errno); unlink(tmp); }
return ok;
}
static void load_anchors(void) {
char path[4096];
if (!anchor_path(path, sizeof(path))) return;
FILE *f = fopen(path, "r");
if (!f) return;
char line[1024], *t[10];
unsigned loaded = 0;
while (fgets(line, sizeof(line), f)) {
if (!strchr(line, '\n') && !feof(f)) { int c; while ((c=fgetc(f)) != EOF && c != '\n') {} continue; }
XrUuidEXT uuid; float v[7];
int n = room_tokens(line, t, 10);
if (n != 9 || strcmp(t[0], "anchor") || !room_uuid(t[1], &uuid) || !room_values(t, 2, 7, v)) continue;
float norm = sqrtf(v[3]*v[3]+v[4]*v[4]+v[5]*v[5]+v[6]*v[6]);
if (norm < 0.0001f) continue;
int duplicate = 0;
for (int i = 0; i < entity_count; ++i) duplicate |= uuid_equal(&uuid, &entities[i].uuid);
if (duplicate) continue;
int i = add_entity((XrPosef){{v[3]/norm,v[4]/norm,v[5]/norm,v[6]/norm},{v[0],v[1],v[2]}}, NULL, 0, 0, 0, 1);
if (i < 0) break;
entities[i].uuid = uuid; entities[i].saved = 1; ++loaded;
}
fclose(f);
LOG("scene: loaded %u saved anchors", loaded);
}
// Batch writes are atomic both on disk and in memory. Failure never silently destroys the old anchor set.
static XrResult persist_spaces(uint32_t count, const XrSpace *spaces, uint32_t uuid_count, const XrUuidEXT *uuids, int erase) {
unsigned char selected[MAX_ENTITIES] = {0}, old_saved[MAX_ENTITIES], old_erased[MAX_ENTITIES];
for (uint32_t k = 0; k < count; ++k) {
int i = fake_space_index(spaces[k]);
if (i < 0) return XR_ERROR_HANDLE_INVALID;
if (entities[i].is_anchor) selected[i] = 1;
}
if (erase) for (int i = 0; i < entity_count; ++i)
if (entities[i].is_anchor && !entities[i].erased && uuid_matches(uuid_count, uuids, &entities[i])) selected[i] = 1;
int changed = 0;
for (int i = 0; i < entity_count; ++i) {
old_saved[i] = (unsigned char)entities[i].saved; old_erased[i] = (unsigned char)entities[i].erased;
if (selected[i]) { entities[i].saved = !erase; entities[i].erased = erase; changed = 1; }
}
if (changed && !write_anchors()) {
for (int i = 0; i < entity_count; ++i) { entities[i].saved = old_saved[i]; entities[i].erased = old_erased[i]; }
return XR_ERROR_RUNTIME_FAILURE;
}
if (changed) LOG("scene: anchors %s", erase ? "erased" : "saved");
return XR_SUCCESS;
}
+432 -117
View File
@@ -1,16 +1,19 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by frame_adapter.c. Emulates Meta's scene / spatial-entity extensions (XR_FB_spatial_entity,
// _query, _storage, _container, XR_FB_scene, XR_FB_scene_capture) for mixed-reality apps on runtimes that
// lack them (Steam Frame). The "room" is a box built from the runtime's STAGE bounds (the guardian /
// chaperone rectangle): floor, ceiling and four walls, reported as one room anchor with a layout.
// lack them (Steam Frame). Room files use STAGE coordinates; absent/invalid files retain the guardian box.
// Entity spaces are fake handles; every function that consumes a space is intercepted for them.
static float scene_min_size = 1.5f; // smallest room edge if the guardian is tiny or unavailable
static int emulate_scene; // app requested the scene extensions and the runtime lacks them
static int scene_app_mesh; // app enabled XR_META_spatial_entity_mesh: only then is the global mesh an entity
// (an older SDK's scene loader fails on an anchor it can't place)
static const char *const scene_extensions[] = {
"XR_FB_spatial_entity", "XR_FB_spatial_entity_query", "XR_FB_spatial_entity_storage",
"XR_FB_spatial_entity_container", "XR_FB_scene", "XR_FB_scene_capture",
"XR_META_spatial_entity_mesh", "XR_META_spatial_entity_discovery", "XR_META_spatial_entity_persistence",
"XR_META_boundary_visibility", // MR games hide the guardian before starting (NOPE asks every frame)
};
#define SCENE_EXTENSION_COUNT (sizeof(scene_extensions) / sizeof(scene_extensions[0]))
@@ -42,38 +45,58 @@ static XrPosef pose_inv(XrPosef a) {
// ---------------------------------------------------------------- entities
enum { ENT_ROOM, ENT_FLOOR, ENT_CEILING, ENT_WALL0, ENT_WALL1, ENT_WALL2, ENT_WALL3, ENT_FIXED_COUNT };
#define MAX_ENTITIES 64
#define MAX_ENTITIES 2048
#define MAX_OUTLINE 1024
typedef struct {
XrSpace handle; // fake handle given to the app
XrUuidEXT uuid;
XrPosef pose_in_stage;
const char *label; // semantic label, NULL if none
char label[256]; // comma-separated Meta semantic labels
XrRect2Df bounds; // 2D bounds in the entity's local XY plane
int has_bounds, is_room, is_anchor;
XrRect3DfFB bounds3;
XrVector2f *boundary;
uint32_t boundary_count;
XrVector3f *vertices;
uint32_t *indices, vertex_count, index_count;
int has_bounds, has_bounds3, is_room, is_anchor, is_wall, erased, saved;
} scene_entity;
static pthread_mutex_t scene_lock = PTHREAD_MUTEX_INITIALIZER;
static scene_entity entities[MAX_ENTITIES];
static int entity_count;
static int entity_count, room_entity_count, wall_count, room_from_file;
static char room_source[32];
static uint64_t room_universe;
static XrSpace scene_stage = XR_NULL_HANDLE; // real STAGE space used to anchor everything
static XrSession scene_session = XR_NULL_HANDLE;
static uint64_t next_request = 0x5ce0000001ull;
#define FAKE_SPACE_TAG 0x5ce0000000000000ull
// Fit the tag in uintptr_t as the adapter is also built for AArch32. Session generations prevent stale
// handles from aliasing new room entities when an engine recreates its session in the same process.
#if UINTPTR_MAX > 0xffffffffu
#define FAKE_SPACE_TAG ((uintptr_t)0x5ce0000000000000ull)
#define FAKE_SPACE_MASK ((uintptr_t)0xfff0000000000000ull)
#else
#define FAKE_SPACE_TAG ((uintptr_t)0x5ce00000u)
#define FAKE_SPACE_MASK ((uintptr_t)0xfff00000u)
#endif
static unsigned scene_generation;
static int is_fake_space(XrSpace space) { return ((uintptr_t)space & FAKE_SPACE_MASK) == FAKE_SPACE_TAG; }
static int fake_space_index(XrSpace space) {
uint64_t v = (uint64_t)(uintptr_t)space;
if ((v & 0xfff0000000000000ull) != FAKE_SPACE_TAG) return -1;
int i = (int)(v & 0xffff);
return i < entity_count ? i : -1;
uintptr_t v = (uintptr_t)space;
if ((v & FAKE_SPACE_MASK) != FAKE_SPACE_TAG || ((v >> 12) & 255) != scene_generation) return -1;
int i = (int)(v & 4095);
return i < entity_count && !entities[i].erased ? i : -1;
}
static XrSpace fake_space(int index) { return (XrSpace)(uintptr_t)(FAKE_SPACE_TAG | (uint64_t)index); }
static XrSpace fake_space(int index) { return (XrSpace)(FAKE_SPACE_TAG | ((uintptr_t)scene_generation << 12) | (uintptr_t)index); }
static XrUuidEXT make_uuid(int index) {
XrUuidEXT u;
for (int i = 0; i < XR_UUID_SIZE_EXT; ++i) u.data[i] = (uint8_t)(0x5c ^ (i * 37) ^ (index * 101));
u.data[6] = (uint8_t)((u.data[6] & 0x0f) | 0x40); // version 4 style
u.data[15] = (uint8_t)index;
u.data[14] ^= (uint8_t)((unsigned)index >> 8);
return u;
}
static int uuid_equal(const XrUuidEXT *a, const XrUuidEXT *b) { return !memcmp(a->data, b->data, XR_UUID_SIZE_EXT); }
@@ -86,7 +109,7 @@ static int add_entity(XrPosef pose, const char *label, float w, float h, int is_
e->handle = fake_space(i);
e->uuid = make_uuid(i);
e->pose_in_stage = pose;
e->label = label;
if (label) snprintf(e->label, sizeof(e->label), "%s", label);
e->has_bounds = w > 0 && h > 0;
e->bounds = (XrRect2Df){{-w / 2, -h / 2}, {w, h}};
e->is_room = is_room;
@@ -94,9 +117,23 @@ static int add_entity(XrPosef pose, const char *label, float w, float h, int is_
return i;
}
static void reset_room(void) {
for (int i = 0; i < entity_count; ++i) {
free(entities[i].boundary);
free(entities[i].vertices);
free(entities[i].indices);
}
memset(entities, 0, sizeof(entities));
entity_count = room_entity_count = wall_count = room_from_file = 0;
room_universe = 0;
}
#include "scene_room.c"
// Build the room once the session exists: guardian rectangle -> floor/ceiling/walls.
static void build_room(void) {
if (entity_count) return;
if (load_room_file()) return;
float width = 0, depth = 0;
PFN_xrGetReferenceSpaceBoundsRect bounds_fn =
(PFN_xrGetReferenceSpaceBoundsRect)lookup(active_instance, "xrGetReferenceSpaceBoundsRect");
@@ -117,10 +154,14 @@ static void build_room(void) {
add_entity((XrPosef){{0, 0, 0, 1}, {0, 0, 0}}, NULL, 0, 0, 1, 0); // ENT_ROOM
add_entity((XrPosef){{-s, 0, 0, s}, {0, 0, 0}}, "FLOOR", width, depth, 0, 0); // +Z (normal) -> +Y
add_entity((XrPosef){{s, 0, 0, s}, {0, h, 0}}, "CEILING", width, depth, 0, 0); // +Z -> -Y
// Walls in loop order around the floor, as Meta's ROOM_LAYOUT lists them (Demeter walks them as an outline).
add_entity((XrPosef){{0, 0, 0, 1}, {0, h / 2, -depth / 2}}, "WALL_FACE", width, h, 0, 0); // faces +Z
add_entity((XrPosef){{0, -s, 0, s}, {width / 2, h / 2, 0}}, "WALL_FACE", depth, h, 0, 0); // faces -X
add_entity((XrPosef){{0, 1, 0, 0}, {0, h / 2, depth / 2}}, "WALL_FACE", width, h, 0, 0); // faces -Z
add_entity((XrPosef){{0, s, 0, s}, {-width / 2, h / 2, 0}}, "WALL_FACE", depth, h, 0, 0); // faces +X
add_entity((XrPosef){{0, -s, 0, s}, {width / 2, h / 2, 0}}, "WALL_FACE", depth, h, 0, 0); // faces -X
wall_count = 4;
room_entity_count = entity_count;
for (int i = ENT_WALL0; i < entity_count; ++i) entities[i].is_wall = 1;
}
// Supported components per entity.
@@ -128,12 +169,17 @@ static int entity_components(const scene_entity *e, XrSpaceComponentTypeFB *out)
int n = 0;
out[n++] = XR_SPACE_COMPONENT_TYPE_LOCATABLE_FB;
out[n++] = XR_SPACE_COMPONENT_TYPE_STORABLE_FB;
out[n++] = XR_SPACE_COMPONENT_TYPE_SHARABLE_FB; // Meta's runtime lists it; nothing is shared from here
if (e->is_room) {
out[n++] = XR_SPACE_COMPONENT_TYPE_ROOM_LAYOUT_FB;
out[n++] = XR_SPACE_COMPONENT_TYPE_SPACE_CONTAINER_FB;
} else if (!e->is_anchor) {
out[n++] = XR_SPACE_COMPONENT_TYPE_BOUNDED_2D_FB;
out[n++] = XR_SPACE_COMPONENT_TYPE_SEMANTIC_LABELS_FB;
if (e->has_bounds) {
out[n++] = XR_SPACE_COMPONENT_TYPE_BOUNDED_2D_FB;
}
if (e->has_bounds3) out[n++] = XR_SPACE_COMPONENT_TYPE_BOUNDED_3D_FB;
if (e->vertex_count) out[n++] = XR_SPACE_COMPONENT_TYPE_TRIANGLE_MESH_META;
if (e->label[0]) out[n++] = XR_SPACE_COMPONENT_TYPE_SEMANTIC_LABELS_FB;
}
return n;
}
@@ -148,13 +194,37 @@ static int entity_has_component(const scene_entity *e, XrSpaceComponentTypeFB ty
#define MAX_EVENTS 64
static XrEventDataBuffer scene_events[MAX_EVENTS];
static int event_head, event_tail;
static int event_free(void) { return (event_head - event_tail - 1 + MAX_EVENTS) % MAX_EVENTS; }
static void push_event(const void *event, size_t size) {
if ((event_tail + 1) % MAX_EVENTS == event_head) return;
memset(&scene_events[event_tail], 0, sizeof(XrEventDataBuffer));
memcpy(&scene_events[event_tail], event, size);
event_tail = (event_tail + 1) % MAX_EVENTS;
}
// A real scan takes a while; an answer in the same frame may be missed by a game's prompt logic (Pinball FX asked
// again at once). The completion waits this long in its own slot. Caller of both holds scene_lock.
static int scene_capture_delay_ms = 1500;
static XrEventDataSceneCaptureCompleteFB pending_capture;
static uint64_t pending_capture_at; // CLOCK_MONOTONIC ns, 0 = none
static uint64_t scene_now_ns(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec;
}
static int scan_waiting;
static uint64_t scan_started_at, scan_poll_at;
static int scan_acked;
static void poll_scan(uint64_t now);
static int pop_event(XrEventDataBuffer *out) { // caller holds scene_lock
uint64_t now = scene_now_ns();
if (scan_waiting) poll_scan(now);
if (!scan_waiting && pending_capture_at && now >= pending_capture_at) {
memset(out, 0, sizeof(*out));
memcpy(out, &pending_capture, sizeof(pending_capture));
pending_capture_at = 0;
LOG("scene: capture complete delivered (request %llu)", (unsigned long long)pending_capture.requestId);
return 1;
}
if (event_head == event_tail) return 0;
memcpy(out, &scene_events[event_head], sizeof(XrEventDataBuffer));
event_head = (event_head + 1) % MAX_EVENTS;
@@ -171,6 +241,7 @@ static int stage_pose(XrSpace space, XrTime time, XrPosef *pose, XrSpaceLocation
XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT | XR_SPACE_LOCATION_POSITION_TRACKED_BIT;
return 1;
}
if (is_fake_space(space)) return 0; // erased or from a prior session: never pass a fake handle to SteamVR
PFN_xrLocateSpace locate = (PFN_xrLocateSpace)lookup(active_instance, "xrLocateSpace");
if (!locate || !scene_stage) return 0;
XrSpaceLocation loc = {XR_TYPE_SPACE_LOCATION, NULL, 0, {{0, 0, 0, 1}, {0, 0, 0}}};
@@ -181,7 +252,7 @@ static int stage_pose(XrSpace space, XrTime time, XrPosef *pose, XrSpaceLocation
}
// Locate `space` in `base` when either is fake. Returns 0 if neither is fake (caller forwards).
static int locate_fake(XrSpace space, XrSpace base, XrTime time, XrPosef *pose, XrSpaceLocationFlags *flags) {
if (fake_space_index(space) < 0 && fake_space_index(base) < 0) return 0;
if (!is_fake_space(space) && !is_fake_space(base)) return 0;
XrPosef s, b;
XrSpaceLocationFlags fs = 0, fb = 0;
if (!stage_pose(space, time, &s, &fs) || !stage_pose(base, time, &b, &fb)) {
@@ -195,99 +266,99 @@ static int locate_fake(XrSpace space, XrSpace base, XrTime time, XrPosef *pose,
}
// ---------------------------------------------------------------- emulated entry points
static XRAPI_ATTR XrResult XRAPI_CALL emu_query_spaces(XrSession session, const XrSpaceQueryInfoBaseHeaderFB *info,
XrAsyncRequestIdFB *request) {
if (!info || !request) return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
scene_session = session;
build_room();
*request = next_request++;
XrEventDataSpaceQueryResultsAvailableFB avail = {XR_TYPE_EVENT_DATA_SPACE_QUERY_RESULTS_AVAILABLE_FB, NULL, *request};
XrEventDataSpaceQueryCompleteFB done = {XR_TYPE_EVENT_DATA_SPACE_QUERY_COMPLETE_FB, NULL, *request, XR_SUCCESS};
push_event(&avail, sizeof(avail));
push_event(&done, sizeof(done));
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
// Query filters are kept per request so retrieve can apply them.
// Results are snapshotted before their available event is queued, so callers on another thread cannot
// observe a half-stored filter. A bounded request history supports the count/data calls independently.
#define MAX_QUERIES 64
typedef struct {
XrAsyncRequestIdFB id;
int has_component;
XrSpaceComponentTypeFB component;
uint32_t uuid_count;
XrUuidEXT uuids[64];
} query_filter;
static query_filter filters[16];
static int filter_next;
static void store_filter(XrAsyncRequestIdFB id, const XrSpaceQueryInfoFB *q) {
query_filter *f = &filters[filter_next++ % 16];
memset(f, 0, sizeof(*f));
f->id = id;
const XrSpaceFilterInfoBaseHeaderFB *flt = q->filter;
if (flt && flt->type == XR_TYPE_SPACE_COMPONENT_FILTER_INFO_FB) {
f->has_component = 1;
f->component = ((const XrSpaceComponentFilterInfoFB *)flt)->componentType;
} else if (flt && flt->type == XR_TYPE_SPACE_UUID_FILTER_INFO_FB) {
const XrSpaceUuidFilterInfoFB *u = (const XrSpaceUuidFilterInfoFB *)flt;
f->uuid_count = u->uuidCount < 64 ? u->uuidCount : 64;
for (uint32_t i = 0; i < f->uuid_count; ++i) f->uuids[i] = u->uuids[i];
}
uint32_t count;
XrSpaceQueryResultFB *results;
} scene_query;
static scene_query queries[MAX_QUERIES];
static unsigned query_next;
static scene_query *new_query(void) {
scene_query *q = &queries[query_next % MAX_QUERIES];
XrSpaceQueryResultFB *results = malloc((size_t)entity_count * sizeof(*results));
if (!results) return NULL;
++query_next;
free(q->results);
*q = (scene_query){next_request++, 0, results};
return q;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_query_spaces_filtered(XrSession session, const XrSpaceQueryInfoBaseHeaderFB *info,
XrAsyncRequestIdFB *request) {
XrResult r = emu_query_spaces(session, info, request);
if (XR_SUCCEEDED(r) && info->type == XR_TYPE_SPACE_QUERY_INFO_FB) {
pthread_mutex_lock(&scene_lock);
store_filter(*request, (const XrSpaceQueryInfoFB *)info);
const XrSpaceQueryInfoFB *q = (const XrSpaceQueryInfoFB *)info;
LOG("scene: query request=%llu filter=%d", (unsigned long long)*request, q->filter ? q->filter->type : 0);
pthread_mutex_unlock(&scene_lock);
}
return r;
static scene_query *find_query(XrAsyncRequestIdFB id) {
for (int i = 0; i < MAX_QUERIES; ++i) if (queries[i].id == id) return &queries[i];
return NULL;
}
static int uuid_matches(uint32_t n, const XrUuidEXT *uuids, const scene_entity *e) {
for (uint32_t i = 0; i < n; ++i) if (uuid_equal(&uuids[i], &e->uuid)) return 1;
return 0;
}
#include "scene_anchors.c"
#include "scene_scan.c"
static int filter_matches(const query_filter *f, const scene_entity *e) {
static int fb_filter_matches(const XrSpaceFilterInfoBaseHeaderFB *f, const scene_entity *e) {
if (!f) return 1;
if (f->has_component) return entity_has_component(e, f->component);
if (f->uuid_count) {
for (uint32_t i = 0; i < f->uuid_count; ++i) if (uuid_equal(&f->uuids[i], &e->uuid)) return 1;
return 0;
if (f->type == XR_TYPE_SPACE_COMPONENT_FILTER_INFO_FB)
return entity_has_component(e, ((const XrSpaceComponentFilterInfoFB *)f)->componentType);
if (f->type == XR_TYPE_SPACE_UUID_FILTER_INFO_FB) {
const XrSpaceUuidFilterInfoFB *u = (const XrSpaceUuidFilterInfoFB *)f;
return u->uuids && uuid_matches(u->uuidCount, u->uuids, e);
}
return 1;
}
static void log_head_in_room(void); // scene_meta.c
static XRAPI_ATTR XrResult XRAPI_CALL emu_query_spaces_filtered(XrSession session, const XrSpaceQueryInfoBaseHeaderFB *info,
XrAsyncRequestIdFB *request) {
if (!info || !request || info->type != XR_TYPE_SPACE_QUERY_INFO_FB) return XR_ERROR_VALIDATION_FAILURE;
const XrSpaceQueryInfoFB *info_fb = (const XrSpaceQueryInfoFB *)info;
pthread_mutex_lock(&scene_lock);
scene_session = session;
build_room();
if (event_free() < 2) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
scene_query *q = new_query();
if (!q) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_OUT_OF_MEMORY; }
for (int i = 0; i < entity_count && q->count < info_fb->maxResultCount; ++i) {
scene_entity *e = &entities[i];
if (!e->erased && fb_filter_matches(info_fb->filter, e))
q->results[q->count++] = (XrSpaceQueryResultFB){e->handle, e->uuid};
}
*request = q->id;
XrEventDataSpaceQueryResultsAvailableFB avail = {XR_TYPE_EVENT_DATA_SPACE_QUERY_RESULTS_AVAILABLE_FB, NULL, *request};
XrEventDataSpaceQueryCompleteFB done = {XR_TYPE_EVENT_DATA_SPACE_QUERY_COMPLETE_FB, NULL, *request, XR_SUCCESS};
push_event(&avail, sizeof(avail)); push_event(&done, sizeof(done));
LOG("scene: query request=%llu results=%u", (unsigned long long)*request, q->count);
log_head_in_room();
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_retrieve_results(XrSession session, XrAsyncRequestIdFB request,
XrSpaceQueryResultsFB *results) {
(void)session;
if (!results) return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
const query_filter *f = NULL;
for (int i = 0; i < 16; ++i) if (filters[i].id == request) f = &filters[i];
uint32_t n = 0;
for (int i = 0; i < entity_count; ++i) {
if (!filter_matches(f, &entities[i])) continue;
if (results->resultCapacityInput && n < results->resultCapacityInput && results->results) {
results->results[n].space = entities[i].handle;
results->results[n].uuid = entities[i].uuid;
scene_query *q = find_query(request);
XrResult r = XR_SUCCESS;
if (!q) r = XR_ERROR_VALIDATION_FAILURE;
else {
results->resultCountOutput = q->count;
if (results->resultCapacityInput && results->resultCapacityInput < q->count) r = XR_ERROR_SIZE_INSUFFICIENT;
else if (results->resultCapacityInput) {
if (!results->results) r = XR_ERROR_VALIDATION_FAILURE;
else memcpy(results->results, q->results, q->count * sizeof(*q->results));
}
++n;
}
results->resultCountOutput = n;
pthread_mutex_unlock(&scene_lock);
return (results->resultCapacityInput && results->resultCapacityInput < n) ? XR_ERROR_SIZE_INSUFFICIENT : XR_SUCCESS;
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_uuid(XrSpace space, XrUuidEXT *uuid) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_uuid(XrSpace space, XrUuidEXT *uuid) {
int i = fake_space_index(space);
if (i < 0 || !uuid) return XR_ERROR_HANDLE_INVALID;
*uuid = entities[i].uuid;
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_enumerate_components(XrSpace space, uint32_t capacity, uint32_t *count,
static XRAPI_ATTR XrResult XRAPI_CALL scene_enumerate_components(XrSpace space, uint32_t capacity, uint32_t *count,
XrSpaceComponentTypeFB *types) {
int i = fake_space_index(space);
if (i < 0 || !count) return XR_ERROR_HANDLE_INVALID;
@@ -296,16 +367,33 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_enumerate_components(XrSpace space, ui
*count = (uint32_t)n;
if (!capacity) return XR_SUCCESS;
if (capacity < (uint32_t)n) return XR_ERROR_SIZE_INSUFFICIENT;
if (!types) return XR_ERROR_VALIDATION_FAILURE;
for (int k = 0; k < n; ++k) types[k] = list[k];
return XR_SUCCESS;
}
// Which entity refused which component (the first ones only): an app's scene loader gives up on the whole room over
// one of these, and OVRPlugin's own error line doesn't name the component (Demeter, 2026-10-16).
static void log_refused_component(const char *op, int i, XrSpaceComponentTypeFB type) {
static int logged;
if (logged >= 20) return;
++logged;
const scene_entity *e = &entities[i];
LOG("scene: %s component %d refused on %s (entity %d)", op, (int)type,
e->is_room ? "room" : e->is_anchor ? "app anchor" : e->label[0] ? e->label : "unlabelled", i);
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_set_component_status(XrSpace space, const XrSpaceComponentStatusSetInfoFB *info,
XrAsyncRequestIdFB *request) {
int i = fake_space_index(space);
if (i < 0 || !info || !request) return XR_ERROR_HANDLE_INVALID;
if (!entity_has_component(&entities[i], info->componentType)) return XR_ERROR_SPACE_COMPONENT_NOT_SUPPORTED_FB;
pthread_mutex_lock(&scene_lock);
int i = fake_space_index(space);
if (i < 0 || !info || !request) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_HANDLE_INVALID; }
if (!entity_has_component(&entities[i], info->componentType)) {
log_refused_component("set", i, info->componentType);
pthread_mutex_unlock(&scene_lock);
return XR_ERROR_SPACE_COMPONENT_NOT_SUPPORTED_FB;
}
if (!event_free()) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
*request = next_request++;
XrEventDataSpaceSetStatusCompleteFB e = {XR_TYPE_EVENT_DATA_SPACE_SET_STATUS_COMPLETE_FB, NULL, *request, XR_SUCCESS,
space, entities[i].uuid, info->componentType, info->enabled};
@@ -314,44 +402,49 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_set_component_status(XrSpace space, co
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_component_status(XrSpace space, XrSpaceComponentTypeFB type,
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_component_status(XrSpace space, XrSpaceComponentTypeFB type,
XrSpaceComponentStatusFB *status) {
int i = fake_space_index(space);
if (i < 0 || !status) return XR_ERROR_HANDLE_INVALID;
if (!entity_has_component(&entities[i], type)) return XR_ERROR_SPACE_COMPONENT_NOT_SUPPORTED_FB;
if (!entity_has_component(&entities[i], type)) {
log_refused_component("get", i, type);
return XR_ERROR_SPACE_COMPONENT_NOT_SUPPORTED_FB;
}
status->enabled = XR_TRUE;
status->changePending = XR_FALSE;
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_container(XrSession session, XrSpace space, XrSpaceContainerFB *container) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_container(XrSession session, XrSpace space, XrSpaceContainerFB *container) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !container) return XR_ERROR_HANDLE_INVALID;
if (!entities[i].is_room) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
uint32_t n = ENT_FIXED_COUNT - 1; // everything except the room itself
uint32_t n = room_entity_count - 1; // everything except the room itself
container->uuidCountOutput = n;
if (!container->uuidCapacityInput) return XR_SUCCESS;
if (container->uuidCapacityInput < n) return XR_ERROR_SIZE_INSUFFICIENT;
if (!container->uuids) return XR_ERROR_VALIDATION_FAILURE;
for (uint32_t k = 0; k < n; ++k) container->uuids[k] = entities[ENT_FLOOR + k].uuid;
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_room_layout(XrSession session, XrSpace space, XrRoomLayoutFB *layout) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_room_layout(XrSession session, XrSpace space, XrRoomLayoutFB *layout) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !layout) return XR_ERROR_HANDLE_INVALID;
if (!entities[i].is_room) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
layout->floorUuid = entities[ENT_FLOOR].uuid;
layout->ceilingUuid = entities[ENT_CEILING].uuid;
layout->wallUuidCountOutput = 4;
layout->wallUuidCountOutput = (uint32_t)wall_count;
if (!layout->wallUuidCapacityInput) return XR_SUCCESS;
if (layout->wallUuidCapacityInput < 4) return XR_ERROR_SIZE_INSUFFICIENT;
for (int k = 0; k < 4; ++k) layout->wallUuids[k] = entities[ENT_WALL0 + k].uuid;
if (layout->wallUuidCapacityInput < (uint32_t)wall_count) return XR_ERROR_SIZE_INSUFFICIENT;
if (!layout->wallUuids) return XR_ERROR_VALIDATION_FAILURE;
for (int k = 0; k < wall_count; ++k) layout->wallUuids[k] = entities[ENT_WALL0 + k].uuid;
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_bounding_box_2d(XrSession session, XrSpace space, XrRect2Df *rect) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_bounding_box_2d(XrSession session, XrSpace space, XrRect2Df *rect) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !rect) return XR_ERROR_HANDLE_INVALID;
@@ -360,19 +453,29 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_get_bounding_box_2d(XrSession session,
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_bounding_box_3d(XrSession session, XrSpace space, XrRect3DfFB *box) {
(void)session; (void)space; (void)box;
return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_bounding_box_3d(XrSession session, XrSpace space, XrRect3DfFB *box) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !box) return XR_ERROR_HANDLE_INVALID;
if (!entities[i].has_bounds3) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
*box = entities[i].bounds3;
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_boundary_2d(XrSession session, XrSpace space, XrBoundary2DFB *boundary) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_boundary_2d(XrSession session, XrSpace space, XrBoundary2DFB *boundary) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !boundary) return XR_ERROR_HANDLE_INVALID;
if (!entities[i].has_bounds) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
boundary->vertexCountOutput = 4;
uint32_t n = entities[i].boundary_count ? entities[i].boundary_count : 4;
boundary->vertexCountOutput = n;
if (!boundary->vertexCapacityInput) return XR_SUCCESS;
if (boundary->vertexCapacityInput < 4) return XR_ERROR_SIZE_INSUFFICIENT;
if (boundary->vertexCapacityInput < n) return XR_ERROR_SIZE_INSUFFICIENT;
if (!boundary->vertices) return XR_ERROR_VALIDATION_FAILURE;
if (entities[i].boundary_count) {
memcpy(boundary->vertices, entities[i].boundary, n * sizeof(*boundary->vertices));
return XR_SUCCESS;
}
const XrRect2Df b = entities[i].bounds;
boundary->vertices[0] = (XrVector2f){b.offset.x, b.offset.y};
boundary->vertices[1] = (XrVector2f){b.offset.x + b.extent.width, b.offset.y};
@@ -381,16 +484,86 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_get_boundary_2d(XrSession session, XrS
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_semantic_labels(XrSession session, XrSpace space, XrSemanticLabelsFB *labels) {
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_triangle_mesh(XrSpace space, const XrSpaceTriangleMeshGetInfoMETA *info,
XrSpaceTriangleMeshMETA *mesh) {
(void)info;
int i = fake_space_index(space);
if (i < 0 || !mesh) return XR_ERROR_HANDLE_INVALID;
const scene_entity *e = &entities[i];
if (!e->vertex_count) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
mesh->vertexCountOutput = e->vertex_count;
mesh->indexCountOutput = e->index_count;
if ((mesh->vertexCapacityInput && mesh->vertexCapacityInput < e->vertex_count) ||
(mesh->indexCapacityInput && mesh->indexCapacityInput < e->index_count)) return XR_ERROR_SIZE_INSUFFICIENT;
if ((mesh->vertexCapacityInput && !mesh->vertices) || (mesh->indexCapacityInput && !mesh->indices))
return XR_ERROR_VALIDATION_FAILURE;
if (mesh->vertexCapacityInput) memcpy(mesh->vertices, e->vertices, e->vertex_count * sizeof(*e->vertices));
if (mesh->indexCapacityInput) memcpy(mesh->indices, e->indices, e->index_count * sizeof(*e->indices));
return XR_SUCCESS;
}
// Labels as Meta's runtime hands them to an app (XR_FB_scene): an app that sends no XrSemanticLabelsSupportInfoFB
// knows only the first label set; one that does gets "OTHER" for labels missing from its recognizedLabels, WALL_FACE
// for invisible walls unless it accepts them, and DESK for TABLE unless it accepts that migration. Older Meta SDKs
// (Demeter, 2026-10-16) rejected a whole scanned room over one INVISIBLE_WALL_FACE and kept asking for Space Setup.
static const char *const scene_legacy_labels = "DESK,COUCH,FLOOR,CEILING,WALL_FACE,DOOR_FRAME,WINDOW_FRAME,OTHER";
static int label_listed(const char *list, const char *label, size_t len) {
for (const char *p = list; p && *p;) {
const char *end = strchr(p, ',');
size_t n = end ? (size_t)(end - p) : strlen(p);
if (n == len && !strncmp(p, label, len)) return 1;
p = end ? end + 1 : NULL;
}
return 0;
}
static void scene_app_labels(const char *label, const XrSemanticLabelsFB *labels, char *out, size_t size) {
const XrSemanticLabelsSupportInfoFB *support = NULL;
for (const XrBaseInStructure *n = labels->next; n; n = n->next)
if (n->type == XR_TYPE_SEMANTIC_LABELS_SUPPORT_INFO_FB) support = (const XrSemanticLabelsSupportInfoFB *)n;
const char *known = support ? support->recognizedLabels : scene_legacy_labels;
XrSemanticLabelsSupportFlagsFB flags = support ? support->flags : 0;
out[0] = 0;
for (const char *p = label; *p;) {
const char *end = strchr(p, ',');
size_t len = end ? (size_t)(end - p) : strlen(p);
char one[64];
snprintf(one, sizeof(one), "%.*s", (int)len, p);
if (!strcmp(one, "INVISIBLE_WALL_FACE") && !(flags & XR_SEMANTIC_LABELS_SUPPORT_ACCEPT_INVISIBLE_WALL_FACE_BIT_FB))
snprintf(one, sizeof(one), "WALL_FACE");
if (!strcmp(one, "TABLE") && !(flags & XR_SEMANTIC_LABELS_SUPPORT_ACCEPT_DESK_TO_TABLE_MIGRATION_BIT_FB))
snprintf(one, sizeof(one), "DESK");
if (known && *known && !label_listed(known, one, strlen(one))) snprintf(one, sizeof(one), "OTHER");
if (!label_listed(out, one, strlen(one)))
snprintf(out + strlen(out), size - strlen(out), "%s%s", out[0] ? "," : "", one);
p = end ? end + 1 : p + len;
}
}
static XRAPI_ATTR XrResult XRAPI_CALL scene_get_semantic_labels(XrSession session, XrSpace space, XrSemanticLabelsFB *labels) {
(void)session;
int i = fake_space_index(space);
if (i < 0 || !labels) return XR_ERROR_HANDLE_INVALID;
if (!entities[i].label) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
uint32_t n = (uint32_t)strlen(entities[i].label) + 1;
if (!entities[i].label[0]) return XR_ERROR_SPACE_COMPONENT_NOT_ENABLED_FB;
char text[sizeof(entities[0].label)];
scene_app_labels(entities[i].label, labels, text, sizeof(text));
uint32_t n = (uint32_t)strlen(text) + 1;
labels->bufferCountOutput = n;
if (!labels->bufferCapacityInput) return XR_SUCCESS;
if (labels->bufferCapacityInput < n) return XR_ERROR_SIZE_INSUFFICIENT;
memcpy(labels->buffer, entities[i].label, n);
if (!labels->buffer) return XR_ERROR_VALIDATION_FAILURE;
memcpy(labels->buffer, text, n);
static unsigned char logged[MAX_ENTITIES]; // diagnostics: what each entity is called for this app, once
if (!logged[i]) {
logged[i] = 1;
const XrSemanticLabelsSupportInfoFB *support = NULL;
for (const XrBaseInStructure *s = labels->next; s; s = s->next)
if (s->type == XR_TYPE_SEMANTIC_LABELS_SUPPORT_INFO_FB) support = (const XrSemanticLabelsSupportInfoFB *)s;
LOG("scene: labels of entity %d: %s -> \"%s\" (app's labels: %s, flags 0x%llx)", i, entities[i].label, text,
support && support->recognizedLabels ? support->recognizedLabels : "none sent, legacy set",
support ? (unsigned long long)support->flags : 0ULL);
}
return XR_SUCCESS;
}
@@ -401,11 +574,22 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_request_scene_capture(XrSession sessio
pthread_mutex_lock(&scene_lock);
scene_session = session;
build_room();
if (!event_free() || pending_capture_at) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
*request = next_request++;
XrEventDataSceneCaptureCompleteFB e = {XR_TYPE_EVENT_DATA_SCENE_CAPTURE_COMPLETE_FB, NULL, *request, XR_SUCCESS};
push_event(&e, sizeof(e));
if (scene_scan_request && start_scan(*request)) {
pending_capture = e;
pending_capture_at = scan_started_at + (uint64_t)scene_capture_delay_ms * 1000000ull;
} else if (scene_capture_delay_ms > 0) {
pending_capture = e;
pending_capture_at = scene_now_ns() + (uint64_t)scene_capture_delay_ms * 1000000ull;
} else {
push_event(&e, sizeof(e));
}
if (scan_waiting) LOG("scene: capture requested -> waiting for FramePort");
else LOG("scene: capture requested -> %s (complete in %d ms)", room_from_file ? "scanned room" : "guardian box",
scene_capture_delay_ms);
pthread_mutex_unlock(&scene_lock);
LOG("scene: capture requested -> using guardian room");
return XR_SUCCESS;
}
@@ -417,9 +601,16 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_create_anchor(XrSession session, const
build_room();
XrPosef base = {{0, 0, 0, 1}, {0, 0, 0}};
XrSpaceLocationFlags flags = 0;
stage_pose(info->space, info->time, &base, &flags);
if (event_free() < 1) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
if (!stage_pose(info->space, info->time, &base, &flags) ||
!(flags & XR_SPACE_LOCATION_POSITION_VALID_BIT) || !(flags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT)) {
pthread_mutex_unlock(&scene_lock); return XR_ERROR_POSE_INVALID;
}
XrUuidEXT uuid;
if (!anchor_uuid(&uuid)) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_RUNTIME_FAILURE; }
int i = add_entity(pose_mul(base, info->poseInSpace), NULL, 0, 0, 0, 1);
*request = next_request++;
if (i >= 0) entities[i].uuid = uuid;
XrEventDataSpatialAnchorCreateCompleteFB e = {XR_TYPE_EVENT_DATA_SPATIAL_ANCHOR_CREATE_COMPLETE_FB, NULL, *request,
i >= 0 ? XR_SUCCESS : XR_ERROR_LIMIT_REACHED, i >= 0 ? entities[i].handle : XR_NULL_HANDLE,
i >= 0 ? entities[i].uuid : (XrUuidEXT){{0}}};
@@ -431,31 +622,136 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_create_anchor(XrSession session, const
static XRAPI_ATTR XrResult XRAPI_CALL emu_save_space(XrSession session, const XrSpaceSaveInfoFB *info, XrAsyncRequestIdFB *request) {
(void)session;
if (!info || !request) return XR_ERROR_VALIDATION_FAILURE;
int i = fake_space_index(info->space);
pthread_mutex_lock(&scene_lock);
int i = fake_space_index(info->space);
if (i < 0) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_HANDLE_INVALID; }
if (!event_free()) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
XrResult r = persist_spaces(1, &info->space, 0, NULL, 0);
*request = next_request++;
XrEventDataSpaceSaveCompleteFB e = {XR_TYPE_EVENT_DATA_SPACE_SAVE_COMPLETE_FB, NULL, *request, XR_SUCCESS, info->space,
i >= 0 ? entities[i].uuid : (XrUuidEXT){{0}}, info->location};
XrEventDataSpaceSaveCompleteFB e = {XR_TYPE_EVENT_DATA_SPACE_SAVE_COMPLETE_FB, NULL, *request, r, info->space,
entities[i].uuid, info->location};
push_event(&e, sizeof(e));
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_erase_space(XrSession session, const XrSpaceEraseInfoFB *info, XrAsyncRequestIdFB *request) {
(void)session;
if (!info || !request) return XR_ERROR_VALIDATION_FAILURE;
int i = fake_space_index(info->space);
pthread_mutex_lock(&scene_lock);
int i = fake_space_index(info->space);
if (i < 0) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_HANDLE_INVALID; }
if (!event_free()) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
XrResult r = persist_spaces(1, &info->space, 0, NULL, 1);
*request = next_request++;
XrEventDataSpaceEraseCompleteFB e = {XR_TYPE_EVENT_DATA_SPACE_ERASE_COMPLETE_FB, NULL, *request, XR_SUCCESS, info->space,
i >= 0 ? entities[i].uuid : (XrUuidEXT){{0}}, info->location};
XrEventDataSpaceEraseCompleteFB e = {XR_TYPE_EVENT_DATA_SPACE_ERASE_COMPLETE_FB, NULL, *request, r, info->space,
entities[i].uuid, info->location};
push_event(&e, sizeof(e));
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
// Reads share the mutation lock: engine worker threads may retrieve geometry while anchors are saved.
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_uuid(XrSpace space, XrUuidEXT *uuid) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_uuid(space, uuid);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_enumerate_components(XrSpace space, uint32_t capacity, uint32_t *count, XrSpaceComponentTypeFB *types) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_enumerate_components(space, capacity, count, types);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_component_status(XrSpace space, XrSpaceComponentTypeFB type, XrSpaceComponentStatusFB *status) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_component_status(space, type, status);
pthread_mutex_unlock(&scene_lock);
return r;
}
// Diagnostics for MR games that reject the room (Pinball FX asks for a scan again and again): every kind of room
// query is logged once per entity with its result. Caller holds scene_lock.
enum { TR_CONTAINER, TR_LAYOUT, TR_BOX2D, TR_BOX3D, TR_BOUNDARY, TR_MESH, TR_COUNT };
static void trace_call(XrSpace space, int kind, XrResult r) {
static const char *const names[TR_COUNT] = {"container", "room layout", "2D box", "3D box", "2D outline", "mesh"};
static unsigned char seen[MAX_ENTITIES];
int i = fake_space_index(space);
if (i < 0 || (seen[i] & (1u << kind))) return;
seen[i] |= (unsigned char)(1u << kind);
LOG("scene: %s of entity %d (%s): result %d", names[kind], i,
entities[i].is_room ? "ROOM" : entities[i].label[0] ? entities[i].label : "anchor", r);
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_container(XrSession session, XrSpace space, XrSpaceContainerFB *container) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_container(session, space, container);
trace_call(space, TR_CONTAINER, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_room_layout(XrSession session, XrSpace space, XrRoomLayoutFB *layout) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_room_layout(session, space, layout);
trace_call(space, TR_LAYOUT, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_bounding_box_2d(XrSession session, XrSpace space, XrRect2Df *rect) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_bounding_box_2d(session, space, rect);
trace_call(space, TR_BOX2D, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_bounding_box_3d(XrSession session, XrSpace space, XrRect3DfFB *box) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_bounding_box_3d(session, space, box);
trace_call(space, TR_BOX3D, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_boundary_2d(XrSession session, XrSpace space, XrBoundary2DFB *boundary) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_boundary_2d(session, space, boundary);
trace_call(space, TR_BOUNDARY, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_triangle_mesh(XrSpace space, const XrSpaceTriangleMeshGetInfoMETA *info, XrSpaceTriangleMeshMETA *mesh) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_triangle_mesh(space, info, mesh);
trace_call(space, TR_MESH, r);
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_get_semantic_labels(XrSession session, XrSpace space, XrSemanticLabelsFB *labels) {
pthread_mutex_lock(&scene_lock);
XrResult r = scene_get_semantic_labels(session, space, labels);
pthread_mutex_unlock(&scene_lock);
return r;
}
// The app's STAGE bounds: the Frame reports a 1x1 m rect but OVRPlugin can't give its outline ("ovrpBoundary_Outer
// is not supported"), so Unity games that trust a configured boundary index an empty array (Demeter's
// RemoveObjectNotInGuardian). With an emulated room there is no play area for the app; build_room asks the runtime.
static XRAPI_ATTR XrResult XRAPI_CALL emu_stage_bounds(XrSession session, XrReferenceSpaceType type, XrExtent2Df *bounds) {
if (type == XR_REFERENCE_SPACE_TYPE_STAGE) {
if (!bounds) return XR_ERROR_VALIDATION_FAILURE;
static int logged;
if (!logged++) LOG("scene: STAGE bounds hidden from the app (no play-area outline on this runtime)");
*bounds = (XrExtent2Df){0, 0};
return XR_SPACE_BOUNDS_UNAVAILABLE;
}
PFN_xrGetReferenceSpaceBoundsRect fn =
(PFN_xrGetReferenceSpaceBoundsRect)lookup(active_instance, "xrGetReferenceSpaceBoundsRect");
return fn ? fn(session, type, bounds) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
#include "scene_meta.c"
static PFN_xrVoidFunction scene_emulation(const char *name) {
if (!emulate_scene) return NULL;
if (!strcmp(name, "xrGetReferenceSpaceBoundsRect")) // only for games that trust the 1x1 m rect (NOPE stalled)
return scene_hide_bounds ? (PFN_xrVoidFunction)emu_stage_bounds : NULL;
static const struct { const char *name; PFN_xrVoidFunction fn; } table[] = {
{"xrQuerySpacesFB", (PFN_xrVoidFunction)emu_query_spaces_filtered},
{"xrRetrieveSpaceQueryResultsFB", (PFN_xrVoidFunction)emu_retrieve_results},
@@ -468,11 +764,17 @@ static PFN_xrVoidFunction scene_emulation(const char *name) {
{"xrGetSpaceBoundingBox2DFB", (PFN_xrVoidFunction)emu_get_bounding_box_2d},
{"xrGetSpaceBoundingBox3DFB", (PFN_xrVoidFunction)emu_get_bounding_box_3d},
{"xrGetSpaceBoundary2DFB", (PFN_xrVoidFunction)emu_get_boundary_2d},
{"xrGetSpaceTriangleMeshMETA", (PFN_xrVoidFunction)emu_get_triangle_mesh},
{"xrGetSpaceSemanticLabelsFB", (PFN_xrVoidFunction)emu_get_semantic_labels},
{"xrRequestSceneCaptureFB", (PFN_xrVoidFunction)emu_request_scene_capture},
{"xrCreateSpatialAnchorFB", (PFN_xrVoidFunction)emu_create_anchor},
{"xrSaveSpaceFB", (PFN_xrVoidFunction)emu_save_space},
{"xrEraseSpaceFB", (PFN_xrVoidFunction)emu_erase_space},
{"xrDiscoverSpacesMETA", (PFN_xrVoidFunction)emu_discover_spaces},
{"xrRetrieveSpaceDiscoveryResultsMETA", (PFN_xrVoidFunction)emu_retrieve_discovery},
{"xrSaveSpacesMETA", (PFN_xrVoidFunction)emu_save_spaces},
{"xrEraseSpacesMETA", (PFN_xrVoidFunction)emu_erase_spaces},
{"xrRequestBoundaryVisibilityMETA", (PFN_xrVoidFunction)emu_request_boundary_visibility},
};
for (size_t i = 0; i < sizeof(table) / sizeof(table[0]); ++i)
if (!strcmp(name, table[i].name)) return table[i].fn;
@@ -485,6 +787,19 @@ static void scene_on_create_session(XrSession session) {
PFN_xrCreateReferenceSpace create = (PFN_xrCreateReferenceSpace)lookup(active_instance, "xrCreateReferenceSpace");
XrReferenceSpaceCreateInfo ci = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO, NULL, XR_REFERENCE_SPACE_TYPE_STAGE,
{{0, 0, 0, 1}, {0, 0, 0}}};
pthread_mutex_lock(&scene_lock);
if (scan_waiting) clean_scan_files();
scan_waiting = 0;
pending_capture_at = 0;
reset_room();
scene_generation = (scene_generation + 1) & 255;
event_head = event_tail = 0;
for (int i = 0; i < MAX_QUERIES; ++i) { free(queries[i].results); memset(&queries[i], 0, sizeof(queries[i])); }
query_next = 0;
scene_session = session;
scene_stage = XR_NULL_HANDLE;
if (create && XR_SUCCEEDED(create(session, &ci, &scene_stage))) LOG("scene: emulation active (anchored to STAGE)");
build_room();
load_anchors();
pthread_mutex_unlock(&scene_lock);
}
+140
View File
@@ -0,0 +1,140 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by scene_emu.c, using the cached OpenXR registry declarations (no private ABI copies).
static int meta_filter_matches(const XrSpaceDiscoveryInfoMETA *info, const scene_entity *e) {
for (uint32_t k = 0; k < info->filterCount; ++k) {
const XrSpaceFilterBaseHeaderMETA *f = info->filters[k];
if (!f) continue;
if (f->type == XR_TYPE_SPACE_FILTER_COMPONENT_META &&
!entity_has_component(e, ((const XrSpaceFilterComponentMETA *)f)->componentType)) return 0;
if (f->type == XR_TYPE_SPACE_FILTER_UUID_META) {
const XrSpaceFilterUuidMETA *u = (const XrSpaceFilterUuidMETA *)f;
if (!u->uuids || !uuid_matches(u->uuidCount, u->uuids, e)) return 0;
}
}
return 1;
}
// Diagnostics: MRUK takes the room whose floor outline contains the head as the "current room"; outside it, a game
// can ask for a room scan again and again (Pinball FX). Logs the head against the floor outline (3 times per session).
static void log_head_in_room(void) {
static int logged;
static XrSpace view = XR_NULL_HANDLE;
const scene_entity *f = &entities[ENT_FLOOR];
if (logged >= 3 || !scene_session || !scene_stage || !last_predicted_time || entity_count <= ENT_FLOOR ||
!f->has_bounds) return;
if (!view) {
PFN_xrCreateReferenceSpace create = (PFN_xrCreateReferenceSpace)lookup(active_instance, "xrCreateReferenceSpace");
XrReferenceSpaceCreateInfo ci = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO, NULL, XR_REFERENCE_SPACE_TYPE_VIEW,
{{0, 0, 0, 1}, {0, 0, 0}}};
if (!create || XR_FAILED(create(scene_session, &ci, &view))) return;
}
XrPosef head;
XrSpaceLocationFlags flags = 0;
if (!stage_pose(view, last_predicted_time, &head, &flags)) return;
XrVector3f p = pose_mul(pose_inv(f->pose_in_stage), head).position; // floor plane: x, y; its normal: z
XrVector2f rect[4], *v = f->boundary;
uint32_t n = f->boundary_count;
if (!n) {
const XrRect2Df b = f->bounds;
rect[0] = (XrVector2f){b.offset.x, b.offset.y};
rect[1] = (XrVector2f){b.offset.x + b.extent.width, b.offset.y};
rect[2] = (XrVector2f){b.offset.x + b.extent.width, b.offset.y + b.extent.height};
rect[3] = (XrVector2f){b.offset.x, b.offset.y + b.extent.height};
v = rect;
n = 4;
}
int inside = 0;
for (uint32_t a = 0, b = n - 1; a < n; b = a++)
if ((v[a].y > p.y) != (v[b].y > p.y) && p.x < (v[b].x - v[a].x) * (p.y - v[a].y) / (v[b].y - v[a].y) + v[a].x)
inside = !inside;
LOG("scene: head in STAGE %.2f,%.2f,%.2f (flags 0x%llx), floor-local %.2f,%.2f: %s the floor outline (%u points)",
head.position.x, head.position.y, head.position.z, (unsigned long long)flags, p.x, p.y,
inside ? "inside" : "OUTSIDE", n);
++logged;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_discover_spaces(XrSession session, const XrSpaceDiscoveryInfoMETA *info,
XrAsyncRequestIdFB *request) {
if (!info || !request || (info->filterCount && !info->filters)) return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
scene_session = session;
build_room();
if (event_free() < 2) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
scene_query *q = new_query();
if (!q) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_OUT_OF_MEMORY; }
for (int i = 0; i < entity_count; ++i) {
scene_entity *e = &entities[i];
if (!e->erased && meta_filter_matches(info, e))
q->results[q->count++] = (XrSpaceQueryResultFB){e->handle, e->uuid};
}
*request = q->id;
XrEventDataSpaceDiscoveryResultsAvailableMETA avail = {XR_TYPE_EVENT_DATA_SPACE_DISCOVERY_RESULTS_AVAILABLE_META, NULL, *request};
XrEventDataSpaceDiscoveryCompleteMETA done = {XR_TYPE_EVENT_DATA_SPACE_DISCOVERY_COMPLETE_META, NULL, *request, XR_SUCCESS};
push_event(&avail, sizeof(avail)); push_event(&done, sizeof(done));
LOG("scene: discovery request=%llu results=%u", (unsigned long long)*request, q->count);
log_head_in_room();
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_retrieve_discovery(XrSession session, XrAsyncRequestIdFB request,
XrSpaceDiscoveryResultsMETA *results) {
(void)session;
if (!results) return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
scene_query *q = find_query(request);
XrResult r = XR_SUCCESS;
if (!q) r = XR_ERROR_VALIDATION_FAILURE;
else {
results->resultCountOutput = q->count;
if (results->resultCapacityInput && results->resultCapacityInput < q->count) r = XR_ERROR_SIZE_INSUFFICIENT;
else if (results->resultCapacityInput) {
if (!results->results) r = XR_ERROR_VALIDATION_FAILURE;
else for (uint32_t i = 0; i < q->count; ++i)
results->results[i] = (XrSpaceDiscoveryResultMETA){q->results[i].space, q->results[i].uuid};
}
}
pthread_mutex_unlock(&scene_lock);
return r;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_save_spaces(XrSession session, const XrSpacesSaveInfoMETA *info,
XrAsyncRequestIdFB *request) {
(void)session;
if (!info || !request || !info->spaceCount || !info->spaces) return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
if (!event_free()) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
XrResult r = persist_spaces(info->spaceCount, info->spaces, 0, NULL, 0);
*request = next_request++;
XrEventDataSpacesSaveResultMETA done = {XR_TYPE_EVENT_DATA_SPACES_SAVE_RESULT_META, NULL, *request, r};
push_event(&done, sizeof(done));
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
// XR_META_boundary_visibility: the Frame shows no guardian over an app anyway, so a request always succeeds; the
// change event (once per new state, like Meta's runtime) is what MR games wait for before they start.
static XrBoundaryVisibilityMETA boundary_visibility = XR_BOUNDARY_VISIBILITY_NOT_SUPPRESSED_META;
static XRAPI_ATTR XrResult XRAPI_CALL emu_request_boundary_visibility(XrSession session, XrBoundaryVisibilityMETA v) {
(void)session;
if (v != XR_BOUNDARY_VISIBILITY_SUPPRESSED_META && v != XR_BOUNDARY_VISIBILITY_NOT_SUPPRESSED_META)
return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
if (v != boundary_visibility) {
boundary_visibility = v;
XrEventDataBoundaryVisibilityChangedMETA e = {XR_TYPE_EVENT_DATA_BOUNDARY_VISIBILITY_CHANGED_META, NULL, v};
push_event(&e, sizeof(e));
LOG("scene: boundary %s", v == XR_BOUNDARY_VISIBILITY_SUPPRESSED_META ? "suppressed" : "shown");
}
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_erase_spaces(XrSession session, const XrSpacesEraseInfoMETA *info,
XrAsyncRequestIdFB *request) {
(void)session;
if (!info || !request || (info->spaceCount && !info->spaces) || (info->uuidCount && !info->uuids))
return XR_ERROR_VALIDATION_FAILURE;
pthread_mutex_lock(&scene_lock);
if (!event_free()) { pthread_mutex_unlock(&scene_lock); return XR_ERROR_LIMIT_REACHED; }
XrResult r = persist_spaces(info->spaceCount, info->spaces, info->uuidCount, info->uuids, 1);
*request = next_request++;
XrEventDataSpacesEraseResultMETA done = {XR_TYPE_EVENT_DATA_SPACES_ERASE_RESULT_META, NULL, *request, r};
push_event(&done, sizeof(done));
pthread_mutex_unlock(&scene_lock);
return XR_SUCCESS;
}
+268
View File
@@ -0,0 +1,268 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by scene_emu.c. Bounded parsing keeps corrupt scan files from exhausting an Android game.
#include <errno.h>
#include <stdint.h>
static int room_uuid(const char *s, XrUuidEXT *u) {
if (strlen(s) != 32) return 0;
for (int i = 0; i < 16; ++i) {
unsigned n = 0;
for (int k = 0; k < 2; ++k) {
char c = s[i * 2 + k];
if (c >= '0' && c <= '9') n = n * 16 + (unsigned)(c - '0');
else if (c >= 'a' && c <= 'f') n = n * 16 + (unsigned)(c - 'a' + 10);
else return 0;
}
u->data[i] = (uint8_t)n;
}
return 1;
}
static int room_float(const char *s, float *v) {
char *end;
errno = 0;
*v = strtof(s, &end);
return *s && !*end && !errno && isfinite(*v) && fabsf(*v) <= 100000;
}
static int room_uint(const char *s, uint64_t *v) {
char *end;
if (*s < '0' || *s > '9') return 0;
errno = 0;
*v = strtoull(s, &end, 10);
return !*end && !errno;
}
static int room_tokens(char *s, char **t, int cap) {
char *comment = strchr(s, '#');
if (comment) *comment = 0;
int n = 0;
char *save;
for (char *p = strtok_r(s, " \t\r\n", &save); p; p = strtok_r(NULL, " \t\r\n", &save)) {
if (n == cap) return -1;
t[n++] = p;
}
return n;
}
static int room_values(char **t, int start, int count, float *v) {
for (int i = 0; i < count; ++i) if (!room_float(t[start + i], &v[i])) return 0;
return 1;
}
static int room_path(char *out, size_t size, const char *name) {
const char *config = getenv("FRAMEBRIDGE_CONFIG");
if (!config || !*config || strchr(name, '/') || strchr(name, '\\') || !strcmp(name, "..")) return 0;
const char *slash = strrchr(config, '/');
int n = slash ? snprintf(out, size, "%.*s/%s", (int)(slash - config), config, name)
: snprintf(out, size, "%s", name);
return n >= 0 && (size_t)n < size;
}
static uint32_t room_u32(const unsigned char *p) {
return (uint32_t)p[0] | (uint32_t)p[1] << 8 | (uint32_t)p[2] << 16 | (uint32_t)p[3] << 24;
}
static int room_mesh(scene_entity *e, const char *name, uint64_t nv, uint64_t ni) {
// At most 200k triangles and 16 MiB of payload; validate every index before publishing the mesh.
if (!nv || nv > 600000 || !ni || ni > 600000 || ni % 3 || nv * 12 + ni * 4 > 16 * 1024 * 1024) return 0;
char path[4096];
if (!room_path(path, sizeof(path), name)) return 0;
FILE *f = fopen(path, "rb");
if (!f) return 0;
unsigned char header[16], word[4];
int ok = fread(header, 1, 16, f) == 16 && !memcmp(header, "FPRM", 4) && room_u32(header + 4) == 1 &&
room_u32(header + 8) == nv && room_u32(header + 12) == ni;
XrVector3f *verts = ok ? malloc((size_t)nv * sizeof(*verts)) : NULL;
uint32_t *indices = ok ? malloc((size_t)ni * sizeof(*indices)) : NULL;
ok = ok && verts && indices;
for (uint64_t i = 0; ok && i < nv; ++i) {
float xyz[3];
for (int k = 0; ok && k < 3; ++k) {
ok = fread(word, 1, 4, f) == 4;
uint32_t bits = ok ? room_u32(word) : 0;
memcpy(&xyz[k], &bits, 4);
ok = ok && isfinite(xyz[k]) && fabsf(xyz[k]) <= 100000;
}
if (ok) verts[i] = (XrVector3f){xyz[0], xyz[1], xyz[2]};
}
for (uint64_t i = 0; ok && i < ni; ++i) {
ok = fread(word, 1, 4, f) == 4;
indices[i] = ok ? room_u32(word) : 0;
ok = ok && indices[i] < nv;
}
ok = ok && fgetc(f) == EOF && !ferror(f);
fclose(f);
if (!ok) { free(verts); free(indices); return 0; }
e->vertices = verts; e->indices = indices;
e->vertex_count = (uint32_t)nv; e->index_count = (uint32_t)ni;
return 1;
}
static int room_set_uuid(int i, XrUuidEXT uuid) {
for (int k = 0; k < entity_count; ++k)
if (k != i && uuid_equal(&entities[k].uuid, &uuid)) return 0;
entities[i].uuid = uuid;
return 1;
}
static int room_wall(XrUuidEXT uuid, const float *v, const char *label) {
float dx = v[2] - v[0], dz = v[3] - v[1], length = hypotf(dx, dz);
if (length < 0.001f || v[5] <= v[4]) return 0;
float angle = atan2f(-dz, dx) / 2;
int i = add_entity((XrPosef){{0, sinf(angle), 0, cosf(angle)},
{(v[0] + v[2]) / 2, (v[4] + v[5]) / 2, (v[1] + v[3]) / 2}},
label, length, v[5] - v[4], 0, 0);
if (i < 0) return 0;
if (!room_set_uuid(i, uuid)) { --entity_count; return 0; }
entities[i].is_wall = 1;
return 1;
}
static int room_outline(scene_entity *e, const XrVector2f *points, int n, float cx, float cz, int ceiling) {
e->boundary = malloc((size_t)n * sizeof(*e->boundary));
if (!e->boundary) return 0;
e->boundary_count = (uint32_t)n;
float xmin = INFINITY, ymin = INFINITY, xmax = -INFINITY, ymax = -INFINITY;
for (int i = 0; i < n; ++i) {
XrVector2f p = {points[i].x - cx, (points[i].y - cz) * (ceiling ? 1 : -1)};
e->boundary[i] = p;
xmin = fminf(xmin, p.x); xmax = fmaxf(xmax, p.x);
ymin = fminf(ymin, p.y); ymax = fmaxf(ymax, p.y);
}
e->has_bounds = 1;
e->bounds = (XrRect2Df){{xmin, ymin}, {xmax - xmin, ymax - ymin}};
return 1;
}
static int load_room_file(void) {
char path[4096];
if (!room_path(path, sizeof(path), "room.txt")) return 0;
FILE *f = fopen(path, "r");
if (!f) return 0;
// Long enough for a 1024-point outline. Reject whole overlong lines, never interpret their tails.
char *line = malloc(65536);
char **t = malloc((2 * MAX_OUTLINE + 4) * sizeof(*t));
if (!line || !t) { free(line); free(t); fclose(f); return 0; }
int ok = fgets(line, 65536, f) != NULL;
int n = ok ? room_tokens(line, t, 2 * MAX_OUTLINE + 4) : 0;
if (n != 2 || strcmp(t[0], "frameport-room") || strcmp(t[1], "1")) {
LOG("scene: bad room magic/version -> guardian box");
free(line); free(t); fclose(f); return 0;
}
const float s = 0.70710678f;
add_entity((XrPosef){{0, 0, 0, 1}, {0, 0, 0}}, NULL, 0, 0, 1, 0);
add_entity((XrPosef){{-s, 0, 0, s}, {0, 0, 0}}, "FLOOR", 0, 0, 0, 0);
add_entity((XrPosef){{s, 0, 0, s}, {0, 2.5f, 0}}, "CEILING", 0, 0, 0, 0);
XrVector2f outline[MAX_OUTLINE];
int outline_n = 0, ceiling_set = 0, volumes = 0, planes = 0, triangles = 0;
unsigned lineno = 1;
snprintf(room_source, sizeof(room_source), "manual");
while (fgets(line, 65536, f)) {
++lineno;
if (!strchr(line, '\n') && !feof(f)) {
int c; while ((c = fgetc(f)) != EOF && c != '\n') {}
LOG("scene: bad room line %u (too long)", lineno); continue;
}
n = room_tokens(line, t, 2 * MAX_OUTLINE + 4);
if (!n) continue;
ok = n > 0;
float v[10]; XrUuidEXT uuid; uint64_t a, b;
if (n < 0) ok = 0;
else if (!strcmp(t[0], "source")) {
ok = n == 2 && (!strcmp(t[1], "scan") || !strcmp(t[1], "boundary") || !strcmp(t[1], "box") || !strcmp(t[1], "manual"));
if (ok) snprintf(room_source, sizeof(room_source), "%s", t[1]);
} else if (!strcmp(t[0], "universe")) {
ok = n == 2 && room_uint(t[1], &a);
if (ok) room_universe = a;
} else if (!strcmp(t[0], "room") || !strcmp(t[0], "floor") || !strcmp(t[0], "ceiling")) {
int i = !strcmp(t[0], "room") ? ENT_ROOM : !strcmp(t[0], "floor") ? ENT_FLOOR : ENT_CEILING;
ok = n == (i == ENT_ROOM ? 2 : 3) && room_uuid(t[1], &uuid) &&
(i == ENT_ROOM || room_float(t[2], v)) && room_set_uuid(i, uuid);
if (ok && i != ENT_ROOM) entities[i].pose_in_stage.position.y = v[0];
if (ok && i == ENT_CEILING) ceiling_set = 1;
} else if (!strcmp(t[0], "outline")) {
ok = n >= 2 && room_uint(t[1], &a) && a >= 3 && a <= MAX_OUTLINE && n == 2 + 2 * (int)a;
XrVector2f parsed[MAX_OUTLINE];
for (uint64_t k = 0; ok && k < a; ++k) ok = room_float(t[2 + k * 2], &parsed[k].x) && room_float(t[3 + k * 2], &parsed[k].y);
if (ok) { memcpy(outline, parsed, (size_t)a * sizeof(*outline)); outline_n = (int)a; }
} else if (!strcmp(t[0], "wall")) {
// optional 8th field: WALL_FACE (default) or INVISIBLE_WALL_FACE for an edge the scanner closed itself
const char *label = n == 9 ? t[8] : "WALL_FACE";
ok = (n == 8 || (n == 9 && (!strcmp(label, "WALL_FACE") || !strcmp(label, "INVISIBLE_WALL_FACE")))) &&
room_uuid(t[1], &uuid) && room_values(t, 2, 6, v) && room_wall(uuid, v, label);
} else if (!strcmp(t[0], "volume") || !strcmp(t[0], "plane")) {
int volume = !strcmp(t[0], "volume");
ok = n == (volume ? 10 : 12) && room_uuid(t[1], &uuid) && strlen(t[2]) < sizeof(entities[0].label) && room_values(t, 3, volume ? 7 : 9, v);
XrPosef pose = {{0, 0, 0, 1}, {0, 0, 0}};
float w = 0, h = 0;
if (ok && volume) {
float yaw = v[3] * 0.00872664626f;
pose = (XrPosef){quat_mul((XrQuaternionf){0, sinf(yaw), 0, cosf(yaw)}, (XrQuaternionf){-s, 0, 0, s}), {v[0], v[2], v[1]}};
w = v[4]; h = v[5]; ok = v[6] > 0;
} else if (ok) {
float norm = sqrtf(v[3]*v[3]+v[4]*v[4]+v[5]*v[5]+v[6]*v[6]);
ok = norm > 0.0001f;
if (ok) pose = (XrPosef){{v[3]/norm,v[4]/norm,v[5]/norm,v[6]/norm}, {v[0],v[1],v[2]}};
w = v[7]; h = v[8];
}
int i = ok && w > 0 && h > 0 ? add_entity(pose, t[2], w, h, 0, 0) : -1;
ok = i >= 0;
if (ok && !room_set_uuid(i, uuid)) { --entity_count; ok = 0; }
if (ok && volume) {
entities[i].has_bounds3 = 1;
entities[i].bounds3 = (XrRect3DfFB){{-w/2, -h/2, -v[6]}, {w, h, v[6]}};
++volumes;
} else if (ok) ++planes;
} else if (!strcmp(t[0], "mesh")) {
if (!scene_app_mesh) continue; // the app didn't enable XR_META_spatial_entity_mesh
ok = n == 5 && room_uuid(t[1], &uuid) && room_uint(t[3], &a) && room_uint(t[4], &b) && !triangles;
int i = ok ? add_entity((XrPosef){{0,0,0,1},{0,0,0}}, "GLOBAL_MESH", 0, 0, 0, 0) : -1;
ok = i >= 0 && room_set_uuid(i, uuid) && room_mesh(&entities[i], t[2], a, b);
if (ok) triangles = (int)b / 3;
else if (i >= 0) --entity_count;
}
if (!ok) LOG("scene: bad room line %u (skipped)", lineno);
}
ok = !ferror(f);
free(line); free(t); fclose(f);
// Layout walls must be contiguous and in file order, regardless of where other records occurred.
for (int i = ENT_WALL0; i < entity_count; ++i) if (entities[i].is_wall) {
scene_entity wall = entities[i];
memmove(&entities[ENT_WALL0 + wall_count + 1], &entities[ENT_WALL0 + wall_count],
(size_t)(i - ENT_WALL0 - wall_count) * sizeof(wall));
entities[ENT_WALL0 + wall_count++] = wall;
}
if (!outline_n && wall_count) {
float xmin = INFINITY, zmin = INFINITY, xmax = -INFINITY, zmax = -INFINITY;
for (int i = ENT_WALL0; i < ENT_WALL0 + wall_count; ++i) {
scene_entity *e = &entities[i];
XrVector3f d = quat_rotate(e->pose_in_stage.orientation, (XrVector3f){e->bounds.extent.width/2,0,0});
xmin = fminf(xmin, e->pose_in_stage.position.x - fabsf(d.x)); xmax = fmaxf(xmax, e->pose_in_stage.position.x + fabsf(d.x));
zmin = fminf(zmin, e->pose_in_stage.position.z - fabsf(d.z)); zmax = fmaxf(zmax, e->pose_in_stage.position.z + fabsf(d.z));
}
outline[0]=(XrVector2f){xmin,zmin}; outline[1]=(XrVector2f){xmax,zmin};
outline[2]=(XrVector2f){xmax,zmax}; outline[3]=(XrVector2f){xmin,zmax}; outline_n=4;
}
double area = 0, cx = 0, cz = 0;
for (int i = 0; i < outline_n; ++i) {
XrVector2f a = outline[i], b = outline[(i+1)%outline_n];
double cross = (double)a.x*b.y-(double)b.x*a.y;
area += cross; cx += (a.x+b.x)*cross; cz += (a.y+b.y)*cross;
}
if (!ok || outline_n < 3 || fabs(area) < 0.00001) { reset_room(); return 0; }
cx /= 3*area; cz /= 3*area;
if (!ceiling_set) entities[ENT_CEILING].pose_in_stage.position.y = entities[ENT_FLOOR].pose_in_stage.position.y + 2.5f;
for (int i = ENT_FLOOR; i <= ENT_CEILING; ++i) {
entities[i].pose_in_stage.position.x = (float)cx; entities[i].pose_in_stage.position.z = (float)cz;
if (!room_outline(&entities[i], outline, outline_n, (float)cx, (float)cz, i == ENT_CEILING)) { reset_room(); return 0; }
}
if (!wall_count) for (int i = 0; i < outline_n; ++i) {
XrVector2f a = outline[i], b = outline[(i+1)%outline_n];
float v[] = {a.x,a.y,b.x,b.y,entities[ENT_FLOOR].pose_in_stage.position.y,entities[ENT_CEILING].pose_in_stage.position.y};
if (!room_wall(make_uuid(entity_count), v, "WALL_FACE")) { reset_room(); return 0; }
++wall_count;
}
// Move generated walls ahead of furniture as well.
if (wall_count && !entities[ENT_WALL0].is_wall) {
scene_entity *walls = malloc((size_t)wall_count * sizeof(*walls));
if (!walls) { reset_room(); return 0; }
memcpy(walls, &entities[entity_count-wall_count], (size_t)wall_count*sizeof(*walls));
memmove(&entities[ENT_WALL0+wall_count], &entities[ENT_WALL0], (size_t)(entity_count-wall_count-ENT_WALL0)*sizeof(*walls));
memcpy(&entities[ENT_WALL0], walls, (size_t)wall_count*sizeof(*walls)); free(walls);
}
for (int i = 0; i < entity_count; ++i) entities[i].handle = fake_space(i);
room_entity_count = entity_count; room_from_file = 1;
LOG("scene: room from %s file (%d walls, %d volumes, %d planes, %d mesh triangles, universe %llu)",
room_source, wall_count, volumes, planes, triangles, (unsigned long long)room_universe);
return 1;
}
+107
View File
@@ -0,0 +1,107 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by scene_emu.c; all scan state is protected by scene_lock.
static int scan_path(char *path, size_t size, const char *kind) {
char anchor[4096];
if (!anchor_path(anchor, sizeof(anchor))) return 0;
char *slash = strrchr(anchor, '/');
if (!slash) return 0;
*slash = 0;
int n = snprintf(path, size, "%s/framebridge-scan-%s", anchor, kind);
return n > 0 && (size_t)n < size;
}
static void clean_scan_files(void) {
const char *kinds[] = {"request", "ack", "result"};
char path[4096];
for (unsigned i = 0; i < 3; ++i) if (scan_path(path, sizeof(path), kinds[i])) unlink(path);
}
static int start_scan(XrAsyncRequestIdFB id) {
char path[4096], tmp[4128];
if (!scan_path(path, sizeof(path), "request")) return 0;
clean_scan_files();
snprintf(tmp, sizeof(tmp), "%s.tmp-XXXXXX", path);
int fd = mkstemp(tmp);
if (fd < 0) { LOG("scene: scan request write failed (%d), using the stored room", errno); return 0; }
FILE *f = fdopen(fd, "w");
if (!f) { close(fd); unlink(tmp); return 0; }
int ok = fprintf(f, "request %llu %llu\n", (unsigned long long)id, (unsigned long long)time(NULL)) > 0;
if (fflush(f) || fsync(fd)) ok = 0;
if (fclose(f)) ok = 0;
if (ok && rename(tmp, path)) ok = 0;
if (!ok) { unlink(tmp); LOG("scene: scan request write failed (%d), using the stored room", errno); return 0; }
scan_started_at = scene_now_ns();
scan_poll_at = 0;
scan_waiting = 1;
scan_acked = 0;
LOG("scene: scan requested from FramePort (request %llu)", (unsigned long long)id);
return 1;
}
static int scan_message(const char *kind, char *status, char *reason, size_t size) {
char path[4096], line[512];
struct stat st;
if (!scan_path(path, sizeof(path), kind) || stat(path, &st)) return 0;
FILE *f = fopen(path, "r");
if (!f) return 0;
int ok = fgets(line, sizeof(line), f) != NULL;
fclose(f);
unsigned long long id;
int used = 0;
if (!ok || sscanf(line, "%15s %llu %n", status, &id, &used) < 2 || id != pending_capture.requestId) return 0;
snprintf(reason, size, "%s", line + used);
reason[strcspn(reason, "\r\n")] = 0;
return 1;
}
static int reload_scanned_room(void) {
scene_entity *old = malloc(sizeof(entities));
if (!old) return 0;
memcpy(old, entities, sizeof(entities));
int old_count = entity_count, old_room_count = room_entity_count, old_walls = wall_count, old_file = room_from_file;
uint64_t old_universe = room_universe;
unsigned old_generation = scene_generation;
char old_source[sizeof(room_source)];
memcpy(old_source, room_source, sizeof(room_source));
memset(entities, 0, sizeof(entities));
entity_count = room_entity_count = wall_count = room_from_file = 0;
room_universe = 0;
scene_generation = (scene_generation + 1) & 255;
if (!load_room_file()) {
reset_room();
memcpy(entities, old, sizeof(entities));
entity_count = old_count; room_entity_count = old_room_count; wall_count = old_walls; room_from_file = old_file;
room_universe = old_universe; scene_generation = old_generation;
memcpy(room_source, old_source, sizeof(room_source));
free(old);
return 0;
}
for (int i = 0; i < old_count; ++i) {
free(old[i].boundary); free(old[i].vertices); free(old[i].indices);
}
free(old);
load_anchors();
return 1;
}
static void poll_scan(uint64_t now) {
if (now < scan_poll_at) return;
scan_poll_at = now + 250000000ull;
char status[16], reason[512];
if (!scan_acked && scan_message("ack", status, reason, sizeof(reason)) && !strcmp(status, "ack")) {
scan_acked = 1;
scan_started_at = now; // allow the scanner a full six minutes after acknowledging
LOG("scene: scan started by FramePort");
}
if (scan_message("result", status, reason, sizeof(reason)) &&
(!strcmp(status, "ok") || !strcmp(status, "failed"))) {
int ok = !strcmp(status, "ok") && reload_scanned_room();
pending_capture_at = now;
pending_capture.result = ok ? XR_SUCCESS : XR_ERROR_RUNTIME_FAILURE;
if (ok) LOG("scene: scanned room loaded (%d entities)", entity_count);
else LOG("scene: scan failed (%s)", !strcmp(status, "failed") ? reason : "could not load scanned room");
} else if (!scan_acked && now - scan_started_at >= 5000000000ull) {
LOG("scene: no scanner answered, using the stored room");
} else if (scan_acked && now - scan_started_at >= 360000000000ull) {
pending_capture_at = now;
pending_capture.result = XR_ERROR_RUNTIME_FAILURE;
LOG("scene: scan failed (timeout)");
} else return;
scan_waiting = 0;
clean_scan_files();
}
+389
View File
@@ -0,0 +1,389 @@
// Included after layer_emul_gl.c: reuses its dynamically loaded GLES entry points.
// Proactive at eight app swapchains: an atlas cannot be allocated after the runtime cap.
#include "pool_rect.h"
#define POOL_MAX 128
#define POOL_SIDE 4096
static XrSession pool_session;
static struct {
XrSwapchain handle;
GLuint images[16];
uint32_t count, index;
int acquired, waited;
} pool_atlas[2];
static struct {
XrSwapchain handle;
GLuint tex;
PoolRect rect;
int atlas, state, released, copied;
GLsync fence, read_fence;
} pool_images[POOL_MAX];
static unsigned pool_errors;
static XrSwapchain pool_runtime[256];
static unsigned pool_runtime_count;
static void pool_note_runtime_locked(XrSession session, XrSwapchain handle) {
if (!swapchain_pool || session != pool_session) return;
for (int i = 0; i < 256; ++i)
if (!pool_runtime[i]) {
pool_runtime[i] = handle;
++pool_runtime_count;
return;
}
}
static void pool_forget_runtime_locked(XrSwapchain handle) {
for (int i = 0; i < 256; ++i)
if (pool_runtime[i] == handle && handle) {
pool_runtime[i] = XR_NULL_HANDLE;
--pool_runtime_count;
return;
}
}
static int pool_should_create_locked(XrSession session) {
return swapchain_pool && session == pool_session && pool_runtime_count >= 8;
}
static int pool_find_locked(XrSwapchain h) {
for (int i = 0; h && i < POOL_MAX; ++i)
if (pool_images[i].handle == h) return i;
return -1;
}
static void pool_error(void) {
if (pool_errors++ % 240 == 0) LOG("swapchain_pool: atlas update failed; omitting affected layers");
}
static int pool_create_locked(XrSession session, const XrSwapchainCreateInfo *ci, XrSwapchain *out, int limited) {
if (!swapchain_pool || session != pool_session || !out || ci->next || ci->faceCount != 1 || ci->arraySize != 1 ||
ci->sampleCount != 1 || ci->mipCount != 1 || !(ci->usageFlags & XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT) ||
(ci->createFlags & ~XR_SWAPCHAIN_CREATE_STATIC_IMAGE_BIT) ||
(ci->format != GL_RGBA8 && ci->format != GL_SRGB8_ALPHA8) || !ci->width || !ci->height ||
ci->width > POOL_SIDE || ci->height > POOL_SIDE)
return 0;
if (!p_eglCreateContext && !emul_load_gl()) return 0;
if (p_eglGetCurrentContext() == EGL_NO_CONTEXT) return 0;
// Like emul_virtual_create: stale application GL errors must not look like allocation failures.
for (int i = 0; i < 8 && p_glGetError() != GL_NO_ERROR; ++i) {
}
int slot = -1;
for (int i = 0; i < POOL_MAX; ++i)
if (!pool_images[i].handle) {
slot = i;
break;
}
if (slot < 0) return 0;
for (int a = 0; a < 2; ++a) {
PoolRect used[POOL_MAX], rect;
int n = 0;
for (int i = 0; i < POOL_MAX; ++i)
if (pool_images[i].handle && pool_images[i].atlas == a) used[n++] = pool_images[i].rect;
if (!pool_place(used, n, POOL_SIDE, ci->width, ci->height, &rect)) continue;
if (!pool_atlas[a].handle) {
XrSwapchainCreateInfo ai = {XR_TYPE_SWAPCHAIN_CREATE_INFO};
ai.width = ai.height = POOL_SIDE;
ai.arraySize = ai.faceCount = ai.sampleCount = ai.mipCount = 1;
ai.format = GL_SRGB8_ALPHA8;
ai.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_TRANSFER_DST_BIT;
PFN_xrCreateSwapchain create = (PFN_xrCreateSwapchain)lookup(active_instance, "xrCreateSwapchain");
PFN_xrEnumerateSwapchainImages enumerate =
(PFN_xrEnumerateSwapchainImages)lookup(active_instance, "xrEnumerateSwapchainImages");
if (XR_FAILED(create(session, &ai, &pool_atlas[a].handle))) return 0;
emul_gles_image images[16];
for (int j = 0; j < 16; ++j) images[j] = (emul_gles_image){XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR};
if (XR_FAILED(enumerate(pool_atlas[a].handle, 16, &pool_atlas[a].count, (void *)images)) ||
!pool_atlas[a].count || pool_atlas[a].count > 16) {
((PFN_xrDestroySwapchain)lookup(active_instance, "xrDestroySwapchain"))(pool_atlas[a].handle);
memset(&pool_atlas[a], 0, sizeof pool_atlas[a]);
return 0;
}
for (uint32_t j = 0; j < pool_atlas[a].count; ++j) pool_atlas[a].images[j] = images[j].image;
}
GLint old;
GLuint tex = 0;
p_glGetIntegerv(GL_TEXTURE_BINDING_2D, &old);
p_glGenTextures(1, &tex);
p_glBindTexture(GL_TEXTURE_2D, tex);
p_glTexStorage2D(GL_TEXTURE_2D, 1, ci->format, ci->width, ci->height);
GLenum err = p_glGetError();
p_glBindTexture(GL_TEXTURE_2D, old);
if (err != GL_NO_ERROR) {
p_glDeleteTextures(1, &tex);
return 0;
}
pool_images[slot].handle = FAKE_HANDLE(XrSwapchain);
pool_images[slot].tex = tex;
pool_images[slot].rect = rect;
pool_images[slot].atlas = a;
*out = pool_images[slot].handle;
note_swapchain_size(*out, ci->width, ci->height);
remember_swapchain(*out);
emul_on_create_swapchain(*out, ci);
LOG("swapchain_pool: virtual %ux%u (%s)", ci->width, ci->height,
limited ? "runtime limit reached" : "runtime limit proactively avoided");
return 1;
}
return 0;
}
static void pool_destroy_locked(int i) {
emul_on_destroy_swapchain(pool_images[i].handle);
if (pool_images[i].fence && p_eglGetCurrentContext && p_eglGetCurrentContext() != EGL_NO_CONTEXT)
p_glDeleteSync(pool_images[i].fence);
if (p_eglGetCurrentContext && p_eglGetCurrentContext() != EGL_NO_CONTEXT)
p_glDeleteTextures(1, &pool_images[i].tex);
if (pool_images[i].read_fence && p_eglGetCurrentContext && p_eglGetCurrentContext() != EGL_NO_CONTEXT)
p_glDeleteSync(pool_images[i].read_fence);
forget_swapchain(pool_images[i].handle);
memset(&pool_images[i], 0, sizeof pool_images[i]);
}
static XrResult pool_enumerate_locked(int i, uint32_t cap, uint32_t *count, XrSwapchainImageBaseHeader *images) {
if (!count) return XR_ERROR_VALIDATION_FAILURE;
*count = 1;
if (!cap) return XR_SUCCESS;
if (!images || images->type != XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR) return XR_ERROR_VALIDATION_FAILURE;
((emul_gles_image *)images)->image = pool_images[i].tex;
return XR_SUCCESS;
}
static XrResult pool_cycle_locked(int i, int op, uint32_t *index) {
int *state = &pool_images[i].state;
if (op == 0) {
if (!index) return XR_ERROR_VALIDATION_FAILURE;
if (*state) return XR_ERROR_CALL_ORDER_INVALID;
if (pool_images[i].read_fence) {
if (p_eglGetCurrentContext() == EGL_NO_CONTEXT) return XR_ERROR_RUNTIME_FAILURE;
p_glWaitSync(pool_images[i].read_fence, 0, GL_TIMEOUT_IGNORED);
}
*index = 0;
*state = 1;
}
if (op == 1) {
if (*state != 1) return XR_ERROR_CALL_ORDER_INVALID;
*state = 2;
}
if (op == 2) {
if (*state != 2) return XR_ERROR_CALL_ORDER_INVALID;
if (p_eglGetCurrentContext() == EGL_NO_CONTEXT) return XR_ERROR_RUNTIME_FAILURE;
if (pool_images[i].fence) p_glDeleteSync(pool_images[i].fence);
pool_images[i].fence = p_glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
if (!pool_images[i].fence) return XR_ERROR_RUNTIME_FAILURE;
p_glFlush();
*state = 0;
pool_images[i].released = 1;
}
return XR_SUCCESS;
}
static void pool_cleanup_locked(XrSession session) {
if (session != pool_session) return;
for (int i = 0; i < POOL_MAX; ++i)
if (pool_images[i].handle) pool_destroy_locked(i);
for (int a = 0; a < 2; ++a)
if (pool_atlas[a].handle) {
((PFN_xrDestroySwapchain)lookup(active_instance, "xrDestroySwapchain"))(pool_atlas[a].handle);
memset(&pool_atlas[a], 0, sizeof pool_atlas[a]);
}
memset(pool_runtime, 0, sizeof pool_runtime);
pool_runtime_count = 0;
pool_session = XR_NULL_HANDLE;
}
// Called on final layers, after strips, flips and rect_clamp. Copy every referenced image
// each frame: runtime atlas images rotate, and untouched areas are not preserved.
static void pool_begin_frame_locked(void) {
for (int i = 0; i < POOL_MAX; ++i) pool_images[i].copied = 0;
}
static int pool_subimage_locked(XrSwapchainSubImage *sub) {
int i = pool_find_locked(sub->swapchain);
if (i < 0) return 1;
int a = pool_images[i].atlas;
if (!pool_images[i].released || pool_images[i].state || sub->imageArrayIndex) return 0;
PFN_xrAcquireSwapchainImage acquire =
(PFN_xrAcquireSwapchainImage)lookup(active_instance, "xrAcquireSwapchainImage");
PFN_xrWaitSwapchainImage wait = (PFN_xrWaitSwapchainImage)lookup(active_instance, "xrWaitSwapchainImage");
if (!pool_atlas[a].acquired) {
XrSwapchainImageAcquireInfo ai = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
if (XR_FAILED(acquire(pool_atlas[a].handle, &ai, &pool_atlas[a].index))) return 0;
pool_atlas[a].acquired = 1;
}
if (!pool_atlas[a].waited) {
XrSwapchainImageWaitInfo wi = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO, NULL, XR_INFINITE_DURATION};
if (wait(pool_atlas[a].handle, &wi) != XR_SUCCESS) return 0;
pool_atlas[a].waited = 1;
}
if (p_eglGetCurrentContext() == EGL_NO_CONTEXT) return 0;
PoolRect r = pool_images[i].rect;
if (!pool_images[i].copied) {
for (int e = 0; e < 8 && p_glGetError() != GL_NO_ERROR; ++e) {
}
if (pool_images[i].fence) p_glWaitSync(pool_images[i].fence, 0, GL_TIMEOUT_IGNORED);
GLint read, draw;
GLuint fb[2];
GLboolean scissor = p_glIsEnabled(GL_SCISSOR_TEST);
p_glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &read);
p_glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &draw);
p_glGenFramebuffers(2, fb);
p_glDisable(GL_SCISSOR_TEST);
p_glBindFramebuffer(GL_READ_FRAMEBUFFER, fb[0]);
p_glFramebufferTexture2D(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, pool_images[i].tex, 0);
p_glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fb[1]);
p_glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
pool_atlas[a].images[pool_atlas[a].index], 0);
int ok = p_glCheckFramebufferStatus(GL_READ_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE &&
p_glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE;
if (ok) p_glBlitFramebuffer(0, 0, r.w, r.h, r.x, r.y, r.x + r.w, r.y + r.h, GL_COLOR_BUFFER_BIT, GL_NEAREST);
ok &= p_glGetError() == GL_NO_ERROR;
p_glBindFramebuffer(GL_READ_FRAMEBUFFER, read);
p_glBindFramebuffer(GL_DRAW_FRAMEBUFFER, draw);
if (scissor) p_glEnable(GL_SCISSOR_TEST);
p_glDeleteFramebuffers(2, fb);
if (!ok) return 0;
if (pool_images[i].read_fence) p_glDeleteSync(pool_images[i].read_fence);
pool_images[i].read_fence = p_glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);
if (!pool_images[i].read_fence) return 0;
pool_images[i].copied = 1;
}
clamp_rect(&sub->imageRect, r.w, r.h);
if (sub->imageRect.offset.x < 0 || sub->imageRect.offset.y < 0 ||
(int64_t)sub->imageRect.offset.x + sub->imageRect.extent.width > r.w ||
(int64_t)sub->imageRect.offset.y + sub->imageRect.extent.height > r.h)
return 0;
sub->swapchain = pool_atlas[a].handle;
sub->imageRect.offset.x += r.x;
sub->imageRect.offset.y += r.y;
return 1;
}
static int pool_finish_locked(void) {
int ok = 1;
for (int a = 0; a < 2; ++a)
if (pool_atlas[a].acquired && pool_atlas[a].waited) {
p_glFlush();
XrSwapchainImageReleaseInfo ri = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
XrResult r = ((PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage"))(
pool_atlas[a].handle, &ri);
if (XR_FAILED(r))
ok = 0;
else
pool_atlas[a].acquired = pool_atlas[a].waited = 0;
}
return ok;
}
// Each OpenXR call may run on a different thread, including creates of different panels.
static pthread_mutex_t pool_lock = PTHREAD_MUTEX_INITIALIZER;
static int pool_find(XrSwapchain h) {
pthread_mutex_lock(&pool_lock);
int result = pool_find_locked(h);
pthread_mutex_unlock(&pool_lock);
return result;
}
static int pool_create(XrSession session, const XrSwapchainCreateInfo *ci, XrSwapchain *out, int limited) {
pthread_mutex_lock(&pool_lock);
int result = pool_create_locked(session, ci, out, limited);
pthread_mutex_unlock(&pool_lock);
return result;
}
static void pool_destroy(int i) {
pthread_mutex_lock(&pool_lock);
pool_destroy_locked(i);
pthread_mutex_unlock(&pool_lock);
}
static void pool_cleanup(XrSession session) {
pthread_mutex_lock(&pool_lock);
pool_cleanup_locked(session);
pthread_mutex_unlock(&pool_lock);
}
static XrResult pool_cycle(int i, int op, uint32_t *index) {
pthread_mutex_lock(&pool_lock);
XrResult result = pool_cycle_locked(i, op, index);
pthread_mutex_unlock(&pool_lock);
return result;
}
static XrResult pool_enumerate(int i, uint32_t cap, uint32_t *count, XrSwapchainImageBaseHeader *images) {
pthread_mutex_lock(&pool_lock);
XrResult result = pool_enumerate_locked(i, cap, count, images);
pthread_mutex_unlock(&pool_lock);
return result;
}
static void pool_begin_frame(void) {
pthread_mutex_lock(&pool_lock);
pool_begin_frame_locked();
pthread_mutex_unlock(&pool_lock);
}
static int pool_subimage(XrSwapchainSubImage *sub) {
pthread_mutex_lock(&pool_lock);
int result = pool_subimage_locked(sub);
pthread_mutex_unlock(&pool_lock);
return result;
}
static int pool_finish(void) {
pthread_mutex_lock(&pool_lock);
int result = pool_finish_locked();
pthread_mutex_unlock(&pool_lock);
return result;
}
static void pool_note_runtime(XrSession session, XrSwapchain handle) {
pthread_mutex_lock(&pool_lock);
pool_note_runtime_locked(session, handle);
pthread_mutex_unlock(&pool_lock);
}
static void pool_forget_runtime(XrSwapchain handle) {
pthread_mutex_lock(&pool_lock);
pool_forget_runtime_locked(handle);
pthread_mutex_unlock(&pool_lock);
}
static int pool_should_create(XrSession session) {
pthread_mutex_lock(&pool_lock);
int result = pool_should_create_locked(session);
pthread_mutex_unlock(&pool_lock);
return result;
}
// Caller-owned storage stays live until the runtime's xrEndFrame returns. The input layer
// structs are const: never mutate them, including repeated references from strips or stereo.
typedef struct {
XrCompositionLayerQuad quads[64];
XrCompositionLayerCylinderKHR cylinders[64];
XrCompositionLayerProjection projections[64];
XrCompositionLayerProjectionView views[64][2];
} PoolFrame;
static uint32_t pool_rewrite_layers(const XrCompositionLayerBaseHeader **layers, uint32_t count, PoolFrame *frame) {
pool_begin_frame();
uint32_t n = 0;
for (uint32_t j = 0; j < count; ++j) {
const XrCompositionLayerBaseHeader *l = layers[j];
int ok = 1;
if (l->type == XR_TYPE_COMPOSITION_LAYER_QUAD) {
frame->quads[j] = *(const XrCompositionLayerQuad *)l;
ok = pool_subimage(&frame->quads[j].subImage);
l = (const void *)&frame->quads[j];
} else if (l->type == XR_TYPE_COMPOSITION_LAYER_CYLINDER_KHR) {
frame->cylinders[j] = *(const XrCompositionLayerCylinderKHR *)l;
ok = pool_subimage(&frame->cylinders[j].subImage);
l = (const void *)&frame->cylinders[j];
} else if (l->type == XR_TYPE_COMPOSITION_LAYER_PROJECTION) {
const XrCompositionLayerProjection *p = (const void *)l;
if (p->viewCount <= 2) {
frame->projections[j] = *p;
for (uint32_t v = 0; v < p->viewCount; ++v) {
frame->views[j][v] = p->views[v];
ok &= pool_subimage(&frame->views[j][v].subImage);
}
frame->projections[j].views = frame->views[j];
l = (const void *)&frame->projections[j];
}
}
if (ok)
layers[n++] = l;
else
pool_error();
}
if (!pool_finish()) {
pool_error();
return 0;
}
return n;
}
+27 -5
View File
@@ -10,7 +10,7 @@ timefix OpenXR layer for Proton games (linux-arm64, glibc; the NDK's clang build
OculusHMDConnected helper for Rift games under Proton (win-x64 PE; the NDK's clang + lld-link, no Windows SDK), the
live view's hardware H.264 encoder fp_venc (linux-arm64-bin, static freestanding executable), and rewrites artifacts/SHA256SUMS. Run `frameport parity` afterwards to see which games change.
python native/build.py [--only adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,zinkfix,foveation,ovrpshim,tlskeys,vrsettings,venc,vroverlay_probe] [--ndk PATH]
python native/build.py [--only adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrruntime,xrshim,vkshim,zinkfix,foveation,ovrpshim,tlskeys,vrsettings,venc,vroverlay_probe] [--ndk PATH]
"""
from __future__ import annotations
@@ -201,6 +201,18 @@ def build_venc(tc: Path):
f"-ffile-prefix-map={HERE}=native", "fp_venc.c", "convert.c", "sys.c", "-o", out / "fp_venc"], cwd=src)
def build_roomscan(tc: Path):
"""Offline room reconstruction (Linux aarch64). No libc, headers, generated tables or fetched dependencies."""
src = HERE / "roomscan"
out = ART / "linux-arm64-bin"
out.mkdir(parents=True, exist_ok=True)
run([tc / "bin/clang", "--target=aarch64-linux-gnu", "-ffreestanding", "-nostdlib", "-nostdlibinc", "-static",
"-fno-stack-protector", "-fno-builtin", "-ffp-contract=off", "-fno-math-errno", "-O2", "-Wall", "-Wextra", "-Werror",
"-fuse-ld=lld", "-Wl,--build-id=none", "-Wl,-z,max-page-size=65536", "-e", "_start",
f"-ffile-prefix-map={HERE}=native", "main.c", "fusion.c", "model.c", "mesh.c", "refine.c", "rules.c",
"roomio.c", "platform.c", "-o", out / "fp_roomscan"], cwd=src)
OCULUSHMD_IMPORTS = ("CreateEventW", "CreateJobObjectW", "AssignProcessToJobObject", "QueryInformationJobObject",
"CreateProcessW", "ResumeThread", "WaitForSingleObject", "GetExitCodeProcess", "CloseHandle",
"Sleep", "GetCommandLineW", "GetLastError", "GetStdHandle", "WriteFile", "ExitProcess")
@@ -331,6 +343,15 @@ def build_ovrtrace(tc: Path):
"-o", ART / "arm64-v8a/libfp_ovrtrace.so"], cwd=src)
def build_xrruntime(tc: Path):
src = HERE / "xrruntime"
run([exe(tc, "aarch64-linux-android29-clang"), "-shared", "-O2", "-fPIC", "-Wall", "-Wextra", "-Werror",
f"-ffile-prefix-map={ROOT}=.", "-fvisibility=hidden", "-Wl,-Bsymbolic",
"-I", openxr_include("adapter"), "-Wl,-soname,libfp_xrruntime.so",
"-Wl,-z,max-page-size=16384", "xrruntime.c", "-ldl", "-llog",
"-o", ART / "arm64-v8a/libfp_xrruntime.so"], cwd=src)
def build_xrshim(tc: Path):
"""FrameBridge extension shim (see xrshim/xrshim.c): DT_NEEDED-injected in front of overport's libopenxr_loader.so."""
src = HERE / "xrshim"
@@ -440,7 +461,7 @@ def write_sums():
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--only", default="adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,zinkfix,foveation,ovrpshim,tlskeys,luajit,vrsettings,venc,a32")
ap.add_argument("--only", default="adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrruntime,xrshim,vkshim,zinkfix,foveation,ovrpshim,tlskeys,luajit,vrsettings,venc,roomscan,a32")
ap.add_argument("--ndk")
args = ap.parse_args()
parts = set(args.only.split(","))
@@ -448,7 +469,7 @@ def main():
(ART / d).mkdir(parents=True, exist_ok=True)
tc = clang_dir(ndk(args.ndk)) if parts - {"dex"} else None
steps = {"adapter": lambda: build_adapter(tc), "bridge": lambda: build_bridge(tc), "compat": lambda: build_compat(tc),
"langpack": lambda: build_langpack(tc),
"xrruntime": lambda: build_xrruntime(tc), "langpack": lambda: build_langpack(tc),
"glshim": lambda: build_glshim(tc), "eglfmt": lambda: build_eglfmt(tc), "glmv": lambda: build_glmv(tc), "ovrtrace": lambda: build_ovrtrace(tc), "xrshim": lambda: build_xrshim(tc), "dex": build_dex, "xrlayer": lambda: build_xrlayer(tc),
"oculushmd": lambda: build_oculushmd(tc), "vkshim": lambda: build_vkshim(tc),
"zinkfix": lambda: build_zinkfix(tc),
@@ -456,9 +477,10 @@ def main():
"ovrpshim": lambda: build_ovrpshim(tc), "tlskeys": lambda: build_tlskeys(tc),
"luajit": lambda: build_luajit(tc),
"vrsettings": lambda: build_vrsettings(tc), "venc": lambda: build_venc(tc),
"roomscan": lambda: build_roomscan(tc),
"vroverlay_probe": lambda: build_vroverlay_probe(tc), "a32": lambda: build_a32(tc)}
for name in ("adapter", "bridge", "compat", "langpack", "glshim", "eglfmt", "glmv", "ovrtrace", "xrshim", "vkshim", "zinkfix", "foveation", "ovrpshim", "tlskeys", "luajit", "dex", "xrlayer", "oculushmd", "vrsettings", "venc",
"vroverlay_probe", "a32"):
for name in ("adapter", "bridge", "compat", "langpack", "glshim", "eglfmt", "glmv", "ovrtrace", "xrruntime", "xrshim", "vkshim", "zinkfix", "foveation", "ovrpshim", "tlskeys", "luajit", "dex", "xrlayer", "oculushmd", "vrsettings", "venc",
"roomscan", "vroverlay_probe", "a32"):
if name in parts:
log(f"build {name}")
steps[name]()
+25 -4
View File
@@ -1,11 +1,32 @@
# Shared hardware video decoding for Lepton
`frameport_hevc.cpp` implements Android OMX decoders for HEVC Main, H.264
(Constrained) Baseline/Main/(Constrained) High and VP9 Profile 0 using Iris
`frameport_hevc.cpp` implements Android OMX decoders for HEVC Main and Main 10,
H.264 (Constrained) Baseline/Main/(Constrained) High and VP9 Profile 0 using Iris
stateful V4L2 through FFmpeg. Android reports constrained H.264 streams under
separate profile values, so they are listed as stock decoders list them.
These are 8-bit 4:2:0 decoders. Format-aware clients keep Android's existing
codecs for Main10/HDR and protected content; clients selecting by MIME alone
Output is 8-bit 4:2:0. HEVC Main 10 decodes in FFmpeg's software decoder inside
the component and is narrowed to 8 bits: Iris on the Frame's kernel (6.18)
switches a Main 10 session to its compressed `Q10C` capture format by itself
(the capture width becomes a multiple of 192), so an NV12 session returned no
usable pictures, hung and blocked later decoder starts in the app. Mainline
Iris gained linear P010 in Linux 7.2; `Q10C`'s compression isn't public.
`hevc_profile.h` reads the profile from the parameter sets (CODECCONFIG or the
first access unit); streams beyond Main never open Iris. Measured on the Frame
(software, 4 threads): a 1080p Main 10 episode at 232 fps, a 4K Main 10 test
clip at 72 fps.
HDR pictures (PQ/HDR10 or HLG, from the bitstream or the container) are tone
mapped to SDR BT.709 while they are narrowed (`hdr_tonemap.h`): BT.2390's
EETF puts HDR reference white (203 cd/m²) at SDR white and rolls highlights
off up to the content's MaxCLL (else the mastering display's peak, else
1000 cd/m²); colour is converted per chroma sample through linear light into
BT.709, and colours outside it move toward grey of the same luminance. The
component then reports BT.709 (bitstream aspects preferred) so surfaces and
its own RGBA conversion treat the picture as SDR. The work runs in the colour
helper threads; one big core takes about 10 ms per 1080p picture and 40-50 ms
per 4K picture, so 4K HDR is split over four threads. Other 10-bit pictures
are rounded to 8 bits.
Protected content keeps Android's existing codecs; clients selecting by MIME alone
must still respect the component's advertised profiles. Other codec types
retain Android's existing codecs. Applications that explicitly select their
own software decoder bypass this integration. Byte-buffer clients receive full-size
+1 -1
View File
@@ -252,7 +252,7 @@ def build(parser, repo: Path, ndk_root: Path, runtime: Path):
path = artifacts / (name + ".txt" if name == "podman.py" else name)
files[name] = hashlib.sha256(path.read_bytes()).hexdigest()
# bump the revision with every change: a Frame keeps an installed codec of the same or a newer revision
manifest = {"revision": 8, "runtime_sha256": RUNTIME_SHA, "files": files,
manifest = {"revision": 10, "runtime_sha256": RUNTIME_SHA, "files": files,
"codecs": ["video/hevc", "video/avc", "video/x-vnd.on2.vp9"],
"build": {"ndk_revision": NDK_REVISION, "ffmpeg_source_sha256": FFMPEG_SHA}}
(artifacts / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
+124 -19
View File
@@ -21,9 +21,12 @@
#include <poll.h>
#include "yuv_copy.h"
#include "yuv_color.h"
#include "hevc_profile.h"
#include "hdr_tonemap.h"
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavutil/opt.h>
#include <libavutil/mastering_display_metadata.h>
extern const YuvConstants kYvuH709Constants, kYvuI601Constants;
extern const YuvConstants kYvuJPEGConstants;
void NV21ToARGBRow_Any_NEON(const uint8_t*,const uint8_t*,uint8_t*,const YuvConstants*,int);
@@ -35,7 +38,10 @@ int frameport_v4l2_export_frame(const AVFrame*);
#include "dma_copy.h"
namespace android {
static const CodecProfileLevel hevc_profiles[]={{OMX_VIDEO_HEVCProfileMain,OMX_VIDEO_HEVCMainTierLevel62}};
// Main 10 decodes in software (Iris on the Frame's kernel has no 10-bit
// capture format) and is narrowed to the 8-bit outputs.
static const CodecProfileLevel hevc_profiles[]={{OMX_VIDEO_HEVCProfileMain,OMX_VIDEO_HEVCMainTierLevel62},
{OMX_VIDEO_HEVCProfileMain10,OMX_VIDEO_HEVCMainTierLevel51}};
// Android reports constrained AVC streams under their own profile values;
// stock decoders list them so format-aware clients can match them.
static const CodecProfileLevel avc_profiles[]={
@@ -57,7 +63,7 @@ struct DecoderKind {
};
static const DecoderKind kinds[]={
{"OMX.frameport.hevc.decoder","video_decoder.hevc","video/hevc","hevc_v4l2m2m","hevc",
OMX_VIDEO_CodingHEVC,V4L2_PIX_FMT_HEVC,hevc_profiles,1,8192,8192},
OMX_VIDEO_CodingHEVC,V4L2_PIX_FMT_HEVC,hevc_profiles,2,8192,8192},
{"OMX.frameport.avc.decoder","video_decoder.avc","video/avc","h264_v4l2m2m","h264",
OMX_VIDEO_CodingAVC,V4L2_PIX_FMT_H264,avc_profiles,5,8192,8192},
{"OMX.frameport.vp9.decoder","video_decoder.vp9","video/x-vnd.on2.vp9","vp9_v4l2m2m","vp9",
@@ -74,8 +80,10 @@ class ColorWorkers {
AVFrame *frame=nullptr;uint8_t *output=nullptr;size_t stride=0;
const YuvConstants *matrix=nullptr;
FramePortYuvPlane planes[3]={};bool native_yuv=false;
void (*job)(void*,unsigned,unsigned)=nullptr;void *job_context=nullptr;
void rows(unsigned band) {
unsigned bands=count+1;
if(job){job(job_context,band,bands);return;}
if(frame->format==AV_PIX_FMT_YUV420P || frame->format==AV_PIX_FMT_YUVJ420P){
unsigned begin=(frame->height/2)*band/bands,end=(frame->height/2)*(band+1)/bands;
if(native_yuv){
@@ -116,21 +124,35 @@ class ColorWorkers {
}
pthread_mutex_unlock(&pool.mutex);return nullptr;
}
void start() {
if(started)return;
started=true;
for(unsigned i=0;i<3;i++) {
// Only this thread advances the generation, so a helper
// created now waits for the next picture, never a stale one.
workers[i]={this,i+1,generation,{}};
if(pthread_create(&workers[i].thread,nullptr,thread,&workers[i]))break;
count++;
}
}
public:
/** Run fn(context, band, bands) over every band, this thread taking band 0. */
void run(void (*fn)(void*,unsigned,unsigned),void *context) {
start();
pthread_mutex_lock(&mutex);
job=fn;job_context=context;complete=0;generation++;
pthread_cond_broadcast(&work);pthread_mutex_unlock(&mutex);
rows(0);
pthread_mutex_lock(&mutex);
while(complete<count)pthread_cond_wait(&done,&mutex);
job=nullptr;
pthread_mutex_unlock(&mutex);
}
void convert(AVFrame *source,uint8_t *destination,size_t pitch,const YuvConstants *constants,
const AHardwareBuffer_Planes *native=nullptr) {
// Start helpers when a large picture first appears, including after
// an adaptive stream begins at a lower resolution.
if(!started && source->width*source->height>=4096*2048) {
started=true;
for(unsigned i=0;i<3;i++) {
// Only this thread advances the generation, so a helper
// created now waits for the next picture, never a stale one.
workers[i]={this,i+1,generation,{}};
if(pthread_create(&workers[i].thread,nullptr,thread,&workers[i]))break;
count++;
}
}
if(source->width*source->height>=4096*2048)start();
pthread_mutex_lock(&mutex);
frame=source;output=destination;stride=pitch;matrix=constants;complete=0;generation++;
native_yuv=native!=nullptr;
@@ -155,7 +177,10 @@ class FramePortVideo final : public SoftVideoDecoderOMXComponent {
bool gpu_allowed=true;
AVCodecContext *codec=nullptr;
AVCodecParserContext *parser=nullptr;
AVFrame *frame=av_frame_alloc();
AVFrame *frame=av_frame_alloc(),*narrow=av_frame_alloc();
// HDR pictures are tone mapped to SDR BT.709 while narrowed (hdr_tonemap.h); the output then reports BT.709
fp_tonemap *tonemap=nullptr;
bool tone_mapped=false;
uint8_t *config=nullptr;
size_t config_size=0;
bool failed=false,held=false,drained=false,replace_config=false,surface=false;
@@ -249,7 +274,7 @@ class FramePortVideo final : public SoftVideoDecoderOMXComponent {
}
int openCodec(const char *name,bool hardware) {
const AVCodec *implementation=avcodec_find_decoder_by_name(name);
if(!implementation || !frame)return AVERROR(ENOMEM);
if(!implementation || !frame || !narrow)return AVERROR(ENOMEM);
codec=avcodec_alloc_context3(implementation);
if(!codec)return AVERROR(ENOMEM);
const auto &input=editPortInfo(kInputPortIndex)->mDef.format.video;
@@ -273,10 +298,21 @@ class FramePortVideo final : public SoftVideoDecoderOMXComponent {
int result=avcodec_open2(codec,implementation,&options);av_dict_free(&options);
return result;
}
bool openDecoder() {
bool openDecoder(const uint8_t *first,size_t first_size) {
if(codec)return true;
const auto &size=editPortInfo(kInputPortIndex)->mDef.format.video;
int result=AVERROR(ENODEV);
// Iris accepts a Main 10 session but returns no pictures: decode
// anything beyond 8-bit Main in software. Parameter sets come as
// CODECCONFIG or in-band in the first access unit.
if(kind.coding==OMX_VIDEO_CodingHEVC && !prefer_software) {
int profile=config_size?hevc_profile(config,config_size):-1;
if(profile<0)profile=hevc_profile(first,first_size);
if(hevc_needs_software(profile)) {
ALOGW("HEVC profile %d is beyond Iris's 8-bit Main; using software decoding",profile);
prefer_software=true;
}
}
if(size.nFrameWidth>kind.max_width || size.nFrameHeight>kind.max_height)
ALOGW("%s %ux%u is above what Iris decodes (%ux%u); using software decoding",kind.mime,
(unsigned)size.nFrameWidth,(unsigned)size.nFrameHeight,kind.max_width,kind.max_height);
@@ -353,10 +389,78 @@ class FramePortVideo final : public SoftVideoDecoderOMXComponent {
if(aspects.mRange==ColorAspects::RangeFull)return bt709?&frameport_full709_rgba:&kYvuJPEGConstants;
return bt709?&kYvuH709Constants:&kYvuI601Constants;
}
struct ToneMapJob {const fp_tonemap *tm;AVFrame *src,*dst;};
static void toneMapBand(void *context,unsigned band,unsigned bands) {
auto &job=*static_cast<ToneMapJob*>(context);
int rows=(job.src->height+1)/2,begin=rows*band/bands,end=rows*(band+1)/bands;
const uint8_t *src[3]={job.src->data[0],job.src->data[1],job.src->data[2]};
uint8_t *dst[3]={job.dst->data[0],job.dst->data[1],job.dst->data[2]};
fp_tonemap_rows(job.tm,job.src->width,job.src->height,begin,end,src,job.src->linesize,dst,job.dst->linesize);
}
int hdrTransfer() {
// the bitstream's transfer, else the container's
if(frame->color_trc==AVCOL_TRC_SMPTE2084)return FP_TM_PQ;
if(frame->color_trc==AVCOL_TRC_ARIB_STD_B67)return FP_TM_HLG;
if(frame->color_trc!=AVCOL_TRC_UNSPECIFIED)return FP_TM_NONE;
Mutex::Autolock lock(mColorAspectsLock);
if(mDefaultColorAspects.mTransfer==ColorAspects::TransferST2084)return FP_TM_PQ;
if(mDefaultColorAspects.mTransfer==ColorAspects::TransferHLG)return FP_TM_HLG;
return FP_TM_NONE;
}
float contentPeak() {
// MaxCLL, else the mastering display's peak (0: unknown, 1000 cd/m² is assumed)
if(auto *side=av_frame_get_side_data(frame,AV_FRAME_DATA_CONTENT_LIGHT_LEVEL)){
auto *light=reinterpret_cast<const AVContentLightMetadata*>(side->data);
if(light->MaxCLL)return (float)light->MaxCLL;
}
if(auto *side=av_frame_get_side_data(frame,AV_FRAME_DATA_MASTERING_DISPLAY_METADATA)){
auto *display=reinterpret_cast<const AVMasteringDisplayMetadata*>(side->data);
if(display->has_luminance)return (float)av_q2d(display->max_luminance);
}
return 0;
}
bool narrowTo8Bit() {
// Software Main 10 pictures are 10-bit planar; every output here is
// 8-bit SDR, so HDR is tone mapped to BT.709 and other pictures are rounded.
av_frame_unref(narrow);
narrow->format=AV_PIX_FMT_YUV420P;narrow->width=frame->width;narrow->height=frame->height;
int result=av_frame_get_buffer(narrow,64);
if(result>=0)result=av_frame_copy_props(narrow,frame);
if(result<0){fail(result,"10-bit picture conversion");return false;}
int transfer=hdrTransfer();
if(transfer!=FP_TM_NONE) {
bool full=frame->color_range==AVCOL_RANGE_JPEG;
bool matrix2020=frame->colorspace==AVCOL_SPC_BT2020_NCL || frame->colorspace==AVCOL_SPC_UNSPECIFIED;
bool primaries2020=frame->color_primaries==AVCOL_PRI_BT2020 || frame->color_primaries==AVCOL_PRI_UNSPECIFIED;
float peak=transfer==FP_TM_PQ?contentPeak():0;
if(!tonemap)tonemap=new fp_tonemap;
if(!tone_mapped || tonemap->transfer!=transfer || tonemap->full_range!=full || tonemap->matrix2020!=matrix2020 ||
tonemap->primaries2020!=primaries2020 || tonemap->peak!=fp_tonemap_peak(transfer,peak)) {
fp_tonemap_init(tonemap,transfer,full,matrix2020,primaries2020,peak);
ALOGI("%s HDR (%s, peak %.0f cd/m²): tone mapping to SDR BT.709",kind.mime,
transfer==FP_TM_PQ?"PQ":"HLG",tonemap->peak);
}
tone_mapped=true;
ToneMapJob job={tonemap,frame,narrow};
color.run(toneMapBand,&job);
narrow->color_primaries=AVCOL_PRI_BT709;narrow->color_trc=AVCOL_TRC_BT709;
narrow->colorspace=AVCOL_SPC_BT709;narrow->color_range=AVCOL_RANGE_MPEG;
} else for(int plane=0;plane<3;plane++) {
int width=plane?(frame->width+1)/2:frame->width,height=plane?(frame->height+1)/2:frame->height;
for(int row=0;row<height;row++) {
const uint16_t *src=(const uint16_t*)(frame->data[plane]+row*frame->linesize[plane]);
uint8_t *dst=narrow->data[plane]+row*narrow->linesize[plane];
for(int x=0;x<width;x++){unsigned value=(src[x]+2u)>>2;dst[x]=value>255?255:value;}
}
}
av_frame_unref(frame);av_frame_move_ref(frame,narrow);
return true;
}
bool outputFrame() {
// Hardware errors are recovered in software. Software decoders
// conceal damage themselves; output their picture as stock decoders do.
if(frame->decode_error_flags && hardware_active)return recoverHardware("decoded picture");
if(frame->format==AV_PIX_FMT_YUV420P10LE && !narrowTo8Bit())return false;
reportBitstreamColor();
bool reset=false;
handlePortSettingsChange(&reset,frame->width,frame->height,
@@ -531,7 +635,8 @@ protected:
OMX_ERRORTYPE result=SoftVideoDecoderOMXComponent::internalSetParameter(index,params);
rgbaPort();return result;
}
int getColorAspectPreference() override {return kPreferContainer;}
// a tone mapped picture is BT.709 whatever the container says
int getColorAspectPreference() override {return tone_mapped?kPreferBitstream:kPreferContainer;}
void onQueueFilled(OMX_U32) override {
if(failed || mOutputPortSettingsChange!=NONE || drained)return;
auto &inputs=getPortQueue(kInputPortIndex);auto &outputs=getPortQueue(kOutputPortIndex);
@@ -591,7 +696,7 @@ protected:
if(codec)prepend_config=annexB();
consumeInput(info);continue;
}
if(!openDecoder())return;
if(!openDecoder(header->pBuffer+header->nOffset,header->nFilledLen))return;
if(header->nFilledLen) {
AVPacket *packet=av_packet_alloc();
if(!packet){fail(AVERROR(ENOMEM),"input packet");return;}
@@ -676,14 +781,14 @@ protected:
void onReset() override {
gpu_allowed=true;
av_frame_unref(frame);av_parser_close(parser);parser=nullptr;gpu.clearBuffers();avcodec_free_context(&codec);av_freep(&config);config_size=0;clearHistory();
hardware_active=false;prefer_software=false;eos_submitted=false;replay_eos=false;
hardware_active=false;prefer_software=false;eos_submitted=false;replay_eos=false;tone_mapped=false;
last_output_pts=discard_until=AV_NOPTS_VALUE;
held=false;drained=false;failed=false;replace_config=false;
SoftVideoDecoderOMXComponent::onReset();
surface=false;native_enabled=false;metadata=false;mOutputFormat=OMX_COLOR_FormatYUV420Planar;
updatePortDefinitions(true,false);
}
~FramePortVideo() override {av_frame_free(&frame);av_parser_close(parser);gpu.clearBuffers();avcodec_free_context(&codec);av_freep(&config);clearHistory();}
~FramePortVideo() override {delete tonemap;av_frame_free(&frame);av_frame_free(&narrow);av_parser_close(parser);gpu.clearBuffers();avcodec_free_context(&codec);av_freep(&config);clearHistory();}
public:
FramePortVideo(const DecoderKind &type,const OMX_CALLBACKTYPE *callbacks,OMX_PTR appData,OMX_COMPONENTTYPE **component)
:SoftVideoDecoderOMXComponent(type.name,type.role,type.coding,
+196
View File
@@ -0,0 +1,196 @@
// SPDX-License-Identifier: GPL-3.0-only
// HDR to SDR for the software Main 10 path (frameport_hevc.cpp): every output of the component is 8-bit BT.709
// SDR, so PQ (HDR10) and HLG pictures are tone mapped and BT.2020 colours are mapped into BT.709 while the 10-bit
// planes are narrowed. Pure C, so the host test (tests/fixtures/src/hdr_tonemap_test.c) can check it.
//
// Brightness follows ITU-R BT.2390's EETF in the PQ domain: HDR reference white (203 cd/m², BT.2408) becomes SDR
// white, darker tones keep their level and highlights up to the content's peak roll off instead of clipping.
// Colour is converted per 2x2 block (one chroma sample): BT.2020 R'G'B' to linear light, scaled by the tone curve
// at the block's luminance (hue kept), BT.2020 to BT.709 primaries, pulled toward the grey of the same luminance
// where it falls outside BT.709 or above white, encoded for a BT.1886 display.
// Each pixel's luma keeps its own detail and is moved by its block's change, so luma and chroma agree.
#pragma once
#include <math.h>
#include <stddef.h>
#include <stdint.h>
enum { FP_TM_NONE = 0, FP_TM_PQ = 1, FP_TM_HLG = 2 };
#define FP_TM_WHITE 203.0f // cd/m²: HDR reference white, shown as SDR white
#define FP_TM_STEPS 4095 // the tone curve table, over sqrt(L / peak)
#define FP_TM_ENC 65535 // the display encoding table, over SDR linear light
struct fp_tonemap {
int transfer, full_range, matrix2020, primaries2020;
float peak; // cd/m² the content reaches (PQ), 1000 for HLG
float luma[1024]; // 10-bit Y' -> SDR Y' (0..1), the per-pixel curve
int16_t luma8[1024]; // the same as 8-bit limited-range code values
float chan[1024]; // E' (0..1 in 1023 steps) -> linear cd/m² / peak
float ratio[FP_TM_STEPS + 1]; // sqrt(L / peak) -> SDR linear (white = 1) per unit of L / peak
uint8_t enc[FP_TM_ENC + 1]; // SDR linear (0..1) -> E' * 219 for a BT.1886 display (gamma 2.4)
};
static inline float fp_tm_pq_eotf(float e) { // PQ signal -> cd/m²
const float m1 = 2610.0f / 16384, m2 = 2523.0f / 4096 * 128, c1 = 3424.0f / 4096, c2 = 2413.0f / 4096 * 32,
c3 = 2392.0f / 4096 * 32;
if (e <= 0) return 0;
float p = powf(e, 1 / m2), n = p - c1;
return n <= 0 ? 0 : 10000 * powf(n / (c2 - c3 * p), 1 / m1);
}
static inline float fp_tm_pq_oetf(float l) { // cd/m² -> PQ signal
const float m1 = 2610.0f / 16384, m2 = 2523.0f / 4096 * 128, c1 = 3424.0f / 4096, c2 = 2413.0f / 4096 * 32,
c3 = 2392.0f / 4096 * 32;
if (l <= 0) return 0;
float y = powf(l / 10000, m1);
return powf((c1 + c2 * y) / (1 + c3 * y), m2);
}
static inline float fp_tm_hlg_linear(float e) { // HLG signal -> display light / peak (OOTF with system gamma 1.2)
const float a = 0.17883277f, b = 0.28466892f, c = 0.55991073f;
if (e <= 0) return 0;
float s = e <= 0.5f ? e * e / 3 : (expf((e - c) / a) + b) / 12;
return powf(s, 1.2f);
}
// BT.2390 EETF: content luminance (cd/m², up to `peak`) -> SDR luminance relative to white (0..1)
static inline float fp_tm_eetf(float l, float peak) {
if (peak <= FP_TM_WHITE) return fminf(l / FP_TM_WHITE, 1);
float w = fp_tm_pq_oetf(peak), e1 = fp_tm_pq_oetf(l) / w, max_lum = fp_tm_pq_oetf(FP_TM_WHITE) / w;
float ks = fmaxf(1.5f * max_lum - 0.5f, 0), e2 = e1;
if (e1 >= 1) e2 = max_lum;
else if (e1 > ks) {
float t = (e1 - ks) / (1 - ks), t2 = t * t, t3 = t2 * t;
e2 = (2 * t3 - 3 * t2 + 1) * ks + (t3 - 2 * t2 + t) * (1 - ks) + (-2 * t3 + 3 * t2) * max_lum;
}
return fminf(fp_tm_pq_eotf(e2 * w) / FP_TM_WHITE, 1);
}
static inline float fp_tm_signal_nits(const struct fp_tonemap *tm, float e) {
return tm->transfer == FP_TM_PQ ? fp_tm_pq_eotf(e) : tm->peak * fp_tm_hlg_linear(e);
}
/** The peak the tables use: PQ content's MaxCLL or mastering peak (0 = unknown -> 1000, kept within 400..10000),
* 1000 cd/m² for HLG. */
static inline float fp_tonemap_peak(int transfer, float peak) {
return transfer == FP_TM_HLG || !(peak > 0) ? 1000 : fminf(fmaxf(peak, 400), 10000);
}
/** Build the tables. transfer FP_TM_PQ/FP_TM_HLG; peak in cd/m² (PQ: MaxCLL or the mastering display's maximum,
* 0 = unknown -> 1000; HLG always uses 1000). */
static inline void fp_tonemap_init(struct fp_tonemap *tm, int transfer, int full_range, int matrix2020,
int primaries2020, float peak) {
tm->transfer = transfer;
tm->full_range = full_range;
tm->matrix2020 = matrix2020;
tm->primaries2020 = primaries2020;
tm->peak = fp_tonemap_peak(transfer, peak);
for (int i = 0; i < 1024; i++) tm->chan[i] = fp_tm_signal_nits(tm, i / 1023.0f) / tm->peak;
for (int code = 0; code < 1024; code++) {
float e = full_range ? code / 1023.0f : (code - 64) / 876.0f;
e = fminf(fmaxf(e, 0), 1);
tm->luma[code] = powf(fp_tm_eetf(fp_tm_signal_nits(tm, e), tm->peak), 1 / 2.4f);
tm->luma8[code] = (int16_t)(16 + 219 * tm->luma[code] + 0.5f);
}
for (int i = 0; i <= FP_TM_STEPS; i++) {
float s = (float)i / FP_TM_STEPS, rel = s * s; // L / peak
tm->ratio[i] = i ? fp_tm_eetf(rel * tm->peak, tm->peak) / rel : tm->peak / FP_TM_WHITE;
}
for (int i = 0; i <= FP_TM_ENC; i++) tm->enc[i] = (uint8_t)(219 * powf((float)i / FP_TM_ENC, 1 / 2.4f) + 0.5f);
}
static inline float fp_tm_clamp01(float x) { return x < 0 ? 0 : x > 1 ? 1 : x; }
static inline int fp_tm_index(float x01, int steps) { return (int)(fp_tm_clamp01(x01) * steps + 0.5f); }
static inline uint8_t fp_tm_u8(int v) { return v < 0 ? 0 : v > 255 ? 255 : (uint8_t)v; }
#define FP_TM_CHUNK 128 // blocks per pass: the passes alternate plain arithmetic (NEON) and table lookups
/** Tone map chroma rows [crow0, crow1) of a 10-bit 4:2:0 picture (and the luma rows they cover) into 8-bit 4:2:0
* limited range BT.709. Strides are in bytes. Rows are independent, so callers may split a picture into bands. */
static inline void fp_tonemap_rows(const struct fp_tonemap *tm, int width, int height, int crow0, int crow1,
const uint8_t *const src[3], const int src_stride[3],
uint8_t *const dst[3], const int dst_stride[3]) {
const float kr = tm->matrix2020 ? 0.2627f : 0.2126f, kb = tm->matrix2020 ? 0.0593f : 0.0722f;
const float kg = 1 - kr - kb, inv_kg = 1 / kg, cr_r = 2 - 2 * kr, cb_b = 2 - 2 * kb;
const float yscale = tm->full_range ? 1023.0f : 876.0f, yoff = tm->full_range ? 0 : 64;
const float cscale = tm->full_range ? 1023.0f : 896.0f;
// BT.2020 -> BT.709 primaries in linear light (ITU-R BT.2087)
static const float m2020[9] = {1.6605f, -0.5876f, -0.0728f, -0.1246f, 1.1329f, -0.0083f,
-0.0182f, -0.1006f, 1.1187f};
static const float ident[9] = {1, 0, 0, 0, 1, 0, 0, 0, 1};
const float *m = tm->primaries2020 ? m2020 : ident;
const int cw = (width + 1) / 2, pairs = width / 2;
int c0[FP_TM_CHUNK], c1[FP_TM_CHUNK], c2[FP_TM_CHUNK], c3[FP_TM_CHUNK], ir[FP_TM_CHUNK], ig[FP_TM_CHUNK],
ib[FP_TM_CHUNK], shift[FP_TM_CHUNK];
float fr[FP_TM_CHUNK], fg[FP_TM_CHUNK], fb[FP_TM_CHUNK], fk[FP_TM_CHUNK];
for (int cy = crow0; cy < crow1; cy++) {
const int y0 = cy * 2, y1 = y0 + 1 < height ? y0 + 1 : y0;
const uint16_t *l0 = (const uint16_t *)(src[0] + (size_t)y0 * src_stride[0]);
const uint16_t *l1 = (const uint16_t *)(src[0] + (size_t)y1 * src_stride[0]);
const uint16_t *us = (const uint16_t *)(src[1] + (size_t)cy * src_stride[1]);
const uint16_t *vs = (const uint16_t *)(src[2] + (size_t)cy * src_stride[2]);
uint8_t *d0 = dst[0] + (size_t)y0 * dst_stride[0], *d1 = dst[0] + (size_t)y1 * dst_stride[0];
uint8_t *du = dst[1] + (size_t)cy * dst_stride[1], *dv = dst[2] + (size_t)cy * dst_stride[2];
for (int start = 0; start < cw; start += FP_TM_CHUNK) {
const int n = cw - start < FP_TM_CHUNK ? cw - start : FP_TM_CHUNK;
const int full = pairs - start < n ? (pairs - start > 0 ? pairs - start : 0) : n;
// the block's four luma codes (an odd width's last block repeats its column)
for (int i = 0; i < full; i++) {
const int x = (start + i) * 2;
c0[i] = l0[x] & 1023; c1[i] = l0[x + 1] & 1023; c2[i] = l1[x] & 1023; c3[i] = l1[x + 1] & 1023;
}
for (int i = full; i < n; i++) {
const int x = (start + i) * 2;
c0[i] = c1[i] = l0[x] & 1023; c2[i] = c3[i] = l1[x] & 1023;
}
// block R'G'B' (0..1) -> table indices
for (int i = 0; i < n; i++) {
float ye = ((c0[i] + c1[i] + c2[i] + c3[i]) * 0.25f - yoff) / yscale;
float cb = ((us[start + i] & 1023) - 512) / cscale, cr = ((vs[start + i] & 1023) - 512) / cscale;
float r = ye + cr_r * cr, b = ye + cb_b * cb, g = (ye - kr * r - kb * b) * inv_kg;
ir[i] = fp_tm_index(r, 1023); ig[i] = fp_tm_index(g, 1023); ib[i] = fp_tm_index(b, 1023);
}
for (int i = 0; i < n; i++) { fr[i] = tm->chan[ir[i]]; fg[i] = tm->chan[ig[i]]; fb[i] = tm->chan[ib[i]]; }
// linear light / peak -> the tone curve's index at the block's luminance
for (int i = 0; i < n; i++) ir[i] = fp_tm_index(sqrtf(kr * fr[i] + kg * fg[i] + kb * fb[i]), FP_TM_STEPS);
for (int i = 0; i < n; i++) fk[i] = tm->ratio[ir[i]];
// scaled by the tone curve (hue kept), BT.709 primaries; colours outside BT.709 or above white move
// toward the grey of the same luminance until they fit (clipping a channel would darken them)
for (int i = 0; i < n; i++) {
float r = fr[i], g = fg[i], b = fb[i], k = fk[i];
float R = (m[0] * r + m[1] * g + m[2] * b) * k, G = (m[3] * r + m[4] * g + m[5] * b) * k,
B = (m[6] * r + m[7] * g + m[8] * b) * k;
float y = fp_tm_clamp01(0.2126f * R + 0.7152f * G + 0.0722f * B);
float lo = R < G ? (R < B ? R : B) : (G < B ? G : B), hi = R > G ? (R > B ? R : B) : (G > B ? G : B);
float t = 1;
if (lo < 0) t = y / (y - lo);
if (hi > 1 && (1 - y) < t * (hi - y)) t = (1 - y) / (hi - y);
ir[i] = fp_tm_index(y + t * (R - y), FP_TM_ENC);
ig[i] = fp_tm_index(y + t * (G - y), FP_TM_ENC);
ib[i] = fp_tm_index(y + t * (B - y), FP_TM_ENC);
}
for (int i = 0; i < n; i++) { ir[i] = tm->enc[ir[i]]; ig[i] = tm->enc[ig[i]]; ib[i] = tm->enc[ib[i]]; }
// chroma, and how much the colour work moved the block's luma from the per-pixel curve
for (int i = 0; i < n; i++) {
float er = (float)ir[i], eg = (float)ig[i], eb = (float)ib[i]; // E' * 219
float ey = 0.2126f * er + 0.7152f * eg + 0.0722f * eb;
du[start + i] = fp_tm_u8((int)(128.5f + (224.0f / 219 / 1.8556f) * (eb - ey)));
dv[start + i] = fp_tm_u8((int)(128.5f + (224.0f / 219 / 1.5748f) * (er - ey)));
shift[i] = (int)(16.5f + ey);
}
for (int i = 0; i < n; i++) {
c0[i] = tm->luma8[c0[i]]; c1[i] = tm->luma8[c1[i]]; c2[i] = tm->luma8[c2[i]]; c3[i] = tm->luma8[c3[i]];
}
// each pixel keeps its own curve value, moved with its block
for (int i = 0; i < full; i++) {
const int x = (start + i) * 2, sh = shift[i] - ((c0[i] + c1[i] + c2[i] + c3[i] + 2) >> 2);
d0[x] = fp_tm_u8(c0[i] + sh); d0[x + 1] = fp_tm_u8(c1[i] + sh);
d1[x] = fp_tm_u8(c2[i] + sh); d1[x + 1] = fp_tm_u8(c3[i] + sh);
}
for (int i = full; i < n; i++) {
const int x = (start + i) * 2, sh = shift[i] - ((c0[i] + c2[i] + 1) >> 1);
d0[x] = fp_tm_u8(c0[i] + sh); d1[x] = fp_tm_u8(c2[i] + sh);
}
}
}
}
+24
View File
@@ -0,0 +1,24 @@
// SPDX-License-Identifier: GPL-3.0-only
// HEVC profile of a stream's parameter sets, to keep streams Iris can't decode
// into NV12 on the software path (frameport_hevc.cpp). Pure, so the host test
// (tests/fixtures/src/hevc_profile_test.c) can check it.
#pragma once
#include <stddef.h>
#include <stdint.h>
// general_profile_idc of the first SPS (1 Main, 2 Main 10, 3 Main Still
// Picture, 4+ range extensions) in Annex B data or an hvcC record, or -1.
// The field precedes any possible emulation-prevention byte.
static inline int hevc_profile(const uint8_t *data, size_t size) {
if (size >= 2 && data[0] == 1) return data[1] & 0x1f; // hvcC: configurationVersion, then profile space/tier/idc
for (size_t i = 0; i + 7 <= size; i++)
if (!data[i] && !data[i + 1] && data[i + 2] == 1 && ((data[i + 3] >> 1) & 0x3f) == 33)
return data[i + 6] & 0x1f;
return -1;
}
// Iris decodes 8-bit Main (and Main Still Picture) into NV12; a stream with no
// profile found keeps trying it.
static inline int hevc_needs_software(int profile) {
return profile > 1 && profile != 3;
}
+339
View File
@@ -0,0 +1,339 @@
/* SPDX-License-Identifier: GPL-3.0-only */
#include "scan.h"
#define MIN_TRIANGLE_TWICE_AREA 1e-8f
#define MIN_RANGE .1f
#define MAX_COORDINATE 10000.f
#define QUATERNION_NORM_TOLERANCE .1f
#define SAMPLE_SPACING_VOXELS .75f
#define MAX_SURFEL_WEIGHT 10.f
#define FEW_FRAMES 8
#define MAX_GRID_XZ 8.f
#define MAX_GRID_Y 4.f
#define GRID_PADDING_VOXELS 5.f
#define NORMAL_PATCH_SIZE 12
#define MIN_PATCH_POINTS 24
float root(float x)
{
if (x <= 0) return 0;
/* -fno-math-errno makes this one hardware sqrt, with no libc/libm symbol on either build. */
return __builtin_sqrtf(x);
}
Vec unit(Vec p) { float n = root(dot(p,p)); return n > 1e-12f ? mul(p,1/n) : vec(0,0,0); }
int flo(float x) { int i = (int)x; return x < (float)i ? i-1 : i; }
float sine(float x)
{
while (x > PI) x -= 2*PI;
while (x < -PI) x += 2*PI;
if (x > PI/2) x = PI-x;
if (x < -PI/2) x = -PI-x;
float q = x*x;
return x*(1-q/6+q*q/120-q*q*q/5040+q*q*q*q/362880);
}
float cosine(float x) { return sine(x+PI/2); }
float angle(float y, float x)
{
/* atan approximation, maximum error < .012 degrees. */
float a = mn(ab(x),ab(y)) / mx(mx(ab(x),ab(y)),1e-20f), q = a*a;
float r = a*(.999866f+q*(-.3302995f+q*(.180141f+q*(-.085133f+q*.0208351f))));
if (ab(y) > ab(x)) r = PI/2-r;
if (x < 0) r = PI-r;
return y < 0 ? -r : r;
}
int same(const char *a, const char *b)
{
while (*a && *a == *b) a++,b++;
return *a == *b;
}
u32 random_next(Scan *s) { s->rng ^= s->rng<<13; s->rng ^= s->rng>>17; s->rng ^= s->rng<<5; return s->rng; }
void warn(Scan *s, const char *m)
{
for (int i = 0; i < s->nwarnings; i++) if (same(s->warnings[i],m)) return;
if (s->nwarnings < 24) s->warnings[s->nwarnings++] = m;
}
/* The head path (where the user stood and walked) for the refinement's walked-volume pass. Bounded: every
* head_stride-th tracked frame, halving the kept points and doubling the stride when the buffer is full. */
static void record_head(Scan *s, Vec eye)
{
if(!s->head_stride) s->head_stride=1;
if(s->head_frames++%s->head_stride) return;
if(s->nheads==MAX_HEADS) {
for(int i=0;i<MAX_HEADS/2;i++) s->heads[i]=s->heads[2*i];
s->nheads=MAX_HEADS/2;s->head_stride*=2;
if((s->head_frames-1)%s->head_stride) return;
}
s->heads[s->nheads++]=eye;
}
static u32 u32le(const u8 *p) { return (u32)p[0]|(u32)p[1]<<8|(u32)p[2]<<16|(u32)p[3]<<24; }
static float f32le(const u8 *p) { union { u32 u; float f; } v = {u32le(p)}; return v.f; }
/* 1 = complete, 0 = clean EOF, -1 = I/O error or partial record. */
static int exact(int fd, void *p, usize n)
{
usize done = 0;
while (done < n) {
long r = io_read(fd,(u8 *)p+done,n-done);
if (r < 0 || (!r && done)) return -1;
if (!r) return 0;
done += (usize)r;
}
return 1;
}
Vec grid_point(Scan *s, int x, int y, int z)
{
return add(s->origin,vec(x*s->opt.voxel,y*s->opt.voxel,z*s->opt.voxel));
}
int cell_index(Scan *s, Vec p)
{
Vec q = mul(sub(p,s->origin),1/s->opt.voxel);
int x = flo(q.x+.5f), y = flo(q.y+.5f), z = flo(q.z+.5f);
if (x < 0 || y < 0 || z < 0 || x >= s->nx || y >= s->ny || z >= s->nz) return -1;
return (y*s->nz+z)*s->nx+x;
}
static void sample(Scan *s, Vec p, Vec n, float w)
{
int i = cell_index(s,p);
if (i < 0) { s->clipped++; return; }
Cell *c = &s->cells[i];
/* Cap repeated views' influence without throwing away later measurements. */
float scale = c->weight > MAX_SURFEL_WEIGHT ? MAX_SURFEL_WEIGHT/c->weight : 1;
c->p = add(mul(c->p,scale),mul(p,w));
c->n = add(mul(c->n,scale),mul(n,w));
c->weight = c->weight*scale+w;
if(c->last_frame!=s->fuse_frame) { c->last_frame=s->fuse_frame;c->views++; }
}
static void triangle(Scan *s, Vec a, Vec b, Vec c, Vec eye, Vec patch_normal, int fuse)
{
Vec abv = sub(b,a), ac = sub(c,a), bc = sub(c,b);
float longest = root(mx(dot(abv,abv),mx(dot(ac,ac),dot(bc,bc))));
Vec p = mul(add(a,add(b,c)),1.f/3), n = cross(abv,ac);
float twice_area = root(dot(n,n));
Vec view = sub(eye,p);
float distance = root(dot(view,view));
if (twice_area < MIN_TRIANGLE_TWICE_AREA || longest > s->opt.max_edge || distance > s->opt.range || distance < MIN_RANGE) {
if (!fuse) s->rejected++;
return;
}
n = mul(n,1/twice_area);
if(dot(patch_normal,patch_normal)>.5f) n=patch_normal;
float facing = dot(n,view)/distance;
if (ab(facing) < s->opt.grazing) { if (!fuse) s->rejected++; return; }
if (facing < 0) n = mul(n,-1);
if (!fuse) {
s->accepted++;
s->seen++;
int slot = s->seen <= MAX_SAMPLES ? (int)s->seen-1 : (int)(random_next(s)%s->seen);
if (slot < MAX_SAMPLES) s->samples[slot] = p;
s->nsamples = (int)mn((float)s->seen,MAX_SAMPLES);
} else {
int steps = (int)(longest/(s->opt.voxel*SAMPLE_SPACING_VOXELS))+1;
float weight = twice_area*facing*facing / ((steps+1)*(steps+2)*.5f);
for (int j = 0; j <= steps; j++) for (int k = 0; k <= steps-j; k++) {
Vec q = add(a,add(mul(abv,(float)j/steps),mul(ac,(float)k/steps)));
sample(s,q,n,weight);
}
}
}
static Vec patch_normal(Vec *points, float *depths, int width, int x0, int y0, int x1, int y1,
float tolerance)
{
Vec mean=vec(0,0,0);int count=0;
/* Shift before accumulating the covariance: room positions can be large relative to patch residuals. */
Vec origin=points[y0*width+x0];
for(int y=y0;y<y1;y++) for(int x=x0;x<x1;x++) if(depths[y*width+x]>0) {
mean=add(mean,sub(points[y*width+x],origin));count++;
}
if(count<MIN_PATCH_POINTS) return vec(0,0,0);
mean=mul(mean,1.f/count);
float cov[3][3]={{0}},basis[3][3]={{1,0,0},{0,1,0},{0,0,1}};
for(int y=y0;y<y1;y++) for(int x=x0;x<x1;x++) if(depths[y*width+x]>0) {
Vec p=sub(sub(points[y*width+x],origin),mean);float a[3]={p.x,p.y,p.z};
for(int i=0;i<3;i++) for(int j=0;j<3;j++) cov[i][j]+=a[i]*a[j];
}
for(int sweep=0;sweep<8;sweep++) {
int p=0,q=1;
if(ab(cov[0][2])>ab(cov[p][q])) { p=0;q=2; }
if(ab(cov[1][2])>ab(cov[p][q])) { p=1;q=2; }
if(ab(cov[p][q])<1e-10f) break;
float phi=.5f*angle(2*cov[p][q],cov[q][q]-cov[p][p]),co=cosine(phi),si=sine(phi);
float aa=cov[p][p],bb=cov[q][q],abv=cov[p][q];
for(int k=0;k<3;k++) if(k!=p && k!=q) {
float a=cov[k][p],b=cov[k][q];
cov[k][p]=cov[p][k]=co*a-si*b;cov[k][q]=cov[q][k]=si*a+co*b;
}
cov[p][p]=co*co*aa-2*si*co*abv+si*si*bb;
cov[q][q]=si*si*aa+2*si*co*abv+co*co*bb;cov[p][q]=cov[q][p]=0;
for(int k=0;k<3;k++) {
float a=basis[k][p],b=basis[k][q];basis[k][p]=co*a-si*b;basis[k][q]=si*a+co*b;
}
}
int least=0;
for(int i=1;i<3;i++) if(cov[i][i]<cov[least][least]) least=i;
if(cov[least][least]/count>tolerance*tolerance) return vec(0,0,0);
return unit(vec(basis[0][least],basis[1][least],basis[2][least]));
}
static int grid_normals(Scan *s, Vec *points, float *depths, Vec *normals, u32 nv, u32 ni, const u8 *indices)
{
if(ni<6 || u32le(indices)!=0 || u32le(indices+8)!=1) return 0;
u32 width=u32le(indices+4);
if(width<3 || width>nv || nv%width || ni!=6*(width-1)*(nv/width-1)) return 0;
/* Only a complete regular fan uses patch normals. Other meshes retain ordinary triangle normals. */
u32 at=0;
for(u32 row=0;row+1<nv/width;row++) for(u32 col=0;col+1<width;col++) {
u32 a=row*width+col,expected[6]={a,a+width,a+1,a+1,a+width,a+width+1};
for(int k=0;k<6;k++) if(u32le(indices+4*at++)!=expected[k]) return 0;
}
int height=(int)(nv/width);
for(int y=0;y<height;y+=NORMAL_PATCH_SIZE) for(int x=0;x<(int)width;x+=NORMAL_PATCH_SIZE) {
int x1=(int)mn(x+NORMAL_PATCH_SIZE,(float)width),y1=(int)mn(y+NORMAL_PATCH_SIZE,height);
Vec n=patch_normal(points,depths,(int)width,x,y,x1,y1,s->opt.patch_residual);
for(int yy=y;yy<y1;yy++) for(int xx=x;xx<x1;xx++) normals[yy*width+xx]=n;
}
return 1;
}
const char *read_dump(Scan *s, int fuse)
{
int fd = io_open(s->opt.input,0);
if (fd < 0) return "cannot open scan dump";
u8 header[44];
const char *error = 0;
Vec *vertices = 0,*normals=0;
float *depths = 0;
u8 *indices = 0;
usize vs = 0, is = 0, ds = 0;
if (exact(fd,header,8) != 1 || header[0]!='F' || header[1]!='P' || header[2]!='R' || header[3]!='S' ||
u32le(header+4) != 1) { error = "invalid FPRS header"; goto done; }
unsigned frames = 0;
for (;;) {
int r = exact(fd,header,44);
if (r == 0) break;
if (r < 0) { error = "truncated frame header"; break; }
/* Tracking can start before poses are available. Consume the entire bounded record before skipping,
* so one NaN cannot hide a truncated payload or desynchronize the next frame. */
int invalid=(u32le(header+4)&0x7ff00000u) == 0x7ff00000u;
float pose[7];
for (int i = 0; i < 7; i++) {
pose[i] = f32le(header+8+i*4);
if (!finite(pose[i]) || ab(pose[i]) > MAX_COORDINATE) invalid=1;
}
float norm = 0;
for (int i = 3; i < 7; i++) norm += pose[i]*pose[i];
if (!finite(norm) || ab(norm-1) > QUATERNION_NORM_TOLERANCE) invalid=1;
if(!invalid) for (int i = 3; i < 7; i++) pose[i] /= root(norm);
if(!fuse && !invalid) record_head(s,vec(pose[0],pose[1],pose[2]));
u32 nv = u32le(header+36), ni = u32le(header+40);
if (nv > MAX_FRAME_VERTICES || ni > MAX_FRAME_INDICES || ni%3 || (!nv && ni)) {
error = "invalid frame counts"; break;
}
usize needv = (usize)nv*12, needi = (usize)ni*4;
if (needv > vs) {
io_free(vertices,vs);io_free(normals,vs);vs=needv;vertices=io_alloc(vs);normals=io_alloc(vs);
}
if (needi > is) { io_free(indices,is); is=needi; indices=io_alloc(is); }
if((usize)nv*4>ds) { io_free(depths,ds);ds=(usize)nv*4;depths=io_alloc(ds); }
if ((nv && (!vertices || !depths || !normals)) || (ni && !indices)) { error = "frame allocation failed"; break; }
if ((needv && exact(fd,vertices,needv)!=1) || (needi && exact(fd,indices,needi)!=1)) {
error = "truncated frame data"; break;
}
Vec eye = vec(pose[0],pose[1],pose[2]), q = vec(pose[3],pose[4],pose[5]);
u64 far=0;
for (u32 i = 0; i < nv; i++) {
u8 *p = (u8 *)&vertices[i];
Vec v = vec(f32le(p),f32le(p+4),f32le(p+8));
if (!finite(v.x) || !finite(v.y) || !finite(v.z) ||
ab(v.x)>MAX_COORDINATE || ab(v.y)>MAX_COORDINATE || ab(v.z)>MAX_COORDINATE) {
invalid=1;break;
}
depths[i]=root(dot(v,v));
if(depths[i]>s->opt.range || depths[i]<MIN_RANGE) { depths[i]=-1;far++;continue; }
Vec t = mul(cross(q,v),2);
vertices[i] = add(eye,add(v,add(mul(t,pose[6]),cross(q,t))));
}
int regular=!invalid && grid_normals(s,vertices,depths,normals,nv,ni,indices);
/* Structural errors are fatal even in a frame whose tracking/depth is unusable. */
s->fuse_frame=frames+1;
for (u32 i = 0; i < ni; i+=3) {
u32 a=u32le(indices+4*i), b=u32le(indices+4*(i+1)), c=u32le(indices+4*(i+2));
if (a>=nv || b>=nv || c>=nv) { error = "index outside vertex array"; break; }
if(invalid) continue;
if(depths[a]<0 || depths[b]<0 || depths[c]<0) { if(!fuse) s->rejected++;continue; }
float near=mn(depths[a],mn(depths[b],depths[c])),far_depth=mx(depths[a],mx(depths[b],depths[c]));
if(far_depth-near>s->opt.depth_jump+s->opt.depth_ratio*near) {
if(!fuse) { s->rejected++;s->discontinuities++; }continue;
}
Vec normal=regular ? normals[a] : vec(0,0,0);
if(regular && dot(normal,normal)<.5f) { if(!fuse) s->rejected++;continue; }
triangle(s,vertices[a],vertices[b],vertices[c],eye,normal,fuse);
}
if (error) break;
if(!fuse) { if(invalid) s->skipped_frames++;else s->far_vertices+=far; }
frames++;
}
if (!error && !frames) error = "scan contains no frames";
if (!error && !fuse) s->frames = frames;
done:
io_free(vertices,vs);
io_free(normals,vs);
io_free(indices,is);
io_free(depths,ds);
io_close(fd);
return error;
}
static float median(Scan *s, int axis)
{
/* In-place quickselect on the bounded reservoir; no dump-sized allocations. */
int lo=0, hi=s->nsamples-1, mid=s->nsamples/2;
while (lo < hi) {
float pivot = ((float *)&s->samples[(lo+hi)/2])[axis];
int i=lo,j=hi;
while (i<=j) {
while (((float *)&s->samples[i])[axis] < pivot) i++;
while (((float *)&s->samples[j])[axis] > pivot) j--;
if (i<=j) { Vec t=s->samples[i]; s->samples[i++]=s->samples[j]; s->samples[j--]=t; }
}
if (mid<=j) hi=j;
else if (mid>=i) lo=i;
else break;
}
return ((float *)&s->samples[mid])[axis];
}
const char *init_grid(Scan *s)
{
if (!s->nsamples) return "scan contains no usable triangles";
float x=median(s,0), y=median(s,1), z=median(s,2), v=s->opt.voxel;
Vec low=s->samples[0],high=low;
for(int i=1;i<s->nsamples;i++) {
Vec p=s->samples[i];
low=vec(mn(low.x,p.x),mn(low.y,p.y),mn(low.z,p.z));
high=vec(mx(high.x,p.x),mx(high.y,p.y),mx(high.z,p.z));
}
float pad=v*mx(GRID_PADDING_VOXELS,s->opt.truncation+2);
low=sub(low,vec(pad,pad,pad));high=add(high,vec(pad,pad,pad));
s->origin=vec(mx(flo(low.x/v)*v,flo((x-MAX_GRID_XZ/2)/v)*v),
mx(flo(low.y/v)*v,flo((y-MAX_GRID_Y/2)/v)*v),
mx(flo(low.z/v)*v,flo((z-MAX_GRID_XZ/2)/v)*v));
s->nx=(int)mn(-flo(-(high.x-s->origin.x)/v)+1,(int)(MAX_GRID_XZ/v)+1);
s->ny=(int)mn(-flo(-(high.y-s->origin.y)/v)+1,(int)(MAX_GRID_Y/v)+1);
s->nz=(int)mn(-flo(-(high.z-s->origin.z)/v)+1,(int)(MAX_GRID_XZ/v)+1);
usize count = (usize)s->nx*s->ny*s->nz;
if (count > MAX_CELLS) return "voxel grid exceeds memory budget; increase --voxel";
s->count = (int)count;
s->cells = io_alloc(count*sizeof(Cell));
return s->cells ? 0 : "grid allocation failed";
}
void finish_fusion(Scan *s)
{
/* A short look-around cannot expose each surface in many frames. Cap the target at ten percent of usable
* views (at least one); long recordings still require repeated evidence rather than transient depth. */
unsigned required=(unsigned)mn(s->opt.min_views,mx(1,(s->frames-s->skipped_frames)/10.f));
for (int i=0; i<s->count; i++) {
Cell *c=&s->cells[i];
if(c->views<required) { c->weight=0;continue; }
if (c->weight) { c->p=mul(c->p,1/c->weight); c->n=unit(c->n); }
}
if (s->clipped) warn(s,"scan clipped to 8 x 8 x 4 metre grid");
if (s->frames < FEW_FRAMES) warn(s,"few frames; scan more viewpoints");
if(s->skipped_frames) warn(s,"non-finite or invalid frames skipped");
if(s->far_vertices) warn(s,"out-of-range depth vertices skipped");
if (s->accepted < s->rejected) warn(s,"many triangles rejected; check depth quality");
}
+395
View File
@@ -0,0 +1,395 @@
/* SPDX-License-Identifier: GPL-3.0-only */
#include "refine.h"
static int error_json(const char *error)
{
Buf t;buf_init(&t,4096);put(&t,"{\"ok\":false,\"error\":\"");put(&t,error);put(&t,"\"}\n");
io_write(1,t.data,t.n);buf_free(&t);return 2;
}
static int parse_float(const char *p,float *out)
{
int digits=0;float n=0,scale=1;
while(*p>='0' && *p<='9') { n=n*10+*p++-'0';digits++;if(n>10000) return 0; }
if(*p=='.') { p++;while(*p>='0' && *p<='9') { scale*=.1f;n+=(*p++-'0')*scale;digits++; } }
if(!digits || *p || !finite(n)) return 0;
*out=n;return 1;
}
static int parse_universe(const char *p,u64 *out)
{
u64 n=0;if(!*p) return 0;
for(;*p;p++) {
if(*p<'0' || *p>'9') return 0;
unsigned digit=(unsigned)(*p-'0');
if(n>(~(u64)0-digit)/10) return 0;
n=n*10+digit;
}
*out=n;return 1;
}
/* "x,y,z,r;x,y,z,r;..." (metres, STAGE space) */
static int parse_erase(const char *value, Refine *o)
{
char *p=(char *)value;o->nerase=0;
while(*p) {
if(o->nerase==MAX_ERASE) return 0;
float v[4];
for(int i=0;i<4;i++) {
if(!parse_number(&p,&v[i])) return 0;
if(i<3) { if(*p!=',') return 0;p++; }
}
if(v[3]<=0 || v[3]>5) return 0;
o->erase[o->nerase++]=(Sphere){v[0],v[1],v[2],v[3]};
while(*p==' ') p++;
if(*p==';') p++;
else if(*p) return 0;
}
return 1;
}
static const char *options(Options *o,int argc,char **argv)
{
*o=(Options){.voxel=.05f,.ceiling_default=2.5f,.truncation=3,.max_edge=.6f,.grazing=.12f,.range=4.f,
.depth_jump=.25f,.depth_ratio=.12f,.min_views=5,.patch_residual=.03f,
.plane_distance=.045f,.min_plane_area=.45f,.wall_height=1.5f,.snap_degrees=5,.floor_band=.35f,
.ceiling_min=1.8f,.min_volume=.05f,.cluster_gap=.10f,.gap=.65f,.horizontal=.85f,
/* The owner's refinement settings (room-scan cleanup page, 2026-10-18). */
.rf={.refine=1,.walls=2,.protect=0,.label_rules=1,.smooth=8,.band=.35f,.facing_deg=60,.band_in=.12f,
.floor_in=.08f,.ceil_in=.25f,.top_strip=.5f,.near=.03f,.clip=.05f,.clip_open=.10f,.island=.20f,
.tops=.10f,.tubes=3,.tube_thick=.15f,.walked=.40f}};
for(int i=1;i<argc;i++) {
const char *arg=argv[i];
if(same(arg,"--json")) continue;
if(i+1==argc) return "option requires a value";
const char *value=argv[++i];
float number=0;
if(same(arg,"--offline")) o->input=value;
else if(same(arg,"--out")) o->out=value;
else if(same(arg,"--hint")) o->hint=value;
else if(same(arg,"--refine-only")) o->refine_only=value;
else if(same(arg,"--universe")) { if(!parse_universe(value,&o->universe)) return "invalid universe"; }
else if(same(arg,"--walls")) {
if(same(value,"off")) o->rf.walls=0;
else if(same(value,"snap")) o->rf.walls=1;
else if(same(value,"panels")) o->rf.walls=2;
else return "option outside supported range";
}
else if(same(arg,"--erase")) { if(!parse_erase(value,&o->rf)) return "invalid erase spheres"; }
else if(same(arg,"--refine") || same(arg,"--protect") || same(arg,"--label-rules") || same(arg,"--smooth")) {
int high=same(arg,"--smooth") ? 50 : 1;
if(!parse_float(value,&number) || number!=(float)(int)number || number>high) return "option outside supported range";
int *dest=same(arg,"--refine") ? &o->rf.refine : same(arg,"--protect") ? &o->rf.protect :
same(arg,"--label-rules") ? &o->rf.label_rules : &o->rf.smooth;
*dest=(int)number;
}
else {
float *dest=0,low=0,high=100;
if(same(arg,"--voxel")) { dest=&o->voxel;low=.04f;high=.2f; }
else if(same(arg,"--ceiling-default")) { dest=&o->ceiling_default;low=1.8f;high=3.8f; }
else if(same(arg,"--truncation")) { dest=&o->truncation;low=1;high=5; }
else if(same(arg,"--max-edge")) { dest=&o->max_edge;low=.05f;high=1.5f; }
else if(same(arg,"--grazing")) { dest=&o->grazing;low=.01f;high=.9f; }
else if(same(arg,"--range")) { dest=&o->range;low=1;high=10; }
else if(same(arg,"--depth-jump")) { dest=&o->depth_jump;low=.02f;high=1; }
else if(same(arg,"--depth-ratio")) { dest=&o->depth_ratio;high=.5f; }
else if(same(arg,"--min-views")) { dest=&o->min_views;low=1;high=16; }
else if(same(arg,"--patch-residual")) { dest=&o->patch_residual;low=.005f;high=.1f; }
else if(same(arg,"--plane-distance")) { dest=&o->plane_distance;low=.01f;high=.15f; }
else if(same(arg,"--min-plane-area")) { dest=&o->min_plane_area;low=.1f;high=4; }
else if(same(arg,"--wall-height")) { dest=&o->wall_height;low=.5f;high=3; }
else if(same(arg,"--snap-degrees")) { dest=&o->snap_degrees;high=15; }
else if(same(arg,"--floor-band")) { dest=&o->floor_band;low=.05f;high=1; }
else if(same(arg,"--ceiling-min")) { dest=&o->ceiling_min;low=1.5f;high=3; }
else if(same(arg,"--min-volume")) { dest=&o->min_volume;low=.005f;high=1; }
else if(same(arg,"--cluster-gap")) { dest=&o->cluster_gap;low=.04f;high=.2f; }
else if(same(arg,"--gap")) { dest=&o->gap;low=.1f;high=1.5f; }
else if(same(arg,"--horizontal")) { dest=&o->horizontal;low=.6f;high=.99f; }
else if(same(arg,"--band")) { dest=&o->rf.band;low=.02f;high=1; }
else if(same(arg,"--facing-deg")) { dest=&o->rf.facing_deg;low=5;high=89; }
else if(same(arg,"--band-in")) { dest=&o->rf.band_in;high=1; }
else if(same(arg,"--floor-in")) { dest=&o->rf.floor_in;high=1; }
else if(same(arg,"--ceil-in")) { dest=&o->rf.ceil_in;high=1; }
else if(same(arg,"--top-strip")) { dest=&o->rf.top_strip;high=2; }
else if(same(arg,"--near")) { dest=&o->rf.near;high=.3f; }
else if(same(arg,"--clip")) { dest=&o->rf.clip;high=1; }
else if(same(arg,"--clip-open")) { dest=&o->rf.clip_open;high=2; }
else if(same(arg,"--island")) { dest=&o->rf.island;high=5; }
else if(same(arg,"--tops")) { dest=&o->rf.tops;high=.5f; }
else if(same(arg,"--tubes")) { dest=&o->rf.tubes;high=20; }
else if(same(arg,"--tube-thick")) { dest=&o->rf.tube_thick;low=.01f;high=1; }
else if(same(arg,"--walked")) { dest=&o->rf.walked;high=1.5f; }
else return "unknown option";
if(!parse_float(value,dest) || *dest<low || *dest>high) return "option outside supported range";
}
}
if(o->rf.walked>0 && o->rf.walked<.05f) return "option outside supported range";
if(o->refine_only) return o->input ? "--refine-only takes no --offline dump" : 0;
return o->input && o->out ? 0 : "required: --offline dump.fprs --out directory";
}
/* ------------------------------------------------------------------------------------------------ refinement */
typedef struct {
Report rep;
Label labels[MAX_VOLUMES];
int nlabels, changed, removed, nsuggest, out_triangles, out_volumes;
float ceiling_y;
i64 ms;
} Outcome;
static const char *copy_name(char *dst, const char *src, usize cap)
{
usize i=0;while(src[i] && i+1<cap) { dst[i]=src[i];i++; }
dst[i]=0;return dst;
}
static void box_json(Buf *b, const RVol *v)
{
put(b,"{\"cx\":");put_dec(b,v->cx);put(b,",\"cz\":");put_dec(b,v->cz);put(b,",\"top_y\":");put_dec(b,v->top);
put(b,",\"yaw_deg\":");put_dec(b,v->yaw);put(b,",\"w\":");put_dec(b,v->w);put(b,",\"d\":");put_dec(b,v->d);
put(b,",\"h\":");put_dec(b,v->h);put(b,"}");
}
static void scan_json(Buf *b, const Refine *o, const Vec *heads, int nheads, const Outcome *r)
{
put(b,"{\"version\":1,\"heads\":");heads_json(b,heads,nheads);
put(b,",\"refine\":");settings_json(b,o);
put(b,",\"report\":");report_json(b,&r->rep,r->nlabels,r->changed,r->removed,r->nsuggest);
put(b,",\"labels\":[");
for(int i=0;i<r->nlabels;i++) {
const Label *l=&r->labels[i];
if(i) put(b,",");
put(b,"{\"from\":\"");put(b,l->from);put(b,"\",\"to\":");
if(l->to[0]) { put(b,"\"");put(b,l->to);put(b,"\""); } else put(b,"null");
put(b,",\"reason\":\"");put(b,l->reason);put(b,"\"");
if(l->suggest[0]) { put(b,",\"suggest\":\"");put(b,l->suggest);put(b,"\""); }
put(b,",\"id\":\"");put(b,l->box.id);put(b,"\",\"box\":");box_json(b,&l->box);put(b,"}");
}
put(b,"],\"suggestions\":[");
int first=1;
for(int i=0;i<r->nlabels;i++) {
const Label *l=&r->labels[i];
if(!l->suggest[0]) continue;
if(!first) put(b,",");
first=0;
put(b,"{\"kind\":\"");put(b,l->suggest);put(b,"\",\"box\":");box_json(b,&l->box);
put(b,",\"wall\":");put_int(b,l->wall);put(b,",\"along\":");put_dec(b,l->along);
put(b,",\"reason\":\"");put(b,l->reason);put(b,"\"}");
}
put(b,"]}\n");
}
/* Refines `raw` (as read back from its own text) and publishes: [raw mesh, raw text,] scan.json, room-mesh.bin,
* room.txt. Everything is computed before the first file is written, so a failure leaves the old files. */
static const char *refine_publish(const char *dir, const Room *raw, const Vec *v, const u32 *idx, int nv, int ni,
const Vec *heads, int nheads, const Refine *o, int write_raw, Outcome *res)
{
static Room refined;
static RVol ruled[MAX_VOLUMES];
const char *error=0;
Prep prep={0};
Vec *rv=0;u32 *ri=0;int rn=0;
u8 *raw_mesh=0,*mesh=0;usize raw_size=0,mesh_size=0;
Buf raw_text,text,json;
buf_init(&raw_text,1u<<20);buf_init(&text,1u<<20);buf_init(&json,MAX_JSON_FILE);
i64 t0=io_time(1);
memset(res,0,sizeof(*res));
refined=*raw;
copy_name(refined.mesh_name,"room-mesh.bin",sizeof(refined.mesh_name));
if(o->refine) {
error=prep_mesh(&prep,v,idx,nv,ni);
int nruled=raw->nvols;
if(!error) {
if(o->label_rules) res->nlabels=rules_apply(raw,&prep,ruled,&nruled,res->labels);
else memcpy(ruled,raw->vols,sizeof(RVol)*(usize)raw->nvols);
}
if(!error) error=refine_run(o,raw,&prep,ruled,nruled,heads,nheads,&rv,&ri,&rn,&res->rep);
if(!error) {
refined.ceiling=res->rep.ceiling_y;
for(int i=0;i<refined.nwalls;i++) if(refined.walls[i].y1==raw->ceiling) refined.walls[i].y1=refined.ceiling;
memcpy(refined.vols,ruled,sizeof(RVol)*(usize)nruled);refined.nvols=nruled;
refined.mesh_nv=refined.mesh_ni=(u32)rn;refined.has_mesh=1;
for(int i=0;i<res->nlabels;i++) {
Label *l=&res->labels[i];
if(l->suggest[0]) res->nsuggest++;
if(!l->to[0]) res->removed++;
else if(!same(l->to,l->from)) res->changed++;
}
}
}
if(!error) error=mesh_bytes(rv ? rv : v,rv ? ri : idx,rv ? rn : nv,rv ? rn : ni,&mesh,&mesh_size);
if(!error && write_raw) error=mesh_bytes(v,idx,nv,ni,&raw_mesh,&raw_size);
if(!error && write_raw && !room_text(raw,&raw_text)) error="room text exceeds output budget";
if(!error && !room_text(&refined,&text)) error="room text exceeds output budget";
if(!error) { scan_json(&json,o,heads,nheads,res);if(!json.ok) error="scan.json exceeds output budget"; }
res->ms=io_time(1)-t0;
res->out_triangles=(int)refined.mesh_ni/3;res->out_volumes=refined.nvols;res->ceiling_y=refined.ceiling;
if(!error && !io_mkdir(dir)) error="cannot create output directory";
if(!error && write_raw) error=publish(dir,"room-mesh-raw.bin",raw_mesh,raw_size);
if(!error && write_raw) error=publish(dir,"room-raw.txt",raw_text.data,raw_text.n);
if(!error) error=publish(dir,"scan.json",json.data,json.n);
if(!error) error=publish(dir,"room-mesh.bin",mesh,mesh_size);
if(!error) error=publish(dir,"room.txt",text.data,text.n);
prep_free(&prep);
io_free(rv,(usize)MAX_MESH_VERTICES*sizeof(Vec));io_free(ri,(usize)MAX_TRIANGLES*3*sizeof(u32));
io_free(raw_mesh,raw_size);io_free(mesh,mesh_size);
buf_free(&raw_text);buf_free(&text);buf_free(&json);
return error;
}
static void refine_summary(Buf *t, const Outcome *r)
{
put(t,"\"refine\":");report_json(t,&r->rep,r->nlabels,r->changed,r->removed,r->nsuggest);
if(t->ok && t->n) t->n--; /* reopen the object for the output counts */
put(t,",\"output_triangles\":");put_int(t,r->out_triangles);
put(t,",\"output_volumes\":");put_int(t,r->out_volumes);put(t,"}");
}
/* ------------------------------------------------------------------------------------------------ summaries */
static void duration(Buf *t,i64 a,i64 b) { put_dec(t,(float)(b-a)/1000000); }
static void warnings_json(Buf *t, const char **w, int n)
{
put(t,",\"warnings\":[");
for(int i=0;i<n;i++) { if(i) put(t,",");put(t,"\"");put(t,w[i]);put(t,"\""); }
put(t,"]");
}
static void summary(Scan *s, const Outcome *r)
{
Buf t;buf_init(&t,1u<<16);put(&t,"{\"ok\":true,\"frames\":");put_u64(&t,s->frames);
put(&t,",\"skipped_frames\":");put_u64(&t,s->skipped_frames);
put(&t,",\"far_vertices\":");put_u64(&t,s->far_vertices);
put(&t,",\"discontinuity_triangles\":");put_u64(&t,s->discontinuities);
put(&t,",\"walls\":");put_u64(&t,(u64)s->noutline);
put(&t,",\"detected_walls\":");put_u64(&t,(u64)s->nplanes);
put(&t,",\"volumes\":");put_u64(&t,(u64)s->nvolumes);
put(&t,",\"floor_y\":");put_dec(&t,s->floor);put(&t,",\"ceiling_y\":");put_dec(&t,s->ceiling);
put(&t,",\"coverage_percent\":");put_dec(&t,s->coverage);
put(&t,",\"hint_used\":");put(&t,s->hint_used ? "true" : "false");
put(&t,",\"triangles\":");put_u64(&t,(u64)s->ni/3);
put(&t,",\"accepted_triangles\":");put_u64(&t,s->accepted);
put(&t,",\"rejected_triangles\":");put_u64(&t,s->rejected);
put(&t,",\"heads\":");put_int(&t,s->nheads);put(&t,",");
refine_summary(&t,r);
warnings_json(&t,s->warnings,s->nwarnings);
put(&t,",\"timings_ms\":{\"read\":");duration(&t,s->t0,s->tread);
put(&t,",\"fusion\":");duration(&t,s->tread,s->tfuse);
put(&t,",\"model\":");duration(&t,s->tfuse,s->tmodel);
put(&t,",\"mesh\":");duration(&t,s->tmodel,s->tmesh);
put(&t,",\"refine\":");duration(&t,s->tmesh,s->trefine);
put(&t,",\"total\":");duration(&t,s->t0,io_time(1));put(&t,"}}\n");io_write(1,t.data,t.n);buf_free(&t);
}
/* ------------------------------------------------------------------------------------------------ scan path */
static const char *scan_outputs(Scan *s, Outcome *out)
{
static Room built,raw;
static Vec heads[MAX_HEADS];
const char *error=room_from_scan(s,&built);
if(error) return error;
copy_name(built.mesh_name,"room-mesh-raw.bin",sizeof(built.mesh_name));
/* Refine what the raw files will hold, so --refine-only on them gives the same refined files. */
Buf t;buf_init(&t,1u<<20);
if(!room_text(&built,&t)) { buf_free(&t);return "room text exceeds output budget"; }
error=room_parse(t.data,t.n,&raw);
buf_free(&t);
if(error) return error;
Buf h;buf_init(&h,1u<<20);heads_json(&h,s->heads,s->nheads);put(&h,"}");
Buf wrap;buf_init(&wrap,1u<<20);put(&wrap,"{\"heads\":");put_n(&wrap,h.data,h.n);
int nheads=wrap.ok ? heads_parse(wrap.data,wrap.n,heads) : 0;
buf_free(&h);buf_free(&wrap);
return refine_publish(s->opt.out,&raw,s->vertices,s->indices,s->nv,s->ni,heads,nheads,&s->opt.rf,1,out);
}
/* ------------------------------------------------------------------------------------------------ --refine-only */
static int refine_only(Options *o)
{
static Room raw;
static Vec heads[MAX_HEADS];
static Outcome res;
const char *warns[4];int nwarn=0;
i64 t0=io_time(1);
const char *dir=o->refine_only,*error=0;
char path[4096];
u8 *text=0,*bin=0,*json=0;usize text_n=0,bin_n=0,json_n=0;
Vec *v=0;u32 *idx=0;int nv=0,ni=0,nheads=0,write_raw=0;
if(!join_path(path,dir,"room-raw.txt")) return error_json("output path too long");
error=read_file(path,1u<<20,&text,&text_n);
if(error) {
/* An older room folder: its room.txt + mesh are the unrefined scan. */
write_raw=1;
if(!join_path(path,dir,"room.txt")) return error_json("output path too long");
error=read_file(path,1u<<20,&text,&text_n);
if(error) return error_json("room folder has neither room-raw.txt nor room.txt");
}
error=room_parse((const char *)text,text_n,&raw);
io_free(text,(1u<<20)+1);
if(error) return error_json(error);
if(!raw.has_mesh) return error_json("room has no mesh to refine");
if(!join_path(path,dir,write_raw ? raw.mesh_name : "room-mesh-raw.bin")) return error_json("output path too long");
error=read_file(path,MAX_RAW_FILE,&bin,&bin_n);
if(!error) error=mesh_parse(bin,bin_n,&v,&idx,&nv,&ni);
io_free(bin,MAX_RAW_FILE+1);
if(error) return error_json(error);
if((u32)nv!=raw.mesh_nv || (u32)ni!=raw.mesh_ni) {
io_free(v,(usize)MAX_MESH_VERTICES*sizeof(Vec));io_free(idx,(usize)MAX_TRIANGLES*3*sizeof(u32));
return error_json("mesh record does not match the mesh file");
}
copy_name(raw.mesh_name,"room-mesh-raw.bin",sizeof(raw.mesh_name));
if(join_path(path,dir,"scan.json") && !read_file(path,MAX_JSON_FILE,&json,&json_n)) {
nheads=heads_parse((const char *)json,json_n,heads);
io_free(json,MAX_JSON_FILE+1);
}
if(!nheads && o->rf.walked>0) warns[nwarn++]="no head path; walked volume skipped";
if(write_raw) warns[nwarn++]="no raw files; the current room was taken as the unrefined scan";
error=refine_publish(dir,&raw,v,idx,nv,ni,heads,nheads,&o->rf,write_raw,&res);
io_free(v,(usize)MAX_MESH_VERTICES*sizeof(Vec));io_free(idx,(usize)MAX_TRIANGLES*3*sizeof(u32));
if(error) return error_json(error);
if(res.rep.truncated) warns[nwarn++]="refined mesh truncated to the triangle budget";
Buf t;buf_init(&t,1u<<16);
put(&t,"{\"ok\":true,\"refine_only\":true,\"raw_triangles\":");put_int(&t,ni/3);
put(&t,",\"volumes\":");put_int(&t,raw.nvols);put(&t,",\"floor_y\":");put_dec(&t,raw.floor);
put(&t,",\"ceiling_y\":");put_dec(&t,res.ceiling_y);put(&t,",\"heads\":");put_int(&t,nheads);put(&t,",");
refine_summary(&t,&res);
warnings_json(&t,warns,nwarn);
put(&t,",\"timings_ms\":{\"refine\":");put_dec(&t,(float)res.ms/1000000);
put(&t,",\"total\":");duration(&t,t0,io_time(1));put(&t,"}}\n");
io_write(1,t.data,t.n);buf_free(&t);
return 0;
}
int main(int argc,char **argv)
{
if(argc==2 && same(argv[1],"--help")) {
const char usage[]="fp_roomscan --offline dump.fprs --out directory [--universe N] [--voxel 0.05]\n"
" [--ceiling-default 2.5] [--hint outline.txt] [--json]\n"
"fp_roomscan --refine-only room-directory [refinement options] [--json]\n"
"Tuning: --truncation --max-edge --grazing --range --depth-jump --depth-ratio --plane-distance --min-plane-area\n"
" --min-views --patch-residual --wall-height --snap-degrees --floor-band --ceiling-min\n"
" --min-volume --cluster-gap --gap --horizontal\n"
"Refinement: --refine 0|1 --walls off|snap|panels --band --facing-deg --band-in --floor-in --ceil-in\n"
" --top-strip --near --clip --clip-open --island --smooth --tops --protect 0|1 --tubes --tube-thick\n"
" --walked --label-rules 0|1 --erase \"x,y,z,r;...\"\n"
"Always prints one JSON summary; errors exit 2. See docs/mr/scan.md for units and ranges.\n";
io_write(1,usage,sizeof(usage)-1);return 0;
}
Scan *s=io_alloc(sizeof(Scan));
if(!s) return error_json("scan allocation failed");
s->rng=0x714ce3u;s->t0=io_time(1);
const char *error=options(&s->opt,argc,argv);
if(!error && s->opt.refine_only) {
int result=refine_only(&s->opt);
io_free(s,sizeof(Scan));return result;
}
static Outcome outcome;
if(!error) error=read_dump(s,0);
s->tread=io_time(1);
if(!error) error=init_grid(s);
if(!error) error=read_dump(s,1);
if(!error) finish_fusion(s);
s->tfuse=io_time(1);
if(!error) error=make_model(s);
s->tmodel=io_time(1);
if(!error) error=make_mesh(s);
s->tmesh=io_time(1);
/* The grid is not needed after the mesh: freeing it keeps the refinement inside the same memory budget. */
io_free(s->cells,(usize)s->count*sizeof(Cell));s->cells=0;
if(!error) error=scan_outputs(s,&outcome);
s->trefine=io_time(1);
if(!error && outcome.rep.truncated) warn(s,"refined mesh truncated to the triangle budget");
int result=0;
if(error) result=error_json(error);else summary(s,&outcome);
io_free(s->vertices,(usize)MAX_MESH_VERTICES*sizeof(Vec));
io_free(s->indices,(usize)MAX_TRIANGLES*3*sizeof(u32));
io_free(s,sizeof(Scan));return result;
}
+112
View File
@@ -0,0 +1,112 @@
/* SPDX-License-Identifier: GPL-3.0-only */
#include "scan.h"
#define TANGENT_RADIUS_SQUARED 2.25f
#define TANGENT_WEIGHT_RADIUS_SQUARED 2.5f
#define MAX_EXTRACTION_STRIDE 4
/* Marching cubes without a 256-case table: trace the zero contour on the six faces, then fan each closed loop.
* Ambiguous faces choose their diagonal from the face centre. Adjacent cubes therefore make the same choice. */
static const int corners[8][3]={{0,0,0},{1,0,0},{1,0,1},{0,0,1},{0,1,0},{1,1,0},{1,1,1},{0,1,1}};
static const int edges[12][2]={{0,1},{1,2},{2,3},{3,0},{4,5},{5,6},{6,7},{7,4},{0,4},{1,5},{2,6},{3,7}};
static const int faces[6][4]={{0,1,2,3},{4,5,6,7},{0,9,4,8},{1,10,5,9},{2,11,6,10},{3,8,7,11}};
static void tsdf(Scan *s)
{
float v=s->opt.voxel,trunc=v*s->opt.truncation;
int radius=(int)s->opt.truncation+2;
for (int i=0;i<s->count;i++) {
Cell *c=&s->cells[i];
if (!c->weight) continue;
int x=i%s->nx,z=(i/s->nx)%s->nz,y=i/(s->nx*s->nz);
for (int dy=-radius;dy<=radius;dy++) for (int dz=-radius;dz<=radius;dz++) for (int dx=-radius;dx<=radius;dx++) {
int xx=x+dx,yy=y+dy,zz=z+dz;
if (xx<0 || yy<0 || zz<0 || xx>=s->nx || yy>=s->ny || zz>=s->nz) continue;
Vec delta=sub(grid_point(s,xx,yy,zz),c->p);
float d=dot(delta,c->n),tangent=dot(delta,delta)-d*d;
if (ab(d)>trunc || tangent>v*v*TANGENT_RADIUS_SQUARED) continue;
Cell *dest=&s->cells[(yy*s->nz+zz)*s->nx+xx];
float w=(1-tangent/(v*v*TANGENT_WEIGHT_RADIUS_SQUARED))*mn(c->weight/(v*v),1.f);
dest->sdf+=d*w;dest->sw+=w;
}
}
for (int i=0;i<s->count;i++) if(s->cells[i].sw) s->cells[i].sdf/=s->cells[i].sw;
}
static void connect(int links[12][2], int a, int b)
{
if(links[a][0]<0) links[a][0]=b;else links[a][1]=b;
if(links[b][0]<0) links[b][0]=a;else links[b][1]=a;
}
static int cube(Scan *s, int x, int y, int z, int stride)
{
float f[8];Vec p[8],normal=vec(0,0,0);
for(int k=0;k<8;k++) {
int xx=x+corners[k][0]*stride,yy=y+corners[k][1]*stride,zz=z+corners[k][2]*stride;
Cell *c=&s->cells[(yy*s->nz+zz)*s->nx+xx];
if (c->sw<=0) return 1; /* Unknown space never creates an invented surface. */
f[k]=c->sdf;p[k]=grid_point(s,xx,yy,zz);
}
Vec cuts[12];int crossing[12],links[12][2],visited[12]={0};
for(int e=0;e<12;e++) {
int a=edges[e][0],b=edges[e][1];
crossing[e]=(f[a]<0)!=(f[b]<0);links[e][0]=links[e][1]=-1;
if(crossing[e]) cuts[e]=add(p[a],mul(sub(p[b],p[a]),f[a]/(f[a]-f[b])));
}
/* Gradient points to positive TSDF: toward the observed free room, including furniture's exterior. */
normal.x=f[1]+f[2]+f[5]+f[6]-f[0]-f[3]-f[4]-f[7];
normal.y=f[4]+f[5]+f[6]+f[7]-f[0]-f[1]-f[2]-f[3];
normal.z=f[2]+f[3]+f[6]+f[7]-f[0]-f[1]-f[4]-f[5];
for(int face=0;face<6;face++) {
int hit[4],n=0;float centre=0;
for(int j=0;j<4;j++) {
int e=faces[face][j];
if(crossing[e]) hit[n++]=e;
centre+=f[edges[e][0]]+f[edges[e][1]];
}
if(n==2) connect(links,hit[0],hit[1]);
else if(n==4) {
/* The shared first edge's first corner fixes the phase on this face. */
int first=edges[hit[0]][0],second=edges[hit[0]][1];
int common=(edges[hit[1]][0]==second || edges[hit[1]][1]==second) ? second : first;
if((centre<0)==(f[common]<0)) {
connect(links,hit[0],hit[3]);connect(links,hit[1],hit[2]);
} else { connect(links,hit[0],hit[1]);connect(links,hit[2],hit[3]); }
}
}
for(int e=0;e<12;e++) if(crossing[e] && !visited[e]) {
int loop[12],n=0,prev=-1,cur=e;
while(cur>=0 && !visited[cur] && n<12) {
visited[cur]=1;loop[n++]=cur;
int next=links[cur][0]==prev ? links[cur][1] : links[cur][0];prev=cur;cur=next;
}
if(cur!=e) continue;
for(int k=1;k+1<n;k++) {
Vec a=cuts[loop[0]],b=cuts[loop[k]],c=cuts[loop[k+1]];
Vec norm=cross(sub(b,a),sub(c,a));
if(dot(norm,norm)<1e-14f) continue;
if(s->ni/3>=MAX_TRIANGLES) return 0;
if(dot(norm,normal)<0) { Vec t=b;b=c;c=t; }
s->vertices[s->nv]=a;s->vertices[s->nv+1]=b;s->vertices[s->nv+2]=c;
for(int j=0;j<3;j++) s->indices[s->ni++]=(u32)s->nv++;
}
}
return 1;
}
const char *make_mesh(Scan *s)
{
s->vertices=io_alloc((usize)MAX_MESH_VERTICES*sizeof(Vec));
s->indices=io_alloc((usize)MAX_TRIANGLES*3*sizeof(u32));
if(!s->vertices || !s->indices) return "mesh allocation failed";
tsdf(s);
for(int stride=1;stride<=MAX_EXTRACTION_STRIDE;stride++) {
s->nv=s->ni=0;int ok=1;
for(int y=0;y+stride<s->ny && ok;y+=stride)
for(int z=0;z+stride<s->nz && ok;z+=stride)
for(int x=0;x+stride<s->nx && ok;x+=stride) ok=cube(s,x,y,z,stride);
if(ok) {
if(stride>1) warn(s,"mesh decimated by coarsening extraction grid");
if(!s->ni) warn(s,"no mesh surface; scan denser views");
return 0;
}
}
return "mesh exceeds triangle budget";
}
+672
View File
@@ -0,0 +1,672 @@
/* SPDX-License-Identifier: GPL-3.0-only */
#include "scan.h"
#define RANSAC_TRIALS 96
#define NORMAL_AGREEMENT .90f
#define MIN_WALL_SPAN .60f
#define FURNITURE_MARGIN .12f
#define NEAR_WALL .35f
#define MIN_TOP_AREA .25f
#define BOX_TRIM .02f
#define VERTICAL_NORMAL_Y .35f
#define BOX_SIDE_NORMAL_Y .30f
#define BOX_EDGE_STRENGTH .40f
#define MIN_CLUSTER_SURFELS 12
#define MIN_BOX_EDGE_VOTES 12
#define MIN_OUTLINE_EXTENT 1.f
#define MIN_OUTLINE_TWICE_AREA .5f
#define FLOOR_INFER_DISTANCE .3f
#define TABLE_MIN_HEIGHT .55f
#define TABLE_MAX_HEIGHT 1.1f
#define COUCH_MIN_HEIGHT .35f
#define COUCH_MAX_HEIGHT .9f
#define COUCH_MIN_LENGTH 1.4f
#define BED_MIN_HEIGHT .3f
#define BED_MAX_HEIGHT .8f
#define BED_MIN_AREA 2.f
#define STORAGE_MIN_HEIGHT 1.1f
#define SCREEN_MAX_DEPTH .22f
#define SCREEN_MIN_WIDTH .4f
#define SCREEN_MAX_WIDTH 2.f
#define DEFAULT_CIRCLE_MIN_POINTS 8
#define DEFAULT_CIRCLE_RADIAL_CV .05f
#define LOW_COVERAGE_PERCENT 80.f
#define WALL_MERGE_DISTANCE .25f
#define WALL_MERGE_COSINE .97f
#define RELATIVE_WALL_AREA .20f
#define DOMINANT_WALL_TOLERANCE 12.f
#define ORTHOGONAL_SUPPORT_FRACTION .65f
#define CEILING_MIN_AREA .08f /* m^2 of fused down-facing cells at one height (fused surfaces are sparse: the
* owner's floor was 0.7 m^2 of cells, the ceiling 0.11 m^2 after a 2 s look up) */
#define CEILING_MIN_HEIGHT 2.0f /* m above the floor: lower ones are shelves, door frames, table undersides */
#define CEILING_MAX_HEIGHT 4.0f
#define BOX_AXIS_COS .96f /* walls within ~16 deg of the two room axes (smeared depth tilted a real
* wall 12 deg in the owner's scan) */
#define BOX_SNAP .5f /* m: a rectangle side snaps to a detected wall this close */
#define BOX_LOW_SHARE .02f /* surfaces beyond the 2nd/98th percentile along an axis are outliers */
#define WALL_BAND_TOP 2.4f /* m above the floor: walls are fitted below this (and below the ceiling) */
#define MIN_FURNITURE_TOP .25f
#define WALL_FURNITURE_CLEARANCE .15f
#define MIN_TOP_DENSITY .30f
static void consolidate_walls(Scan *s)
{
if(!s->nplanes) return;
Plane result[MAX_PLANES]={0};int count=0;
float largest=0;
for(int i=0;i<s->nplanes;i++) {
Plane p=s->planes[i];int merge=-1;
largest=mx(largest,p.area);
for(int j=0;j<count;j++) if(dot(p.n,result[j].n)>WALL_MERGE_COSINE &&
ab(p.d-result[j].d)<WALL_MERGE_DISTANCE) { merge=j;break; }
if(merge<0) result[count++]=p;
else {
Plane *r=&result[merge];
r->area+=p.area;r->bottom=mn(r->bottom,p.bottom);r->top=mx(r->top,p.top);
r->lo=mn(r->lo,p.lo);r->hi=mx(r->hi,p.hi);
}
}
float base=angle(result[0].n.z,result[0].n.x),aligned=0,total=0;
for(int i=0;i<count;i++) {
Plane *p=&result[i];float a=angle(p->n.z,p->n.x)-base;
float delta=ab(a-flo(a/(PI/2)+.5f)*(PI/2));
total+=p->area;
if(delta<DOMINANT_WALL_TOLERANCE*PI/180) aligned+=p->area;
}
int orthogonal=aligned>total*ORTHOGONAL_SUPPORT_FRACTION && s->opt.snap_degrees>0;
s->nplanes=0;
for(int i=0;i<count;i++) {
Plane p=result[i];float a=angle(p.n.z,p.n.x)-base;
float delta=ab(a-flo(a/(PI/2)+.5f)*(PI/2));
if(p.area<largest*RELATIVE_WALL_AREA) continue;
if(orthogonal && delta>DOMINANT_WALL_TOLERANCE*PI/180) continue;
s->planes[s->nplanes++]=p;
}
}
static float plane_height(Scan *s, int ceiling, int *found)
{
float hist[512]={0}, step=s->opt.voxel*.5f;
int bins=(int)(4/step)+2;
if (bins>512) bins=512;
for (int i=0;i<s->count;i++) {
Cell *c=&s->cells[i];
if (!c->weight || (ceiling ? c->n.y > -s->opt.horizontal : c->n.y < s->opt.horizontal)) continue;
if (ceiling ? c->p.y<s->opt.ceiling_min : ab(c->p.y)>s->opt.floor_band) continue;
int b=flo((c->p.y-s->origin.y)/step);
if (b>=0 && b<bins) hist[b]+=1;
}
int best=0;
float score=0;
for (int b=1;b<bins-1;b++) {
float h=hist[b-1]+hist[b]+hist[b+1];
if (h>score) { score=h; best=b; }
}
float y=s->origin.y+(best+.5f)*step, total=0, sum=0,upward=0;
for (int i=0;i<s->count;i++) {
Cell *c=&s->cells[i];
if(c->weight && c->n.y>s->opt.horizontal) upward++;
if (!c->weight || ab(c->p.y-y)>s->opt.plane_distance) continue;
if (ceiling ? c->n.y > -s->opt.horizontal : c->n.y < s->opt.horizontal) continue;
total++; sum+=c->p.y;
}
*found=(ceiling || score* s->opt.voxel*s->opt.voxel >= s->opt.min_plane_area) && total>0;
/* A ceiling is judged on its own area and height, not against the floor: players glance up for a second and look
* at the floor all the time, so a share-of-upward-surfaces rule rejected every real ceiling (owner's scans,
* 2026-10-16: down-facing surfaces at ~2.9 m in both recordings, "ceiling not seen"). */
(void)upward;
if(ceiling && (total*s->opt.voxel*s->opt.voxel<CEILING_MIN_AREA || y-s->floor<CEILING_MIN_HEIGHT ||
y-s->floor>CEILING_MAX_HEIGHT)) *found=0;
return *found ? sum/total : (ceiling ? s->floor+s->opt.ceiling_default : 0);
}
static int on_plane(Scan *s, Cell *c, Plane *p)
{
return ab(dot(c->p,p->n)-p->d)<s->opt.plane_distance && dot(c->n,p->n)>NORMAL_AGREEMENT;
}
static void refine(Scan *s, int *ids, int n, Plane *p)
{
Vec avg=vec(0,0,0), normal=vec(0,0,0);
float count=0;
for (int i=0;i<n;i++) {
Cell *c=&s->cells[ids[i]];
if (!c->mark && on_plane(s,c,p)) { avg=add(avg,c->p); normal=add(normal,c->n); count++; }
}
if (!count) return;
avg=mul(avg,1/count);
/* Fit positions, rather than averaging noisy per-triangle normals. */
float xx=0,zz=0,xz=0;
for (int i=0;i<n;i++) {
Cell *c=&s->cells[ids[i]];
if (!c->mark && on_plane(s,c,p)) {
float x=c->p.x-avg.x,z=c->p.z-avg.z;
xx+=x*x; zz+=z*z; xz+=x*z;
}
}
float yaw=.5f*angle(2*xz,xx-zz);
Vec nn=vec(-sine(yaw),0,cosine(yaw));
if (dot(nn,normal)<0) nn=mul(nn,-1);
p->n=nn; p->d=dot(nn,avg);
p->lo=1e10f; p->hi=-1e10f; p->bottom=1e10f; p->top=-1e10f;
count=0;
for (int i=0;i<n;i++) {
Cell *c=&s->cells[ids[i]];
if (!c->mark && on_plane(s,c,p)) {
float t=c->p.x*p->n.z-c->p.z*p->n.x;
p->lo=mn(p->lo,t); p->hi=mx(p->hi,t);
p->bottom=mn(p->bottom,c->p.y); p->top=mx(p->top,c->p.y); count++;
}
}
p->area=count*s->opt.voxel*s->opt.voxel;
}
static void walls(Scan *s, int *ids, int n)
{
/* Walls are fitted in a fixed band above the floor whatever the ceiling: Valve's smoothed depth bends where wall
* meets ceiling, and with a detected 2.9 m ceiling that band confused the fit (owner's scan, 2026-10-16). */
float top=mn(s->ceiling,s->floor+WALL_BAND_TOP)-FURNITURE_MARGIN;
for (int i=0;i<n;i++) {
Cell *c=&s->cells[ids[i]];
c->mark=ab(c->n.y)>VERTICAL_NORMAL_Y || c->p.y<s->floor+FURNITURE_MARGIN || c->p.y>top;
}
for (int attempt=0;attempt<MAX_PLANES*2 && s->nplanes<MAX_PLANES;attempt++) {
Plane best={0}; int best_count=0;
for (int trial=0;trial<RANSAC_TRIALS;trial++) {
Cell *seed=&s->cells[ids[random_next(s)%(unsigned)n]];
if (seed->mark) continue;
Plane p={0}; p.n=unit(vec(seed->n.x,0,seed->n.z)); p.d=dot(seed->p,p.n);
int count=0;
for (int j=0;j<n;j++) {
Cell *c=&s->cells[ids[j]];
if (!c->mark && on_plane(s,c,&p)) count++;
}
if (count>best_count) { best_count=count; best=p; }
}
if (best_count*s->opt.voxel*s->opt.voxel<s->opt.min_plane_area) break;
refine(s,ids,n,&best);
/* Height separates large furniture faces from room walls. */
int wall=best.area>=s->opt.min_plane_area && best.top-best.bottom>=s->opt.wall_height &&
best.hi-best.lo>=MIN_WALL_SPAN;
for (int j=0;j<n;j++) {
Cell *c=&s->cells[ids[j]];
if (!c->mark && on_plane(s,c,&best)) c->mark=wall ? 2 : 3;
}
if (wall) s->planes[s->nplanes++]=best;
/* Rejected candidates are consumed too, ensuring bounded progress. */
}
if (s->nplanes==MAX_PLANES) warn(s,"wall limit reached");
if (s->nplanes) {
float base=angle(s->planes[0].n.z,s->planes[0].n.x);
for (int k=0;k<s->nplanes;k++) {
Plane *p=&s->planes[k];
float a=angle(p->n.z,p->n.x), diff=a-base;
int quarter=flo(diff/(PI/2)+.5f);
float target=base+quarter*(PI/2);
if (ab(a-target)<s->opt.snap_degrees*PI/180) {
Vec nn=vec(cosine(target),0,sine(target));
float sum=0,count=0,lo=1e10f,hi=-1e10f;
for (int j=0;j<n;j++) {
Cell *c=&s->cells[ids[j]];
if (on_plane(s,c,p)) {
sum+=dot(c->p,nn); count++;
float t=c->p.x*nn.z-c->p.z*nn.x;
lo=mn(lo,t); hi=mx(hi,t);
}
}
if (count) { p->n=nn; p->d=sum/count; p->lo=lo; p->hi=hi; }
}
}
}
consolidate_walls(s);
}
static Pt endpoint(Plane *p, int end)
{
float t=end ? p->hi : p->lo;
return (Pt){p->n.x*p->d+p->n.z*t,p->n.z*p->d-p->n.x*t};
}
static float distance2(Pt a, Pt b) { float x=a.x-b.x,z=a.z-b.z; return x*x+z*z; }
static int intersect(Plane *a, Plane *b, Pt *p)
{
float det=a->n.x*b->n.z-a->n.z*b->n.x;
if (ab(det)<.1f) return 0;
p->x=(a->d*b->n.z-a->n.z*b->d)/det;
p->z=(a->n.x*b->d-a->d*b->n.x)/det;
return 1;
}
static void push_outline(Scan *s, Pt p, int invisible)
{
if (s->noutline && distance2(s->outline[s->noutline-1],p)<.0001f) return;
if (s->noutline<MAX_OUTLINE) {
s->outline[s->noutline]=p;
s->invisible[s->noutline++]=invisible;
}
}
static void rectangle(Scan *s, int *ids, int n)
{
float x0=1e10f,x1=-1e10f,z0=1e10f,z1=-1e10f;
for (int i=0;i<n;i++) {
Vec p=s->cells[ids[i]].p;
x0=mn(x0,p.x);x1=mx(x1,p.x);z0=mn(z0,p.z);z1=mx(z1,p.z);
}
if (x1-x0<MIN_OUTLINE_EXTENT) { float m=(x0+x1)*.5f; x0=m-MIN_OUTLINE_EXTENT/2; x1=m+MIN_OUTLINE_EXTENT/2; }
if (z1-z0<MIN_OUTLINE_EXTENT) { float m=(z0+z1)*.5f; z0=m-MIN_OUTLINE_EXTENT/2; z1=m+MIN_OUTLINE_EXTENT/2; }
push_outline(s,(Pt){x0,z0},1);push_outline(s,(Pt){x1,z0},1);
push_outline(s,(Pt){x1,z1},1);push_outline(s,(Pt){x0,z1},1);
warn(s,"incomplete walls; synthesized outline");
}
static int segment_cross(Pt a, Pt b, Pt c, Pt d)
{
float p=(b.x-a.x)*(c.z-a.z)-(b.z-a.z)*(c.x-a.x);
float q=(b.x-a.x)*(d.z-a.z)-(b.z-a.z)*(d.x-a.x);
float r=(d.x-c.x)*(a.z-c.z)-(d.z-c.z)*(a.x-c.x);
float t=(d.x-c.x)*(b.z-c.z)-(d.z-c.z)*(b.x-c.x);
return p*q<-.000001f && r*t<-.000001f;
}
static int valid_polygon(Scan *s)
{
float area=0;
for (int i=0;i<s->noutline;i++) {
Pt a=s->outline[i], b=s->outline[(i+1)%s->noutline];
area+=a.x*b.z-b.x*a.z;
for (int j=i+2;j<s->noutline;j++) {
if ((j+1)%s->noutline==i) continue;
if (segment_cross(a,b,s->outline[j],s->outline[(j+1)%s->noutline])) return 0;
}
}
return s->noutline>=3 && area>MIN_OUTLINE_TWICE_AREA;
}
static int convex_fallback(Scan *s)
{
if(s->nplanes<3 || s->nplanes>4) return 0;
/* A concave multi-wall room must use directed segments. A small set of enclosure planes can instead be
* clipped as inward half-planes when smeared endpoints prevent reliable corner ordering. */
Pt a[MAX_OUTLINE],b[MAX_OUTLINE];float x0=1e10f,x1=-1e10f,z0=1e10f,z1=-1e10f;
for(int i=0;i<s->nplanes;i++) for(int end=0;end<2;end++) {
Pt p=endpoint(&s->planes[i],end);x0=mn(x0,p.x);x1=mx(x1,p.x);z0=mn(z0,p.z);z1=mx(z1,p.z);
}
int n=4;a[0]=(Pt){x0,z0};a[1]=(Pt){x1,z0};a[2]=(Pt){x1,z1};a[3]=(Pt){x0,z1};
for(int i=0;i<s->nplanes;i++) {
Plane *p=&s->planes[i];int count=0;
for(int j=0;j<n;j++) {
Pt u=a[j],v=a[(j+1)%n];float du=p->n.x*u.x+p->n.z*u.z-p->d,dv=p->n.x*v.x+p->n.z*v.z-p->d;
if(du>=0) b[count++]=u;
if((du>=0)!=(dv>=0)) {
float t=du/(du-dv);b[count++]=(Pt){u.x+t*(v.x-u.x),u.z+t*(v.z-u.z)};
}
}
if(count<3 || count>=MAX_OUTLINE) return 0;
n=count;memcpy(a,b,(usize)n*sizeof(Pt));
}
s->noutline=n;
for(int j=0;j<n;j++) {
s->outline[j]=a[j];s->invisible[j]=1;Pt v=a[(j+1)%n];
for(int i=0;i<s->nplanes;i++) {
Plane *p=&s->planes[i];
if(ab(p->n.x*a[j].x+p->n.z*a[j].z-p->d)<s->opt.plane_distance &&
ab(p->n.x*v.x+p->n.z*v.z-p->d)<s->opt.plane_distance) s->invisible[j]=0;
}
}
if(!valid_polygon(s)) { s->noutline=0;return 0; }
warn(s,"wall endpoints uncertain; using plane intersections");return 1;
}
/* Value below which `share` of the projections fall (2 cm bins over +-8 m around the data). */
static float projection_percentile(Scan *s, int *ids, int n, Pt axis, float share)
{
static u32 hist[800];
memset(hist,0,sizeof(hist));
for (int i=0;i<n;i++) {
Vec p=s->cells[ids[i]].p;
int b=(int)((p.x*axis.x+p.z*axis.z+8)/.02f);
if (b>=0 && b<800) hist[b]++;
}
u64 total=0,run=0;
for (int b=0;b<800;b++) total+=hist[b];
for (int b=0;b<800;b++) { run+=hist[b]; if (run>=share*total) return b*.02f-8+.01f; }
return 8;
}
/* Most rooms are rectangles. When every detected wall runs along one of two perpendicular axes (2-4 walls), the
* outline is that rectangle: its sides at the 2nd/98th percentile of the surfaces along each axis, each snapped to a
* detected wall when one lies within BOX_SNAP. Directed wall chaining made a skewed quadrilateral of the owner's
* room from three smeared walls (2026-10-16). L-shaped rooms have more walls and keep the chaining below. */
static int orthogonal_box(Scan *s, int *ids, int n)
{
if (s->nplanes<2 || s->nplanes>4) return 0;
Vec u=s->planes[0].n; float len=root(u.x*u.x+u.z*u.z);
if (len<.5f) return 0;
Pt axis[2]={{u.x/len,u.z/len},{-u.z/len,u.x/len}};
float side[2][2]; int wall[2][2]={{0,0},{0,0}}, snapped=0;
for (int k=0;k<2;k++) {
side[k][0]=projection_percentile(s,ids,n,axis[k],BOX_LOW_SHARE);
side[k][1]=projection_percentile(s,ids,n,axis[k],1-BOX_LOW_SHARE);
}
for (int i=0;i<s->nplanes;i++) {
Plane *p=&s->planes[i];
float c0=p->n.x*axis[0].x+p->n.z*axis[0].z, c1=p->n.x*axis[1].x+p->n.z*axis[1].z;
int k=ab(c0)>=BOX_AXIS_COS ? 0 : ab(c1)>=BOX_AXIS_COS ? 1 : -1;
if (k<0) return 0; // a wall off both axes: not a rectangle
float offset=p->d*(k ? c1 : c0)/ab(k ? c1 : c0);
int end=ab(offset-side[k][0])<ab(offset-side[k][1]) ? 0 : 1;
if (ab(offset-side[k][end])<BOX_SNAP) { side[k][end]=offset; snapped+=!wall[k][end]; wall[k][end]=1; }
}
if (snapped<2 || side[0][1]-side[0][0]<MIN_OUTLINE_EXTENT || side[1][1]-side[1][0]<MIN_OUTLINE_EXTENT) return 0;
/* corners (s,t) -> world x = a0.x s + a1.x t, z = a0.z s + a1.z t; edge flags = whether its side is a wall */
int order[4][2]={{0,0},{1,0},{1,1},{0,1}}, edge[4][2]={{1,0},{0,1},{1,1},{0,0}}; // edge k lies on side[a][e]
float twice=0;
Pt c[4];
for (int j=0;j<4;j++) {
float a=side[0][order[j][0]], b=side[1][order[j][1]];
c[j]=(Pt){axis[0].x*a+axis[1].x*b, axis[0].z*a+axis[1].z*b};
}
for (int j=0;j<4;j++) twice+=c[j].x*c[(j+1)%4].z-c[(j+1)%4].x*c[j].z;
s->noutline=0;
if (twice>0) for (int j=0;j<4;j++) push_outline(s,c[j],!wall[edge[j][0]][edge[j][1]]);
else for (int j=0;j<4;j++) { // reversed: corner 0,3,2,1; edge from corner k to k-1 is the old edge k-1
int k=(4-j)%4, e=(k+3)%4;
push_outline(s,c[k],!wall[edge[e][0]][edge[e][1]]);
}
if (!valid_polygon(s)) { s->noutline=0; return 0; }
if (snapped<4) warn(s,"gaps closed with invisible walls");
return 1;
}
static void outline(Scan *s, int *ids, int n)
{
if (orthogonal_box(s,ids,n)) return;
if (s->nplanes<3) { rectangle(s,ids,n); return; }
int order[MAX_PLANES]={0};
s->planes[0].used=1;
for (int k=1;k<s->nplanes;k++) {
Pt end=endpoint(&s->planes[order[k-1]],1);
float best=1e10f; int which=-1;
for (int j=0;j<s->nplanes;j++) if (!s->planes[j].used) {
float d=distance2(end,endpoint(&s->planes[j],0));
if (d<best) { best=d;which=j; }
}
order[k]=which;s->planes[which].used=1;
}
Pt starts[MAX_PLANES], ends[MAX_PLANES];
for (int k=0;k<s->nplanes;k++) {
Plane *p=&s->planes[order[k]], *next=&s->planes[order[(k+1)%s->nplanes]];
ends[k]=endpoint(p,1); starts[(k+1)%s->nplanes]=endpoint(next,0);
Pt corner;
if (intersect(p,next,&corner) && distance2(corner,ends[k])<s->opt.gap*s->opt.gap &&
distance2(corner,starts[(k+1)%s->nplanes])<s->opt.gap*s->opt.gap)
ends[k]=starts[(k+1)%s->nplanes]=corner;
}
for (int k=0;k<s->nplanes;k++) {
push_outline(s,starts[k],0);
if (distance2(ends[k],starts[(k+1)%s->nplanes])>.0001f) {
push_outline(s,ends[k],1);warn(s,"gaps closed with invisible walls");
}
}
if (!valid_polygon(s)) {
s->noutline=0;
if(!convex_fallback(s)) { rectangle(s,ids,n);warn(s,"wall ordering ambiguous"); }
}
}
static float wall_distance(Scan *s, Vec p)
{
float best=1e10f;
for (int i=0;i<s->nplanes;i++) best=mn(best,ab(dot(s->planes[i].n,p)-s->planes[i].d));
return best;
}
static void trimmed_bounds(Scan *s, int *queue, int n, Vec axis, float *lo, float *hi)
{
int hist[512]={0};float start=*lo,range=*hi-start;
if(range<.001f) return;
for(int i=0;i<n;i++) {
float value=dot(s->cells[queue[i]].p,axis);
int bin=(int)clamp((value-start)/range*511,0,511);hist[bin]++;
}
int count=0,target=(int)(n*BOX_TRIM);
for(int b=0;b<512;b++) { count+=hist[b];if(count>target) { *lo=start+range*b/511;break; } }
count=0;
for(int b=511;b>=0;b--) { count+=hist[b];if(count>target) { *hi=start+range*(b+1)/511;break; } }
}
static float top_height(Scan *s, int *queue, int n, float low, float high)
{
int histogram[512]={0},bins=(int)mn((high-low)/(s->opt.voxel*.25f)+2,512);
float step=s->opt.voxel*.25f;
for(int i=0;i<n;i++) {
int bin=flo((s->cells[queue[i]].p.y-low)/step);
if(bin>=0 && bin<bins) histogram[bin]++;
}
int count=0;
for(int bin=0;bin<bins;bin++) {
count+=histogram[bin];
/* The median resists a softened lip without selecting one of several noisy sub-peaks as a new top. */
if(count>=n/2) return low+(bin+.5f)*step;
}
return (low+high)*.5f;
}
static void furniture(Scan *s, int *ids, int n, int *queue)
{
/* Tops remain separable when a softened sofa/table edge blends into its neighbouring wall. Vertical-only
* components are considered separately for screens; they cannot turn a whole wall into an OTHER volume. */
for(int mode=0;mode<2;mode++) {
float histogram[256]={0},levels[32];int nlevels=0;
if(!mode) {
for(int j=0;j<n;j++) {
Cell *c=&s->cells[ids[j]];
if(c->n.y>s->opt.horizontal && c->p.y>s->floor+MIN_FURNITURE_TOP &&
c->p.y<s->ceiling-FURNITURE_MARGIN && wall_distance(s,c->p)>WALL_FURNITURE_CLEARANCE) {
int bin=flo((c->p.y-s->floor)/s->opt.voxel);
if(bin>=0 && bin<256) histogram[bin]++;
}
}
for(int k=0;k<32;k++) {
int best=0;float score=0;
for(int bin=1;bin<255;bin++) {
float count=histogram[bin-1]+histogram[bin]+histogram[bin+1];
if(count>score) { score=count;best=bin; }
}
if(score*s->opt.voxel*s->opt.voxel<MIN_TOP_AREA) break;
levels[nlevels++]=s->floor+(best+.5f)*s->opt.voxel;
int radius=(int)(s->opt.plane_distance*2/s->opt.voxel)+1;
for(int bin=best-radius;bin<=best+radius;bin++) if(bin>=0 && bin<256) histogram[bin]=0;
}
}
for (int j=0;j<n;j++) {
Cell *c=&s->cells[ids[j]];
int normal=mode ? ab(c->n.y)<BOX_SIDE_NORMAL_Y : c->n.y>s->opt.horizontal;
c->mark=(normal && c->p.y>s->floor+MIN_FURNITURE_TOP && c->p.y<s->ceiling-FURNITURE_MARGIN &&
wall_distance(s,c->p)>WALL_FURNITURE_CLEARANCE) ? 1 : 0;
if(c->mark && !mode) {
c->mark=0;float best=s->opt.plane_distance*2;
for(int k=0;k<nlevels;k++) if(ab(c->p.y-levels[k])<best) {
best=ab(c->p.y-levels[k]);c->mark=k+1;
}
}
}
int radius=(int)(s->opt.cluster_gap/s->opt.voxel)+1;
for (int j=0;j<n;j++) {
int seed=ids[j];
if (s->cells[seed].mark<=0) continue;
int group=s->cells[seed].mark;
int head=0,tail=1;queue[0]=seed;s->cells[seed].mark=-1;
while (head<tail) {
int id=queue[head++],x=id%s->nx,z=(id/s->nx)%s->nz,y=id/(s->nx*s->nz);
for (int dy=-radius;dy<=radius;dy++) for (int dz=-radius;dz<=radius;dz++) for (int dx=-radius;dx<=radius;dx++) {
if (dx*dx+dy*dy+dz*dz>radius*radius) continue;
int xx=x+dx,yy=y+dy,zz=z+dz;
if (xx<0 || yy<0 || zz<0 || xx>=s->nx || yy>=s->ny || zz>=s->nz) continue;
int next=(yy*s->nz+zz)*s->nx+xx;
Cell *c=&s->cells[next];
if (c->mark==group) { c->mark=-1;queue[tail++]=next; }
}
}
if (tail<MIN_CLUSTER_SURFELS) continue;
Vec avg=vec(0,0,0);float top=-1e10f,bottom=1e10f;
for (int k=0;k<tail;k++) {
Vec p=s->cells[queue[k]].p;avg=add(avg,p);top=mx(top,p.y);bottom=mn(bottom,p.y);
}
avg=mul(avg,1.f/tail);
float xx=0,zz=0,xz=0,edge_x=0,edge_y=0,edge_count=0;
for (int k=0;k<tail;k++) {
Cell *c=&s->cells[queue[k]];
Vec p=sub(c->p,avg);xx+=p.x*p.x;zz+=p.z*p.z;xz+=p.x*p.z;
if(ab(c->n.y)<BOX_SIDE_NORMAL_Y) {
Vec norm=unit(vec(c->n.x,0,c->n.z));
float co2=norm.x*norm.x-norm.z*norm.z,si2=2*norm.x*norm.z;
edge_x+=co2*co2-si2*si2;edge_y+=2*co2*si2;edge_count++;
}
}
float yaw=.5f*angle(2*xz,xx-zz);
/* Opposing faces and perpendicular edges vote for the same fourth harmonic. Unlike PCA, this is not
* biased by one side being more visible. Non-box objects fall back to position covariance. */
if(edge_count>MIN_BOX_EDGE_VOTES && root(edge_x*edge_x+edge_y*edge_y)>edge_count*BOX_EDGE_STRENGTH)
yaw=.25f*angle(edge_y,edge_x);
if(!mode) {
float smallest=1e10f;
for(int degree=0;degree<90;degree++) {
float a=degree*PI/180,co=cosine(a),si=sine(a),u0=1e10f,u1=-1e10f,v0=1e10f,v1=-1e10f;
for(int k=0;k<tail;k++) {
Vec p=s->cells[queue[k]].p;float u=p.x*co+p.z*si,v=-p.x*si+p.z*co;
u0=mn(u0,u);u1=mx(u1,u);v0=mn(v0,v);v1=mx(v1,v);
}
float area=(u1-u0)*(v1-v0);
if(area<smallest) { smallest=area;yaw=a; }
}
}
float co=cosine(yaw),si=sine(yaw);
float u0=1e10f,u1=-1e10f,v0=1e10f,v1=-1e10f,top_count=0;
for (int k=0;k<tail;k++) {
Cell *c=&s->cells[queue[k]];Vec p=sub(c->p,avg);
float u=p.x*co+p.z*si,v=-p.x*si+p.z*co;
u0=mn(u0,u);u1=mx(u1,u);v0=mn(v0,v);v1=mx(v1,v);
if (c->n.y>s->opt.horizontal && c->p.y>top-s->opt.plane_distance*2) top_count++;
}
/* Stereo discontinuities leave a low-density halo even after long-edge rejection. Trim the outer
* two percent before measuring boxes; otherwise a table can become wide enough to look like a couch. */
float up=dot(avg,vec(co,0,si)),vp=dot(avg,vec(-si,0,co));
u0+=up;u1+=up;v0+=vp;v1+=vp;
trimmed_bounds(s,queue,tail,vec(co,0,si),&u0,&u1);
trimmed_bounds(s,queue,tail,vec(-si,0,co),&v0,&v1);
u0-=up;u1-=up;v0-=vp;v1-=vp;
trimmed_bounds(s,queue,tail,vec(0,1,0),&bottom,&top);
if(!mode) top=top_height(s,queue,tail,bottom,top);
float w=u1-u0,d=v1-v0,h=top-(bottom<s->floor+FLOOR_INFER_DISTANCE ? s->floor : mn(bottom,s->floor+FURNITURE_MARGIN));
if(d>w) { float t=w;w=d;d=t;yaw+=PI/2; }
while(yaw>PI/2) yaw-=PI;
if(!mode) h=top-s->floor;
if (w*d*h<s->opt.min_volume) continue;
float height=top-s->floor,area=w*d,top_area=(mode ? top_count : tail)*s->opt.voxel*s->opt.voxel;
if(!mode && (top_area<MIN_TOP_AREA || top_area<area*MIN_TOP_DENSITY)) continue;
const char *label="OTHER";
if (height>=BED_MIN_HEIGHT && height<=BED_MAX_HEIGHT && area>=BED_MIN_AREA) label="BED";
else if (height>=COUCH_MIN_HEIGHT && height<=COUCH_MAX_HEIGHT && mx(w,d)+2*s->opt.voxel>=COUCH_MIN_LENGTH) label="COUCH";
else if (height>=TABLE_MIN_HEIGHT && height<=TABLE_MAX_HEIGHT && top_area>=MIN_TOP_AREA) label="TABLE";
else if (mn(w,d)<SCREEN_MAX_DEPTH && mx(w,d)>=SCREEN_MIN_WIDTH && mx(w,d)<=SCREEN_MAX_WIDTH &&
wall_distance(s,avg)<NEAR_WALL) label="SCREEN";
else if (h>=STORAGE_MIN_HEIGHT && wall_distance(s,avg)<NEAR_WALL) label="STORAGE";
if(mode && !same(label,"SCREEN")) continue;
if(!mode && same(label,"OTHER") && height>TABLE_MAX_HEIGHT) continue;
if (s->nvolumes>=MAX_VOLUMES) { warn(s,"furniture limit reached");break; }
Vec centre=vec(avg.x+co*(u0+u1)*.5f-si*(v0+v1)*.5f,top,
avg.z+si*(u0+u1)*.5f+co*(v0+v1)*.5f);
/* ROOM_FORMAT yaw is an active right-handed rotation about +Y. */
s->volumes[s->nvolumes++]=(Volume){centre,-yaw*180/PI,w,d,h,label};
}
}
}
/* A hint is a fallback for missing walls, never a constraint on observed geometry. */
static const char *hint(Scan *s);
const char *make_model(Scan *s)
{
int found;
s->floor=plane_height(s,0,&found);
if (!found) warn(s,"floor not seen; using STAGE floor");
s->ceiling=plane_height(s,1,&found);
if (!found) warn(s,"ceiling not seen");
usize bytes=(usize)s->count*sizeof(int);
int *ids=io_alloc(bytes),*queue=io_alloc(bytes);
if (!ids || !queue) { io_free(ids,bytes);io_free(queue,bytes);return "model allocation failed"; }
int n=0;
for (int i=0;i<s->count;i++) if (s->cells[i].weight) ids[n++]=i;
if (!n) { io_free(ids,bytes);io_free(queue,bytes);return "no surfaces in reconstruction bounds"; }
walls(s,ids,n);outline(s,ids,n);
const char *error=hint(s);
if (!error) furniture(s,ids,n,queue);
float perimeter=0,observed=0;
for (int i=0;i<s->noutline;i++) {
float d=root(distance2(s->outline[i],s->outline[(i+1)%s->noutline]));perimeter+=d;
if (!s->invisible[i]) observed+=d;
}
s->coverage=perimeter ? 100*observed/perimeter : 0;
if (s->coverage<LOW_COVERAGE_PERCENT) warn(s,"low wall coverage; scan all walls");
io_free(ids,bytes);io_free(queue,bytes);
return error;
}
int parse_number(char **cursor, float *value)
{
char *p=*cursor;while (*p==' ' || *p=='\t' || *p=='\r') p++;
int sign=1,digits=0;if (*p=='-' || *p=='+') { if (*p=='-') sign=-1;p++; }
float v=0,scale=1;
while (*p>='0' && *p<='9') { v=v*10+*p++-'0';digits++;if(v>10000) return 0; }
if (*p=='.') { p++;while (*p>='0' && *p<='9') { scale*=.1f;v+=(*p++-'0')*scale;digits++; } }
if(*p=='e' || *p=='E') {
p++;int exponent=0,negative=0,edigits=0;
if(*p=='-' || *p=='+') { negative=*p=='-';p++; }
while(*p>='0' && *p<='9') { exponent=exponent*10+*p++-'0';edigits++;if(exponent>38) return 0; }
if(!edigits) return 0;
while(exponent--) v=negative ? v*.1f : v*10;
}
if (!digits || !finite(v) || v>10000) return 0;
*value=sign*v;*cursor=p;return 1;
}
static const char *hint(Scan *s)
{
if (!s->opt.hint) return 0;
int fd=io_open(s->opt.hint,0);
if (fd<0) return "cannot open outline hint";
char data[8192];usize size=0;
for (;;) {
long r=io_read(fd,data+size,sizeof(data)-1-size);
if (r<0 || (r>0 && size+(usize)r==sizeof(data)-1)) { io_close(fd);return "outline hint too large or unreadable"; }
if (!r) break;
size+=(usize)r;
}
io_close(fd);data[size]=0;
Pt pts[MAX_OUTLINE];int n=0;char *p=data;
while (*p) {
while (*p==' ' || *p=='\t' || *p=='\r' || *p=='\n') p++;
if (!*p) break;
if (*p=='#') { while (*p && *p!='\n') p++;continue; }
if (n==MAX_OUTLINE || !parse_number(&p,&pts[n].x) || !parse_number(&p,&pts[n].z)) return "invalid outline hint";
n++;while (*p==' ' || *p=='\t' || *p=='\r') p++;
if (*p && *p!='\n' && *p!='#') return "invalid outline hint";
if (*p=='#') while (*p && *p!='\n') p++;
}
if (n<3) return "outline hint needs at least three points";
Pt centre={0};for (int i=0;i<n;i++) { centre.x+=pts[i].x/n;centre.z+=pts[i].z/n; }
float mean=0,variance=0;
for (int i=0;i<n;i++) mean+=root(distance2(pts[i],centre))/n;
for (int i=0;i<n;i++) { float r=root(distance2(pts[i],centre))-mean;variance+=r*r/n; }
if (n>=DEFAULT_CIRCLE_MIN_POINTS && root(variance)<mean*DEFAULT_CIRCLE_RADIAL_CV) {
warn(s,"default circle hint ignored");return 0;
}
int missing=0;for(int i=0;i<s->noutline;i++) missing+=s->invisible[i];
if (!missing) return 0;
Pt saved[MAX_OUTLINE];int inv[MAX_OUTLINE],old=s->noutline;
memcpy(saved,s->outline,sizeof(saved));memcpy(inv,s->invisible,sizeof(inv));
float area=0;for(int i=0;i<n;i++) area+=pts[i].x*pts[(i+1)%n].z-pts[(i+1)%n].x*pts[i].z;
s->noutline=n;
for(int i=0;i<n;i++) { s->outline[i]=pts[area>0 ? i : n-1-i];s->invisible[i]=1; }
if (!valid_polygon(s)) {
memcpy(s->outline,saved,sizeof(saved));memcpy(s->invisible,inv,sizeof(inv));s->noutline=old;
return "invalid outline hint polygon";
}
/* Only use a hint compatible with every observed wall. Chaperone can be an inset play area. */
for(int i=0;i<s->nplanes;i++) {
Plane *w=&s->planes[i];int match=0;
for(int k=0;k<n;k++) {
Pt a=s->outline[k],b=s->outline[(k+1)%n];
if(ab(a.x*w->n.x+a.z*w->n.z-w->d)<s->opt.gap &&
ab(b.x*w->n.x+b.z*w->n.z-w->d)<s->opt.gap) { match=1;s->invisible[k]=0; }
}
if (!match) {
memcpy(s->outline,saved,sizeof(saved));memcpy(s->invisible,inv,sizeof(inv));s->noutline=old;
warn(s,"hint disagrees with observed walls; ignored");return 0;
}
}
s->hint_used=1;warn(s,"incomplete walls closed using outline hint");return 0;
}
+145
View File
@@ -0,0 +1,145 @@
/* SPDX-License-Identifier: GPL-3.0-only */
#include "platform.h"
#ifdef FP_HOSTED
#define _DEFAULT_SOURCE 1
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/mman.h>
#include <sys/random.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
int io_open(const char *p, int w) { return open(p, w ? O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW : O_RDONLY, 0600); }
long io_read(int fd, void *p, usize n)
{
long r;
do r = read(fd, p, n); while (r < 0 && errno == EINTR);
return r;
}
static long put(int fd, const void *p, usize n)
{
long r;
do r = write(fd, p, n); while (r < 0 && errno == EINTR);
return r;
}
int io_close(int fd) { return close(fd); }
int io_rename(const char *a, const char *b) { return rename(a, b); }
void io_remove(const char *p) { unlink(p); }
int io_mkdir(const char *p) { return mkdir(p, 0700) == 0 || errno == EEXIST; }
void *io_alloc(usize n)
{
void *p = mmap(0, n, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
return p == MAP_FAILED ? 0 : p;
}
void io_free(void *p, usize n) { if (p) munmap(p, n); }
int io_random(void *p, usize n)
{
usize done = 0;
while (done < n) {
long r = getrandom((u8 *)p + done, n - done, 0);
if (r < 0 && errno == EINTR) continue;
if (r <= 0) return 0;
done += (usize)r;
}
return 1;
}
i64 io_time(int mono)
{
struct timespec t;
if (clock_gettime(mono ? CLOCK_MONOTONIC : CLOCK_REALTIME, &t)) return 0;
return (i64)t.tv_sec * 1000000000 + t.tv_nsec;
}
#else
/* asm-generic Linux syscall numbers: this target has no libc or UAPI dependency. */
static long call(long n, long a, long b, long c, long d, long e, long f)
{
register long x8 __asm__("x8") = n;
register long x0 __asm__("x0") = a;
register long x1 __asm__("x1") = b;
register long x2 __asm__("x2") = c;
register long x3 __asm__("x3") = d;
register long x4 __asm__("x4") = e;
register long x5 __asm__("x5") = f;
__asm__ volatile("svc #0" : "+r"(x0) : "r"(x8), "r"(x1), "r"(x2), "r"(x3), "r"(x4), "r"(x5) : "memory", "cc");
return x0;
}
int io_open(const char *p, int w) { return (int)call(56, -100, (long)p, w ? 1|64|128|131072 : 0, 0600, 0, 0); }
long io_read(int fd, void *p, usize n)
{
long r;
do r = call(63, fd, (long)p, (long)n, 0, 0, 0); while (r == -4);
return r;
}
static long put(int fd, const void *p, usize n)
{
long r;
do r = call(64, fd, (long)p, (long)n, 0, 0, 0); while (r == -4);
return r;
}
int io_close(int fd) { return (int)call(57, fd, 0, 0, 0, 0, 0); }
int io_rename(const char *a, const char *b) { return (int)call(38, -100, (long)a, -100, (long)b, 0, 0); }
void io_remove(const char *p) { call(35, -100, (long)p, 0, 0, 0, 0); }
int io_mkdir(const char *p)
{
long r = call(34, -100, (long)p, 0700, 0, 0, 0);
return r == 0 || r == -17;
}
void *io_alloc(usize n)
{
long p = call(222, 0, (long)n, 3, 0x22, -1, 0);
return (unsigned long)p > (unsigned long)-4096 ? 0 : (void *)p;
}
void io_free(void *p, usize n) { if (p) call(215, (long)p, (long)n, 0, 0, 0, 0); }
int io_random(void *p, usize n)
{
usize done = 0;
while (done < n) {
long r = call(278, (long)((u8 *)p + done), (long)(n - done), 0, 0, 0, 0);
if (r == -4) continue;
if (r <= 0) return 0;
done += (usize)r;
}
return 1;
}
i64 io_time(int mono)
{
struct { long sec, ns; } t = {0, 0};
call(113, mono ? 1 : 0, (long)&t, 0, 0, 0, 0);
return (i64)t.sec * 1000000000 + t.ns;
}
void *memcpy(void *d, const void *s, usize n)
{
u8 *a = d;
const u8 *b = s;
while (n--) *a++ = *b++;
return d;
}
void *memset(void *d, int c, usize n)
{
u8 *a = d;
while (n--) *a++ = (u8)c;
return d;
}
extern int main(int argc, char **argv);
__attribute__((used, noreturn)) void start_c(long *sp)
{
int code = main((int)sp[0], (char **)(sp + 1));
for (;;) call(94, code, 0, 0, 0, 0, 0);
}
__asm__(".text\n.global _start\n.type _start,%function\n_start:\n"
"mov x29,#0\nmov x30,#0\nmov x0,sp\nbl start_c\nbrk #0\n");
#endif
int io_write(int fd, const void *p, usize n)
{
while (n) {
long r = put(fd, p, n);
if (r <= 0) return 0;
p = (const u8 *)p + r;
n -= (usize)r;
}
return 1;
}
Loaded 100 of 193 files, more files were not shown because too many files have changed in this diff. Show more