The panel recording of 20261014-010727 showed one eye at a time displaying an image it had shown 3-5
refreshes before (race: left 10.4 %, right 7.0 %, both together 0.02 %, eyes out of step 19.4 %; menus 0),
at the old pixels while the image moved: SteamVR resolves each eye's image when it composes, after our
alternating pair was already acquired again. The mock's compositor model (lazyacq) reproduces it with two
pairs (left 24.6 %, right 15.1 %, eyes from different sets 18.8 %) and shows none with six.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mock race without other load: 60.0 game fps and 75 slow frames in 74 s with the clamp, 58.2 fps and
376 without (nice -15 alone), 58.8 fps and 286 with neither: the game thread's 11 to 14 ms of logic
a frame wants the Frame's one 1024-capacity core. sched_setattr needs no privilege for this.
KARTFRAME_UCLAMP=0 leaves the clamps alone; the "thread priority:" lines show what was set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
H3: a PrimeFrame build during the session starved the game thread (2095 slow game frames, runq
waits). The pacing thread raises itself, the game thread, the GX thread, Aurora's frame worker and
the GPU watcher to the lowest nice RealtimeKit grants (at most -15), else what RLIMIT_NICE allows
(KARTFRAME_THREADPRIO=high|nice|off; "thread priority:" log lines). Aurora publishes the frame
worker's thread id and names it on Linux.
frame/perf/run.sh takes PrimeFrame's build lock as well as KartFrame's; frame/perf/bgload.sh is a
background compile load for runs; hudrace.sh keeps its runs under kf-laneH.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
H3: lane G's end-at-locate rule gave every set shown a refresh late (24 % of the race) a display
time one refresh older than its cycle's; SteamVR's mispresented frames rose from 116 to 1284 and
reused app frames from 0.2 to 4.4 %. Every xrEndFrame now carries its cycle's predicted time
(KARTFRAME_END_TIME=locate restores lane G's rule). The "vr display times" line counts sets shown a
refresh or more late apart from SteamVR's +-0.02 ms prediction jitter (which made nearly every set
"located for another time").
xrmock: the Frame's own per-eye FOVs by default, predictedDisplayTime 3 refreshes after the wake
(MOCK_XR_LEAD; SteamVR's lead was 25 to 27 ms), 0.02 ms prediction jitter by default, an ERROR for
every xrEndFrame whose display time is not its frame's predicted time ("xrmock: end times"), and
MOCK_XR_DUMP_STEP for full-resolution dumps. The GPU watcher publishes its thread id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second headset session: 'the pointer and controller models didn't render when it was a 2D screen'.
Since lane E the menus are a room-fixed quad and nothing was drawn into the eyes. Now a menu set also
renders the room around the screen into a packed two-eye image (3456x1728, left eye left half): plain
background, the Frame controller models, the laser pointers and their target marks, and the screen's
area left transparent black. The backend shows it as one projection layer with the set's own poses,
under the screen's quad (the quad stays the sharp, reprojected picture), plus the panel quad; never
two projection layers. KARTFRAME_MENU_ENV=over puts it over the quad, alpha-blended (a controller in
front of the screen then stays in front where the compositor keeps the layer order), =0 turns it off.
Same release-after-GPU rule (separate swapchain pair, swapped like the eyes').
Plumbing without new public API: target 1 of a virtual-screen set is the environment image
(OpenXRPresentation::menu_environment), the packet's 'window' flag (meaningless outside immersive
replay) asks Aurora for it, vrui::render draws one eye into its half (viewport) with the hole.
Mock (MOCK_XR_HANDS=1, Frame profile, warm caches, menus 30-90 s): 120.5 sets/s, 0 late, GPU 2.60 /
2.99 ms a set (off: 1.25 / 5.89), encode 0.52 / 1.04 ms (off 0.15 / 0.60); 0 mock errors, every
release checked finished; the dump shows both eyes' controllers and lasers ending on the screen's
hole. Menus to race to menus in one run: 0 errors, 12866 race eye images with matching stamps.
The frame stamp is written for immersive sets only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second headset session: images doubled on head motion although every rate was clean.
- KARTFRAME_FRAME_STAMP=1 (tests): the Vulkan bridge writes the display time an eye set was located
for and the eye into a corner block of each eye image; the mock (MOCK_XR_STAMP) reads it back at
every release and compares it with the locate of the poses each projection layer submits.
Pipelined and inline, with and without prediction jitter: 10064-14318 eye images per run, all
rendered for the locate of their submitted poses. The image/pose pairing is not the cause.
- A layer is ended with the display time its poses were located for (not this cycle's prediction,
which on SteamVR need not be one period after the last), keeping display times increasing; repeats
keep the cycle's time. KARTFRAME_END_TIME=cycle restores the old behaviour. Mock with
MOCK_XR_PREDICT_JITTER=2: before 0 of 7155 fresh layers carried their poses' time, after 7157 of
7159. New log line every 10 s: 'vr display times' (prediction step error, display lead, sets shown
for another time than their cycle's); the eye fields of view are logged once.
- VR interpolation never interpolated with pipelined pacing: a set's display time lies 2-3 refreshes
after the newest game frame's boundary, so the weight was clamped to 1 for 98 % of race sets (mock:
between 2-54 of ~1180 per 10 s) and each game frame showed on two refreshes: 60 Hz scenery on a
120 Hz display, moving things doubled. stereo::SceneClock samples the scene at the display time
minus a learnt delay (running mean of display - boundary, minus half an interval): mock race now
80-87 % of sets between two frames. KARTFRAME_SCENE_CLOCK=0 restores the old rule. New aurora line
'VR interpolation weights (10 s)'.
- xrmock: MOCK_XR_STAMP, MOCK_XR_PREDICT_JITTER, increasing display time check, asymmetric
Frame-like FOV by default (MOCK_XR_FOV), MOCK_XR_HANDS, scripted presses (MOCK_XR_PRESS).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aurora's frame worker sat out the producer's prepare permit (about 10 ms of
every game frame) and an eye set published meanwhile waited with it, so
inline sets were late 86 % of the time. With the stereo provider and
interpolation active the sealed frame is now encoded before the permit wait
and the wait serves retained replays. The VR menu sorted ~9000 controller
triangles per eye by moving them and switched material almost every
triangle; it now sorts keys in 5 mm slices by material (encode 6.5 -> 3 ms,
GPU 5 -> 2.6 ms per set in menus). Pacing: 5 s warm-up before the half-rate
rule counts, KARTFRAME_HALF_RATE=0|1, KARTFRAME_PACING_DEBUG timelines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[vr] frame_pacing = auto|inline|pipelined (KARTFRAME_PACING wins): one
xrLocateViews per set for the time it is displayed, screens, panel and
pointer from those poses, late sets never released early (the retained
layer is repeated), half rate only on real misses (>12 of 240, back at <=6
of 600 after 10 s). VR interpolation available on desktop Linux with the
CLOCK_MONOTONIC fallback; projected menus on the Frame; 120 Hz requested
through XR_FB_display_refresh_rate; focus blips under 1 s ignored; the
Frame controller extension enabled. One 'vr pacing:' line a second.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After Aurora's queue.Submit and the bridge's release barrier, an empty
vkQueueSubmit under Dawn's device lock signals a timeline semaphore; a
watcher thread records each set's completion (submit-to-complete time) and
WaitForSubmissionUntil succeeds only once the GPU finished. FinishFrame
refuses to release an unfinished set. KARTFRAME_NO_GPU_WAIT=1 is the
negative test. Adds per-refresh eye-set primitives (BeginCycle, PrepareSet,
EndCycleWithSet, EndCycleRetained, CancelSet, DropSet) for inline and
pipelined pacing, logs the Vulkan layer chain, and reports sets Aurora
rendered from a retained frame. Replay test covers the sets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build aurora's Vulkan interop on Linux with the patched Dawn linked statically, extend the Vulkan OpenXR backend guards to desktop Linux, and route openxr_integration to it with XR_KHR_vulkan_enable2 and timespec display-time conversion. Re-applied from the earlier native Linux commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The pacing thread aims each immersive window eye through the window itself (AimEyesThroughWindow):
it keeps its position but looks square-on at the window's plane through an off-axis frustum just
around it, so the eye image is the window, at the display's pixel density (about 680x380 per eye
at render_scale 0.8 instead of 1344x1408), with a two-pixel border the mask leaves transparent.
The frame's views carry that pose and field of view to the projection layer.
- vulkan_interop.cpp copies an eye smaller than its AHardwareBuffer into the buffer's corner, and
the Quest layer's imageRect is the rendered part of the swapchain image.
- These eyes are not foveated: their field of view follows the head, which would rebuild the
density map every frame.
- Quest only (kWindowShapedEyesSupported); the PC backends copy whole eyes and keep masking them.
debug.wiicompiled.window_eyes 0 renders them whole and masked again for A/B timing.
- Quest 3, paused Retro Rewind race, render_scale 1.0: Immersive window runs the GPU at level 1
(456 MHz, app GPU 11.0 ms, eyes 8.4 ms) where Immersive needs level 3 (599-640 MHz, 13.4 ms,
10.3 ms), about 42% fewer GPU cycles. No edge artifacts, image as sharp. Docs: OPENXR.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Introduced a new configuration option for immersive window mode in runtime_config.h.
- Updated the parsing and setting functions to handle the immersive window state.
- Modified the OpenXR backend to support rendering with the immersive window, blending the race view with the surrounding environment.
- Enhanced the settings overlay to allow users to select between immersive, immersive window, and flat screen race views.
- Implemented GPU rendering logic for the immersive window mask, ensuring correct visual output in various rendering paths.
- Added tests to validate the immersive window functionality and its interaction with existing race view settings.
- Eyes render under a VK_EXT_fragment_density_map: full rate around each eye's forward direction,
2x2 then 4x4 pixel blocks towards the edges ([vr] foveation = off|low|medium|high, default off).
XR_FB_foveation cannot help here: the runtime's maps only shape passes drawing into its
swapchain, and the eyes reach it through a copy.
- aurora-main/patches/dawn/aurora_fdm.inc: Dawn enables the extension only on request and for
dynamic rendering, flags every render pipeline, and chains an immutable RG8 map into any pass
whose first color attachment is a view bound to one (ABI: include/aurora/dawn_fdm_abi.h).
- android/Build-QuestDawn.ps1 builds the pinned Dawn revision with those patches for arm64
(dawn-build CI flags, protobuf off) into a cached package; Build-Quest.ps1 links it
(-StockDawn opts out) and AuroraDawnProvider.cmake enables the ABI from its manifest.
- lib/gfx/foveation.hpp generates the maps (32 px per texel, densities 255/127/63); an eye is
foveated only when single_pass_eyes draws it in one render pass. Menus never are.
- Live level from the headset panel's VR tab and the launcher; the launch decides whether the
device has maps. debug.wiicompiled.foveation and debug.wiicompiled.fdm for A/B.
- Tests: Foveation cases in gx_fifo_tests, mkw_vr_config_tests. Docs: OPENXR.md, quest-port.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Added eye_pass_plan.hpp: an eye keeps drawing in the render pass it has open across the frame's
GX copies (only the mono render performs them), skips passes a later clear of the whole color and
depth erases, and splits only for a clear of color alone or depth alone.
- render_stereo_eye follows that plan; the cockpit fallback is drawn inside the last pass instead of
a render pass of its own that loaded the eye back.
- The stencil is now cleared where the eye actually starts, so a cockpit mask drawn in an erased pass
can no longer punch holes in the HUD.
- [vr] single_pass_eyes (default on, live from F10) and debug.wiicompiled.eye_passes 0/1 on the Quest
switch back to one render pass per recorded pass for A/B timing.
- Each new plan shape is logged once ("Eye replay plan: ...").
- Tests: EyePassPlan cases in gx_fifo_tests; stereo_multiplayer_smoke and stereo_frame_worker_smoke
pass on D3D12.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First person already seats you in the cockpit, so hand steering belongs with it:
the stick keeps steering until a grip actually takes hold of the wheel, and
nothing else changes until it does. "yaw_pitch" becomes the default anchor for
the same reason - from the driver's seat the vehicle's own climb reads as the
ground rising, where a level horizon reads as the kart sinking away.
Both defaults move together in runtime_config.h, the fresh-config template, the
F10 menu's reset, and both launchers, which mirror the runtime's constants. The
menu's reset also puts the rotation combo back on the default rather than on
"yaw", and now covers hand steering with the other cockpit keys.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The panel was drawn into the eye images, so below a 1.00x OpenXR
resolution its 1440x1080 canvas was minified into a small eye region and
the text became hard to read. It is now submitted as a quad layer of its
own over the scene's projection or menu quad, which the compositor samples
directly at any render scale.
Each backend (D3D12, Windows Vulkan, Quest) makes the panel's swapchain
pair (plus two shared buffers on the Quest) the first time the panel
opens. While it is open, the frame hands Aurora one more stereo target
after the eyes, which the bridge fills with the panel texture or a
transparent image. The panel image follows the eyes' displayed/retained
pairing, so a cancelled frame never shows an unwritten panel. The quad
hangs where the pointer's hits are tested. Aurora leaves the panel out of
the eyes in layer mode, and a backend that cannot make the layer falls
back to drawing it into the eyes.
On the Quest, debug.wiicompiled.panel_layer 0 selects the old path at
run time. Measured there at a race start (render_scale 0.8), the layer
costs nothing while the panel is closed; while it is open, app GPU time is
10.5 ms against 9.7 ms drawn into the eyes, with unchanged frame rates.
The replay tests cover lazy creation, cancelled frames, closed and
unplaced panels, and render-first pacing on D3D12 and Windows Vulkan.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Clicking the right thumbstick flips first person exactly as the F10
checkbox does, and saves it the same way. It works on the VR controllers
in either presentation (with a short tick on the right controller) and on
any other gamepad while VR is running. It never reached the game: SDL's
stick click only lands in Aurora's extended buttons.
A click counts on release, and only if the left thumbstick stayed up and
the headset settings panel stayed closed throughout, so the two-thumbstick
panel chord in gamepad mode never toggles the camera. A gamepad whose
right stick click is bound to a GameCube control on its port, as a button
or in an input expression, keeps it for the game. The XR side only posts
a request; the toggle itself runs on the game thread with the checkbox.
[vr] first_person_toggle_click (default true) and a checkbox under the
camera toggle turn the click off. mkw_vr_camera_toggle_tests covers the
click rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ported from heurazy's mario-kart-wii-VR-port. F10 > Controllers > USB
wheel and pedals (also in the headset panel) picks and calibrates the
steering axis and both pedals by moving them, and assigns buttons by
pressing them, over raw SDL joysticks: no per-model table or gamepad
mapping, and separate pedals, reversed axes and combined pedal axes all
calibrate the same way. Settings live in PhysicalWheel.toml beside
Config.toml; heurazy's five-button files load unchanged.
The wheel is player 1's GameCube controller. In a race it owns port 0
(steering on the stick, the accelerator on A, the brake pedal braking then
reversing over A and drift, the paddles on R and L), keeping only the
other source's pause and item aim. In menus it adds only deliberate
presses. It arms once every pedal and button is released, stays neutral
in a race while its setup is incomplete, and nothing is opened until it
is enabled. Optional light rumble follows the game's own, capped at 15 %.
Added here: its confirm, a new back button and the steering device's
first hat (D-pad) work the menus, since our VR controllers default to a
Wii Remote and the wheel needs to navigate on its own. Devices SDL
classifies as wheels are marked in the list.
In the VR cockpit the wheel follows the hardware steering and hand
steering steps aside while it drives. README documents setup and
Logitech notes (G HUB, the G29's PS3 switch, shifter gears as buttons);
not yet tried on a physical wheel.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every XR frame in the cockpit seat the pacing thread locates both grips in
the seated frame (the immersive base turned by the lean-back angle, the
frame the eye transforms place the vehicle in) and publishes a driving
snapshot. The wheel or handlebar shows the left stick's steering at the
configured full-lock angle, eased; the vehicle's own wheel reads that angle
on the guest thread.
With hand_steering on (off by default), squeezing a grip near the wheel or
handlebar takes hold of it (a short pulse on grab and release); one or two
hands turn it through heurazy's SteeringWheel, and while it is held the
wheel replaces the left stick's X axis in both the Wii Remote and gamepad
presentations, the stick's Y still aims items, and a holding grip no
longer presses C or a shoulder. The settings panel withholds it like any
other input.
The stereo packet carries the cockpit overlay: the hands, in the runtime's
hand mesh (XR_EXT_hand_tracking + XR_FB_hand_tracking_mesh, requested only
when hand steering is on at launch) or procedural gloves, and the separate
VR wheel or handlebar whenever the vehicle's own is not the one turning.
first_person_seat = "cockpit" (the new default) places the head at the
driver's own eyes, measured once from the character's head bone while the
kart drives straight and undamaged, kept behind the steering wheel, at a
life-size scale (cockpit_units_per_meter, grown with the character's
height and the player's size). "yaw" takes the kart's driving direction
from a level seat frame that follows the simulation's position and
direction, so damage spins and tricks do not turn the seat.
first_person_seat = "custom" keeps the previous offset placement.
At the race draw boundary the vehicle's steering wheel disc (karts) or
handle part (bikes, re-seated level while the bike banks) is decoded from
its MDL0, turned on a copy, and posted to the GX thread for Aurora to
substitute into the player's own draws; the copies are dropped after the
frame's draws. The anchor carries the wheel or handlebar geometry in the
seated frame and the frame's exact scale, which reaches the policy and
aurora_set_stereo_scene_anchor_scaled. If no draw takes the copies for 30
frames the vehicle falls back to the VR wheel, and the log says so.
The wheel copy is matched against RaceCamera::GetViewMtx with no dolly
offset, since the scene camera is only set once the draws run; the log
reports its distance from the scene camera once a second.
Nothing turns the wheel yet: the angle comes from the XR side's driving
snapshot, which the next change publishes.
Guest offsets ported from heurazy's mario-kart-wii-VR-port, each
re-checked against the generated leaf getters and the decompilation.
The pure pieces of the first-person cockpit and hand steering, ported from
heurazy's mario-kart-wii-VR-port: the SteeringWheel grab/turn model, the
native wheel vertex rotation, the level seat stabiliser, the seated-eye and
wheel/handlebar geometry, and the XR_FB_hand_tracking_mesh loader. Adds
openxr_driving.h, the OpenXR-free snapshot the pacing thread will publish for
the guest thread, with the hand-off rule (a held wheel replaces the left
stick's X and releases that hand's grip for the game) and the wheel's
displayed angle.
New [vr] keys: first_person_seat (cockpit), cockpit_units_per_meter (100),
steering_wheel (true), native_steering_wheel (true), hand_steering (false)
and the seven wheel_* tuning keys. Nothing reads them yet.
The fresh-config template now writes the first-person defaults the
constants hold (50 / 1.5 / 0); d86dcb0 updated the constants but not the
template.
Tests: mkw_steering_wheel_tests (the fork's), mkw_vr_cockpit_tests,
mkw_vr_hand_steering_tests.
- Introduced a new GX thread to handle the rendering pipeline, allowing the game thread to post commands without blocking.
- Added `gx_thread.h` and `gx_thread.cpp` to manage the command ring buffer and thread synchronization.
- Updated `vi.cpp` to utilize the GX thread for rendering tasks, improving frame pacing and responsiveness.
- Modified `main.cpp` to configure and start the GX thread, ensuring it integrates with the existing rendering workflow.
- Enhanced `openxr_integration.cpp` to register the GX thread with OpenXR for better performance in VR scenarios.
- Refactored `settings_overlay.cpp` to remove unnecessary waits for the frame worker, as the overlay now draws directly into the game thread's ImGui frame.
- Improved error handling and logging in the GX thread to capture exceptions during command execution.
