Commit Graph
29 Commits
Author SHA1 Message Date
InstanceBrick 932b9ea7d9 Add optional desktop mirror and native ARM64 release packaging 2026-10-16 15:43:07 -04:00
InstanceBrick 7dc4f55642 Add native ARM64 Steam Frame VR and eye-tracked rendering
Integrate Frame OpenXR input, SteamVR video controls, gaze foveation, native audio, frame pacing and source-based packaging.
2026-10-16 15:31:49 -04:00
dim-ghub 1854e00934 feat(vr): add SMAA as a vr.anti_aliasing choice
vr.anti_aliasing now takes fxaa, smaa or off. SMAA runs the reference
implementation (third_party/smaa, MIT) at its High preset: luma edge
detection and blending weights into per-eye images, then neighborhood
blending into the runtime's swapchain through FXAA's final pass. Its area and
search lookup textures are uploaded once per device. If SMAA cannot be set
up, the eyes fall back to FXAA.

Also export vr.anti_aliasing from config.yaml: the VR backend reads the
process environment, so the setting only worked as an environment variable.

On a Quest 2 through SteamVR at 120 Hz (2136x2236 per eye), SMAA ran at 118.6
fps against 118.8 for FXAA, with p99 frame times of 10.1 and 9.5 ms.
2026-10-16 01:12:21 -05:00
dim-ghub 753a3f8106 feat(vr): anti-alias the headset eyes with FXAA
Roblox's VR debug device renders both eyes into single-sample textures, so
edges were jagged in the headset. When the eyes go to the runtime's
swapchain, an FXAA draw now replaces the plain copy. It samples the eye
directly, and re-encodes for sRGB swapchains so the guest's bytes round-trip
exactly. Anything it cannot set up falls back to the copy.

vr.anti_aliasing (fxaa, the default, or off) controls it. The SPIR-V is
generated from shaders/vr by scripts/generate_vr_shaders.py and checked in,
so builds need no shader compiler; a test checks it against the sources when
glslc is installed.

On a Quest 2 through SteamVR at 90 Hz (2244x2352 per eye) frame times did not
change. Captured output keeps 97 to 98 percent of pixels byte-identical and
the average brightness unchanged, and smooths the edges.
2026-10-15 20:19:17 -05:00
dim-ghub ea248b1ecf feat(vr): add vr.mode with on, off and auto
vr.mode in config.yaml picks how Nightcap starts: off on the desktop, on in
VR, auto in VR only when this user's SteamVR is running and its OpenXR
runtime reports a headset. auto never starts SteamVR, and asks its runtime
from a short-lived copy of the process so the runtime's libraries do not load
before the VR backend arms. vr.enabled still works as on/off, and --vr or
--no-vr still override the file for one launch.
2026-10-15 19:37:32 -05:00
dim-ghub df025d2eb7 fix(vr): publish the shared-memory mirror a frame later instead of waiting
With the eye copies no longer waited for, MOCKTAIL_VR_MIRROR_SHM made the CPU
wait for the whole frame before publishing, which cut the frame rate from 117
to 82 fps at 120 Hz. Publish each readback at the next copy, when the GPU has
finished it. The external viewer sees each frame one frame later.
2026-10-15 14:58:26 -05:00
dim-ghub dea1d5fd59 test(vr): cover handing present dependencies to the GPU
Move the empty wait-and-signal batch into internal::ChainPresentDependencies,
like WaitForPresentDependencies, and test that it waits on every present
semaphore, signals the replacement, never blocks the CPU, and leaves a failed
submit to the guest.
2026-10-15 11:43:51 -05:00
dim-ghub 7296520064 perf(vr): order eye copies and the desktop mirror on the GPU
Every frame the CPU waited for the GPU to finish Roblox's whole frame in the
desktop mirror, then again for the eye copy, so CPU and GPU work never
overlapped. The mirror blit now consumes the guest's present semaphores and
signals a per-image semaphore that the present waits on. Present preparation
hands remaining waits to the GPU the same way. Eye copies are submitted from a
small ring of command buffers and fences and rely on queue order, so the CPU
only waits when the GPU is three frames behind or the guest destroys an image
a copy still reads.

On Monado at 120 Hz with 1280x1440 eyes this removed 2 to 5 ms of waiting per
frame and kept the frame rate at 118 to 119 fps instead of 103 to 118.

Also free the copy resources when the guest device goes away without a
session, so a later device never reuses stale handles.
2026-10-15 04:52:37 -05:00
dim-ghub e094a4c89d feat(vr): time the desktop mirror, present prepare and frame interval
MOCKTAIL_VR_PERF now reports how long the CPU waits in the desktop mirror and
in present preparation, and the time between presents, so per-frame stalls
outside the XR frame cycle show up in the numbers.
2026-10-15 01:37:12 -05:00
dim-ghub 42b5138b11 fix(vr): deliver hand-motor and HapticEffect haptics the way games aim them
The Android client drops HapticService:SetMotor on the Gamepad1 LeftHand and
RightHand motors: its haptics backend only consumes Large and Small. VR games
such as VR Hands drive those motors, so they never vibrated.

- Interpose HapticService::SetMotor through its reflection descriptor, call
  the original, and forward Gamepad1 LeftHand/RightHand to OpenXR. The
  descriptor and the engine's intensity reader are found by pattern at load,
  for both the 2998 (tuple span) and 3222 (shared tuple) reader ABIs. The held
  level renews a 50 ms lease every presented frame.
- Report the hand motors at x = -1/+1 in GetMotors. Positional HapticEffects
  aim at a hand there with a 0.5 radius (VOXEL DESTRUCT 2 does), so at +-0.2
  every effect mixed to zero.
- Play SetMotorState buffers sample by sample with trailing silence trimmed.
  Roblox sends 100 x 25 ms look-ahead buffers; taking the peak for the whole
  buffer turned a short bump into 2.5 s at full strength.
- Stop vibrating from the DebugDeviceVR haptic slot. The engine pulls it every
  frame but with each mix buffer's first sample, held about a second, so it
  duplicated the buffers as a long buzz.
- Enable SupportGenericHapticsOnVR and FixRegisterMotors with the other haptic
  flags.
2026-10-15 01:12:01 -05:00
vii 2eaba9048b fix(vr): reload the OpenXR runtime with its own Vulkan loader instead of pinning it
The Vulkan adapter exports vk* symbols globally, so a runtime loaded again after it bound its Vulkan calls to the adapter and deadlocked the join. Pinning avoided the reload but left SteamVR with its old VkInstance, which aborted the join with "vkGetInstanceProcAddr: Invalid instance". Later instances now load the runtime libraries with RTLD_DEEPBIND and let them unload with their instance.
2026-10-14 22:31:55 -05:00
vii 2be319a979 fix(vr): keep the OpenXR runtime loaded so a game join does not deadlock 2026-10-14 04:19:58 -05:00
vii 1ff05e4e6f fix(vr): do not recover an XR session when the restarted runtime uses another GPU 2026-10-13 18:37:35 -05:00
vii 7ec8304d8f fix(vr): stop keyboard click-through, redraw churn and upload leaks; build its drawing only with VR 2026-10-13 18:37:34 -05:00
vii 7294a46e8a fix(vr): turn on Vulkan queue locking when the adapter loads after the backend 2026-10-13 18:37:19 -05:00
dim-ghub 228121157b vr: drop test hooks and dead code, document every VR variable
- Remove the MOCKTAIL_VR_TEST_SESSION_LOSS_AFTER fault injection; nothing
  used it.
- The scripted head pose and the eye evidence dumper only work in builds
  configured with -DMOCKTAIL_VR_DIAGNOSTICS=ON (default OFF); release
  builds ignore their environment variables.
- Remove FindFmodInputCaptureProfile and HasExpectedFmodInputCaptureVtable
  (only their own tests called them) and the never-defined
  PrepareCaptureLocked declaration; the audio bridge files match upstream
  again.
- docs/VR.md lists every MOCKTAIL_VR_* variable with its default, linked
  from the README.
2026-10-13 00:28:26 -05:00
dim-ghub 8106203c8e vr: fix eye colour formats, mirror segment safety and frame-wait locking
- Once the guest eye format is known, the eye swapchains are recreated
  once in the format that keeps its bytes: the eye format, else its
  UNORM/SRGB sibling (raw copy), else the other channel order with the same
  sRGB-ness (a blit that only swizzles). Mixed order-and-gamma cases no
  longer go through a gamma-converting blit; if only that remains, it is
  logged.
- The desktop mirror segment is created with O_EXCL after removing our own
  stale segment (a crashed run's leftover), and must be private to this
  user before eye images are written to it.
- A new frame_mutex_ is held for the whole XR frame cycle and by every path
  that can destroy the session (taken before mutex_). That lets the cycle
  release mutex_ while it blocks in xrWaitFrame, so guest render-thread
  hooks no longer wait out the runtime's pacing. Swapchain-image and fence
  waits stay under mutex_; they return promptly unless the GPU or
  compositor stalls, and they run while guest eye bindings are copied.
2026-10-13 00:08:22 -05:00
dim-ghub 0603734ff6 vr: recover from XR failures instead of giving up on headset output
- Every frame error, including the swapchain-image and copy-fence
  timeouts, now tears the session down recoverably instead of ending XR
  output for the rest of the run.
- Session recovery backs off (1, 2, 4, 8 s) and recreates the XrInstance
  after an instance-loss event, an instance/runtime failure, or three
  failed attempts, so VR returns after SteamVR or WiVRn restarts instead
  of retrying forever.
- While no XR session runs (headset idle, session not READY, recovering)
  presents are paced at 60 Hz outside the backend lock instead of running
  unthrottled.
- A PrepareHostPresent failure that consumed nothing (copy resources
  unavailable, failed fence reset or submit) presents normally instead of
  failing the guest's vkQueuePresentKHR; only device loss is propagated.
- A runtime/guest GPU mismatch falls back to the plain host device and
  keeps the game running without headset output, naming both GPUs.
- A failed EGL attach on the GLES path no longer makes window init fail;
  the game runs without headset output and the bind is not retried.
2026-10-13 00:04:42 -05:00
dim-ghub bafbe4268c vr: name the build and the supported builds when VR is unavailable
An explicit --vr on a Roblox build without VR support now exits with the
build's version and Build ID, the builds VR supports, and how to select one
(--roblox-lib). When VR is only enabled in the config, the launch continues
without VR and logs the same details. The bridge's own Install error names
both as well. The verified build contracts carry their version labels and
move to namespace scope so they can be listed.
2026-10-12 23:55:37 -05:00
dim-ghub 4cc0e41ef3 vr: add in-headset virtual keyboard for TextBox input
When Roblox asks for a keyboard (showKeyboard), a world-locked panel
opens below eye level as an OpenXR quad layer. Controllers point at keys
and the trigger types; text and key presses go through the same platform
events as the desktop keyboard, so the focused TextBox edits identically.

The layout is US QWERTY without the function row or Ctrl/Alt, with one-shot
Shift, Caps Lock, Tab, Backspace auto-repeat, Enter and Close. A trigger
aimed at the panel is withheld from Roblox so a key press is not also a
click. A text field above the keys mirrors the focused TextBox, whose
desktop overlay never reaches the eye images. MOCKTAIL_VR_KEYBOARD=0 turns
it off.
2026-10-12 23:45:51 -05:00
dim-ghub ddc985218b vr: add separate VR frame-rate limit, render scale and unpaced benchmark mode
vr.frame_rate_limit / MOCKTAIL_VR_FRAME_RATE_LIMIT replaces
graphics.frame_rate_limit while VR is active and takes the same values.
It defaults to unlimited so xrWaitFrame paces Roblox at the headset
refresh rate instead of Roblox's own 60 FPS cap.

MOCKTAIL_VR_RENDER_SCALE scales the runtime's recommended eye extent.
SteamVR's recommendation for a Quest 2 at 90 Hz (2244x2352 per eye) missed
the 11.1 ms budget and halved the rate to 45 FPS.

MOCKTAIL_VR_UNPACED=1 stops guest frames from blocking in xrWaitFrame:
the open XR frame spans several guest frames and is submitted once per
display period, for measuring maximum render rate.
2026-10-12 22:44:46 -05:00
dim-ghub 509822974b vr: Recreate OpenXR instance on guest Vulkan instance recreation
SteamVR (vrclient.so) binds its Vulkan renderer to the lifetime of the
XrInstance. When Roblox destroys the LuaApp Vulkan device and creates a
second VkInstance/VkDevice for UGC experiences, reusing the original
XrInstance causes SteamVR to dereference freed device handles in xrEndFrame.

Tear down and recreate the XrInstance whenever a new guest Vulkan instance
is initialized, resetting session state, spaces, and controller actions.
2026-10-12 05:42:33 -05:00
komaruworld 9a1ccbb5ce add roblox 3092 support
Also fix openxr shit
2026-10-03 13:40:24 +03:00
UwieandUwie 5e24bee4ab resolution thing or smth idk (#141)
Co-authored-by: Uwie <Uwie>
2026-09-14 17:14:42 +03:00
komaruworld a014751cd4 bla bla bla 2026-09-14 16:38:31 +03:00
komaruworld 7e057232e4 fixes 2026-09-13 20:36:53 +03:00
komaruworld 3d89f128ce support alvr and opengl 2026-09-13 19:28:28 +03:00
komaruworld b187efbedf Full support camera
NOT TESTED
2026-09-13 18:20:58 +03:00
komaruworld dd23f06a8b init vr 2026-09-12 21:45:48 +03:00