Compare commits

..
66 Commits
Author SHA1 Message Date
vii adccf4e484 Merge branch 'main' into vr 2026-10-16 01:22:27 -05:00
dim-ghub 8370850bb7 docs(changelog): merge the two Changed sections under Unreleased 2026-10-16 01:14:12 -05:00
dim-ghub 1854e00934 feat(vr): add SMAA as a vr.anti_aliasing choice
vr.anti_aliasing now takes fxaa, smaa or off. SMAA runs the reference
implementation (third_party/smaa, MIT) at its High preset: luma edge
detection and blending weights into per-eye images, then neighborhood
blending into the runtime's swapchain through FXAA's final pass. Its area and
search lookup textures are uploaded once per device. If SMAA cannot be set
up, the eyes fall back to FXAA.

Also export vr.anti_aliasing from config.yaml: the VR backend reads the
process environment, so the setting only worked as an environment variable.

On a Quest 2 through SteamVR at 120 Hz (2136x2236 per eye), SMAA ran at 118.6
fps against 118.8 for FXAA, with p99 frame times of 10.1 and 9.5 ms.
2026-10-16 01:12:21 -05:00
dim-ghub 0221738952 Merge remote-tracking branch 'origin/main' into vr 2026-10-16 00:52:17 -05:00
dim-ghub 753a3f8106 feat(vr): anti-alias the headset eyes with FXAA
Roblox's VR debug device renders both eyes into single-sample textures, so
edges were jagged in the headset. When the eyes go to the runtime's
swapchain, an FXAA draw now replaces the plain copy. It samples the eye
directly, and re-encodes for sRGB swapchains so the guest's bytes round-trip
exactly. Anything it cannot set up falls back to the copy.

vr.anti_aliasing (fxaa, the default, or off) controls it. The SPIR-V is
generated from shaders/vr by scripts/generate_vr_shaders.py and checked in,
so builds need no shader compiler; a test checks it against the sources when
glslc is installed.

On a Quest 2 through SteamVR at 90 Hz (2244x2352 per eye) frame times did not
change. Captured output keeps 97 to 98 percent of pixels byte-identical and
the average brightness unchanged, and smooths the edges.
2026-10-15 20:19:17 -05:00
dim-ghub 569df68729 fix(vr): stop trapping the desktop cursor in the window during VR
Roblox's mouse lock and the RMB camera drag put the window into relative
mouse mode with a grab, which locked the cursor inside it while the headset
was in use. In VR the controllers aim and the window only mirrors the left
eye, so the pointer is never captured and the system cursor stays visible.
2026-10-15 19:55:35 -05:00
dim-ghub 043964a5dc fix(vr): say why auto found no headset, and document vr.mode
Without a headset SteamVR refuses the OpenXR client at xrCreateInstance
(VRInitError_Driver_WirelessHmdNotConnected) instead of reporting no system,
so say that it usually means no headset is connected. Describe vr.mode in the
README, docs/VR.md and the changelog.
2026-10-15 19:42:40 -05:00
dim-ghub ea248b1ecf feat(vr): add vr.mode with on, off and auto
vr.mode in config.yaml picks how Nightcap starts: off on the desktop, on in
VR, auto in VR only when this user's SteamVR is running and its OpenXR
runtime reports a headset. auto never starts SteamVR, and asks its runtime
from a short-lived copy of the process so the runtime's libraries do not load
before the VR backend arms. vr.enabled still works as on/off, and --vr or
--no-vr still override the file for one launch.
2026-10-15 19:37:32 -05:00
dim-ghub 35033454b0 docs(vr): note the frame-stall fix and the new perf stages 2026-10-15 17:21:09 -05:00
dim-ghub df025d2eb7 fix(vr): publish the shared-memory mirror a frame later instead of waiting
With the eye copies no longer waited for, MOCKTAIL_VR_MIRROR_SHM made the CPU
wait for the whole frame before publishing, which cut the frame rate from 117
to 82 fps at 120 Hz. Publish each readback at the next copy, when the GPU has
finished it. The external viewer sees each frame one frame later.
2026-10-15 14:58:26 -05:00
dim-ghub dea1d5fd59 test(vr): cover handing present dependencies to the GPU
Move the empty wait-and-signal batch into internal::ChainPresentDependencies,
like WaitForPresentDependencies, and test that it waits on every present
semaphore, signals the replacement, never blocks the CPU, and leaves a failed
submit to the guest.
2026-10-15 11:43:51 -05:00
dim-ghub 9ae5667cd7 perf(vr): read per-frame settings once and skip the lock for other passes
The frame cycle read the pose source, evidence and mirror settings from the
environment every frame. Read them once. Check the thread-local eye request
before taking the backend lock, so render passes that are not eye passes do
not contend with the frame cycle.
2026-10-15 08:19:04 -05:00
dim-ghub 7296520064 perf(vr): order eye copies and the desktop mirror on the GPU
Every frame the CPU waited for the GPU to finish Roblox's whole frame in the
desktop mirror, then again for the eye copy, so CPU and GPU work never
overlapped. The mirror blit now consumes the guest's present semaphores and
signals a per-image semaphore that the present waits on. Present preparation
hands remaining waits to the GPU the same way. Eye copies are submitted from a
small ring of command buffers and fences and rely on queue order, so the CPU
only waits when the GPU is three frames behind or the guest destroys an image
a copy still reads.

On Monado at 120 Hz with 1280x1440 eyes this removed 2 to 5 ms of waiting per
frame and kept the frame rate at 118 to 119 fps instead of 103 to 118.

Also free the copy resources when the guest device goes away without a
session, so a later device never reuses stale handles.
2026-10-15 04:52:37 -05:00
dim-ghub e094a4c89d feat(vr): time the desktop mirror, present prepare and frame interval
MOCKTAIL_VR_PERF now reports how long the CPU waits in the desktop mirror and
in present preparation, and the time between presents, so per-frame stalls
outside the XR frame cycle show up in the numbers.
2026-10-15 01:37:12 -05:00
vii de0ddf7864 feat(update): solve the VR device bridge and compare it with the VR offset finder 2026-10-15 02:20:13 -05:00
vii e553db2001 fix(vr): point 3092 FixRegisterMotors at the flag Roblox reads
The 3092 table had the address of another field in the same registration, so the flag never turned on.
2026-10-15 02:20:13 -05:00
vii 44ec71f826 Merge branch 'main' into vr
# Conflicts:
#	FAQ.md
2026-10-15 01:57:30 -05:00
dim-ghub 42b5138b11 fix(vr): deliver hand-motor and HapticEffect haptics the way games aim them
The Android client drops HapticService:SetMotor on the Gamepad1 LeftHand and
RightHand motors: its haptics backend only consumes Large and Small. VR games
such as VR Hands drive those motors, so they never vibrated.

- Interpose HapticService::SetMotor through its reflection descriptor, call
  the original, and forward Gamepad1 LeftHand/RightHand to OpenXR. The
  descriptor and the engine's intensity reader are found by pattern at load,
  for both the 2998 (tuple span) and 3222 (shared tuple) reader ABIs. The held
  level renews a 50 ms lease every presented frame.
- Report the hand motors at x = -1/+1 in GetMotors. Positional HapticEffects
  aim at a hand there with a 0.5 radius (VOXEL DESTRUCT 2 does), so at +-0.2
  every effect mixed to zero.
- Play SetMotorState buffers sample by sample with trailing silence trimmed.
  Roblox sends 100 x 25 ms look-ahead buffers; taking the peak for the whole
  buffer turned a short bump into 2.5 s at full strength.
- Stop vibrating from the DebugDeviceVR haptic slot. The engine pulls it every
  frame but with each mix buffer's first sample, held about a second, so it
  duplicated the buffers as a long buzz.
- Enable SupportGenericHapticsOnVR and FixRegisterMotors with the other haptic
  flags.
2026-10-15 01:12:01 -05:00
vii 2eaba9048b fix(vr): reload the OpenXR runtime with its own Vulkan loader instead of pinning it
The Vulkan adapter exports vk* symbols globally, so a runtime loaded again after it bound its Vulkan calls to the adapter and deadlocked the join. Pinning avoided the reload but left SteamVR with its old VkInstance, which aborted the join with "vkGetInstanceProcAddr: Invalid instance". Later instances now load the runtime libraries with RTLD_DEEPBIND and let them unload with their instance.
2026-10-14 22:31:55 -05:00
vii a2ffa5c811 Merge branch 'vr' of CoderDayton/nightcap into vr 2026-10-14 04:29:10 -05:00
vii 2be319a979 fix(vr): keep the OpenXR runtime loaded so a game join does not deadlock 2026-10-14 04:19:58 -05:00
vii 6cf641d472 feat(vr): find the haptic flags with the offset finder and enable them on 3092 2026-10-14 03:14:16 -05:00
vii 3788026d10 fix(vr): limit haptics to VR sessions and check flag names before writing
- Register HapticProtocol and report motors only when MOCKTAIL_VR_ENABLED is on.
- Drop the per-message [haptic] dispatch log; log each OpenXR haptic result once per hand.
- Write a fast flag only when its name pointer matches the expected name.
- Remove the state-sync fallback that dlsym can never resolve.
2026-10-14 03:14:16 -05:00
dim-ghub f3e68fc3b7 feat(vr): implement OpenXR controller haptics and bypass mobile suppression
- Implement RobloxHapticProtocolBridge for MessageBus HapticProtocol
  (SupportsHaptics, GetMotors, SetMotorState, UpdateSingletonVibration).
- Wire bridge into RobloxExperienceComposition and direct game runtime.
- Call JNI NativeGLInterface.nativeUpdateMobileHapticsState during startup
  and game completion.
- Enable FFlagKeepHapticsOnMobileWhenGamepadConnected and FFlagVRHaptics in
  VR client settings overrides to bypass gamepad haptic suppression in libroblox.
- Directly activate KeepHaptics and VRHaptics fast flag bytes in memory
  within RobloxVrDeviceBridge::Activate for builds 2998 and 3222.
- Check xrApplyHapticFeedback result code and add diagnostic logging.
2026-10-14 01:32:32 -05:00
vii 0b0d8067d8 docs(vr): list supported builds, move VR detail into docs/VR.md and update the FAQ 2026-10-13 20:20:51 -05:00
vii 87ac6ee572 test(vr): cover the keyboard upload and quad layer on a real OpenXR runtime 2026-10-13 19:58:40 -05:00
vii fcc8b5382a fix(vr): turn off the desktop flyout side panel in VR 2026-10-13 19:58:40 -05:00
vii 9826535102 fix(vr): align the IsVRAppBuild hook so unoptimized builds do not crash Roblox 2026-10-13 19:58:40 -05:00
vii a30f05fc5b fix(vr): keep the bridge and backend alive until exit for guest threads still in a hook 2026-10-13 18:37:35 -05:00
vii 1ff05e4e6f fix(vr): do not recover an XR session when the restarted runtime uses another GPU 2026-10-13 18:37:35 -05:00
vii 123968be89 test(assets): pin overlay priority and path safety when the root is the content folder 2026-10-13 18:37:35 -05:00
vii 393297a439 feat(vr): support Roblox 3222 (2.742.1059) 2026-10-13 18:37:35 -05:00
vii 7ec8304d8f fix(vr): stop keyboard click-through, redraw churn and upload leaks; build its drawing only with VR 2026-10-13 18:37:34 -05:00
vii f9289d9947 fix(vr): read the pointer hand and HeadScale offsets per build and check them at start-up 2026-10-13 18:37:26 -05:00
vii 7294a46e8a fix(vr): turn on Vulkan queue locking when the adapter loads after the backend 2026-10-13 18:37:19 -05:00
vii 4b67176970 fix(scripts): find VR offsets when Roblox moves the pointer fields
Build 3222 moved the hand selector, hand frame and camera HeadScale fields, so the fixed byte patterns found nothing. Match the code shape instead, read the three offsets from it and write them to the output.
2026-10-13 17:15:39 -05:00
dim-ghub 228121157b vr: drop test hooks and dead code, document every VR variable
- Remove the MOCKTAIL_VR_TEST_SESSION_LOSS_AFTER fault injection; nothing
  used it.
- The scripted head pose and the eye evidence dumper only work in builds
  configured with -DMOCKTAIL_VR_DIAGNOSTICS=ON (default OFF); release
  builds ignore their environment variables.
- Remove FindFmodInputCaptureProfile and HasExpectedFmodInputCaptureVtable
  (only their own tests called them) and the never-defined
  PrepareCaptureLocked declaration; the audio bridge files match upstream
  again.
- docs/VR.md lists every MOCKTAIL_VR_* variable with its default, linked
  from the README.
2026-10-13 00:28:26 -05:00
dim-ghub a06c7de93e vr: fix bridge hand math and make its shutdown safe for live guest threads
- MOCKTAIL_VR_HAND_POSE_SOURCE=grip now rotates hand records from the grip
  pose. base_ori started as the identity quaternion, so the norm check
  never fell back to it and hand rotation was dropped.
- Hand records cancel the engine's stored hand pitch (read from
  hand_pitch_storage_rva by the pointer hook, 20 until read) instead of a
  hard-coded 20, so they agree with the pointer ray.
- Shutdown never rewrites the aim-pointer prologue while guest threads may
  be executing it; the jump stays and only forwards once the bridge is gone,
  and a later activation reuses it.
- Guest hooks count themselves in flight before loading the bridge pointer;
  Shutdown unpublishes the bridge and waits for in-flight calls to drain,
  so no hook keeps a pointer to a destroyed bridge.
- Roblox's original state and eye getters are kept for the process
  lifetime. A hook reached with no bridge, including after a failed vtable
  restore, forwards to them instead of returning an unfilled state buffer.
  Restoring the vtable no longer depends on the pointer hook.
- The activation log names the vtable slots in the right order.
2026-10-13 00:24:45 -05:00
dim-ghub 8106203c8e vr: fix eye colour formats, mirror segment safety and frame-wait locking
- Once the guest eye format is known, the eye swapchains are recreated
  once in the format that keeps its bytes: the eye format, else its
  UNORM/SRGB sibling (raw copy), else the other channel order with the same
  sRGB-ness (a blit that only swizzles). Mixed order-and-gamma cases no
  longer go through a gamma-converting blit; if only that remains, it is
  logged.
- The desktop mirror segment is created with O_EXCL after removing our own
  stale segment (a crashed run's leftover), and must be private to this
  user before eye images are written to it.
- A new frame_mutex_ is held for the whole XR frame cycle and by every path
  that can destroy the session (taken before mutex_). That lets the cycle
  release mutex_ while it blocks in xrWaitFrame, so guest render-thread
  hooks no longer wait out the runtime's pacing. Swapchain-image and fence
  waits stay under mutex_; they return promptly unless the GPU or
  compositor stalls, and they run while guest eye bindings are copied.
2026-10-13 00:08:22 -05:00
dim-ghub 0603734ff6 vr: recover from XR failures instead of giving up on headset output
- Every frame error, including the swapchain-image and copy-fence
  timeouts, now tears the session down recoverably instead of ending XR
  output for the rest of the run.
- Session recovery backs off (1, 2, 4, 8 s) and recreates the XrInstance
  after an instance-loss event, an instance/runtime failure, or three
  failed attempts, so VR returns after SteamVR or WiVRn restarts instead
  of retrying forever.
- While no XR session runs (headset idle, session not READY, recovering)
  presents are paced at 60 Hz outside the backend lock instead of running
  unthrottled.
- A PrepareHostPresent failure that consumed nothing (copy resources
  unavailable, failed fence reset or submit) presents normally instead of
  failing the guest's vkQueuePresentKHR; only device loss is propagated.
- A runtime/guest GPU mismatch falls back to the plain host device and
  keeps the game running without headset output, naming both GPUs.
- A failed EGL attach on the GLES path no longer makes window init fail;
  the game runs without headset output and the bind is not retried.
2026-10-13 00:04:42 -05:00
dim-ghub 0b5578d80b vr: keep non-VR launches on their original paths
Queue and device-idle locks in the Vulkan adapter are now taken only after
an armed OpenXR backend calls mocktail_vulkan_enable_queue_sharing(). The
guest synchronizes its own queue use; only the runtime and the eye copies
add a second user. Without VR, and in builds without it, submits and
presents take no locks.

FFlagDebugEnableVREmulator is only written when VR is active, so a user's
own fflags.json value is no longer overridden on every launch.

Config loading is back after the single-instance lock and the support
bundle guard. A non-VR build still rejects --vr before creating any data
or state directories, from the command line alone; a vr.enabled config in
such a build is rejected once the config is loaded.
2026-10-12 23:59:46 -05:00
dim-ghub bafbe4268c vr: name the build and the supported builds when VR is unavailable
An explicit --vr on a Roblox build without VR support now exits with the
build's version and Build ID, the builds VR supports, and how to select one
(--roblox-lib). When VR is only enabled in the config, the launch continues
without VR and logs the same details. The bridge's own Install error names
both as well. The verified build contracts carry their version labels and
move to namespace scope so they can be listed.
2026-10-12 23:55:37 -05:00
dim-ghub 3ef81c3d93 vr: make VR opt-in and build without the OpenXR submodule
VR now stays off at runtime in every build until --vr or vr.enabled: true,
so a normal launch on a VR-capable build never tries to reach an OpenXR
runtime and no longer exits when none is running.

MOCKTAIL_ENABLE_VR defaults to AUTO: OpenXR support is built when the
third_party/OpenXR-SDK submodule is present and skipped with a notice
otherwise, so source packages that do not fetch submodules (Nix, AUR) keep
building. ON still requires the submodule. mocktail-vr-probe is only built
and installed in VR builds.

CI gains an x86_64 job that builds and tests with MOCKTAIL_ENABLE_VR=OFF.
The config docs and README describe the new defaults.
2026-10-12 23:53:31 -05:00
dim-ghub 360ab6af58 vr: fix CI failures from the 3092 profile
Mark the 3092 profile experimental. It still runs when selected with
--roblox-lib, but it no longer becomes the preferred update target for
every x86_64 user, so the shipped payload metadata (2998) is again the
preferred profile.

Select the 2998 entry by Build ID in the partial-input profile test instead
of relying on manifest order, and drop the shipped 3092 entry before the
test re-identifies 2998 as 3092.

Align the example config's VR build wording with the bootstrap template.
2026-10-12 23:48:32 -05:00
dim-ghub 4cc0e41ef3 vr: add in-headset virtual keyboard for TextBox input
When Roblox asks for a keyboard (showKeyboard), a world-locked panel
opens below eye level as an OpenXR quad layer. Controllers point at keys
and the trigger types; text and key presses go through the same platform
events as the desktop keyboard, so the focused TextBox edits identically.

The layout is US QWERTY without the function row or Ctrl/Alt, with one-shot
Shift, Caps Lock, Tab, Backspace auto-repeat, Enter and Close. A trigger
aimed at the panel is withheld from Roblox so a key press is not also a
click. A text field above the keys mirrors the focused TextBox, whose
desktop overlay never reaches the eye images. MOCKTAIL_VR_KEYBOARD=0 turns
it off.
2026-10-12 23:45:51 -05:00
dim-ghub ddc985218b vr: add separate VR frame-rate limit, render scale and unpaced benchmark mode
vr.frame_rate_limit / MOCKTAIL_VR_FRAME_RATE_LIMIT replaces
graphics.frame_rate_limit while VR is active and takes the same values.
It defaults to unlimited so xrWaitFrame paces Roblox at the headset
refresh rate instead of Roblox's own 60 FPS cap.

MOCKTAIL_VR_RENDER_SCALE scales the runtime's recommended eye extent.
SteamVR's recommendation for a Quest 2 at 90 Hz (2244x2352 per eye) missed
the 11.1 ms budget and halved the rate to 45 FPS.

MOCKTAIL_VR_UNPACED=1 stops guest frames from blocking in xrWaitFrame:
the open XR frame spans several guest frames and is submitted once per
display period, for measuring maximum render rate.
2026-10-12 22:44:46 -05:00
dim-ghub 33dd7769f0 Merge remote-tracking branch 'upstream/main' into vr
# Conflicts:
#	stubs/libandroid_stub.cc
2026-10-12 21:50:15 -05:00
dim-ghub 88bb4c940c vr: align pointer ray pitch with hand orientation and reset world scale to 1.0 2026-10-12 21:28:20 -05:00
dim-ghub 68f8ec82d2 vr: fix sRGB double-gamma, home world avatar spawn, and controller aim alignment 2026-10-12 20:25:21 -05:00
dim-ghub 509822974b vr: Recreate OpenXR instance on guest Vulkan instance recreation
SteamVR (vrclient.so) binds its Vulkan renderer to the lifetime of the
XrInstance. When Roblox destroys the LuaApp Vulkan device and creates a
second VkInstance/VkDevice for UGC experiences, reusing the original
XrInstance causes SteamVR to dereference freed device handles in xrEndFrame.

Tear down and recreate the XrInstance whenever a new guest Vulkan instance
is initialized, resetting session state, spaces, and controller actions.
2026-10-12 05:42:33 -05:00
dim-ghub a441fa7110 assets: fix shader pack path resolution and root directory normalization 2026-10-12 02:16:21 -05:00
dim-ghub d54d20de64 audio: align input capture test with shared_ptr atomic store and restore FMOD profile functions 2026-10-12 01:45:08 -05:00
dim-ghub f9fede1649 Merge remote-tracking branch 'mocktail/vr' into vr 2026-10-12 01:39:13 -05:00
komaruworld 9a1ccbb5ce add roblox 3092 support
Also fix openxr shit
2026-10-03 13:40:24 +03:00
daeeseD ecb80b3ee1 vr: capture the microphone for voice chat on Build 2998 (#181)
* vr: capture the microphone for voice chat on Build 2998

On the vr branch Roblox 2.736.1408 joins voice chat and plays other
players, but never opens a microphone: its FmodAudioDevice capture
methods have no working host path, and main's native capture (537828a
"Fix vc") is pinned to 2.738 RVAs.

Port main's NativeInputCapture and interpose the FmodAudioDevice capture
slots for Build 2998. 2.738 inserted two device-list slots (7 and 11),
so the capture methods sit at {4, 11..15} here instead of {4, 13..17}.
The input-device slots {8, 9, 10, 16}, checked against main's input RVAs
for this image, anchor that layout before anything is patched, and each
capture slot must be a distinct method next to the input-device code.

The voice sink is called through its own vtable after checking that the
vtable and its PCM entry point lie in libroblox, and the retained
shared_ptr is released through the libc++ control-block ABI, so no
further Build 2998 RVAs are needed. If any check fails, capture stays
off and the output routes still work.

MOCKTAIL_NATIVE_INPUT_CAPTURE=0 turns the capture interposition off, and
audio.input_device: disabled reports no microphone to Roblox.

* Requested fix at PR 181
2026-10-01 19:25:48 +03:00
komaruworld 6362283ade fix idle 1fps
bob
2026-09-29 19:34:17 +03:00
珍惜andHttpAnimations fbf97df708 Fix VR crashing is the version is not supported (#158)
Co-authored-by: HttpAnimations <shawndouglasyt@gmail.com>
2026-09-22 23:55:55 +03:00
komaruworld 25cfd94c56 BLA BLA BLA 3 2026-09-17 00:07:27 +03:00
komaruworld 3c20ee3c55 Bla fix main menu bla 2026-09-14 18:26:08 +03:00
UwieandUwie 5e24bee4ab resolution thing or smth idk (#141)
Co-authored-by: Uwie <Uwie>
2026-09-14 17:14:42 +03:00
komaruworld a014751cd4 bla bla bla 2026-09-14 16:38:31 +03:00
komaruworld 7e057232e4 fixes 2026-09-13 20:36:53 +03:00
komaruworld f5080e8a99 fixes 2026-09-13 20:19:00 +03:00
komaruworld 3d89f128ce support alvr and opengl 2026-09-13 19:28:28 +03:00
komaruworld b187efbedf Full support camera
NOT TESTED
2026-09-13 18:20:58 +03:00
komaruworld dd23f06a8b init vr 2026-09-12 21:45:48 +03:00
131 changed files with 29445 additions and 137 deletions

No files matched your search

+10 -2
View File
@@ -77,7 +77,7 @@ jobs:
bash tests/anylinux_dependencies_test.sh .
native-build:
name: Native build (${{ matrix.arch }})
name: Native build (${{ matrix.arch }}, VR ${{ matrix.vr }})
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
strategy:
@@ -86,8 +86,15 @@ jobs:
include:
- arch: x86_64
runner: ubuntu-26.04
vr: 'ON'
- arch: aarch64
runner: ubuntu-26.04-arm
vr: 'ON'
# Packaging recipes build without the OpenXR submodule; keep the
# non-VR configuration compiling and passing on its own.
- arch: x86_64
runner: ubuntu-26.04
vr: 'OFF'
steps:
- name: Check out source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -112,7 +119,8 @@ jobs:
run: |
test "$(uname -m)" = "${{ matrix.arch }}"
cmake -S . -B build-ci -G Ninja \
-DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON
-DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON \
-DMOCKTAIL_ENABLE_VR=${{ matrix.vr }}
cmake --build build-ci --parallel 4
./build-ci/mocktail --help
./build-ci/mocktail_updater status
+6
View File
@@ -4,3 +4,9 @@
[submodule "third_party/Vulkan-Headers"]
path = third_party/Vulkan-Headers
url = https://github.com/KhronosGroup/Vulkan-Headers.git
[submodule "third_party/OpenXR-SDK"]
path = third_party/OpenXR-SDK
url = https://github.com/KhronosGroup/OpenXR-SDK.git
[submodule "third_party/smaa"]
path = third_party/smaa
url = https://github.com/iryoku/smaa.git
+11
View File
@@ -17,6 +17,12 @@ Release notes on GitHub are taken from the matching section here.
- `MOCKTAIL_ASSET_TRACE=1` now also prints the files Roblox asks for and does not get, as `[asset] miss`.
- Arch Linux: `nightcap-git` is on the AUR and builds the current `main` branch. Thanks to @dim-ghub, who
maintains it.
- `vr.mode` in `config.yaml`: `off` starts on the desktop, `on` starts in VR, and `auto` starts in VR only
when SteamVR is already running with a connected headset. `auto` never starts SteamVR. `vr.enabled` still
works.
- Anti-aliasing in VR. Roblox draws the VR eyes without it, so edges were jagged in the headset. Each eye is
now filtered on its way to the headset. `vr.anti_aliasing` picks `fxaa` (the default), `smaa`, which finds
edges more precisely and keeps text sharper for a little more GPU time, or `off`.
- The log now names each Roblox Java call that Nightcap has no handler for, once per call, as
`[JNI] no handler for <class>.<method><signature>`. This shows what a new Roblox version needs.
- `--profile` traces now show how much GPU memory decoded ETC2 textures take, as the `emulated texture (MB)`
@@ -34,6 +40,11 @@ Release notes on GitHub are taken from the matching section here.
`FFlagDebugLuauCodegenAll` in `fflags.json` to choose it yourself.
- Nightcap answers Roblox's Java calls with less work per call, and the audio thread no longer allocates
memory for each block of sound it plays.
- VR on Vulkan no longer makes the CPU wait for the GPU each frame. The eye copies, the desktop mirror and
the `MOCKTAIL_VR_MIRROR_SHM` readback are ordered on the GPU, so Roblox prepares the next frame while the
last one renders. On a simulated 120 Hz headset this removed 2 to 5 ms of waiting per frame and steadied
the frame rate. The shared-memory mirror now shows each frame one frame later.
- `MOCKTAIL_VR_PERF` also reports `desktop_mirror`, `present_prepare` and `present_interval`.
## [0.3.4] - 2026-10-11
+4
View File
@@ -126,6 +126,8 @@ set(VULKAN_HEADERS_ENABLE_INSTALL OFF CACHE BOOL
"Install vendored Vulkan-Headers" FORCE)
add_subdirectory(third_party/Vulkan-Headers EXCLUDE_FROM_ALL)
include(${CMAKE_SOURCE_DIR}/cmake/MocktailVR.cmake)
include(CTest)
if(BUILD_TESTING)
# Package builds disable tests and do not fetch Google Test.
@@ -297,6 +299,7 @@ add_library(mocktail_runtime STATIC
src/runtime/roblox_account_bridge.cc
src/runtime/roblox_message_bus_request_bridge.cc
src/runtime/roblox_call_protocol_bridge.cc
src/runtime/roblox_haptic_protocol_bridge.cc
src/runtime/payload_update_preflight.cc
src/runtime/process_launch_policy.cc
src/runtime/process_diagnostics.cc
@@ -600,6 +603,7 @@ target_compile_definitions(mocktail PRIVATE
"MOCKTAIL_DEFAULT_COMPATIBILITY_MANIFEST=\"${MOCKTAIL_DEFAULT_COMPATIBILITY_MANIFEST}\""
)
target_link_libraries(mocktail PRIVATE
Mocktail::VR
mocktail_legacy_runtime
mocktail_auth_runtime_composition
mocktail_failure_dialog_runtime
+2 -4
View File
@@ -75,7 +75,5 @@ Start WiVRn or SteamVR/ALVR, connect your headset, then run:
./build/mocktail --vr
```
That branch's
[README](https://github.com/CoderDayton/nightcap/blob/vr/README.md#experimental-vr)
and [docs/VR.md](https://github.com/CoderDayton/nightcap/blob/vr/docs/VR.md)
have the details, including how to try it without a headset.
The [README](README.md#experimental-vr) and [docs/VR.md](docs/VR.md) have the
details, including how to try it without a headset.
+55
View File
@@ -153,6 +153,8 @@ Edit `~/.config/mocktail/config.yaml`. Useful options:
- `graphics.frame_rate_limit: display` to unlock FPS to your monitor's rate.
- `graphics.vsync: off` for the lowest lag, at the cost of tearing.
- `vr.frame_rate_limit` sets the cap in VR. It defaults to `unlimited`, so the
headset's refresh rate paces frames; use `-1` for Roblox's 60 FPS cap.
- `graphics.backend: opengl` if Vulkan does not work on your GPU.
## Building
@@ -235,6 +237,59 @@ To run the same checks as CI before each commit and push, install
lefthook install
```
## Experimental VR
Experimental OpenXR VR support on Linux `x86_64` using WiVRn or SteamVR/ALVR.
VR works on these Roblox builds. On any other build, `--vr` stops at start-up
and names them:
| Build | Roblox version |
| --- | --- |
| 2998 | 2.736.1408 |
| 3092 | 2.738.1397 |
| 3222 | 2.742.1059 |
### Build with VR
VR support is compiled in when the OpenXR SDK submodule is checked out
(`MOCKTAIL_ENABLE_VR` defaults to `AUTO`). Without the submodule the build
skips VR. Pass `-DMOCKTAIL_ENABLE_VR=ON` to make a missing submodule an error,
or `OFF` to leave VR out:
```bash
git submodule update --init third_party/OpenXR-SDK
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j4
```
VR stays off at runtime until you pass `--vr` or set `vr.mode: on` in
`config.yaml`. `vr.mode: auto` starts in VR only when SteamVR is running with a
headset connected.
### Run
Start WiVRn or SteamVR/ALVR and connect your headset first:
```bash
./build/mocktail --vr
```
The OpenXR runtime is detected automatically. To pick one manually:
```bash
MOCKTAIL_VR_RUNTIME=wivrn ./build/mocktail --vr
# or
MOCKTAIL_VR_RUNTIME=alvr MOCKTAIL_GRAPHICS_BACKEND=direct-vulkan ./build/mocktail --vr
```
If you previously set `XR_RUNTIME_JSON`, unset it before switching runtimes.
Use `--no-vr` for desktop mode.
A keyboard opens in the headset when a text box takes focus. The rest is in
[docs/VR.md](docs/VR.md): menus and typing, every VR environment variable,
adding a Roblox build, and testing without a headset.
## Known limitations
Nightcap does not currently work with `hardened_malloc`. Using
+4
View File
@@ -22,6 +22,7 @@ add_library(mocktail_input_runtime STATIC
${MOCKTAIL_INPUT_ROOT}/src/runtime/roblox_text_font_resolver.cc
${MOCKTAIL_INPUT_ROOT}/src/runtime/roblox_text_surface_overlay.cc
${MOCKTAIL_INPUT_ROOT}/src/runtime/roblox_window_input_runtime.cc
${MOCKTAIL_INPUT_ROOT}/src/runtime/roblox_xr_controller_input.cc
)
add_library(Mocktail::InputRuntime ALIAS mocktail_input_runtime)
target_include_directories(mocktail_input_runtime PUBLIC
@@ -33,12 +34,14 @@ target_include_directories(mocktail_input_runtime PRIVATE
target_link_libraries(mocktail_input_runtime PUBLIC
Mocktail::PlatformSdl
Mocktail::Runtime
Mocktail::VR
mocktail_window
PkgConfig::UTF8PROC
PRIVATE
PkgConfig::FONTCONFIG
SDL3_ttf::SDL3_ttf
nlohmann_json::nlohmann_json
${CMAKE_DL_LIBS}
)
target_compile_features(mocktail_input_runtime PUBLIC cxx_std_17)
mocktail_apply_compile_options(mocktail_input_runtime)
@@ -47,6 +50,7 @@ if(BUILD_TESTING AND TARGET GTest::gtest_main)
add_executable(roblox_input_router_test
${MOCKTAIL_INPUT_ROOT}/tests/roblox_gamepad_input_test.cc
${MOCKTAIL_INPUT_ROOT}/tests/roblox_input_router_test.cc
${MOCKTAIL_INPUT_ROOT}/tests/roblox_xr_controller_input_test.cc
)
target_link_libraries(roblox_input_router_test PRIVATE
Mocktail::InputRuntime
+116
View File
@@ -0,0 +1,116 @@
# Copyright 2026 Mocktail Project Authors
# Licensed under the Apache License, Version 2.0.
include_guard(GLOBAL)
# AUTO builds OpenXR support when the third_party/OpenXR-SDK submodule is
# checked out and skips it otherwise, so source packages that do not fetch
# submodules still build. ON requires the submodule; OFF never builds it.
set(MOCKTAIL_ENABLE_VR "AUTO" CACHE STRING
"Build experimental OpenXR support: AUTO, ON or OFF")
set_property(CACHE MOCKTAIL_ENABLE_VR PROPERTY STRINGS AUTO ON OFF)
set(_mocktail_openxr_sdk "${CMAKE_SOURCE_DIR}/third_party/OpenXR-SDK/CMakeLists.txt")
string(TOUPPER "${MOCKTAIL_ENABLE_VR}" _mocktail_vr_request)
if(_mocktail_vr_request STREQUAL "AUTO")
if(EXISTS "${_mocktail_openxr_sdk}" AND CMAKE_SYSTEM_NAME STREQUAL "Linux")
set(MOCKTAIL_ENABLE_VR ON)
else()
message(STATUS "OpenXR SDK submodule not found; building without VR "
"(git submodule update --init third_party/OpenXR-SDK to enable it)")
set(MOCKTAIL_ENABLE_VR OFF)
endif()
elseif(MOCKTAIL_ENABLE_VR)
set(MOCKTAIL_ENABLE_VR ON)
else()
set(MOCKTAIL_ENABLE_VR OFF)
endif()
add_library(mocktail_vr STATIC
src/vr/diagnostics.cc
src/vr/headset_detection.cc
src/vr/openxr_backend_mode.cc
src/vr/vr_perf.cc
src/vr/vr_mirror.cc
src/vr/xr_controller.cc
src/vr/roblox_vr_device_bridge.cc
src/vr/virtual_keyboard.cc)
add_library(Mocktail::VR ALIAS mocktail_vr)
target_include_directories(mocktail_vr PUBLIC "${CMAKE_SOURCE_DIR}/include")
target_include_directories(mocktail_vr PRIVATE "${CMAKE_SOURCE_DIR}/src")
target_link_libraries(mocktail_vr PUBLIC Vulkan::Headers PRIVATE nlohmann_json::nlohmann_json)
# shm_open for the desktop eye mirror lives in librt on glibc.
find_library(MOCKTAIL_RT_LIBRARY rt)
if(MOCKTAIL_RT_LIBRARY)
target_link_libraries(mocktail_vr PUBLIC ${MOCKTAIL_RT_LIBRARY})
endif()
mocktail_apply_compile_options(mocktail_vr)
if(MOCKTAIL_ENABLE_VR)
if(NOT CMAKE_SYSTEM_NAME STREQUAL "Linux")
message(FATAL_ERROR "Mocktail OpenXR support currently targets Linux")
endif()
if(NOT EXISTS "${_mocktail_openxr_sdk}")
message(FATAL_ERROR
"Initialize OpenXR: git submodule update --init third_party/OpenXR-SDK")
endif()
# Scope upstream's generic option names to this dependency. Static linking
# packages the pinned loader without requiring a matching system loader.
function(mocktail_add_openxr_sdk)
set(BUILD_LOADER ON)
set(DYNAMIC_LOADER OFF)
set(BUILD_WITH_SYSTEM_JSONCPP OFF)
set(BUILD_TESTS OFF)
set(BUILD_API_LAYERS OFF)
set(BUILD_CONFORMANCE_TESTS OFF)
set(BUILD_SDK_TESTS OFF)
set(BUILD_FORCE_GENERATION OFF)
add_subdirectory("${CMAKE_SOURCE_DIR}/third_party/OpenXR-SDK"
"${CMAKE_BINARY_DIR}/third_party/OpenXR-SDK" EXCLUDE_FROM_ALL)
endfunction()
mocktail_add_openxr_sdk()
target_sources(mocktail_vr PRIVATE src/vr/openxr_probe.cc src/vr/openxr_preview.cc
src/vr/openxr_backend.cc src/vr/gles_transport.cc src/vr/xr_actions.cc
src/vr/virtual_keyboard_raster.cc)
# The in-headset keyboard draws its labels with the same SDL3_ttf and
# fontconfig stack as the TextBox overlay.
find_package(SDL3 3.4 REQUIRED CONFIG)
find_package(SDL3_ttf REQUIRED CONFIG)
find_package(PkgConfig REQUIRED)
pkg_check_modules(FONTCONFIG REQUIRED IMPORTED_TARGET fontconfig)
target_link_libraries(mocktail_vr PRIVATE SDL3_ttf::SDL3_ttf SDL3::SDL3
PkgConfig::FONTCONFIG)
# Error unwinding is confined to these native session implementations; their
# public entry points catch errors before returning to the guest runtime.
set_source_files_properties(src/vr/openxr_preview.cc src/vr/openxr_backend.cc
src/vr/xr_actions.cc
PROPERTIES COMPILE_OPTIONS -fexceptions)
target_compile_definitions(mocktail_vr PRIVATE XR_USE_GRAPHICS_API_VULKAN)
option(MOCKTAIL_VR_DIAGNOSTICS
"Compile VR diagnostics: scripted head pose, eye evidence dumps" OFF)
if(MOCKTAIL_VR_DIAGNOSTICS)
target_compile_definitions(mocktail_vr PRIVATE MOCKTAIL_VR_DIAGNOSTICS=1)
endif()
set_source_files_properties(src/vr/openxr_backend.cc PROPERTIES COMPILE_DEFINITIONS "XR_USE_GRAPHICS_API_OPENGL_ES;XR_USE_PLATFORM_EGL")
target_include_directories(mocktail_vr PRIVATE ${MOCKTAIL_EGL_INCLUDE_DIR} ${MOCKTAIL_GLES3_INCLUDE_DIR})
# SMAA's area and search lookup textures for vr.anti_aliasing: smaa.
if(NOT EXISTS "${CMAKE_SOURCE_DIR}/third_party/smaa/Textures/AreaTex.h")
message(FATAL_ERROR
"Initialize SMAA: git submodule update --init third_party/smaa")
endif()
target_include_directories(mocktail_vr PRIVATE "${CMAKE_SOURCE_DIR}/third_party/smaa")
target_link_libraries(mocktail_vr PRIVATE OpenXR::openxr_loader Vulkan::Headers ${CMAKE_DL_LIBS})
else()
target_sources(mocktail_vr PRIVATE src/vr/openxr_disabled.cc
src/vr/xr_actions_disabled.cc)
endif()
if(MOCKTAIL_ENABLE_VR)
add_executable(mocktail_vr_probe src/vr/probe_main.cc)
set_target_properties(mocktail_vr_probe PROPERTIES
OUTPUT_NAME mocktail-vr-probe)
target_link_libraries(mocktail_vr_probe PRIVATE Mocktail::VR)
mocktail_apply_compile_options(mocktail_vr_probe)
install(TARGETS mocktail_vr_probe
RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}")
endif()
+22
View File
@@ -52,6 +52,28 @@ graphics:
# unless frame_rate_limit is unlimited.
# vsync: off
# vr:
# String (default: off): experimental VR through OpenXR on supported Roblox
# builds. off always starts on the desktop. on always starts in VR, like
# --vr. auto starts in VR when SteamVR is already running with a connected
# headset (ALVR included), and on the desktop otherwise. auto never starts
# SteamVR. --vr and --no-vr override this for one launch. The older
# enabled: true/false still works as on/off; set one or the other.
# Standalone OpenXR test scene: mocktail-vr-probe --scene.
# mode: off
# String (default: fxaa): smooth jagged edges in the headset. Roblox draws
# the VR eyes without anti-aliasing, so each eye is filtered on its way to
# the headset. fxaa is the cheapest. smaa finds edges more precisely and
# keeps text and fine detail sharper, at a little more GPU time. off sends
# the eyes as rendered.
# anti_aliasing: fxaa
# Optional string or integer: frame-rate limit used instead of
# graphics.frame_rate_limit while VR is active. Same values: -1 (Roblox's
# own cap, usually 60), display, unlimited, or any positive integer. The
# default, unlimited, lets the OpenXR runtime pace frames at the headset's
# refresh rate.
# frame_rate_limit: unlimited
performance:
# Boolean (default: false): size Roblox scheduler/render queues from every
# physical CPU core. A place's Lua/main thread can still remain serial.
+40 -1
View File
@@ -102,7 +102,46 @@
"input_current_method_rva": "0x32d13a6",
"input_select_method_rva": "0x32d0a56"
},
"reason": "Verified 2026-09-06: exact Build-ID profile passed two isolated no-recovery Tier C Vulkan canaries and real game sessions, including fullscreen state synchronization and host FMOD output-device interposition; legacy executable patches and signal recovery remain disabled."
"vr_debug_device_bridge": {
"vtable_rva": "0x6c87b38",
"constructor_rva": "0x3810614",
"state_getter_rva": "0x38109f6",
"eye_getter_rva": "0x38109e6",
"eye_initializer_rva": "0x3810a18",
"emulator_flag_storage_rva": "0x7369920",
"device_create_framebuffer_vtable_offset": "0x180"
},
"reason": "Verified 2026-09-06: exact Build-ID profile passed two isolated no-recovery Tier C Vulkan canaries and real game sessions, including fullscreen state synchronization and host FMOD output-device interposition; legacy executable patches and signal recovery remain disabled. Experimental vr_debug_device_bridge metadata (DebugDeviceVR constructor/state/eye/initializer RVAs) is machine-validated at activation and gated behind --vr; native stereo output is not yet verified end-to-end."
},
{
"allow_host_abi_bridges": true,
"allow_host_constructor_replay": true,
"allow_legacy_binary_patches": false,
"default_allowed": true,
"elf_build_id": "5f0704edd9064f566ee3d6df2bd2fabbcc709f03",
"fmod_output_device_bridge": {
"count_method_rva": "0x320c676",
"current_method_rva": "0x320c6c6",
"info_method_rva": "0x320c716",
"select_method_rva": "0x320bf78",
"string_constructor_rva": "0x1d7a038",
"vtable_layout_version": 2,
"vtable_rva": "0x6cd3ce0"
},
"reason": "Exact-Build-ID host ABI promoted from the validated probation profile; native stereo, VR state/pointer/haptic contracts and FMOD output-device layout v2 were re-derived for 3092. Experimental: runs when selected explicitly (--roblox-lib) but is never an update target, so non-VR users stay on the default payload.",
"status": "experimental",
"user_game_settings_fullscreen_setter_rva": "0x45c5f42",
"version_code": 3092,
"version_name": "2.738.1397",
"vr_debug_device_bridge": {
"vtable_rva": "0x6d08108",
"constructor_rva": "0x37f9fe4",
"state_getter_rva": "0x37fa3c6",
"eye_getter_rva": "0x37fa3b6",
"eye_initializer_rva": "0x37fa3e8",
"emulator_flag_storage_rva": "0x73f4690",
"device_create_framebuffer_vtable_offset": "0x188"
}
}
]
}
+25
View File
@@ -197,6 +197,31 @@ and your value applies everywhere. To keep the interpreter in the menus too:
framerate cap in charge; Nightcap sets no `DFIntTaskSchedulerTargetFps`
override. A fixed value forwards that number to the scheduler.
In VR, `vr.frame_rate_limit` (`MOCKTAIL_VR_FRAME_RATE_LIMIT`) replaces
`graphics.frame_rate_limit` and takes the same values. Its default is
`unlimited`: the scheduler target goes to 240 and `xrWaitFrame` holds each
frame to the headset's refresh rate. Set `-1` to restore Roblox's own cap, or a
number such as `90` to pin it.
`MOCKTAIL_VR_RENDER_SCALE` (default `1.0`, range `0.25`–`2.0`) scales each
axis of the OpenXR runtime's recommended eye size. SteamVR's recommendation
already includes its own supersampling, so on a Quest 2 at 90 Hz it is about
2244x2352 per eye. If a frame misses the 11.1 ms budget, the runtime halves
the rate to 45 FPS; `0.75` renders about 56% of the pixels.
`MOCKTAIL_VR_UNPACED=1` is a benchmark mode. Roblox no longer waits for the
headset between frames; each XR frame stays open across several Roblox frames
and is submitted with the newest eyes once per display period. Pair it with a
fixed scheduler target above 240 and `MOCKTAIL_VSYNC=off` to find the
maximum rate:
```bash
MOCKTAIL_VR_UNPACED=1 MOCKTAIL_VR_FRAME_RATE_LIMIT=9999 MOCKTAIL_VSYNC=off mocktail --vr
```
Frames beyond the headset refresh are never displayed, and they all reuse the
open frame's head pose, so expect judder. Do not play in this mode.
`graphics.vsync` (default `auto`) selects the presentation mode. `auto` and
`on` both pick the lowest-latency synchronized mode the driver offers, in
order: `FIFO_LATEST_READY`, `MAILBOX`, `FIFO_RELAXED`, `FIFO`. The first two do
+135
View File
@@ -0,0 +1,135 @@
# VR
Experimental OpenXR VR is opt-in: launch with `--vr`, or set `vr.mode` in
`config.yaml`:
- `off` (default) always starts on the desktop.
- `on` always starts in VR.
- `auto` starts in VR when SteamVR is already running with a connected
headset, ALVR included, and on the desktop otherwise. It never starts
SteamVR, so start SteamVR and connect the headset first.
`--vr` and `--no-vr` override `vr.mode` for one launch. The older
`vr.enabled: true`/`false` still works as `on`/`off`. Everything here only
applies while VR is active. The [README](../README.md#experimental-vr) covers
building and running.
## Menus and typing
The home menu uses Roblox's `More` button. The desktop side panel is turned
off in VR, because the VR pointer cannot click it.
When a text box takes focus (chat, search), a keyboard opens in front of you,
below eye level. It is a full US layout without the function row or
Ctrl/Alt. Point at a key and pull the trigger. While you point at the keyboard,
the trigger types and does not click the game behind it. Hold Backspace to
delete repeatedly. Shift applies to the next key and selects its upper symbol;
Caps Lock stays on until you tap it again. Close dismisses the keyboard and
leaves the text box. Set `MOCKTAIL_VR_KEYBOARD=0` to turn the keyboard off.
## Environment variables
## Runtime and display
| Variable | Default | Effect |
| --- | --- | --- |
| `MOCKTAIL_VR_ENABLED` | `0` | Same as `--vr` (`1`) or `--no-vr` (`0`). |
| `MOCKTAIL_VR_RUNTIME` | `auto` | OpenXR runtime: `auto`, `system`, `wivrn`, `steamvr` or `alvr`. `XR_RUNTIME_JSON` overrides it. |
| `MOCKTAIL_VR_BACKEND` | `xr` | `native` keeps Roblox's stereo rendering without an OpenXR runtime, for debugging. |
| `MOCKTAIL_VR_FRAME_RATE_LIMIT` | `unlimited` | Frame-rate cap in VR, same values as `graphics.frame_rate_limit` (also `vr.frame_rate_limit`). With `unlimited` the headset's refresh rate paces frames. |
| `MOCKTAIL_VR_ANTI_ALIASING` | `fxaa` | Same as `vr.anti_aliasing`. Roblox draws the VR eyes without anti-aliasing, so each eye is filtered on its way to the headset (Vulkan only). `fxaa` is the cheapest; `smaa` finds edges more precisely and keeps text sharper; `off` sends them as rendered. |
| `MOCKTAIL_VR_RENDER_SCALE` | `1.0` | Scales each axis of the runtime's recommended eye size, `0.25` to `2.0`. |
| `MOCKTAIL_VR_UNPACED` | `0` | `1` stops waiting for the headset between frames, to measure the maximum frame rate. Frames past the headset refresh are not shown. Not for playing. |
| `MOCKTAIL_VR_DESKTOP_MIRROR` | `1` | `0` stops copying the left eye to the desktop window. |
| `MOCKTAIL_VR_MIRROR_SHM` | unset | A shared-memory name such as `/nightcap-mirror` publishes both eyes for an external viewer. The segment is private to your user. |
| `MOCKTAIL_VR_KEYBOARD` | `1` | `0` turns off the in-headset keyboard. |
## Hands and scale
| Variable | Default | Effect |
| --- | --- | --- |
| `MOCKTAIL_VR_HAND_POSE_SOURCE` | `aim` | `grip` orients hands from the controller grip pose instead of the aim pose. |
| `MOCKTAIL_VR_HAND_PITCH_OFFSET` | `-20` | Degrees added to Roblox's own hand pitch. |
| `MOCKTAIL_VR_HAND_PITCH` | unset | Replaces the total hand pitch, in degrees. |
| `MOCKTAIL_VR_WORLD_SCALE` | `1.0` | Scales head, hand and eye offsets. `MOCKTAIL_VR_EYE_OFFSET_SCALE` is an older name for it. |
## Performance tracing
| Variable | Default | Effect |
| --- | --- | --- |
| `MOCKTAIL_VR_PERF` | unset | `1` logs per-stage frame timings and counters once a second. `present_interval` is the frame time; `desktop_mirror`, `copy_record` and `copy_fence_wait` show time the CPU spent on VR copies. |
| `MOCKTAIL_VR_MEASURE_START_NS`, `MOCKTAIL_VR_MEASURE_END_NS` | unset | Limit `MOCKTAIL_VR_PERF` to a `CLOCK_MONOTONIC` window. |
| `MOCKTAIL_VR_POSE_LOG_INTERVAL` | `240` | Frames between head-pose log lines. |
## Diagnostics builds
These only work in builds configured with `-DMOCKTAIL_VR_DIAGNOSTICS=ON`.
Release builds ignore them.
| Variable | Default | Effect |
| --- | --- | --- |
| `MOCKTAIL_VR_POSE_SOURCE` | `runtime` | `script` drives the head from a scripted motion instead of the headset, for testing without wearing it. |
| `MOCKTAIL_VR_POSE_AMPLITUDE_POSITION`, `_YAW`, `_PITCH` | `0.25`, `0.5`, `0.17` | Scripted motion size, in metres and radians. |
| `MOCKTAIL_VR_POSE_PERIOD_POSITION`, `_ROTATION` | `240`, `360` | Scripted motion period, in frames. |
| `MOCKTAIL_VR_XR_EVIDENCE_DIR` | unset | Writes submitted eye images to this directory. |
| `MOCKTAIL_VR_XR_EVIDENCE_INTERVAL` | `20` | Frames between evidence images. |
## Adding a Roblox build
VR needs addresses that are specific to each Roblox build. To use a supported
build that is not the active one, point `--roblox-lib` at its `libroblox.so`.
To add a build, run the offset finder on its library:
```bash
python3 scripts/find_roblox_vr_offsets.py --elf <libroblox.so> --output offsets.json
```
It needs the `capstone` Python package. Add its values to `kContracts` in
`src/vr/roblox_vr_device_bridge.cc`. Nightcap checks those addresses against
Roblox's code before it hooks anything, and leaves VR off if they do not
match. To run the same checks on real libraries, set
`MOCKTAIL_TEST_VR_PAYLOADS` to a folder of `<build>-<build id>/libroblox.so`
payloads and run `ctest -R VrSupportedBuilds`.
For a second opinion, solve the new build from the previous one and compare
the two answers. It needs the previous build's HostAbi profile and
compatibility manifest:
```bash
python3 scripts/roblox_abi_check.py --reference-lib <old libroblox.so> \
--reference-profile <old profile.json> \
--reference-compatibility config/roblox_compatibility.json \
--candidate-lib <new libroblox.so> --vr
```
It lists every field where the finder and the solver disagree and exits 0
when they agree on all of them.
## Testing without a headset
[Monado](https://monado.freedesktop.org) can stand in for a headset. Its
`qwerty` driver gives a head and two controllers that you move with the
keyboard and mouse:
```bash
QWERTY_ENABLE=1 XRT_DEBUG_GUI=1 monado-service
```
Then, in another terminal:
```bash
XR_RUNTIME_JSON=/usr/share/openxr/1/openxr_monado.json ./build/mocktail --vr
```
The keys work while Monado's debug window has focus. The simulated display is
small, so text in the headset view looks coarse.
Two tests need an OpenXR runtime and skip without one. To run them, start
Monado with no window and opt in:
```bash
XRT_COMPOSITOR_NULL=1 monado-service &
XR_RUNTIME_JSON=/usr/share/openxr/1/openxr_monado.json MOCKTAIL_TEST_XR_GLES=1 \
ctest --test-dir build -R GlesVrTransportTest.OpenXr
```
+15
View File
@@ -53,6 +53,19 @@ struct FmodOutputDeviceBridgeProfile {
}
};
// Exact-build metadata for the experimental native VR bridge. The bridge
// interposes DebugDeviceVR slots and a checked world-pointer trampoline.
// All RVAs are validated against machine-code contracts before first use.
struct VrDebugDeviceBridgeProfile {
std::uintptr_t vtable_rva = 0;
std::uintptr_t constructor_rva = 0;
std::uintptr_t state_getter_rva = 0;
std::uintptr_t eye_getter_rva = 0;
std::uintptr_t eye_initializer_rva = 0;
std::uintptr_t emulator_flag_storage_rva = 0;
std::uintptr_t device_create_framebuffer_vtable_offset = 0;
};
struct BuildProfile {
std::string version_name;
int version_code = 0;
@@ -70,6 +83,8 @@ struct BuildProfile {
// output-device virtual slots and restores them during controlled teardown;
// executable guest code is never modified.
std::optional<FmodOutputDeviceBridgeProfile> fmod_output_device_bridge;
// Optional exact-build experimental VR capability; see the struct above.
std::optional<VrDebugDeviceBridgeProfile> vr_debug_device_bridge;
std::string reason;
};
+35
View File
@@ -0,0 +1,35 @@
#ifndef MOCKTAIL_VR_HEADSET_DETECTION_H_
#define MOCKTAIL_VR_HEADSET_DETECTION_H_
#include <chrono>
#include <string>
namespace mocktail::vr {
// Set on the child process that answers DetectSteamVrHeadset. main() checks
// it first and returns RunHeadsetProbeChild() without starting Roblox.
inline constexpr const char* kHeadsetProbeEnvironment =
"MOCKTAIL_VR_HEADSET_PROBE";
struct HeadsetDetection {
bool headset = false;
std::string reason;
};
// vr.mode: auto. True only when this user's SteamVR is running and its OpenXR
// runtime reports a head-mounted display. SteamVR is never started. The
// runtime is asked from a child process (`self_executable` with
// kHeadsetProbeEnvironment set) so its libraries never load into this one
// before the VR backend arms.
HeadsetDetection DetectSteamVrHeadset(
const std::string& self_executable,
std::chrono::milliseconds timeout = std::chrono::seconds(5),
const std::string& proc_root = "/proc");
// The child side: probes the OpenXR runtime in XR_RUNTIME_JSON and returns
// the probe's exit code (0 when a headset is available).
int RunHeadsetProbeChild();
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_HEADSET_DETECTION_H_
+665
View File
@@ -0,0 +1,665 @@
#ifndef MOCKTAIL_VR_OPENXR_BACKEND_H_
#define MOCKTAIL_VR_OPENXR_BACKEND_H_
#include <vulkan/vulkan.h>
#include <array>
#include <atomic>
#include <cstdint>
#include <mutex>
#include <string>
#include <unordered_map>
#include <vector>
#include <memory>
#include "mocktail/status.h"
#include "mocktail/vr/virtual_keyboard.h"
#include "mocktail/vr/vr_mirror.h"
#include "mocktail/vr/xr_actions.h"
#include "mocktail/vr/xr_controller.h"
namespace mocktail::vr {
namespace internal {
// Swapchain format for guest eyes of `eye` given the runtime's formats:
// `eye` itself, else its UNORM/SRGB sibling (a raw byte copy, since the guest
// writes sRGB-encoded bytes either way), else the other channel order with
// the same sRGB-ness (a blit that only swizzles). VK_FORMAT_UNDEFINED when
// only a gamma-converting blit remains.
VkFormat PreferredEyeSwapchainFormat(VkFormat eye,
const std::vector<std::int64_t>& available);
// Consume binary present dependencies exactly once. On success a later present
// must omit them. VK_NOT_READY means nothing was consumed and the guest's
// present proceeds unchanged; any other failure consumed them (or lost the
// device) and must be propagated, not retried.
VkResult WaitForPresentDependencies(VkDevice device, VkQueue queue,
const VkPresentInfoKHR& present,
VkFence fence, PFN_vkResetFences reset,
PFN_vkQueueSubmit submit,
PFN_vkWaitForFences wait);
// Hands binary present dependencies to the GPU instead: one empty batch on
// `queue` waits on them and signals `signal`, which the present waits on
// instead. Work submitted to `queue` afterwards is ordered after the waits.
// The CPU never blocks. VK_NOT_READY means nothing was consumed.
VkResult ChainPresentDependencies(VkDevice device, VkQueue queue,
const VkPresentInfoKHR& present,
VkSemaphore signal, PFN_vkQueueSubmit submit);
// Makes the Vulkan adapter serialize queue calls for the rest of the process.
// Called when a backend arms; the adapter library may load before or after.
void RequestVulkanQueueSharing();
} // namespace internal
// Resolved --vr output mode. kXrOutput drives the real Roblox stereo images
// into an OpenXR projection layer; kNativeOnly keeps the diagnostic native
// stereo prototype without any runtime requirement. Selected through
// MOCKTAIL_VR_BACKEND ("xr" default, "native" diagnostic).
enum class VrBackendMode {
kDisabled,
kNativeOnly,
kXrOutput,
};
// Explicit XR_RUNTIME_JSON wins. Runtime preference can select SteamVR (ALVR),
// WiVRn, or the registered OpenXR runtime without changing system registration.
std::string SelectVrRuntimeManifest(const std::string& explicit_manifest,
const std::vector<std::string>& candidates);
std::vector<std::string> VrRuntimeManifestCandidates(
const std::string &preference, const std::string &home,
const std::string &config_home, const std::string &config_dirs);
struct VrRuntimeDiscovery {
std::vector<std::string> candidates;
std::string reason;
};
VrRuntimeDiscovery DiscoverVrRuntime(
const std::string& preference, const std::string& home,
const std::string& config_home, const std::string& config_dirs,
const std::string& runtime_dir, const std::string& proc_root = "/proc");
std::string VrRuntimeUnavailableHint(bool user_selected, const std::string& manifest);
// SteamVR's OpenXR manifest when this user's vrserver is running, else empty.
std::string RunningSteamVrManifest(const std::string& proc_root = "/proc");
VrBackendMode ResolveVrBackendMode(bool vr_enabled);
const char* VrBackendModeName(VrBackendMode mode);
// One deterministic head pose sample. Position is metres, orientation is a
// quaternion (x, y, z, w) in the OpenXR LOCAL reference space.
// VrHandPose (tracked controller hands) is defined in xr_controller.h.
struct ScriptedPoseSample {
float position[3] = {0.f, 0.f, 0.f};
float orientation[4] = {0.f, 0.f, 0.f, 1.f};
float eye_offset[2][3] = {}; // metres in head space
float eye_fov[2][4] = {}; // left, right, up, down angles
// Index 0 = left, 1 = right (Roblox UserCFrame LeftHand/RightHand).
VrHandPose hands[2];
float floor_position[3] = {};
float floor_orientation[4] = {0.f, 0.f, 0.f, 1.f};
bool floor_valid = false;
// Verified DebugDeviceVR input channels, published with the same pose.
float controller_channels[28] = {};
bool controllers_connected = false;
std::uint64_t frame = 0;
bool valid = false;
};
// Production OpenXR backend for the native stereo prototype.
//
// Lifecycle: main arms the backend before Roblox startup (xrCreateInstance
// and xrGetSystem fail fast with a clear message when no runtime exists).
// The Vulkan adapter delegates the guest's vkCreateInstance/vkCreateDevice
// through xrCreateVulkanInstanceKHR/xrCreateVulkanDeviceKHR so the runtime's
// extension and GPU requirements are satisfied by the very device Roblox
// renders with. The session is created on that device; per-frame XR work runs
// on the render thread from the adapter's host-present notification:
// xrEndFrame(N) with the two eye images of frame N copied into the
// projection swapchains, then xrWaitFrame/xrBeginFrame/xrLocateViews for
// frame N+1 and publishing the scripted/runtime head pose that the device
// bridge injects into Roblox before its eye passes run.
//
// EGL/OpenGL ES uses XR_MNDX_egl_enable on the guest SDL context and GPU
// framebuffer blits into GLES swapchain textures. It shares the same frame
// scheduling, tracking, pose application and session recovery below.
//
// Eye image provenance is recorded in the adapter during the guest eye
// initializer (owner = DebugDeviceVR object) and bound to an eye index only
// through the real per-frame eye-getter request plus the actual render-pass
// attachment, mirroring the validated evidence-layer semantics. Creation
// order or size alone never selects an eye.
//
// All public entry points are exception-free at their boundary; the
// implementation is compiled with exceptions like openxr_preview.cc and
// translates failures into Status/log outcomes.
class GlesTransport;
enum class VrGraphicsApi { kVulkan, kOpenGles };
class OpenXrBackend final {
public:
OpenXrBackend() = default;
~OpenXrBackend();
OpenXrBackend(const OpenXrBackend&) = delete;
OpenXrBackend& operator=(const OpenXrBackend&) = delete;
// Claims the process-wide backend slot and creates XrInstance/system.
Status Arm(VrGraphicsApi graphics_api = VrGraphicsApi::kVulkan);
// EGL context must be current on the calling thread. Resolver bypasses guest
// hooks.
Status AttachGlesContext(void *display, void *config, void *context,
void *egl_get_proc, void *(*resolve)(const char *));
void DetachGlesContext(void *context);
void NoteGlesPresent();
void *WrapGlesProcAddress(const char *name, void *raw);
// Idempotent full teardown (XR resources first, then the instance).
void Disarm();
bool armed() const { return armed_.load(std::memory_order_acquire); }
// Adapter delegation. Return false when the backend does not own creation
// (caller falls back to the plain host loader path).
bool CreateVulkanInstance(const VkInstanceCreateInfo* create_info,
const VkAllocationCallbacks* allocator,
VkInstance* instance, VkResult* result);
bool CreateVulkanDevice(VkPhysicalDevice physical_device,
const VkDeviceCreateInfo* create_info,
const VkAllocationCallbacks* allocator,
VkDevice* device, VkResult* result);
void NoteDeviceDestroyed(VkDevice device);
void NoteInstanceDestroyed(VkInstance instance);
void NoteQueue(VkDevice device, VkQueue queue, std::uint32_t family,
std::uint32_t index);
std::vector<std::uint32_t> DesktopQueueFamilies(VkDevice device) const;
void RecordCommandPool(VkDevice device, VkCommandPool pool,
std::uint32_t family);
void RecordCommandBuffers(VkDevice device, VkCommandPool pool,
std::uint32_t count,
const VkCommandBuffer* buffers);
void ForgetCommandBuffers(std::uint32_t count,
const VkCommandBuffer* buffers);
void ForgetCommandPool(VkCommandPool pool);
void ResetCommandPool(VkCommandPool pool);
void ResetCommandBuffer(VkCommandBuffer buffer);
void NoteCommandBuffersSubmitted(VkQueue queue, std::uint32_t count,
const VkCommandBuffer* buffers);
VkImageLayout NormalizeDesktopBarrier(VkCommandBuffer command, VkImage image,
std::uint32_t* source,
std::uint32_t* destination,
VkImageLayout* old_layout,
VkImageLayout new_layout);
// Resource provenance recording (adapter wrappers).
bool WantsResourceRecords() const {
return recording_.load(std::memory_order_acquire);
}
void RecordImage(VkDevice device, VkImage image,
const VkImageCreateInfo* info);
void RecordImageView(VkImageView view, VkImage image);
void RecordFramebuffer(VkFramebuffer framebuffer,
const VkFramebufferCreateInfo* info);
void RecordRenderPass(VkRenderPass render_pass,
const VkRenderPassCreateInfo* info);
void NoteDestroyImage(VkImage image);
void NoteDestroyImageView(VkImageView view);
void NoteDestroyFramebuffer(VkFramebuffer framebuffer);
void NoteRenderPassBegin(VkCommandBuffer command_buffer,
const VkRenderPassBeginInfo* info);
bool EyeBindingComplete() const {
return eyes_bound_.load(std::memory_order_acquire);
}
// Render-thread frame cycle at host present begin.
void NoteHostPresent(VkQueue queue, VkDevice device);
// VK_NOT_READY leaves the present untouched until eye submission provenance
// is known. VK_SUCCESS means its waits were consumed on the graphics queue,
// so the eye copies that follow there are ordered after them. The present
// must then wait on *ready instead (none when it is VK_NULL_HANDLE).
VkResult PrepareHostPresent(VkQueue queue, VkDevice device,
const VkPresentInfoKHR* info, VkSemaphore* ready);
void RecordDesktopSwapchain(VkDevice device, VkSwapchainKHR swapchain,
const VkSwapchainCreateInfoKHR* info,
const VkImage* images, unsigned count,
bool converted_to_concurrent = false);
// On success the original present semaphores have been consumed by the
// mirror blit on the GPU. The caller must present waiting only on *ready,
// which the blit signals. The CPU never waits for the guest's frame here.
VkResult MirrorDesktop(VkQueue queue, const VkPresentInfoKHR* info,
VkSemaphore* ready);
// Pose published for the frame that is about to render. The device bridge
// reads it once per frame on the render thread and injects it into the
// guest. valid=false until the XR session located a usable head pose.
// hands[0]=left, hands[1]=right carry this frame's tracked controller grip
// poses (metres, LOCAL space) when a controller layer is active.
ScriptedPoseSample PublishedHeadPose() const;
bool RecommendedEyeExtent(std::uint32_t* width, std::uint32_t* height) const;
void NotePoseApplied(void* owner, std::uint64_t frame);
// Controller input delivery, consumed by the window-thread input runtime
// through the C ABI in xr_controller_abi.h. TakeControllerDelivery pops one
// batch produced by the once-per-frame action sync; it returns false when
// no controller layer is active or nothing is queued. The snapshot exposes
// the current levels for overflow resync. Haptics requests are thread-safe.
bool TakeControllerDelivery(ControllerDelivery* out);
ControllerSnapshot ControllerSnapshotForFrame() const;
void RequestControllerHaptics(int hand, float amplitude,
std::uint64_t duration_ns, float frequency_hz);
void StopControllerHaptics(int hand);
// Diagnostics.
std::uint64_t submitted_frames() const {
return submitted_frames_.load(std::memory_order_relaxed);
}
std::uint64_t skipped_frames() const {
return skipped_frames_.load(std::memory_order_relaxed);
}
bool session_running() const {
return session_running_.load(std::memory_order_relaxed);
}
std::string runtime_name() const;
private:
friend struct OpenXrBackendTestAccess;
struct ImageRecord {
VkImage image = VK_NULL_HANDLE;
VkDevice device = VK_NULL_HANDLE;
std::uint32_t width = 0;
std::uint32_t height = 0;
VkFormat format = VK_FORMAT_UNDEFINED;
VkImageUsageFlags usage = 0;
VkSampleCountFlagBits samples = VK_SAMPLE_COUNT_1_BIT;
void* owner = nullptr;
std::uint64_t order = 0;
};
struct EyeBinding {
bool valid = false;
std::uint64_t frame = 0;
VkImage image = VK_NULL_HANDLE;
VkFramebuffer framebuffer = VK_NULL_HANDLE;
VkImageLayout final_layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
void* owner = nullptr;
std::uint32_t width = 0;
std::uint32_t height = 0;
VkFormat format = VK_FORMAT_UNDEFINED;
};
struct EyeSwapchain {
void* swapchain = nullptr; // XrSwapchain (opaque to keep the header XR-free)
std::vector<VkImage> images;
std::vector<std::uint32_t> gl_images;
std::uint32_t acquired_index = 0;
bool image_acquired = false;
std::uint32_t width = 0;
std::uint32_t height = 0;
VkFormat format = VK_FORMAT_UNDEFINED;
};
bool InitializeSessionLocked(VkPhysicalDevice physical_device,
VkDevice device,
const VkDeviceCreateInfo* create_info,
std::string* error);
// Session-owned controller action layer (poses, buttons, axes, haptics).
// Created at instance level, attached per session, detached on teardown so
// a replacement session never reuses stale spaces.
XrActions actions_;
bool actions_created_ = false;
// Shared by action sync, snapshots, haptics and lifecycle operations.
// Acquired after mutex_ on the render thread, but never held across
// xrWaitFrame; the input thread takes only this controller lock.
mutable std::mutex controller_mutex_;
// Syncs actions once per frame at the predicted display time and folds the
// resulting hand poses into published_pose_ before the guest renders.
void SyncControllersLocked(std::uint64_t predicted_display_time);
bool InitializeGlesSessionLocked(std::string *error);
bool InitializeSessionResourcesLocked(std::string *error);
bool CopyGlesEyesIntoSwapchains();
bool EnsureXrInstanceLocked(std::string* error = nullptr);
void DestroyXrInstanceLocked();
void DisarmLocked();
void* ResolveFunction(const char* name) const;
void SetupDeviceAfterCreationLocked(VkPhysicalDevice physical_device,
VkDevice device,
const VkDeviceCreateInfo* create_info);
void TeardownSessionLocked(const char* reason, bool preserve_device = false);
void HandleSessionState(int state);
void HandleReferenceSpaceChange(std::int64_t change_time);
bool RecoverSessionLocked(std::uint64_t now_ns);
// False only when the runtime names a GPU other than the guest device's.
bool RuntimeUsesGuestGpuLocked();
void PollSessionEvents();
bool SupportsPresentQueueLocked(VkQueue queue) const;
VkQueue GraphicsQueueForPresentLocked(VkQueue present) const;
bool unsupported_present_queue_logged_ = false;
// Drops the pose/images of the frame currently in flight without touching
// guest ownership, so the next xrEndFrame submits no layer instead of an
// image paired with a pose that no longer describes the same origin.
void InvalidateInFlightPose(const char* reason);
void RunFrameCycle(VkQueue queue, VkDevice device);
// Returns true when the backend owns this present but ran no XR frame, so
// the caller paces the guest itself.
bool NoteHostPresentLocked(VkQueue queue, VkDevice device);
bool CreateSwapchains(std::string* error);
// Recreates the eye swapchains once in the format that copies the guest's
// eye bytes without a gamma change (see PreferredEyeSwapchainFormat).
void MatchSwapchainFormatToEyesLocked();
void DestroySwapchains();
bool EnsureCopyResources(VkDevice device, std::uint32_t queue_family,
std::string* error);
void DestroyCopyResources();
// Records both eye copies (and, when the mirror is enabled, both staging
// readbacks) into one command buffer and submits it once. Only the shared
// memory mirror waits for it; the headset path leaves it to the GPU.
bool CopyEyesIntoSwapchains(VkQueue queue);
void RecordEyeCopiesLocked(VkCommandBuffer command, const VkImage source[2],
const VkImage destination[2]);
void RecordFilteredEyesLocked(VkCommandBuffer command, bool smaa);
// Per-frame GPU work (desktop mirror, eye copies) is submitted without a CPU
// wait. A slot is reused only after its fence signalled; its images let a
// guest destroy wait for the submissions that still read them.
struct AsyncSubmission {
VkCommandBuffer command = VK_NULL_HANDLE;
VkFence fence = VK_NULL_HANDLE;
bool pending = false;
std::uint64_t serial = 0; // bumped per submission
std::vector<VkImage> images;
};
// The shared-memory mirror's readback from the previous frame copy. It is
// published at the next copy, when the GPU has long finished it, so neither
// the headset nor the mirror makes the CPU wait for the frame.
struct MirrorReadback {
AsyncSubmission* slot = nullptr;
std::uint64_t serial = 0;
std::uint64_t frame = 0;
VkFormat format = VK_FORMAT_UNDEFINED;
std::uint32_t width = 0;
std::uint32_t height = 0;
};
void PublishMirrorReadbackLocked();
// vr.anti_aliasing: fxaa. The eye copy becomes one FXAA draw per eye into
// the runtime's swapchain image. Anything it cannot set up falls back to
// the plain copy, which never loses headset output.
struct FxaaState {
VkFormat target_format = VK_FORMAT_UNDEFINED;
VkShaderModule vertex = VK_NULL_HANDLE;
VkShaderModule fragment = VK_NULL_HANDLE;
VkSampler sampler = VK_NULL_HANDLE;
VkDescriptorSetLayout set_layout = VK_NULL_HANDLE;
VkDescriptorPool pool = VK_NULL_HANDLE;
VkDescriptorSet sets[2] = {};
VkPipelineLayout layout = VK_NULL_HANDLE;
VkRenderPass render_pass = VK_NULL_HANDLE;
VkPipeline pipeline = VK_NULL_HANDLE;
// The eye image each set samples, and its view.
VkImage sources[2] = {};
VkImageView source_views[2] = {};
// Per eye, one view and framebuffer per runtime swapchain image.
std::vector<VkImageView> target_views[2];
std::vector<VkFramebuffer> framebuffers[2];
bool failed = false;
};
// vr.anti_aliasing. SMAA shares FXAA's final pass, targets and eye views.
enum class AntiAliasing { kOff, kFxaa, kSmaa };
AntiAliasing AntiAliasingMode() const;
// Pipeline, targets and eye bindings for this frame; false means copy.
bool PrepareFxaaLocked();
bool CreateFxaaPipelineLocked(VkFormat target_format);
bool CreateFullscreenPipelineLocked(VkShaderModule vertex,
VkShaderModule fragment,
VkPipelineLayout layout,
VkRenderPass render_pass,
VkPipeline* pipeline);
void DestroyFxaaTargetsLocked();
void ForgetFxaaSourceLocked(VkImage image);
void DestroyFxaaLocked();
// SMAA: edge detection and blending weights into per-eye images, then
// neighborhood blending into FXAA's swapchain targets.
struct SmaaPass {
VkDescriptorSetLayout set_layout = VK_NULL_HANDLE;
VkPipelineLayout layout = VK_NULL_HANDLE;
VkRenderPass render_pass = VK_NULL_HANDLE;
VkPipeline pipeline = VK_NULL_HANDLE;
VkDescriptorSet sets[2] = {};
};
struct SmaaImage {
VkImage image = VK_NULL_HANDLE;
VkDeviceMemory memory = VK_NULL_HANDLE;
VkImageView view = VK_NULL_HANDLE;
};
struct SmaaState {
std::vector<VkShaderModule> modules;
VkDescriptorPool pool = VK_NULL_HANDLE;
SmaaPass edges, weights, blend;
SmaaImage area, search;
// Per eye, at the eye's extent.
SmaaImage edge_images[2], weight_images[2];
VkFramebuffer edge_framebuffers[2] = {};
VkFramebuffer weight_framebuffers[2] = {};
std::uint32_t width = 0, height = 0;
VkFormat target_format = VK_FORMAT_UNDEFINED;
// The eye views the edge and blend sets were written with.
VkImageView bound_sources[2] = {};
bool failed = false;
};
bool PrepareSmaaLocked();
bool CreateSmaaPipelinesLocked();
bool CreateSmaaImageLocked(VkFormat format, std::uint32_t width,
std::uint32_t height, VkImageUsageFlags usage,
SmaaImage* out);
bool UploadSmaaLookupLocked(SmaaImage* image, std::uint32_t width,
std::uint32_t height, const void* bytes,
std::size_t size);
void DestroySmaaImageLocked(SmaaImage* image);
void DestroySmaaTargetsLocked();
void DestroySmaaLocked();
// The next slot, recorded-ready, or nullptr when its previous submission
// did not finish in time (the device is hung or lost).
AsyncSubmission* NextAsyncSubmissionLocked();
VkResult SubmitAsyncLocked(VkQueue queue, AsyncSubmission* slot,
const VkSubmitInfo& submit);
// Waits for pending submissions reading `image`, or all of them for
// VK_NULL_HANDLE.
void WaitAsyncSubmissionsLocked(VkImage image);
// Semaphore the present of `image_index` waits on, created on first use.
enum class PresentStep { kMirror, kPrepare };
VkSemaphore PresentReadySemaphoreLocked(VkSwapchainKHR swapchain,
std::uint32_t image_index,
PresentStep step);
void DestroyRetiredSemaphoresLocked();
// In-headset keyboard: show/hide from the host's TextBox request, pointer
// input, and trigger masking. Runs inside the controller sync.
void UpdateVirtualKeyboardLocked();
// Rasterizes and uploads the panel when it changed. Returns true when the
// keyboard swapchain holds a released image ready for a quad layer.
bool PrepareVirtualKeyboardLayerLocked(VkQueue queue);
bool UploadVirtualKeyboardVulkan(VkQueue queue, VkImage image);
void DestroyVirtualKeyboardStaging();
void DestroyVirtualKeyboardResources();
bool CaptureEyeEvidence(VkQueue queue, int eye);
ScriptedPoseSample ComputeScriptedPose(std::uint64_t frame) const;
void EnsureMirrorForExtent(std::uint32_t width, std::uint32_t height);
MirrorVkProcs MirrorProcs() const;
VrGraphicsApi graphics_api_ = VrGraphicsApi::kVulkan;
GlesTransport *gles_ = nullptr;
std::uint64_t gl_min_version_ = 0;
std::uint64_t gl_max_version_ = 0;
// Held for a whole XR frame cycle and by every path that can destroy the
// session; always taken before mutex_.
std::mutex frame_mutex_;
mutable std::mutex mutex_;
// The frame cycle's hold on mutex_, so it can release it around
// xrWaitFrame. Only set while NoteHostPresent runs the cycle.
std::unique_lock<std::mutex>* state_lock_ = nullptr;
std::atomic<bool> armed_{false};
std::atomic<bool> recording_{false};
std::atomic<bool> eyes_bound_{false};
std::atomic<bool> session_running_{false};
std::atomic<std::uint64_t> submitted_frames_{0};
std::atomic<std::uint64_t> skipped_frames_{0};
// XR handles are stored as opaque pointers in the header; the implementation
// casts them back. This keeps vulkan-only consumers free of OpenXR headers.
void* xr_instance_ = nullptr;
std::uint64_t xr_system_ = 0;
void* xr_session_ = nullptr;
void* xr_local_space_ = nullptr;
void* xr_view_space_ = nullptr;
void* xr_stage_space_ = nullptr;
std::uint32_t xr_eye_width_[2] = {};
std::uint32_t xr_eye_height_[2] = {};
EyeSwapchain eye_swapchains_[2];
VirtualKeyboard keyboard_;
std::unique_ptr<VirtualKeyboardRasterizer> keyboard_raster_;
EyeSwapchain keyboard_swapchain_;
bool keyboard_swapchain_failed_ = false;
bool keyboard_image_ready_ = false;
std::uint64_t keyboard_uploaded_revision_ = 0;
// A new showKeyboard request may open the panel; Close disarms it until
// Roblox drops that request.
bool keyboard_armed_ = true;
pose::Pose keyboard_head_{};
bool keyboard_head_valid_ = false;
std::vector<std::uint8_t> keyboard_pixels_;
VkBuffer keyboard_staging_ = VK_NULL_HANDLE;
VkDeviceMemory keyboard_staging_memory_ = VK_NULL_HANDLE;
void* keyboard_staging_mapped_ = nullptr;
int xr_blend_mode_ = 0;
// Runtime name captured at xrCreateInstance so the bind-time diagnostics line
// can report it without re-entering runtime_name() under the held mutex.
std::string runtime_name_;
bool xr_instance_used_ = false;
// Libraries the runtime's first instance loaded; later instances load them
// with RTLD_DEEPBIND (see EnsureXrInstanceLocked).
std::vector<std::string> runtime_libraries_;
std::string user_manifest_;
std::string runtime_manifest_;
VkInstance vk_instance_ = VK_NULL_HANDLE;
VkDevice vk_device_ = VK_NULL_HANDLE;
VkPhysicalDevice vk_physical_device_ = VK_NULL_HANDLE;
std::uint32_t vk_queue_family_ = 0;
VkQueue vk_graphics_queue_ = VK_NULL_HANDLE;
std::vector<std::uint32_t> device_queue_families_;
void* vk_loader_ = nullptr;
struct VkProcs;
VkProcs* vk_ = nullptr;
VkCommandPool copy_pool_ = VK_NULL_HANDLE;
// Synchronous one-off work: keyboard uploads, evidence, present fallback.
VkCommandBuffer copy_command_ = VK_NULL_HANDLE;
VkFence copy_fence_ = VK_NULL_HANDLE;
// Two submissions per frame, so three frames may be in flight.
static constexpr std::size_t kAsyncSubmissions = 6;
std::array<AsyncSubmission, kAsyncSubmissions> async_{};
std::size_t next_async_ = 0;
MirrorReadback mirror_readback_{};
FxaaState fxaa_{};
SmaaState smaa_{};
// The mirror's signal for this present, which PrepareHostPresent passes on.
VkSemaphore mirror_ready_ = VK_NULL_HANDLE;
// Semaphores of destroyed desktop swapchains. A present may still wait on
// one, so they are destroyed only once the device is idle.
std::vector<VkSemaphore> retired_semaphores_;
std::unordered_map<VkFormat, VkFormatFeatureFlags> blit_features_;
// Optional desktop two-eye mirror (MOCKTAIL_VR_MIRROR_SHM). Publishes the
// latest complete pair through shared memory; inert when unset.
VrMirror mirror_;
std::unordered_map<VkImage, ImageRecord> images_;
struct DesktopSwapchain {
VkExtent2D extent;
VkFormat format;
std::vector<VkImage> images;
VkSharingMode sharing_mode = VK_SHARING_MODE_EXCLUSIVE;
std::vector<std::uint32_t> families;
bool converted_to_concurrent = false;
// Two per image, see PresentReadySemaphoreLocked.
std::vector<VkSemaphore> ready;
};
std::unordered_map<VkSwapchainKHR, DesktopSwapchain> desktop_swapchains_;
std::unordered_map<VkImageView, VkImage> image_views_;
std::unordered_map<VkFramebuffer, std::vector<VkImageView>> framebuffer_views_;
std::unordered_map<VkFramebuffer, VkRenderPass> framebuffer_render_pass_;
std::unordered_map<VkRenderPass, std::vector<VkImageLayout>> render_pass_final_layouts_;
std::unordered_map<VkQueue, std::uint32_t> queue_families_;
struct CommandRecord {
VkCommandPool pool = VK_NULL_HANDLE;
std::uint32_t family = UINT32_MAX;
std::vector<std::pair<VkImage, std::uint64_t>> eye_images;
};
std::unordered_map<VkCommandPool, std::uint32_t> command_pool_families_;
std::unordered_map<VkCommandBuffer, CommandRecord> command_records_;
std::unordered_map<VkImage, VkQueue> eye_render_queues_;
std::unordered_map<VkImage, std::uint64_t> eye_render_frames_;
std::uint64_t image_order_ = 0;
EyeBinding eyes_[2];
// Frame cycle state (render thread only).
bool frame_open_ = false;
bool frame_should_render_ = false;
// A running session is hidden in SYNCHRONIZED; shouldRender still governs rendering.
bool visibility_lost_ = false;
bool session_recovery_pending_ = false;
std::uint64_t next_session_retry_ns_ = 0;
// Consecutive failed session recoveries, for backoff and for deciding when
// the XrInstance itself must be recreated (e.g. after a runtime restart).
std::uint32_t session_recovery_failures_ = 0;
bool xr_instance_stale_ = false;
bool present_sync_failure_logged_ = false;
bool gles_attach_failed_ = false;
// Runtime swapchain formats, and the format chosen once the guest eye
// format is known (VK_FORMAT_UNDEFINED until then).
std::vector<std::int64_t> xr_swapchain_formats_;
VkFormat eye_swapchain_format_ = VK_FORMAT_UNDEFINED;
VkFormat eye_format_checked_ = VK_FORMAT_UNDEFINED;
std::uint64_t frame_display_time_ = 0;
// MOCKTAIL_VR_UNPACED benchmark mode: guest presents before this monotonic
// time leave the open XR frame alone instead of blocking in xrWaitFrame.
std::uint64_t frame_due_ns_ = 0;
struct ViewState {
float position[3];
float orientation[4];
float fov[4]; // angleLeft, angleRight, angleUp, angleDown (radians)
bool valid;
};
ViewState frame_views_[2] = {};
struct PoseStorage {
float position[3] = {0.f, 0.f, 0.f};
float orientation[4] = {0.f, 0.f, 0.f, 1.f};
};
PoseStorage frame_view_poses_[2];
bool frame_views_valid_ = false;
// One-shot guard so a canted-view runtime reports the rejection reason once
// instead of once per frame per eye.
bool projection_rejection_logged_ = false;
mutable ScriptedPoseSample published_pose_;
std::uint64_t xr_frame_counter_ = 0;
std::unordered_map<void*, std::uint64_t> applied_poses_;
};
// Process-wide single backend owned by main. Adapter hooks resolve these
// exported C symbols; they are inert while no backend is armed.
OpenXrBackend* ActiveVrBackend();
// Paths of the shared objects mapped into the process.
std::vector<std::string> LoadedSharedObjects();
// Paths mapped now that were not in `before`.
std::vector<std::string> SharedObjectsLoadedSince(
const std::vector<std::string>& before);
// Holds the runtime's libraries open with RTLD_DEEPBIND while the OpenXR
// loader loads the runtime, so its Vulkan calls bind to its own libvulkan
// dependency instead of the Vulkan adapter's exported hooks. Released after
// the loader holds its own reference, so the runtime still unloads with its
// instance.
class DeepBoundLibraries {
public:
explicit DeepBoundLibraries(const std::vector<std::string>& paths);
~DeepBoundLibraries();
DeepBoundLibraries(const DeepBoundLibraries&) = delete;
DeepBoundLibraries& operator=(const DeepBoundLibraries&) = delete;
private:
std::vector<void*> handles_;
};
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_OPENXR_BACKEND_H_
+34
View File
@@ -0,0 +1,34 @@
#ifndef MOCKTAIL_VR_OPENXR_PREVIEW_H_
#define MOCKTAIL_VR_OPENXR_PREVIEW_H_
#include <array>
#include <cstdint>
#include <string>
namespace mocktail::vr {
struct PreviewOptions {
// Zero keeps the scene open until the runtime or user requests exit.
std::uint32_t frame_limit = 0;
};
struct PreviewResult {
bool success = false;
std::string message;
std::uint64_t frames = 0;
std::uint64_t rendered_frames = 0;
std::uint64_t tracked_frames = 0;
std::array<float, 3> head_position{};
std::array<float, 4> head_orientation{0, 0, 0, 1};
};
// Runs a native Vulkan/OpenXR scene with a panel fixed in LOCAL space. The
// compositor projects it into both eyes. Roblox images/input are not connected.
// Call on the main thread: SIGINT/SIGTERM request a graceful session exit, and
// their previous handlers are restored before returning.
PreviewResult RunOpenXrPreview(const PreviewOptions& options = {});
std::string FormatPreviewResult(const PreviewResult& result);
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_OPENXR_PREVIEW_H_
+52
View File
@@ -0,0 +1,52 @@
#ifndef MOCKTAIL_VR_OPENXR_PROBE_H_
#define MOCKTAIL_VR_OPENXR_PROBE_H_
#include <cstdint>
#include <string>
#include <vector>
namespace mocktail::vr {
enum class ProbeStatus {
kReady,
kBuildDisabled,
kRuntimeUnavailable,
kHeadsetUnavailable,
kIncompatibleRuntime,
kError,
};
struct ViewConfiguration {
std::uint32_t width = 0;
std::uint32_t height = 0;
std::uint32_t samples = 0;
};
struct ProbeReport {
ProbeStatus status = ProbeStatus::kError;
std::string message;
std::string sdk_version;
std::string runtime_name;
std::string runtime_version;
std::string system_name;
std::vector<std::string> extensions;
std::vector<ViewConfiguration> views;
bool orientation_tracking = false;
bool position_tracking = false;
std::string minimum_vulkan_version;
std::string maximum_vulkan_version;
};
// Native host discovery only. Does not load Roblox, change the active runtime,
// create a graphics device/session, or claim that VR rendering is implemented.
ProbeReport ProbeOpenXr();
std::string FormatProbeReport(const ProbeReport& report);
int ProbeExitCode(ProbeStatus status);
// False in builds configured without MOCKTAIL_ENABLE_VR; --vr fails closed
// early in that case instead of starting Roblox without VR support.
bool IsOpenXrSupportCompiled();
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_OPENXR_PROBE_H_
@@ -0,0 +1,130 @@
#ifndef MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_H_
#define MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_H_
#include <atomic>
#include <cstdint>
#include <mutex>
#include <string>
#include <string_view>
#include "compat/build_profile.h"
#include "mocktail/status.h"
namespace mocktail::vr {
namespace internal {
struct VrBridgeTestAccess;
struct VrBuildContract;
}
const compat::VrDebugDeviceBridgeProfile* FindRobloxVrDeviceProfile(
std::string_view build_id);
// Human-readable list of the Roblox builds VR supports, e.g.
// "2998 (2.736.1408) and 3092 (2.738.1397)", for error messages.
std::string DescribeSupportedVrBuilds();
// Experimental native stereo bring-up for the exact Build-ID-scoped
// RBX::Graphics::DebugDeviceVR object of the guest client.
//
// The bridge interposes the state and eye getters of the exact guest build.
// Resource readiness is checked on the live object during each state call;
// initialization never dereferences cached guest object/device pointers. The
// guest owns resource destruction. Recreated objects and reused addresses
// therefore need no cache invalidation. Initialization happens only on a thread
// observed presenting.
class RobloxVrDeviceBridge final {
public:
using StateGetterFn = void *(*)(void *result_buffer, void *device_object);
using EyeGetterFn = void *(*)(void *device_object, int eye_index);
using EyeInitializerFn = void (*)(void *device_object, void *graphics_device);
RobloxVrDeviceBridge() = default;
~RobloxVrDeviceBridge();
RobloxVrDeviceBridge(const RobloxVrDeviceBridge &) = delete;
RobloxVrDeviceBridge &operator=(const RobloxVrDeviceBridge &) = delete;
// Arms the bridge for an exact-build profile that carries
// vr_debug_device_bridge metadata. Claims the process-wide bridge slot.
Status Install(const compat::BuildProfile &profile);
// Validates machine contracts at the loaded image base and interposes the
// two DebugDeviceVR vtable slots. Called once, after libroblox relocations
// and before any guest startup code runs.
Status Activate(std::uintptr_t image_base);
// Restores the interposed vtable slots and disarms. Safe to call twice.
void Shutdown();
bool installed() const { return installed_; }
bool active() const { return active_.load(std::memory_order_acquire); }
// Render-thread notification from the Vulkan adapter's host present path.
void NoteHostPresent();
// Guest-side hooks invoked by the interposed vtable stubs.
void OnHapticsCall(void* device_object, int hand, float amplitude);
// HapticService:SetMotor on Gamepad1 LeftHand (0) / RightHand (1); the
// level holds until the game sets it again.
void OnSetMotorCall(int hand, float amplitude);
void OnPointerFrameCall(void* result, void* service);
void OnStateGetterCall(void *result_buffer, void *device_object);
void OnEyeGetterCall(void *device_object, int eye_index, void *framebuffer);
StateGetterFn original_state_getter() const {
return original_state_getter_.load(std::memory_order_acquire);
}
EyeGetterFn original_eye_getter() const {
return original_eye_getter_.load(std::memory_order_acquire);
}
// Cumulative initialization diagnostics (not a count of live devices).
std::uintptr_t image_base() const {
return image_base_.load(std::memory_order_acquire);
}
int captured_device_count() const;
int initialized_device_count() const;
std::uint64_t present_count() const {
return present_count_.load(std::memory_order_relaxed);
}
std::uint64_t eye_getter_call_count(int eye_index) const;
private:
friend struct internal::VrBridgeTestAccess;
static constexpr std::uint64_t kEyeGetterSummaryInterval = 600;
bool ValidateObjectHeader(void *self) const;
void *ValidatedGraphicsDevice(void *self) const;
void TryInitializeEyes(void *self);
bool RestoreVtableLocked();
mutable std::mutex mutex_;
compat::VrDebugDeviceBridgeProfile profile_{};
const internal::VrBuildContract* contract_ = nullptr;
bool installed_ = false;
std::atomic<bool> active_{false};
std::atomic<std::uintptr_t> image_base_{0};
std::atomic<std::uintptr_t> image_size_{0};
std::uintptr_t *vtable_ = nullptr;
std::uintptr_t original_haptic_slot_ = 0;
std::uintptr_t original_state_slot_ = 0;
std::uintptr_t original_eye_slot_ = 0;
std::atomic<StateGetterFn> original_state_getter_{nullptr};
std::atomic<EyeGetterFn> original_eye_getter_{nullptr};
std::atomic<std::uint64_t> present_thread_{0};
std::atomic<std::uint64_t> present_count_{0};
std::atomic<bool> flag_warning_logged_{false};
std::atomic<int> captured_count_{0};
std::atomic<int> initialized_count_{0};
mutable std::atomic<std::uint64_t> eye_call_totals_[2] = {};
};
// Process-wide single active bridge, mirroring the audio output-device
// bridge ownership model. Install claims the slot; the destructor releases
// it. legacy_runtime forwards the loaded libroblox image base through
// NotifyRobloxVrImageLoaded; it is a successful no-op while no bridge is
// armed, and fail-closed once one is.
Status NotifyRobloxVrImageLoaded(std::uintptr_t image_base);
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_H_
+191
View File
@@ -0,0 +1,191 @@
#ifndef MOCKTAIL_VR_VIRTUAL_KEYBOARD_H_
#define MOCKTAIL_VR_VIRTUAL_KEYBOARD_H_
#include <cstdint>
#include <string>
#include <vector>
#include "mocktail/vr/vr_pose_math.h"
namespace mocktail::vr {
// Host side of the in-headset keyboard. The window module owns Roblox's
// TextBox focus and the SDL event path; main wires these callbacks so the VR
// library never links the window library. Every callback may be called from
// the render thread and must be thread-safe.
struct VirtualKeyboardHost {
void* context = nullptr;
// True while Roblox has asked for an on-screen keyboard (showKeyboard).
bool (*input_requested)(void* context) = nullptr;
// Commits UTF-8 text exactly like an SDL_EVENT_TEXT_INPUT.
void (*commit_text)(void* context, const char* utf8) = nullptr;
// Presses and releases one SDL scancode exactly like a physical key.
void (*press_key)(void* context, std::uint32_t scancode) = nullptr;
};
// Installs the process-wide host. Passing nullptr clears it.
void SetVirtualKeyboardHost(const VirtualKeyboardHost* host);
VirtualKeyboardHost CurrentVirtualKeyboardHost();
// The focused TextBox's current contents, shown in the panel's text field.
// The desktop draws TextBox text as a window overlay that never reaches the
// eye images, so the headset would otherwise show an empty box. Safe from any
// thread; the text is sensitive and is never logged.
struct VirtualKeyboardText {
std::string utf8;
int cursor_utf16 = 0;
bool password = false;
std::uint64_t revision = 0;
};
void SetVirtualKeyboardText(const char* utf8, std::size_t size,
int cursor_utf16, bool password);
void ClearVirtualKeyboardText();
VirtualKeyboardText CurrentVirtualKeyboardText();
enum class VirtualKeyAction {
kText, // commits text, or shift_text under Shift/Caps Lock
kBackspace,
kEnter,
kTab,
kShift, // one-shot
kCapsLock,
kSpace,
kClose, // Escape: leaves the TextBox and hides the panel
};
struct VirtualKey {
std::string label;
std::string text; // committed text for kText
std::string shift_text; // committed under Shift (Caps Lock for letters)
VirtualKeyAction action = VirtualKeyAction::kText;
// Panel UV rectangle, origin top-left, both axes 0..1.
float u0 = 0.f, v0 = 0.f, u1 = 0.f, v1 = 0.f;
};
// One controller's pointing state for a frame. Positions are metres in the
// OpenXR LOCAL space; the ray points along the aim pose's -Z axis.
struct VirtualKeyboardPointer {
bool valid = false;
pose::Pose aim{};
float trigger = 0.f; // 0..1
};
// What the keyboard asks the host to do after an Update.
struct VirtualKeyboardOutput {
std::vector<std::string> text; // committed in order
std::vector<std::uint32_t> scancodes; // pressed in order, after text
};
// Ray/quad intersection. The quad is centred on `quad` with its face along
// +Z, `width` x `height` metres. Returns false when the ray misses or points
// away. u/v are panel UVs (origin top-left).
bool IntersectVirtualKeyboardPanel(const pose::Pose& quad, float width,
float height, const pose::Pose& aim,
float* u, float* v, float* distance);
// Pure keyboard state machine: layout, paging, shift, hover, trigger edges and
// backspace auto-repeat. No OpenXR, Vulkan or SDL types, so it is unit-tested
// directly.
class VirtualKeyboard {
public:
// A text field over 15 x 5 normal-key units, 4 cm per unit.
static constexpr float kPanelWidthMetres = 0.60f;
static constexpr float kPanelHeightMetres = 0.248f;
// Texture aspect matches the panel so keys are not stretched.
static constexpr int kTextureWidth = 1500;
static constexpr int kTextureHeight = 620;
// Top share of the panel holding the text field; keys fill the rest.
static constexpr float kFieldFraction = 120.f / 620.f;
VirtualKeyboard();
// Opens the panel in front of `head` (LOCAL space), below eye level and
// tilted up toward the viewer. Resets Shift and hover; keeps Caps Lock.
void Show(const pose::Pose& head);
void Hide();
bool visible() const { return visible_; }
const pose::Pose& panel_pose() const { return panel_; }
// Text shown in the field. Bumps the revision only when it changed.
void SetText(const VirtualKeyboardText& text);
const std::string& text() const { return text_.utf8; }
int text_cursor_utf16() const { return text_.cursor_utf16; }
bool text_password() const { return text_.password; }
// Processes one frame of both pointers. now_ns is monotonic.
VirtualKeyboardOutput Update(std::uint64_t now_ns,
const VirtualKeyboardPointer (&pointers)[2]);
// True when this hand's trigger belongs to the keyboard this frame (aimed
// at the panel, or a press that began on it is still held, including after
// the panel hid mid-press). The backend withholds that trigger from Roblox
// so a key press is not also a click.
bool CapturesTrigger(int hand) const;
// Current layout, with Shift/Caps Lock applied to the labels.
const std::vector<VirtualKey>& keys() const;
// -1 when the hand is not over a key.
int hovered_key(int hand) const { return hands_[hand & 1].hovered; }
int pressed_key(int hand) const { return hands_[hand & 1].pressed; }
// Pointer position on the panel, when the hand is over it.
bool cursor(int hand, float* u, float* v) const;
bool shifted() const { return shift_; }
bool caps_lock() const { return caps_lock_; }
// Advances whenever anything visible changes; the backend re-rasterizes
// and re-uploads only when this moves.
std::uint64_t revision() const { return revision_; }
private:
struct Hand {
int hovered = -1;
int pressed = -1; // key under an active press, -1 otherwise
bool trigger_down = false;
bool captured = false;
bool over_panel = false;
float u = 0.f, v = 0.f;
std::uint64_t repeat_at_ns = 0;
};
void RebuildLayout();
int KeyAt(float u, float v) const;
std::string Output(const VirtualKey& key) const;
void Activate(const VirtualKey& key, VirtualKeyboardOutput* output);
void Touch() { ++revision_; }
bool visible_ = false;
bool shift_ = false;
bool caps_lock_ = false;
pose::Pose panel_{};
VirtualKeyboardText text_;
Hand hands_[2];
// A press the keyboard owned when it hid; held until that trigger releases.
bool release_pending_[2] = {false, false};
std::uint64_t cursor_drawn_ns_ = 0;
std::vector<VirtualKey> keys_;
std::uint64_t revision_ = 1;
};
// Draws the keyboard into a tightly packed RGBA8 buffer
// (VirtualKeyboard::kTextureWidth x kTextureHeight, sRGB-encoded, opaque).
// Glyphs come from SDL3_ttf with fontconfig's sans fallback chain and are
// cached per label, so redraws after the first are plain fills and blits.
class VirtualKeyboardRasterizer {
public:
VirtualKeyboardRasterizer();
~VirtualKeyboardRasterizer();
VirtualKeyboardRasterizer(const VirtualKeyboardRasterizer&) = delete;
VirtualKeyboardRasterizer& operator=(const VirtualKeyboardRasterizer&) =
delete;
// Returns false when no font could be opened; keys still draw, unlabelled.
bool Draw(const VirtualKeyboard& keyboard, std::vector<std::uint8_t>* rgba);
private:
struct Impl;
Impl* impl_ = nullptr;
};
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_VIRTUAL_KEYBOARD_H_
+105
View File
@@ -0,0 +1,105 @@
#ifndef MOCKTAIL_VR_MIRROR_H_
#define MOCKTAIL_VR_MIRROR_H_
#include <vulkan/vulkan.h>
#include <cstdint>
#include <string>
#include "mocktail/vr/vr_mirror_shm.h"
namespace mocktail::vr {
// The subset of device/instance procs the mirror needs, supplied by the XR
// backend from its already-resolved guest-device table. Keeping this explicit
// avoids a second proc table and keeps the mirror independently testable.
struct MirrorVkProcs {
PFN_vkCreateBuffer CreateBuffer = nullptr;
PFN_vkDestroyBuffer DestroyBuffer = nullptr;
PFN_vkGetBufferMemoryRequirements GetBufferMemoryRequirements = nullptr;
PFN_vkAllocateMemory AllocateMemory = nullptr;
PFN_vkFreeMemory FreeMemory = nullptr;
PFN_vkBindBufferMemory BindBufferMemory = nullptr;
PFN_vkMapMemory MapMemory = nullptr;
PFN_vkUnmapMemory UnmapMemory = nullptr;
PFN_vkCmdCopyImageToBuffer CmdCopyImageToBuffer = nullptr;
PFN_vkCmdPipelineBarrier CmdPipelineBarrier = nullptr;
PFN_vkGetPhysicalDeviceMemoryProperties GetPhysicalDeviceMemoryProperties =
nullptr;
};
// Producer side of the desktop two-eye mirror. Maps a POSIX shared-memory
// segment (created with the name from MOCKTAIL_VR_MIRROR_SHM) and publishes
// the latest complete eye pair under a seqlock. It owns two persistent,
// bounded host-visible staging buffers reused across frames; the GPU->staging
// copies are recorded into the caller's single batched command buffer so the
// mirror adds no extra queue submission or fence wait.
//
// The mirror never touches the filesystem on the frame path and keeps only the
// newest pair (one latest slot), so a slow consumer drops rather than queues
// old frames.
class VrMirror {
public:
VrMirror() = default;
~VrMirror();
VrMirror(const VrMirror&) = delete;
VrMirror& operator=(const VrMirror&) = delete;
bool enabled() const { return mapping_ != nullptr; }
// Opens the shared segment. Returns false (and stays disabled) when the name
// is empty or mapping fails; the XR path then runs without a mirror.
bool Open(const std::string& shm_name, std::uint32_t width,
std::uint32_t height, std::string* error);
void Close();
// Ensures the two staging buffers match width*height*4 bytes on this device.
// Allocates once; a size/device change recreates them. Existing contents are
// discarded on resize.
bool EnsureStaging(const MirrorVkProcs& vk, VkDevice device,
VkPhysicalDevice physical, std::uint32_t width,
std::uint32_t height, std::string* error);
void DestroyStaging(const MirrorVkProcs& vk);
// Records a source-eye -> staging-buffer copy for one eye into cmd, adding
// the layout barriers around it. The source is returned to final_layout so
// the next guest render pass sees it unchanged.
void RecordEyeCopy(const MirrorVkProcs& vk, VkCommandBuffer cmd, int eye,
VkImage source, VkImageLayout final_layout,
std::uint32_t width, std::uint32_t height);
// After the batched fence wait completed, copies both staging buffers into
// the shared segment and bumps the seqlock with the frame id and a monotonic
// timestamp. No-op when staging does not cover width*height.
void Publish(std::uint64_t frame, VkFormat format, std::uint32_t width,
std::uint32_t height, std::uint64_t now_ns);
std::uint64_t published_pairs() const { return published_pairs_; }
std::uint32_t staging_width() const { return staging_width_; }
std::uint32_t staging_height() const { return staging_height_; }
const void* mapping() const { return mapping_; }
// Test seam: publish a pair from caller-supplied staging bytes without any
// Vulkan resources, exercising the same seqlock/segment write as the GPU
// path. Both buffers must hold width*height*4 bytes.
void PublishFromBytesForTest(std::uint64_t frame, VkFormat format,
std::uint32_t width, std::uint32_t height,
std::uint64_t now_ns, const void* eye0,
const void* eye1);
private:
void* mapping_ = nullptr;
std::uint64_t mapping_bytes_ = 0;
std::string shm_name_;
VkBuffer staging_[2] = {VK_NULL_HANDLE, VK_NULL_HANDLE};
VkDeviceMemory staging_memory_[2] = {VK_NULL_HANDLE, VK_NULL_HANDLE};
void* staging_map_[2] = {nullptr, nullptr};
std::uint32_t staging_width_ = 0;
std::uint32_t staging_height_ = 0;
VkDevice staging_device_ = VK_NULL_HANDLE;
std::uint64_t published_pairs_ = 0;
};
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_MIRROR_H_
+42
View File
@@ -0,0 +1,42 @@
#ifndef MOCKTAIL_VR_MIRROR_SHM_H_
#define MOCKTAIL_VR_MIRROR_SHM_H_
#include <cstdint>
namespace mocktail::vr {
// Shared-memory transport for the desktop two-eye mirror. The producer (XR
// backend, inside the client process) publishes the latest complete eye pair
// under a seqlock; the consumer (desktop viewer) reads the newest pair without
// any per-frame filesystem transport, image encode/decode, or allocation.
//
// Layout is fixed and little-endian.
// sizeof(VrMirrorHeader) == kVrMirrorHeaderBytes.
struct VrMirrorHeader {
std::uint32_t magic; // kVrMirrorMagic
std::uint32_t header_bytes; // kVrMirrorHeaderBytes (pixel area offset)
std::uint64_t seq; // seqlock: odd while writing, even when stable
std::uint64_t frame; // XR frame id the pair belongs to
std::uint32_t width; // eye width in pixels
std::uint32_t height; // eye height in pixels
std::uint32_t format; // VkFormat of the source eye images
std::uint32_t flags; // bit0: a complete, valid pair is present
std::uint64_t producer_ns; // CLOCK_MONOTONIC publish timestamp
std::uint64_t published_pairs;
std::uint64_t reserved;
};
inline constexpr std::uint32_t kVrMirrorMagic = 0x3152564D; // 'MVR1'
inline constexpr std::uint32_t kVrMirrorHeaderBytes = 64;
inline constexpr std::uint32_t kVrMirrorFlagValid = 1u << 0;
// Total mapped bytes for a given eye extent (header + two RGBA/BGRA images).
inline std::uint64_t VrMirrorBytes(std::uint32_t width, std::uint32_t height) {
return static_cast<std::uint64_t>(kVrMirrorHeaderBytes) +
2ull * static_cast<std::uint64_t>(width) *
static_cast<std::uint64_t>(height) * 4ull;
}
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_MIRROR_SHM_H_
+102
View File
@@ -0,0 +1,102 @@
#ifndef MOCKTAIL_VR_PERF_H_
#define MOCKTAIL_VR_PERF_H_
#include <cstdint>
#include <string>
#include <vector>
namespace mocktail::vr::perf {
// Opt-in, low-overhead stage timing and counter instrumentation. Collection is
// compiled out at runtime unless MOCKTAIL_VR_PERF is set, so the steady-state
// path only pays a single cached boolean test per record.
bool Enabled();
enum class Stage {
kFrameCycle,
kXrWaitFrame,
kXrBeginFrame,
kXrLocateViews,
kSwapchainAcquire,
kSwapchainWait,
kCopyRecord,
kCopySubmit,
kCopyFenceWait,
kMirrorPublish,
kEvidence,
kControllerSync,
kDesktopMirror,
kPresentPrepare,
kPresentInterval,
kStageCount,
};
enum class Counter {
kPresents,
kFramesBegun,
kFramesSubmitted,
kFramesSkipped,
kSkipShouldRender,
kSkipPoseInvalid,
kSkipEyesUnbound,
kSkipEyeFrameMismatch,
kSubmits,
kFenceWaits,
kMirrorPairs,
kMirrorDrops,
kSkipVisibilityLost,
kReferenceSpaceChanges,
kVisibilityLost,
kVisibilityGained,
kFramesInvalidated,
kControllerSyncs,
kCounterCount,
};
const char* StageName(Stage stage);
const char* CounterName(Counter counter);
// Bounded reservoir per stage: the most recent N samples are kept and ranked
// only when a report is emitted (once per interval), never on the frame path.
class Collector {
public:
static constexpr std::size_t kReservoir = 4096;
static constexpr std::uint64_t kIntervalNs = 1'000'000'000ULL;
void Record(Stage stage, std::uint64_t nanos);
void Count(Counter counter, std::uint64_t delta = 1);
// Emits one distribution line per active stage plus a counter line when at
// least kIntervalNs elapsed since the previous emission. Cheap no-op
// otherwise. now_ns is a CLOCK_MONOTONIC timestamp supplied by the caller so
// the collector stays clock-agnostic and testable.
void EmitIfDue(std::uint64_t now_ns, const char* context);
// Test/inspection accessors.
std::uint64_t counter(Counter counter_id) const;
std::vector<std::uint64_t> Percentiles(Stage stage) const; // p50,p95,p99,max
std::size_t Samples(Stage stage) const;
void Reset();
private:
struct StageStat {
std::vector<std::uint64_t> samples; // ring, bounded by kReservoir
std::size_t head = 0;
std::size_t count = 0;
std::uint64_t total_ns = 0;
};
StageStat stages_[static_cast<std::size_t>(Stage::kStageCount)]{};
std::uint64_t counters_[static_cast<std::size_t>(Counter::kCounterCount)]{};
std::uint64_t last_emit_ns_ = 0;
std::uint64_t window_start_ns_ = 0;
};
// Process-wide collector used by the XR backend frame cycle.
Collector& ProcessCollector();
// Monotonic nanoseconds (CLOCK_MONOTONIC), 0 on failure.
std::uint64_t NowNs();
} // namespace mocktail::vr::perf
#endif // MOCKTAIL_VR_PERF_H_
+217
View File
@@ -0,0 +1,217 @@
#ifndef MOCKTAIL_VR_POSE_MATH_H_
#define MOCKTAIL_VR_POSE_MATH_H_
#include <algorithm>
#include <cmath>
#include <cstdint>
namespace mocktail::vr::pose {
// Coordinate spaces and units used across the whole camera path, documented so
// every stage is auditable:
//
// OpenXR head/views metres, right-handed, +Y up, -Z forward, LOCAL space
// -> published pose metres, quaternion (x, y, z, w), unchanged units
// -> guest state metres; the exact-build native helper converts metres
// to studs itself (factor 10/3). This layer must never
// pre-apply that scale.
// -> camera engine composes
// out.rotation = base.rotation * head.rotation
// out.position = base.position
// + base.rotation * head.position * HeadScale
// where HeadScale is an engine value distinct from the
// metres-to-studs conversion factor.
// -> render one pass per eye using the per-eye offset/FOV
// -> XR submission metres again, the very pose used to render the pair.
struct Vec3 {
float x = 0.f, y = 0.f, z = 0.f;
};
struct Quat {
float x = 0.f, y = 0.f, z = 0.f, w = 1.f;
};
struct Pose {
Vec3 position{};
Quat orientation{};
};
inline constexpr float kPi = 3.14159265358979323846f;
inline Quat QuatMultiply(const Quat& a, const Quat& b) {
return Quat{
a.w * b.x + a.x * b.w + a.y * b.z - a.z * b.y,
a.w * b.y - a.x * b.z + a.y * b.w + a.z * b.x,
a.w * b.z + a.x * b.y - a.y * b.x + a.z * b.w,
a.w * b.w - a.x * b.x - a.y * b.y - a.z * b.z,
};
}
inline Quat QuatConjugate(const Quat& q) { return Quat{-q.x, -q.y, -q.z, q.w}; }
inline float QuatNorm(const Quat& q) {
return std::sqrt(q.x * q.x + q.y * q.y + q.z * q.z + q.w * q.w);
}
// Returns false for a non-finite or near-zero quaternion; the output is left
// untouched in that case so callers never publish a fabricated rotation.
inline bool QuatNormalize(const Quat& in, Quat* out) {
if (out == nullptr) {
return false;
}
if (!std::isfinite(in.x) || !std::isfinite(in.y) || !std::isfinite(in.z) ||
!std::isfinite(in.w)) {
return false;
}
const float norm = QuatNorm(in);
if (!(norm > 1e-6f) || !std::isfinite(norm)) {
return false;
}
const float inverse = 1.f / norm;
*out = Quat{in.x * inverse, in.y * inverse, in.z * inverse, in.w * inverse};
return true;
}
// q and -q denote the same rotation. Used to prove the path is sign-invariant.
inline bool QuatSameRotation(const Quat& a, const Quat& b, float tolerance) {
const float dot = a.x * b.x + a.y * b.y + a.z * b.z + a.w * b.w;
return std::abs(std::abs(dot) - 1.f) <= tolerance;
}
inline Vec3 QuatRotate(const Quat& q, const Vec3& v) {
// t = 2 * cross(q.xyz, v); v' = v + q.w * t + cross(q.xyz, t)
const Vec3 u{q.x, q.y, q.z};
const Vec3 cross1{u.y * v.z - u.z * v.y, u.z * v.x - u.x * v.z,
u.x * v.y - u.y * v.x};
const Vec3 t{2.f * cross1.x, 2.f * cross1.y, 2.f * cross1.z};
const Vec3 cross2{u.y * t.z - u.z * t.y, u.z * t.x - u.x * t.z,
u.x * t.y - u.y * t.x};
return Vec3{v.x + q.w * t.x + cross2.x, v.y + q.w * t.y + cross2.y,
v.z + q.w * t.z + cross2.z};
}
// result = a * b in SE(3): apply b first, then a.
inline Pose PoseMultiply(const Pose& a, const Pose& b) {
Pose result;
result.orientation = QuatMultiply(a.orientation, b.orientation);
const Vec3 rotated = QuatRotate(a.orientation, b.position);
result.position = Vec3{a.position.x + rotated.x, a.position.y + rotated.y,
a.position.z + rotated.z};
return result;
}
inline Pose PoseInverse(const Pose& p) {
Pose result;
result.orientation = QuatConjugate(p.orientation);
const Vec3 neg{-p.position.x, -p.position.y, -p.position.z};
result.position = QuatRotate(result.orientation, neg);
return result;
}
// Rotation for a head turned by yaw about +Y then pitched about +X, matching
// the scripted diagnostic motion. Quaternion order is (x, y, z, w).
inline Quat QuatFromYawPitch(float yaw, float pitch) {
const float cy = std::cos(yaw * 0.5f), sy = std::sin(yaw * 0.5f);
const float cp = std::cos(pitch * 0.5f), sp = std::sin(pitch * 0.5f);
return Quat{cy * sp, sy * cp, -sy * sp, cy * cp};
}
inline Quat QuatFromAxisAngle(const Vec3& axis, float angle) {
const float half = angle * 0.5f;
const float s = std::sin(half);
return Quat{axis.x * s, axis.y * s, axis.z * s, std::cos(half)};
}
// Per-eye FOV as the runtime reports it: angleLeft, angleRight, angleUp,
// angleDown in radians, each signed relative to the eye's forward axis.
struct EyeFov {
float angle_left = 0.f;
float angle_right = 0.f;
float angle_up = 0.f;
float angle_down = 0.f;
};
// The guest state stores projection tangents, not angles. The 2998 layout is
// up, down, left, right in that order; down/left are negated because the
// guest's tangent space measures them in the opposite direction. Getting this
// order or a sign wrong flips or shears the image, so it is pinned by test.
struct ProjectionTangents {
float up = 0.f;
float down = 0.f;
float left = 0.f;
float right = 0.f;
};
inline ProjectionTangents FovToTangents(const EyeFov& fov) {
return ProjectionTangents{std::tan(fov.angle_up), std::tan(-fov.angle_down),
std::tan(-fov.angle_left), std::tan(fov.angle_right)};
}
inline constexpr float kMaxEyeFovAngle = 1.56f;
// Enclose the display frustum in a head-parallel camera for the guest renderer.
// The compositor receives this rendered FOV and pose, not the canted display's.
inline bool ParallelEyeFov(const Quat& eye_orientation, const EyeFov& display,
EyeFov* rendered) {
Quat orientation;
if (!rendered || !QuatNormalize(eye_orientation, &orientation) ||
!(display.angle_left < display.angle_right) ||
!(display.angle_down < display.angle_up)) {
return false;
}
const float angles[] = {display.angle_left, display.angle_right,
display.angle_up, display.angle_down};
for (float angle : angles) {
if (!std::isfinite(angle) || std::abs(angle) >= kMaxEyeFovAngle)
return false;
}
if (orientation.x == 0.f && orientation.y == 0.f && orientation.z == 0.f) {
*rendered = display;
return true;
}
const float xs[] = {std::tan(display.angle_left),
std::tan(display.angle_right)};
const float ys[] = {std::tan(display.angle_down), std::tan(display.angle_up)};
float left = 0.f, right = 0.f, down = 0.f, up = 0.f;
bool first = true;
for (float x : xs) {
for (float y : ys) {
const Vec3 ray = QuatRotate(orientation, {x, y, -1.f});
if (!std::isfinite(ray.z) || ray.z >= -1e-6f) return false;
const float horizontal = ray.x / -ray.z;
const float vertical = ray.y / -ray.z;
if (!std::isfinite(horizontal) || !std::isfinite(vertical)) return false;
if (first) {
left = right = horizontal;
down = up = vertical;
first = false;
} else {
left = std::min(left, horizontal);
right = std::max(right, horizontal);
down = std::min(down, vertical);
up = std::max(up, vertical);
}
}
}
const EyeFov result{std::atan(left), std::atan(right), std::atan(up),
std::atan(down)};
const float bounds[] = {result.angle_left, result.angle_right,
result.angle_up, result.angle_down};
for (float angle : bounds) {
if (std::abs(angle) >= kMaxEyeFovAngle) return false;
}
*rendered = result;
return true;
}
// Metres-to-studs factor the exact-build native helper applies itself. Exposed
// only so tests can prove this layer does NOT pre-apply it.
inline constexpr float kMetresToStuds = 10.f / 3.f;
// Expected stud value for a metre offset, for numerical unit checks.
inline float MetresToStuds(float metres) { return metres * kMetresToStuds; }
} // namespace mocktail::vr::pose
#endif // MOCKTAIL_VR_POSE_MATH_H_
+101
View File
@@ -0,0 +1,101 @@
#ifndef MOCKTAIL_VR_XR_ACTIONS_H_
#define MOCKTAIL_VR_XR_ACTIONS_H_
#include <cstdint>
#include <string>
#include "mocktail/status.h"
#include "mocktail/vr/vr_pose_math.h"
#include "mocktail/vr/xr_controller.h"
namespace mocktail::vr {
// Session-owned OpenXR action layer: one action set, left/right subactions,
// grip+aim pose spaces, and interaction-profile bindings for Touch (primary),
// Quest 2, Index, Vive wands and the simple controller. All XR handles stay
// opaque here so non-OpenXR consumers (bridge, tests) need no XR headers.
//
// Frame contract: Sync() runs exactly once per frame from the backend frame
// cycle, immediately after xrWaitFrame returned the predicted display time and
// before the guest renders, so located hand poses share the head/eye frame.
// There is no second frame wait and no polling from guest getters.
class XrActions final {
public:
XrActions() = default;
~XrActions();
XrActions(const XrActions &) = delete;
XrActions &operator=(const XrActions &) = delete;
// Creates the action set/actions and suggests bindings. Instance-level:
// valid before any session exists. Failure is non-fatal for XR output; the
// backend logs and continues without controller input.
Status Create(void *xr_instance);
// Attaches to a session and creates the per-hand pose spaces. Must be
// called after xrCreateSession and before the first Sync; recreated on every
// session replacement (no stale handles survive).
Status Attach(void *xr_instance, void *xr_session, void *xr_local_space);
// Destroys spaces and detaches; the action set survives for reattach.
void Detach();
void Destroy();
bool attached() const { return attached_; }
// One per-frame sync at predicted_display_time. Locates grip and aim poses
// in the session's LOCAL space, reads analog/boolean states, refreshes
// connection state from the runtime's current interaction profile, and
// builds the coherent snapshot + delivery batch. Returns false when actions
// are unavailable (runtime refused sync); input is reset and a disconnect
// remains drainable even while the session is detached.
bool Sync(void *xr_session, std::uint64_t predicted_display_time,
std::uint64_t frame, std::uint64_t session_generation);
// Latest coherent snapshot (frame/session generation tagged).
const ControllerSnapshot &snapshot() const { return snapshot_; }
// Hand poses in metres/LOCAL space for the state-copy records.
const VrHandPose &hand(int index) const { return hands_[index & 1]; }
// Drains the next pending delivery batch built by Sync. Returns false when
// nothing is queued. Thread-safe: Sync runs on the render thread, the drain
// runs on the window thread. Backed by a bounded queue, so a slow consumer
// drops oldest frames and converges via resync rather than growing.
bool TakeDelivery(ControllerDelivery *out);
// Queues a vibration request for one hand. Safe from any thread; applied on
// the next Sync's XR call window. amplitude [0,1], duration in nanoseconds
// (0 = runtime default). A newer request for the same hand replaces the
// pending one. StopHaptics cancels.
void RequestHaptics(int hand, float amplitude, std::uint64_t duration_ns,
float frequency_hz);
void StopHaptics(int hand);
// Interaction-profile diagnostics (refreshed on profile-change events and
// every Sync that observes a change).
std::string ActiveProfile(int hand) const;
// Test seam: publish an externally sourced profile name (event dispatch in
// the backend) without XR calls.
void NoteProfileChanged();
// Called by the backend on focus/session loss, under controller ownership.
void ResetInput();
private:
struct Impl;
Impl *impl_ = nullptr;
bool created_ = false;
bool attached_ = false;
ControllerStateBuilder builder_;
ControllerSnapshot snapshot_;
VrHandPose hands_[2];
ControllerDeliveryQueue delivery_queue_;
std::uint64_t session_generation_ = 0;
// Last connection level handed to the consumer, to mark connection changes.
bool last_delivered_connected_[2] = {false, false};
};
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_XR_ACTIONS_H_
+236
View File
@@ -0,0 +1,236 @@
#ifndef MOCKTAIL_VR_XR_CONTROLLER_H_
#define MOCKTAIL_VR_XR_CONTROLLER_H_
#include "mocktail/vr/xr_controller_abi.h"
#include <array>
#include <cstdint>
#include <deque>
#include <mutex>
#include <string>
#include <vector>
namespace mocktail::vr {
// One tracked controller hand, located at the same predicted display time as
// the head. Grip is the held-object pose; aim is the pointing pose. They are
// not interchangeable: the guest consumes grip for hands/tools and aim for UI
// rays. Position is metres, orientation a quaternion (x, y, z, w) in the
// OpenXR LOCAL reference space. The guest applies its own metres-to-studs
// conversion and hand pitch offset, so neither is pre-applied here.
struct VrHandPose {
float grip_position[3] = {0.f, 0.f, 0.f};
float grip_orientation[4] = {0.f, 0.f, 0.f, 1.f};
float aim_position[3] = {0.f, 0.f, 0.f};
float aim_orientation[4] = {0.f, 0.f, 0.f, 1.f};
// Pose validity, reported separately from button activity and connection so
// a controller can be connected with an untracked pose.
bool grip_valid = false;
bool aim_valid = false;
};
// Engine-facing controller capability, independent of which OpenXR
// interaction profile supplied it. The guest receives buttons and analog axes
// through its verified Android gamepad route, so this layer normalizes every
// supported profile onto that one stable contract.
enum class ControllerButton {
kTrigger, // analog click semantics; value reported separately
kSqueeze, // grip/squeeze
kThumbstick, // stick click
kFaceNorth, // right B / left Y
kFaceSouth, // right A / left X
kFaceEast, // right B equivalent on layouts that have it
kFaceWest, // left X equivalent
kMenu, // application-accessible menu only; never a system button
kCount,
};
// Capacitive contacts are distinct from presses. Some runtimes/profiles do
// not expose them; active=false means unavailable, not an inferred contact.
enum class ControllerTouch {
kTrigger,
kThumbstick,
kThumbrest,
kA,
kB,
kX,
kY,
kCount
};
enum class ControllerAxis {
kTrigger, // 0..1
kSqueeze, // 0..1
kThumbstickX, // -1..1
kThumbstickY, // -1..1
kCount,
};
// Analog trigger/squeeze thresholds used only where the runtime exposes a
// boolean click for an analog control. Hysteresis prevents edge chattering at
// the boundary; native analog values are always preserved alongside it.
struct AnalogClickThresholds {
float press = 0.75f;
float release = 0.6f;
};
// One coherent per-frame controller snapshot. Built once per frame by the
// backend after a single xrSyncActions, then read by native getters without
// re-polling. Button booleans are level state; edge detection happens once, at
// publish time, so repeated native getters never emit duplicate edges.
struct ControllerHandState {
bool connected = false;
bool pose_valid = false;
std::array<bool, static_cast<std::size_t>(ControllerTouch::kCount)> touches{};
std::array<bool, static_cast<std::size_t>(ControllerTouch::kCount)>
touch_active{};
std::array<bool, static_cast<std::size_t>(ControllerButton::kCount)>
buttons{};
std::array<float, static_cast<std::size_t>(ControllerAxis::kCount)> axes{};
// Interaction profile actually reported by the runtime for this hand, for
// diagnostics only (never used to gate input).
std::string profile;
};
struct ControllerSnapshot {
ControllerHandState hands[2]; // 0 = left, 1 = right
std::uint64_t frame = 0;
std::uint64_t session_generation = 0;
};
// Pure state machine: maps raw OpenXR action values onto the normalized
// contract and derives edges exactly once per published frame. No OpenXR or
// JNI types, so it is unit-testable with injected values.
class ControllerStateBuilder {
public:
explicit ControllerStateBuilder(AnalogClickThresholds thresholds = {});
// Records one raw action value for the given hand. Values not set by the
// runtime are neutral when explicitly marked inactive.
void SetTouch(int hand, ControllerTouch touch, bool value, bool active);
void SetButton(int hand, ControllerButton button, bool value, bool active);
void SetAxis(int hand, ControllerAxis axis, float value, bool active);
void SetConnected(int hand, bool connected);
void SetProfile(int hand, std::string profile);
void SetPoseValid(int hand, bool valid);
// Produces the snapshot for this frame and records which buttons changed so
// the caller can emit exactly one native edge per transition. frame must
// advance monotonically; publishing the same frame twice yields no edges.
struct Edge {
int hand = 0;
ControllerButton button = ControllerButton::kTrigger;
bool pressed = false;
};
ControllerSnapshot Publish(std::uint64_t frame, std::uint64_t generation,
std::vector<Edge> *edges);
// Neutralizes all axes and releases held buttons once (focus loss,
// disconnect, session stop). Produces release edges for anything held so no
// button can stay stuck down.
void ReleaseAll(std::uint64_t frame, std::uint64_t generation,
std::vector<Edge> *edges);
// Derives the click state for an analog control with hysteresis: latches on
// at `press`, off at `release`. Call after SetAxis for the same hand. Used
// for profiles that expose a trigger value without a click action; native
// analog values are always preserved regardless.
bool AnalogClick(int hand, ControllerAxis axis);
const ControllerSnapshot &snapshot() const { return snapshot_; }
private:
AnalogClickThresholds thresholds_;
ControllerSnapshot snapshot_;
// Last published level per hand/button, used for edge detection.
std::array<
std::array<bool, static_cast<std::size_t>(ControllerButton::kCount)>, 2>
published_buttons_{};
// Hysteresis latch per (hand, analog control): [hand*2 + axis].
std::array<bool, 4> analog_latch_{};
bool have_published_ = false;
};
// Maps the normalized contract onto the guest's verified Android gamepad
// identifiers, so XR controllers reuse the single authoritative input route
// and cannot double-fire alongside a physical SDL gamepad. hand selects the
// per-hand squeeze/thumbstick/trigger identifiers (0 = left, 1 = right).
// Returns -1 when the control has no keycode (analog click).
int32_t ControllerButtonToAndroidKey(int hand, ControllerButton button);
// Returns the Android MotionEvent axis id for one axis of the given hand
// (left/right trigger differ), or -1 when the control has no analog axis.
int32_t ControllerAxisToAndroidAxis(int hand, ControllerAxis axis);
// How many float components the guest expects for this axis (sticks 2, triggers
// 1).
int32_t ControllerAxisComponentCount(ControllerAxis axis);
// SDL3 gamepad ordinals. XR controller delivery reuses the existing router
// (RobloxInputRouter::HandleEvent) as the single authoritative event route:
// the router performs the verified SDL->Android key/axis conversion, level
// deduplication, focus gating and release-on-disconnect itself. Synthesized
// platform events must therefore carry SDL ordinals, not Android codes.
// hand selects the per-hand squeeze/thumbstick/trigger ordinals.
int32_t ControllerButtonToSdlOrdinal(int hand, ControllerButton button);
// SDL axis ordinal per hand for stick/trigger delivery; -1 when not an axis.
// Left hand: LEFTX/LEFTY/TRIGGERLEFT; right: RIGHTX/RIGHTY/TRIGGERRIGHT.
int32_t ControllerAxisToSdlOrdinal(int hand, ControllerAxis axis);
// One drained delivery batch from the XR backend to the input route. Built
// once per frame after xrSyncActions; edges appear exactly once. `resync`
// requests full level re-delivery (recovery after queue overflow or an
// unfocused drop) — the router deduplicates, so re-delivery is idempotent and
// guarantees no button sticks down.
struct ControllerDelivery {
std::uint64_t session_generation = 0;
bool button_levels[2][static_cast<std::size_t>(ControllerButton::kCount)] =
{};
bool connected[2] = {false, false};
bool connection_changed[2] = {false, false};
struct ButtonEdge {
int hand = 0;
ControllerButton button = ControllerButton::kTrigger;
bool pressed = false;
};
std::vector<ButtonEdge> edges;
// SDL ordinal order: LEFTX, LEFTY, RIGHTX, RIGHTY, TRIGGERLEFT, TRIGGERRIGHT.
float axes[6] = {0.f, 0.f, 0.f, 0.f, 0.f, 0.f};
bool axes_valid[2] = {false, false};
bool resync = false;
std::uint64_t frame = 0;
// Bounded growth guard: the backend caps the edge queue and sets resync
// instead of growing without limit.
static constexpr std::size_t kMaxEdges = 64;
};
bool EncodeNativeControllerChannels(const ControllerSnapshot &snapshot,
float (&channels)[28]);
void EncodeControllerDelivery(const ControllerDelivery &delivery,
MocktailVrControllerDelivery *out);
// Thread-safe bounded queue between the XR render-thread producer (one push
// per frame after xrSyncActions) and the window-thread consumer that drains
// into the verified input route. Overflow drops the oldest frames and arms a
// resync on the next delivered frame, so the consumer always converges to the
// current levels and a full queue can never leave a button stuck down.
class ControllerDeliveryQueue {
public:
static constexpr std::size_t kCapacity = 8;
void Push(ControllerDelivery delivery);
bool Pop(ControllerDelivery *out);
std::size_t size() const;
bool overflowed() const { return overflowed_; }
std::uint64_t dropped_frames() const { return dropped_frames_; }
void Clear();
private:
mutable std::mutex mutex_;
std::deque<ControllerDelivery> queue_;
bool overflowed_ = false;
std::uint64_t dropped_frames_ = 0;
};
} // namespace mocktail::vr
#endif // MOCKTAIL_VR_XR_CONTROLLER_H_
+76
View File
@@ -0,0 +1,76 @@
#ifndef MOCKTAIL_VR_XR_CONTROLLER_ABI_H_
#define MOCKTAIL_VR_XR_CONTROLLER_ABI_H_
#include <stddef.h>
#include <stdint.h>
/* Plain-C mirror of mocktail::vr::ControllerDelivery for crossing the
* dlsym module boundary (XR backend -> input runtime) without exporting C++
* types or requiring the consumer to link OpenXR. Field order and types are
* the ABI; both sides include this header. Keep it POD: no constructors, no
* std, fixed arrays only. */
#ifdef __cplusplus
extern "C" {
#endif
#define MOCKTAIL_VR_CONTROLLER_MAX_EDGES 64
#define MOCKTAIL_VR_CONTROLLER_BUTTON_COUNT 8
#define MOCKTAIL_VR_CONTROLLER_AXIS_COUNT 6
/* Button indices: mirror mocktail::vr::ControllerButton enum order. */
enum {
MOCKTAIL_VR_BUTTON_TRIGGER = 0,
MOCKTAIL_VR_BUTTON_SQUEEZE = 1,
MOCKTAIL_VR_BUTTON_THUMBSTICK = 2,
MOCKTAIL_VR_BUTTON_FACE_NORTH = 3,
MOCKTAIL_VR_BUTTON_FACE_SOUTH = 4,
MOCKTAIL_VR_BUTTON_FACE_EAST = 5,
MOCKTAIL_VR_BUTTON_FACE_WEST = 6,
MOCKTAIL_VR_BUTTON_MENU = 7
};
typedef struct MocktailVrControllerEdge {
int32_t hand; /* 0 = left, 1 = right */
int32_t button; /* MOCKTAIL_VR_BUTTON_* */
int32_t pressed; /* 0/1 */
} MocktailVrControllerEdge;
typedef struct MocktailVrControllerDelivery {
uint64_t frame;
uint64_t session_generation;
int32_t connected[2];
int32_t connection_changed[2];
/* Current button levels per hand (MOCKTAIL_VR_BUTTON_* order). Used for
* resync after queue overflow: the consumer re-delivers every level and the
* router's deduplication makes it idempotent. */
int32_t button_levels[2][MOCKTAIL_VR_CONTROLLER_BUTTON_COUNT];
/* Analog levels in SDL gamepad axis order:
* 0=LEFTX 1=LEFTY 2=RIGHTX 3=RIGHTY 4=TRIGGERLEFT 5=TRIGGERRIGHT. */
float axes[MOCKTAIL_VR_CONTROLLER_AXIS_COUNT];
int32_t axes_valid[2];
int32_t resync;
int32_t edge_count;
MocktailVrControllerEdge edges[MOCKTAIL_VR_CONTROLLER_MAX_EDGES];
} MocktailVrControllerDelivery;
/* Pops the next delivery batch produced by the XR backend's once-per-frame
* action sync. Returns 1 and fills *out when a batch was queued, 0 when the
* queue is empty or no XR controller layer is active. Inert (returns 0) in
* non-VR builds and while no backend is armed. */
typedef int32_t (*MocktailVrControllerPopFn)(MocktailVrControllerDelivery *out);
/* Queues a vibration request for one hand (0=left, 1=right). amplitude is
* clamped to [0,1]; duration_ns 0 means the runtime default. Safe from any
* thread; applied on the next XR sync. No-op when no controller layer is
* active. */
typedef void (*MocktailVrControllerHapticsFn)(int32_t hand, float amplitude,
uint64_t duration_ns,
float frequency_hz);
typedef void (*MocktailVrControllerStopHapticsFn)(int32_t hand);
#ifdef __cplusplus
}
#endif
#endif /* MOCKTAIL_VR_XR_CONTROLLER_ABI_H_ */
+2
View File
@@ -1,6 +1,7 @@
#ifndef MOCKTAIL_RUNTIME_COMMAND_LINE_H_
#define MOCKTAIL_RUNTIME_COMMAND_LINE_H_
#include <optional>
#include <string>
#include <vector>
@@ -21,6 +22,7 @@ enum class WindowMode {
struct CommandLineOptions {
CommandMode mode = CommandMode::kRun;
WindowMode window_mode = WindowMode::kUnspecified;
std::optional<bool> vr_enabled;
std::string program_name = "mocktail";
std::string roblox_library_path;
std::string graphics_backend;
+5
View File
@@ -64,6 +64,11 @@ bool MergeRuntimeClientSettingsOverrides(
std::string_view base_json, std::string* merged_json, std::string* error,
std::vector<std::string>* kept_overrides = nullptr);
// The resolved VR mode owns this flag in both client-settings channels.
bool MergeVrClientSettingsOverrides(bool enabled, std::string_view base_json,
std::string* merged_json,
std::string* error);
// Keeps the legacy test bypass disabled. The host PermissionsProtocol now
// answers microphone authorization from capture configuration. No permission
// query opens a recording device.
@@ -38,6 +38,16 @@ bool MergeDesktopAppPolicyClientSettingsOverride(std::string_view policy_json,
std::string* merged_json,
std::string* error);
// Apply after desktop policy composition. Roblox 2998's Android payload is not
// a VRAppBuild, so its LuaApp enables RunService's idle render throttle
// even with DebugDeviceVR active. Preserve the effective app-policy (the later
// fast-flags channel takes precedence) in both channels, changing only
// ThrottleFramerate. This does not write the setting to appStorage.
bool MergeVrAppPolicyClientSettingsOverrides(
bool enabled, std::string_view client_settings_json,
std::string_view fast_flags_json, std::string* merged_client_settings,
std::string* merged_fast_flags, std::string* error);
} // namespace runtime
} // namespace mocktail
@@ -19,6 +19,7 @@
#include "runtime/roblox_experience_launch_bridge.h"
#include "runtime/roblox_experience_presence.h"
#include "runtime/roblox_fresh_game_launch_controller.h"
#include "runtime/roblox_haptic_protocol_bridge.h"
#include "runtime/roblox_account_bridge.h"
#include "runtime/roblox_permissions_bridge.h"
#include "runtime/roblox_web_view_bridge.h"
@@ -260,6 +261,7 @@ class RobloxExperienceComposition final {
std::unique_ptr<RobloxBrowserServiceBridge> browser_service_bridge_;
std::unique_ptr<RobloxPermissionsBridge> permissions_bridge_;
std::unique_ptr<RobloxCallProtocolBridge> call_protocol_bridge_;
std::unique_ptr<RobloxHapticProtocolBridge> haptic_protocol_bridge_;
// Optional: a payload without the AccountProtocol entrypoints still starts.
std::unique_ptr<RobloxAccountBridge> account_bridge_;
std::shared_ptr<WebViewHelperProcess> web_surface_process_;
@@ -0,0 +1,47 @@
#ifndef MOCKTAIL_RUNTIME_ROBLOX_HAPTIC_PROTOCOL_BRIDGE_H_
#define MOCKTAIL_RUNTIME_ROBLOX_HAPTIC_PROTOCOL_BRIDGE_H_
#include <cstdint>
#include <functional>
#include <memory>
#include <string>
#include "runtime/roblox_message_bus_request_bridge.h"
namespace mocktail {
namespace runtime {
using HapticSinkFn = std::function<void(int hand, float amplitude,
uint64_t duration_ns, float frequency_hz)>;
using HapticStopFn = std::function<void(int hand)>;
using HapticStateSyncFn = std::function<void(JNIEnv* env)>;
class HapticWaveformPlayer;
class RobloxHapticProtocolBridge final {
public:
RobloxHapticProtocolBridge(
JniEnvironmentProvider environment,
RobloxMessageBusSymbols symbols,
RobloxMessageBusObjects objects,
HapticSinkFn haptic_sink = nullptr,
HapticStopFn haptic_stop = nullptr,
HapticStateSyncFn haptic_state_sync = nullptr);
~RobloxHapticProtocolBridge();
Status Initialize();
Status Shutdown();
static std::string GetMotorsJson();
// True during a VR session; main() sets MOCKTAIL_VR_ENABLED to 0 when VR
// cannot start, so the bridge stays off for desktop play.
static bool EnabledForThisSession();
private:
std::shared_ptr<HapticWaveformPlayer> player_;
std::unique_ptr<RobloxMessageBusRequestBridge> bridge_;
};
} // namespace runtime
} // namespace mocktail
#endif
+6
View File
@@ -89,6 +89,12 @@ struct RobloxTextDisplaySink {
UpdateFn update = nullptr;
};
// Process-wide second observer of the focused TextBox display, called
// synchronously after the surface overlay with the same sensitivity contract.
// The VR keyboard uses it to show the text being typed in the headset, where
// the desktop overlay is not visible. Passing an empty sink clears it.
void SetRobloxTextDisplayMirror(RobloxTextDisplaySink sink);
struct RobloxTextSink {
using SyncFn = Status (*)(void* context, const char* utf8, std::size_t size,
int32_t cursor_utf16);
@@ -8,6 +8,7 @@
#include "mocktail/status.h"
#include "runtime/roblox_input_native_adapter.h"
#include "runtime/roblox_text_surface_overlay.h"
#include "runtime/roblox_xr_controller_input.h"
namespace mocktail {
namespace runtime {
@@ -35,20 +36,40 @@ class RobloxWindowInputRuntime final {
Status QueryCurrentTextBoxInfo(RobloxNativeTextBoxInfoQueryResult* result);
Status UpdateTextFocusProperties(uint64_t generation,
const RobloxTextFocusProperties& properties);
// Feeds a synthetic platform event through the same authoritative router as
// real SDL events. Used by the XR controller delivery path so tracked
// controllers reuse the verified gamepad route (single event source, router
// deduplication and focus gating) instead of a second JNI input path.
void InjectPlatformEvent(const platform::PlatformEvent& event);
RobloxInputSnapshot Snapshot() const;
void SetRawMouseEnabled(bool enabled);
// Drains one batch of XR tracked-controller input into the same router.
// Called once per main-loop iteration on the input thread; inert in non-VR
// builds or while no OpenXR controller layer is active.
void DrainXrControllers();
// Routes an engine/experience vibration request to the active XR hand.
void RequestXrControllerHaptics(int hand, float amplitude,
std::uint64_t duration_ns, float frequency_hz);
void StopXrControllerHaptics(int hand);
private:
static void PlatformEventCallback(void* context,
const platform::PlatformEvent& event);
static bool MouseLockQueryCallback(void* context, bool* locked_center);
static void GamepadEventCallback(void* context,
const platform::PlatformEvent& event);
static void XrControllerEventCallback(void* context,
const platform::PlatformEvent& event);
RobloxTextSurfaceOverlay text_surface_overlay_;
std::unique_ptr<platform::TextClipboard> text_clipboard_;
RobloxInputRuntime runtime_;
platform::SdlGamepadManager gamepads_;
// XR tracked-controller producer. Feeds the same runtime_ router as SDL so
// there is exactly one authoritative input route.
RobloxXrControllerInput xr_controllers_;
bool xr_controllers_initialized_ = false;
bool observer_registered_ = false;
bool mouse_lock_query_registered_ = false;
bool initialized_ = false;
@@ -0,0 +1,100 @@
#ifndef MOCKTAIL_RUNTIME_ROBLOX_XR_CONTROLLER_INPUT_H_
#define MOCKTAIL_RUNTIME_ROBLOX_XR_CONTROLLER_INPUT_H_
#include <cstdint>
#include "mocktail/platform/platform_runtime.h"
#include "mocktail/status.h"
#include "mocktail/vr/xr_controller_abi.h"
namespace mocktail {
namespace runtime {
// Delivers OpenXR tracked-controller input through the one authoritative
// Roblox input route (RobloxInputRouter::HandleEvent), the same verified path
// physical SDL gamepads use. Because both sources converge on that single
// route, the router's own level deduplication, focus gating and
// release-on-disconnect logic apply unchanged and an XR controller can never
// double-fire alongside a physical gamepad.
//
// Both hands are merged into ONE synthetic gamepad: the left hand supplies
// LEFTX/LEFTY, the left shoulder (squeeze) and left stick click plus X/Y, the
// right hand supplies RIGHTX/RIGHTY, the right shoulder, right stick click and
// A/B. That matches how Roblox reads VR controller buttons through
// GamepadService, and it means the guest sees a single normal controller
// rather than two competing devices.
//
// The XR backend is resolved through dlsym so this module links and runs in
// non-VR builds and while no OpenXR backend is armed; then every drain is a
// no-op. No OpenXR headers are required here.
class RobloxXrControllerInput final {
public:
using EventSink = void (*)(void *context,
const platform::PlatformEvent &event);
// Synthetic device identity. Deliberately far outside the SDL joystick id
// range so it can never collide with a physical controller.
static constexpr std::int64_t kInstanceLeft =
static_cast<std::int64_t>(0x7FFFFFFFFFFFFFF0LL);
RobloxXrControllerInput() = default;
~RobloxXrControllerInput() = default;
RobloxXrControllerInput(const RobloxXrControllerInput &) = delete;
RobloxXrControllerInput &operator=(const RobloxXrControllerInput &) = delete;
// Resolves the XR controller ABI. Returns false (and stays inert) when the
// symbols are absent, which is the normal non-VR or no-backend case; that is
// not an error condition.
bool Initialize(EventSink sink, void *context);
// Test/injection seam: binds explicit ABI function pointers instead of
// resolving them through dlsym. A null pop makes the module inert, matching
// the non-VR case. Production callers use Initialize().
bool InitializeWithAbi(EventSink sink, void *context,
MocktailVrControllerPopFn pop,
MocktailVrControllerHapticsFn haptics,
MocktailVrControllerStopHapticsFn stop_haptics);
void Shutdown();
bool active() const { return pop_ != nullptr; }
// Drains every queued delivery and forwards it as platform events. Call once
// per main-loop iteration, after window::PumpEvents(), on the same thread
// that delivers SDL input. Bounded: drains at most kMaxDrainPerPump batches.
void Drain();
void ResendState() { force_resync_ = true; }
// Forwards a vibration request to the XR hand output. hand: 0 left, 1 right.
// No-op when the XR layer is inactive. amplitude is clamped by the backend.
void RequestHaptics(int hand, float amplitude, std::uint64_t duration_ns,
float frequency_hz);
void StopHaptics(int hand);
// Last observed state, for diagnostics/tests.
bool connected() const { return connected_; }
std::uint64_t delivered_events() const { return delivered_events_; }
std::uint64_t resyncs() const { return resyncs_; }
private:
static constexpr int kMaxDrainPerPump = 4;
void EmitConnection(bool connected);
void EmitButton(int sdl_button, bool pressed);
void EmitAxis(int sdl_axis, std::int16_t value);
void *context_ = nullptr;
EventSink sink_ = nullptr;
void *pop_ = nullptr;
void *haptics_ = nullptr;
void *stop_haptics_ = nullptr;
bool force_resync_ = false;
std::uint64_t generation_ = 0;
bool connected_ = false;
bool initialized_ = false;
std::uint64_t delivered_events_ = 0;
std::uint64_t resyncs_ = 0;
// Last delivered button levels, for resync and to avoid redundant events.
bool last_buttons_[16] = {};
};
} // namespace runtime
} // namespace mocktail
#endif // MOCKTAIL_RUNTIME_ROBLOX_XR_CONTROLLER_INPUT_H_
+27
View File
@@ -16,6 +16,15 @@
namespace mocktail {
namespace runtime {
// VR is opt-in in every build (--vr or vr.enabled: true). A VR-capable build
// must not try to reach an OpenXR runtime on a normal desktop launch.
inline constexpr bool kDefaultVrEnabled = false;
// vr.mode: on and off force VR or the desktop. auto starts in VR only when
// SteamVR is running with a headset, which main() checks before launch.
enum class VrMode { kOff, kOn, kAuto };
const char* VrModeName(VrMode mode);
enum class GraphicsBackend {
kAuto,
kSystem,
@@ -96,6 +105,10 @@ class RuntimeConfig {
static RuntimeConfig FromEnvironment(const Environment& environment);
bool headless() const { return headless_; }
bool vr_enabled() const { return vr_enabled_; }
// MOCKTAIL_VR_ENABLED (--vr, --no-vr) wins over MOCKTAIL_VR_MODE.
VrMode vr_mode() const { return vr_mode_; }
bool vr_valid() const { return vr_valid_; }
const std::filesystem::path& roblox_library_path() const {
return roblox_library_path_;
}
@@ -121,6 +134,15 @@ class RuntimeConfig {
return roblox_http_user_agent_;
}
const FrameRatePolicy& frame_rate() const { return frame_rate_; }
// Replaces frame_rate() while VR is enabled. Defaults to unlimited so the
// OpenXR runtime's xrWaitFrame paces Roblox at the headset refresh rate.
const FrameRatePolicy& vr_frame_rate() const { return vr_frame_rate_; }
// vr.anti_aliasing: fxaa, smaa or off. The VR backend reads it from the
// exported MOCKTAIL_VR_ANTI_ALIASING.
const std::string& vr_anti_aliasing() const { return vr_anti_aliasing_; }
const FrameRatePolicy& active_frame_rate() const {
return vr_enabled_ ? vr_frame_rate_ : frame_rate_;
}
const std::string& vsync_mode() const { return vsync_mode_; }
const PerformancePolicy& performance() const { return performance_; }
const std::string& audio_output_device() const {
@@ -161,6 +183,9 @@ class RuntimeConfig {
private:
bool headless_ = false;
bool vr_enabled_ = kDefaultVrEnabled;
VrMode vr_mode_ = kDefaultVrEnabled ? VrMode::kOn : VrMode::kOff;
bool vr_valid_ = true;
std::filesystem::path roblox_library_path_ = "rbx_bin/libroblox.so";
GraphicsBackend graphics_backend_ = GraphicsBackend::kVulkan;
std::string graphics_backend_name_ = "direct-vulkan";
@@ -171,6 +196,8 @@ class RuntimeConfig {
bool device_profile_valid_ = true;
std::optional<std::string> roblox_http_user_agent_;
FrameRatePolicy frame_rate_;
FrameRatePolicy vr_frame_rate_{FrameRateLimitMode::kUnlimited, 0};
std::string vr_anti_aliasing_ = "fxaa";
std::string vsync_mode_ = "auto";
PerformancePolicy performance_;
std::string audio_output_device_ = "default";
+1609
View File
File diff suppressed because it is too large. Load diff
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""Regenerate src/vr/vr_shaders_spirv.h from shaders/vr with glslc.
The header is checked in so a build never needs a shader compiler. The SMAA
passes include SMAA.hlsl from the third_party/smaa submodule. Run this
after editing a shader; VrShaders.SpirvMatchesSources checks it when glslc is
installed.
"""
import argparse
import pathlib
import struct
import subprocess
import sys
import tempfile
ROOT = pathlib.Path(__file__).resolve().parent.parent
SHADERS = [
("kFullscreenVertexSpirv", "shaders/vr/fullscreen.vert"),
("kFxaaFragmentSpirv", "shaders/vr/fxaa.frag"),
("kSmaaEdgesVertexSpirv", "shaders/vr/smaa_edges.vert"),
("kSmaaEdgesFragmentSpirv", "shaders/vr/smaa_edges.frag"),
("kSmaaWeightsVertexSpirv", "shaders/vr/smaa_weights.vert"),
("kSmaaWeightsFragmentSpirv", "shaders/vr/smaa_weights.frag"),
("kSmaaBlendVertexSpirv", "shaders/vr/smaa_blend.vert"),
("kSmaaBlendFragmentSpirv", "shaders/vr/smaa_blend.frag"),
]
# SMAA.hlsl comes from the third_party/smaa submodule.
INCLUDES = [ROOT / "third_party/smaa"]
OUTPUT = ROOT / "src/vr/vr_shaders_spirv.h"
def compile_words(glslc, source):
with tempfile.TemporaryDirectory() as directory:
output = pathlib.Path(directory) / "shader.spv"
subprocess.run(
[glslc, "-O", "--target-env=vulkan1.0"] +
["-I" + str(path) for path in INCLUDES] +
[str(ROOT / source), "-o", str(output)],
check=True)
data = output.read_bytes()
return struct.unpack("<%dI" % (len(data) // 4), data)
def render(glslc):
lines = [
"// Generated by scripts/generate_vr_shaders.py from shaders/vr. Do not edit.",
"#ifndef MOCKTAIL_VR_VR_SHADERS_SPIRV_H_",
"#define MOCKTAIL_VR_VR_SHADERS_SPIRV_H_",
"",
"#include <cstdint>",
"",
"namespace mocktail::vr {",
"",
]
for name, source in SHADERS:
words = compile_words(glslc, source)
lines.append("// %s" % source)
lines.append("inline constexpr std::uint32_t %s[] = {" % name)
for start in range(0, len(words), 6):
chunk = ", ".join("0x%08x" % word for word in words[start:start + 6])
lines.append(" %s," % chunk)
lines.append("};")
lines.append("")
lines += ["} // namespace mocktail::vr", "",
"#endif // MOCKTAIL_VR_VR_SHADERS_SPIRV_H_", ""]
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--glslc", default="glslc")
parser.add_argument("--check", action="store_true",
help="exit 1 when the header is out of date")
arguments = parser.parse_args()
text = render(arguments.glslc)
if arguments.check:
current = OUTPUT.read_text() if OUTPUT.exists() else ""
if current != text:
print("%s is out of date; run %s" % (OUTPUT, __file__),
file=sys.stderr)
return 1
return 0
OUTPUT.write_text(text)
return 0
if __name__ == "__main__":
sys.exit(main())
+65 -5
View File
@@ -23,6 +23,10 @@ Usage:
Prints every field where the two disagree. Exit status: 0 agree, 1 differ or
one side failed, 2 both failed.
With --vr, the VR device bridge is compared too. `find_roblox_vr_offsets.py`
finds it in each build on its own: its answer for the reference build is the
solver's input, and its answer for the candidate joins the deriver's side.
"""
from __future__ import annotations
@@ -34,25 +38,73 @@ import sys
from typing import Any, Sequence
def _load_solver():
name = "mocktail_roblox_abi_solver"
def _load_sibling(name: str, file_name: str):
if name in sys.modules:
return sys.modules[name]
path = Path(__file__).resolve().with_name("roblox_abi_solver.py")
path = Path(__file__).resolve().with_name(file_name)
specification = importlib.util.spec_from_file_location(name, path)
if specification is None or specification.loader is None:
raise RuntimeError("cannot load roblox_abi_solver.py")
raise RuntimeError(f"cannot load {file_name}")
module = importlib.util.module_from_spec(specification)
sys.modules[name] = module
specification.loader.exec_module(module)
return module
SOLVER = _load_solver()
SOLVER = _load_sibling("mocktail_roblox_abi_solver", "roblox_abi_solver.py")
FINDER = _load_sibling("mocktail_find_roblox_vr_offsets", "find_roblox_vr_offsets.py")
DERIVER = SOLVER.DERIVER
AnalyzerError = SOLVER.AnalyzerError
format_rva = SOLVER.format_rva
# Where find_roblox_vr_offsets.py reports each VR bridge field.
_FINDER_FIELDS = {
"vtable_rva": ("vr_debug_device_bridge", "vtable_rva"),
"constructor_rva": ("vr_debug_device_bridge", "constructor_rva"),
"state_getter_rva": ("vr_debug_device_bridge", "state_getter_rva"),
"eye_getter_rva": ("vr_debug_device_bridge", "eye_getter_rva"),
"eye_initializer_rva": ("vr_debug_device_bridge", "eye_initializer_rva"),
"emulator_flag_storage_rva": ("vr_debug_device_bridge", "emulator_flag_storage_rva"),
"device_create_framebuffer_vtable_offset": (
"vr_debug_device_bridge", "device_create_framebuffer_vtable_offset"
),
"state_memcpy_rva": ("vr_bridge_supplement", "state_getter_memcpy_rva"),
"pointer_frame_rva": ("vr_bridge_supplement", "world_pointer_getter_rva"),
"workspace_getter_rva": ("vr_bridge_supplement", "workspace_getter_rva"),
"camera_vtable_offset": ("vr_bridge_supplement", "workspace_camera_vtable_offset"),
"hand_pitch_storage_rva": ("vr_bridge_supplement", "new_vr_system_hand_pitch_storage_rva"),
"haptic_sink_rva": ("vr_bridge_supplement", "haptic_sink_rva"),
"pointer_hand_selector_offset": ("vr_bridge_supplement", "usercframe_selector_offset"),
"camera_head_scale_offset": ("vr_bridge_supplement", "camera_head_scale_offset"),
"keep_haptics_flag_rva": ("vr_bridge_supplement", "keep_haptics_flag_rva"),
"vr_haptics_flag_rva": ("vr_bridge_supplement", "vr_haptics_flag_rva"),
"support_generic_haptics_flag_rva": (
"vr_bridge_supplement", "support_generic_haptics_flag_rva"
),
"fix_register_motors_flag_rva": ("vr_bridge_supplement", "fix_register_motors_flag_rva"),
}
def vr_bridge_from_finder(
result: dict[str, Any], fields: Sequence[str] = SOLVER.VR.VR_FIELDS
) -> dict[str, str]:
"""The VR bridge fields from a find_roblox_vr_offsets.py result."""
bridge = {}
for field in fields:
section, key = _FINDER_FIELDS[field]
try:
bridge[field] = result[section][key]
except (KeyError, TypeError) as error:
raise AnalyzerError(f"VR offset finder result lacks {key}") from error
return bridge
def _find_vr_bridge(library: Path) -> dict[str, str]:
try:
return vr_bridge_from_finder(FINDER.find_vr_offsets(library))
except FINDER.AnalyzerError as error:
raise AnalyzerError(f"VR offset finder: {error}") from error
def compare_results(derived: Any, solved: Any, path: str = "") -> list[str]:
"""Returns one line for every leaf value where the two results differ."""
@@ -121,6 +173,7 @@ def check(arguments: argparse.Namespace) -> int:
manifests = [Path(path) for path in arguments.reference_compatibility]
cache_dir = Path(arguments.cache_dir) if arguments.cache_dir else None
derived = solved = deriver_error = solver_error = None
reference_vr = _find_vr_bridge(Path(arguments.reference_lib)) if arguments.vr else None
with DERIVER.ElfImage(Path(arguments.reference_lib)) as reference, DERIVER.ElfImage(
Path(arguments.candidate_lib)
) as candidate:
@@ -128,6 +181,8 @@ def check(arguments: argparse.Namespace) -> int:
profile, anchors = DERIVER.derive_profile(reference, candidate, sidecar)
runtime = DERIVER.derive_runtime_compatibility(reference, candidate, manifests)
derived = {"profile": profile, "derivation_anchors": anchors, "runtime": runtime}
if reference_vr is not None:
derived["vr_device_bridge"] = _find_vr_bridge(Path(arguments.candidate_lib))
except Exception as error:
deriver_error = _describe(error)
try:
@@ -138,6 +193,8 @@ def check(arguments: argparse.Namespace) -> int:
source = DERIVER.load_reference_runtime_profile(manifests, reference.build_id)
runtime = solver.derive_runtime_compatibility(_solver_runtime_source(source))
solved = {"profile": profile, "derivation_anchors": anchors, "runtime": runtime}
if reference_vr is not None:
solved["vr_device_bridge"] = solver.derive_vr(reference_vr)
except Exception as error:
solver_error = _describe(error)
status, lines = check_status(derived, solved, deriver_error, solver_error)
@@ -161,6 +218,9 @@ def main(argv: Sequence[str] | None = None) -> int:
default=str(Path.home() / ".cache" / "mocktail" / "function_index"),
help="where function indexes are kept; empty to disable",
)
parser.add_argument(
"--vr", action="store_true", help="also compare the VR device bridge"
)
arguments = parser.parse_args(sys.argv[1:] if argv is None else argv)
try:
return check(arguments)
+4 -3
View File
@@ -27,8 +27,8 @@ reference function is matched through independent clues:
A match is accepted only with a score of at least ACCEPT_SCORE and a lead of
at least ACCEPT_LEAD over the runner-up. `roblox_abi_targets.py` turns the
matches into a profile with the same shape as the one
`derive_roblox_host_abi_profile.py` writes, and `roblox_abi_check.py`
compares the two on one update.
`derive_roblox_host_abi_profile.py` writes, `roblox_abi_vr.py` into the VR
device bridge, and `roblox_abi_check.py` compares the two on one update.
"""
from __future__ import annotations
@@ -58,6 +58,7 @@ def _load_sibling(name: str, file_name: str):
INDEX = _load_sibling("mocktail_roblox_function_index", "roblox_function_index.py")
TARGETS = _load_sibling("mocktail_roblox_abi_targets", "roblox_abi_targets.py")
VR = _load_sibling("mocktail_roblox_abi_vr", "roblox_abi_vr.py")
DERIVER = INDEX.DERIVER
FunctionIndex = INDEX.FunctionIndex
Instruction = INDEX.Instruction
@@ -136,7 +137,7 @@ def _weighted(parts: Sequence[tuple[float | None, float]]) -> float:
)
class Solver(TARGETS.TargetDerivation):
class Solver(TARGETS.TargetDerivation, VR.VrTargets):
def __init__(self, reference: Any, candidate: Any, cache_dir: Path | None = None):
self._start(FunctionIndex(reference, cache_dir), FunctionIndex(candidate, cache_dir))
+262
View File
@@ -0,0 +1,262 @@
#!/usr/bin/env python3
# Copyright 2026 Mocktail Project Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""VR device bridge targets that `roblox_abi_solver.py` derives.
`VrTargets.derive_vr` takes one build's VR bridge (the `VrBuildContract`
fields in `src/vr/roblox_vr_device_bridge.cc`, as hex strings) and returns
the same fields for the candidate build.
"""
from __future__ import annotations
import importlib.util
from pathlib import Path
import re
import sys
from typing import Any
def _load_index():
name = "mocktail_roblox_function_index"
if name in sys.modules:
return sys.modules[name]
path = Path(__file__).resolve().with_name("roblox_function_index.py")
specification = importlib.util.spec_from_file_location(name, path)
if specification is None or specification.loader is None:
raise RuntimeError("cannot load roblox_function_index.py")
module = importlib.util.module_from_spec(specification)
sys.modules[name] = module
specification.loader.exec_module(module)
return module
DERIVER = _load_index().DERIVER
AnalyzerError = DERIVER.AnalyzerError
format_rva = DERIVER.format_rva
VR_FIELDS = (
"vtable_rva",
"constructor_rva",
"state_getter_rva",
"eye_getter_rva",
"eye_initializer_rva",
"emulator_flag_storage_rva",
"device_create_framebuffer_vtable_offset",
"state_memcpy_rva",
"pointer_frame_rva",
"workspace_getter_rva",
"camera_vtable_offset",
"hand_pitch_storage_rva",
"haptic_sink_rva",
"pointer_hand_selector_offset",
"camera_head_scale_offset",
"keep_haptics_flag_rva",
"vr_haptics_flag_rva",
"support_generic_haptics_flag_rva",
"fix_register_motors_flag_rva",
)
# Fast flag and setting storage, found through the functions that use it.
_STORAGE_FIELDS = (
"emulator_flag_storage_rva",
"hand_pitch_storage_rva",
"keep_haptics_flag_rva",
"vr_haptics_flag_rva",
"support_generic_haptics_flag_rva",
"fix_register_motors_flag_rva",
)
# Storage users tried, smallest first; registration functions are small.
_STORAGE_USER_LIMIT = 8
_VTABLE_SLOT_LIMIT = 0x400
# The constructor stores the vtable address plus the Itanium header.
_VTABLE_REFERENCE_SPAN = 0x40
_NUMBER = r"(0x[0-9a-f]+|\d+)"
_SLOT_LOAD = re.compile(r"^mov (\w+), qword ptr \[\1 \+ ")
# How far back the load of the vtable pointer itself may sit.
_VTABLE_LOAD_WINDOW = 6
def _vtable_call(instructions: list[Any], index: int) -> bool:
"""A call through a vtable slot, or a load of the slot from a vtable
pointer loaded just before into the same register."""
shape = instructions[index].shape
if shape.startswith("call qword ptr ["):
return True
load = _SLOT_LOAD.match(shape)
if load is None:
return False
vtable_load = re.compile(rf"^mov {load.group(1)}, qword ptr \[\w+\]$")
return any(
vtable_load.match(previous.shape)
for previous in instructions[max(0, index - _VTABLE_LOAD_WINDOW) : index]
)
class VrTargets:
"""Mixin for `Solver`: needs `reference`, `candidate`, `matches`,
`match_function`, `map_data` and `_aligned`."""
def map_operand(
self, name: str, function_name: str, start: int, value: int, vtable_call: bool = False
) -> int:
"""Maps a field offset that a matched function uses as a displacement.
Stack-relative uses are ignored; with `vtable_call`, so is every use
that is not a virtual call. An instruction whose shape changed is
looked for between its nearest aligned neighbours.
"""
reference, candidate, mapping = self._aligned(function_name, start)
token = re.compile(rf"(?<![0-9a-fx]){re.escape(hex(value))}(?![0-9a-f])")
found = set()
for index, item in enumerate(reference):
if not token.search(item.shape) or "rsp" in item.shape or "rbp" in item.shape:
continue
if vtable_call and not _vtable_call(reference, index):
continue
pattern = re.compile(
"^" + token.sub(lambda _: _NUMBER, re.escape(item.shape)) + "$"
)
if index in mapping:
window = range(mapping[index], mapping[index] + 1)
else:
below = max((k for k in mapping if k < index), default=None)
above = min((k for k in mapping if k > index), default=None)
low = 0 if below is None else mapping[below] + 1
high = len(candidate) if above is None else mapping[above]
window = range(low, high)
values = []
for position in window:
if vtable_call and not _vtable_call(candidate, position):
continue
matched = pattern.match(candidate[position].shape)
if matched is not None and len({*matched.groups()}) == 1:
values.append(int(matched.group(1), 0))
if len(values) != 1:
raise AnalyzerError(f"{name} has {len(values)} candidate offsets")
found.add(values[0])
if len(found) != 1:
raise AnalyzerError(f"{name} maps to {len(found)} offsets")
return found.pop()
def map_call_target(self, name: str, function_name: str, start: int, target: int) -> int:
"""Maps the target of a direct call inside a matched function."""
reference, candidate, mapping = self._aligned(function_name, start)
found = set()
for index, item in enumerate(reference):
if item.shape != "call" or item.branch_target != target:
continue
if index not in mapping:
raise AnalyzerError(f"{name} call has no counterpart")
found.add(candidate[mapping[index]].branch_target)
if len(found) != 1 or None in found:
raise AnalyzerError(f"{name} has no unique call target")
return found.pop()
def _map_storage(self, name: str, target: int) -> int:
users = sorted(
self.reference.data_users.get(target, ()),
key=lambda start: self.reference.functions[start].size,
)[:_STORAGE_USER_LIMIT]
found = set()
for user in users:
user_name = f"{name} user {format_rva(user)}"
try:
self.match_function(user_name, user)
found.add(self.map_data(name, user_name, user, target))
except AnalyzerError:
continue
if len(found) != 1:
raise AnalyzerError(f"{name} maps to {len(found)} addresses")
return found.pop()
def _map_vtable(self, constructor: int, vtable: int) -> int:
found = set()
for item in self.reference.instructions(constructor):
for target in item.rip_targets:
if 0 <= target - vtable < _VTABLE_REFERENCE_SPAN:
mapped = self.map_data("VR vtable", "vr-constructor", constructor, target)
found.add(mapped - (target - vtable))
if len(found) != 1:
raise AnalyzerError(f"VR vtable maps to {len(found)} addresses")
return found.pop()
@staticmethod
def _slot(relocations: dict[int, int], vtable: int, method: int) -> int:
slots = [
offset
for offset in range(0, _VTABLE_SLOT_LIMIT, 8)
if relocations.get(vtable + offset) == method
]
if not slots:
raise AnalyzerError(f"reference VR vtable does not hold {format_rva(method)}")
return slots[0]
def derive_vr(self, source: dict[str, str]) -> dict[str, str]:
missing = [field for field in VR_FIELDS if field not in source]
if missing:
raise AnalyzerError("reference VR bridge lacks " + ", ".join(missing))
value = {field: DERIVER.parse_rva(source[field], field) for field in VR_FIELDS}
result = {}
constructor = self.match_function("vr-constructor", value["constructor_rva"])
result["constructor_rva"] = constructor
vtable = self._map_vtable(value["constructor_rva"], value["vtable_rva"])
result["vtable_rva"] = vtable
reference_relocations = DERIVER.relative_relocation_map(self.reference.image)
relocations = DERIVER.relative_relocation_map(self.candidate.image)
slots = {
field: self._slot(reference_relocations, value["vtable_rva"], value[field])
for field in (
"state_getter_rva", "eye_getter_rva", "eye_initializer_rva", "haptic_sink_rva"
)
}
self.candidate.image.require_relro_rva(vtable, max(slots.values()) + 8)
for field in ("state_getter_rva", "eye_getter_rva", "eye_initializer_rva"):
result[field] = self.match_function(
f"vr-{field}", value[field], vtable_slot=relocations.get(vtable + slots[field])
)
# The haptic sink is a one-byte ret stub, so only its slot identifies it.
sink = relocations.get(vtable + slots["haptic_sink_rva"])
if sink is None or sink not in self.candidate.functions or (
self.candidate.functions[sink].size != 1
):
raise AnalyzerError("VR haptic sink slot does not hold a ret stub")
result["haptic_sink_rva"] = sink
result["state_memcpy_rva"] = self.map_call_target(
"VR state memcpy", "vr-state_getter_rva",
value["state_getter_rva"], value["state_memcpy_rva"],
)
result["device_create_framebuffer_vtable_offset"] = self.map_operand(
"VR framebuffer slot", "vr-eye_initializer_rva",
value["eye_initializer_rva"], value["device_create_framebuffer_vtable_offset"],
vtable_call=True,
)
result["pointer_frame_rva"] = self.match_function(
"vr-pointer-frame", value["pointer_frame_rva"]
)
result["workspace_getter_rva"] = self.map_call_target(
"VR workspace getter", "vr-pointer-frame",
value["pointer_frame_rva"], value["workspace_getter_rva"],
)
for field in (
"camera_vtable_offset", "pointer_hand_selector_offset", "camera_head_scale_offset"
):
result[field] = self.map_operand(
field, "vr-pointer-frame", value["pointer_frame_rva"], value[field],
vtable_call=field == "camera_vtable_offset",
)
for field in _STORAGE_FIELDS:
result[field] = self._map_storage(field, value[field])
self.candidate.image.require_writable_rva(result[field], 1)
return {field: format_rva(result[field]) for field in VR_FIELDS}
+8
View File
@@ -0,0 +1,8 @@
#version 450
// One triangle that covers the viewport; uv spans 0..1 over the target.
layout(location = 0) out vec2 uv;
void main() {
uv = vec2((gl_VertexIndex << 1) & 2, gl_VertexIndex & 2);
gl_Position = vec4(uv * 2.0 - 1.0, 0.0, 1.0);
}
+59
View File
@@ -0,0 +1,59 @@
#version 450
// FXAA for one VR eye, after Timothy Lottes' FXAA (console variant). The eye
// holds sRGB-encoded bytes, so luma is computed on perceptual values, which
// is what FXAA expects.
layout(set = 0, binding = 0) uniform sampler2D source;
layout(push_constant) uniform Push {
vec2 texel; // 1 / source extent
uint linearize; // 1 when the target view re-encodes to sRGB
} push;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 color;
const float kEdgeThreshold = 1.0 / 8.0;
const float kEdgeThresholdMin = 1.0 / 24.0;
const float kReduceMul = 1.0 / 8.0;
const float kReduceMin = 1.0 / 128.0;
const float kSpanMax = 8.0;
float Luma(vec3 rgb) { return dot(rgb, vec3(0.299, 0.587, 0.114)); }
vec3 Sample(vec2 at) { return textureLod(source, at, 0.0).rgb; }
// The target encodes linear values back to sRGB, which restores these bytes.
vec3 Linearize(vec3 c) {
return mix(c / 12.92, pow((c + 0.055) / 1.055, vec3(2.4)),
step(vec3(0.04045), c));
}
void main() {
const vec4 center = textureLod(source, uv, 0.0);
const float lumaM = Luma(center.rgb);
const float lumaNW = Luma(Sample(uv + vec2(-1.0, -1.0) * push.texel));
const float lumaNE = Luma(Sample(uv + vec2(1.0, -1.0) * push.texel));
const float lumaSW = Luma(Sample(uv + vec2(-1.0, 1.0) * push.texel));
const float lumaSE = Luma(Sample(uv + vec2(1.0, 1.0) * push.texel));
const float lumaMin =
min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
const float lumaMax =
max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
vec3 rgb = center.rgb;
// Flat areas, where nothing aliases, keep their exact pixels.
if (lumaMax - lumaMin >= max(kEdgeThresholdMin, lumaMax * kEdgeThreshold)) {
vec2 dir = vec2(-((lumaNW + lumaNE) - (lumaSW + lumaSE)),
(lumaNW + lumaSW) - (lumaNE + lumaSE));
const float reduce = max((lumaNW + lumaNE + lumaSW + lumaSE) *
(0.25 * kReduceMul),
kReduceMin);
const float rcpDirMin = 1.0 / (min(abs(dir.x), abs(dir.y)) + reduce);
dir = clamp(dir * rcpDirMin, vec2(-kSpanMax), vec2(kSpanMax)) * push.texel;
const vec3 rgbA = 0.5 * (Sample(uv + dir * (1.0 / 3.0 - 0.5)) +
Sample(uv + dir * (2.0 / 3.0 - 0.5)));
const vec3 rgbB =
rgbA * 0.5 + 0.25 * (Sample(uv + dir * -0.5) + Sample(uv + dir * 0.5));
const float lumaB = Luma(rgbB);
rgb = (lumaB < lumaMin || lumaB > lumaMax) ? rgbA : rgbB;
}
color = vec4(push.linearize != 0u ? Linearize(rgb) : rgb, center.a);
}
+21
View File
@@ -0,0 +1,21 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_VS 0
#include "SMAA.hlsl"
layout(set = 0, binding = 0) uniform sampler2D colorTex;
layout(set = 0, binding = 1) uniform sampler2D blendTex;
layout(location = 0) in vec2 texcoord;
layout(location = 1) in vec4 offset;
layout(location = 0) out vec4 color;
// The target encodes linear values back to sRGB, which restores these bytes.
vec3 Linearize(vec3 c) {
return mix(c / 12.92, pow((c + 0.055) / 1.055, vec3(2.4)),
step(vec3(0.04045), c));
}
void main() {
color = SMAANeighborhoodBlendingPS(texcoord, offset, colorTex, blendTex);
if (push.linearize != 0u) color.rgb = Linearize(color.rgb);
}
+13
View File
@@ -0,0 +1,13 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_PS 0
#include "SMAA.hlsl"
layout(location = 0) out vec2 texcoord;
layout(location = 1) out vec4 offset;
void main() {
texcoord = vec2((gl_VertexIndex << 1) & 2, gl_VertexIndex & 2);
gl_Position = vec4(texcoord * 2.0 - 1.0, 0.0, 1.0);
SMAANeighborhoodBlendingVS(texcoord, offset);
}
+11
View File
@@ -0,0 +1,11 @@
// Shared setup for the SMAA passes: the upstream SMAA.hlsl (third_party/smaa)
// in its GLSL 4 mode at the High preset. Vulkan, like Direct3D, puts the
// texture origin at the top left, which is what SMAA and its lookup textures
// expect, so nothing is flipped.
layout(push_constant) uniform Push {
vec4 metrics; // 1/width, 1/height, width, height
uint linearize; // 1 when the target view re-encodes to sRGB
} push;
#define SMAA_RT_METRICS push.metrics
#define SMAA_GLSL_4 1
#define SMAA_PRESET_HIGH 1
+12
View File
@@ -0,0 +1,12 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_VS 0
#include "SMAA.hlsl"
// Luma edges on the eye's sRGB-encoded bytes, as SMAA recommends.
layout(set = 0, binding = 0) uniform sampler2D colorTex;
layout(location = 0) in vec2 texcoord;
layout(location = 1) in vec4 offset[3];
layout(location = 0) out vec2 edges;
void main() { edges = SMAALumaEdgeDetectionPS(texcoord, offset, colorTex); }
+13
View File
@@ -0,0 +1,13 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_PS 0
#include "SMAA.hlsl"
layout(location = 0) out vec2 texcoord;
layout(location = 1) out vec4 offset[3];
void main() {
texcoord = vec2((gl_VertexIndex << 1) & 2, gl_VertexIndex & 2);
gl_Position = vec4(texcoord * 2.0 - 1.0, 0.0, 1.0);
SMAAEdgeDetectionVS(texcoord, offset);
}
+17
View File
@@ -0,0 +1,17 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_VS 0
#include "SMAA.hlsl"
layout(set = 0, binding = 0) uniform sampler2D edgesTex;
layout(set = 0, binding = 1) uniform sampler2D areaTex;
layout(set = 0, binding = 2) uniform sampler2D searchTex;
layout(location = 0) in vec2 texcoord;
layout(location = 1) in vec2 pixcoord;
layout(location = 2) in vec4 offset[3];
layout(location = 0) out vec4 weights;
void main() {
weights = SMAABlendingWeightCalculationPS(texcoord, pixcoord, offset, edgesTex,
areaTex, searchTex, vec4(0.0));
}
+14
View File
@@ -0,0 +1,14 @@
#version 450
#extension GL_GOOGLE_include_directive : require
#include "smaa_common.glsl"
#define SMAA_INCLUDE_PS 0
#include "SMAA.hlsl"
layout(location = 0) out vec2 texcoord;
layout(location = 1) out vec2 pixcoord;
layout(location = 2) out vec4 offset[3];
void main() {
texcoord = vec2((gl_VertexIndex << 1) & 2, gl_VertexIndex & 2);
gl_Position = vec4(texcoord * 2.0 - 1.0, 0.0, 1.0);
SMAABlendingWeightCalculationVS(texcoord, pixcoord, offset);
}
+55 -1
View File
@@ -148,6 +148,57 @@ bool ParseOptionalFmodOutputDeviceBridge(
return input_fields_present == 0 || input_fields_present == 4;
}
bool ParseOptionalVrDebugDeviceBridge(
const Json& object,
std::optional<VrDebugDeviceBridgeProfile>* bridge_profile) {
if (bridge_profile == nullptr) {
return false;
}
bridge_profile->reset();
const auto field = object.find("vr_debug_device_bridge");
if (field == object.end() || field->is_null()) {
return true;
}
if (!field->is_object()) {
return false;
}
std::optional<std::uintptr_t> vtable_rva;
std::optional<std::uintptr_t> constructor_rva;
std::optional<std::uintptr_t> state_getter_rva;
std::optional<std::uintptr_t> eye_getter_rva;
std::optional<std::uintptr_t> eye_initializer_rva;
std::optional<std::uintptr_t> emulator_flag_storage_rva;
std::optional<std::uintptr_t> device_create_framebuffer_vtable_offset;
if (!ParseOptionalRva(*field, "vtable_rva", &vtable_rva) ||
!ParseOptionalRva(*field, "constructor_rva", &constructor_rva) ||
!ParseOptionalRva(*field, "state_getter_rva", &state_getter_rva) ||
!ParseOptionalRva(*field, "eye_getter_rva", &eye_getter_rva) ||
!ParseOptionalRva(*field, "eye_initializer_rva", &eye_initializer_rva) ||
!ParseOptionalRva(*field, "emulator_flag_storage_rva",
&emulator_flag_storage_rva) ||
!ParseOptionalRva(*field, "device_create_framebuffer_vtable_offset",
&device_create_framebuffer_vtable_offset) ||
!vtable_rva.has_value() || !constructor_rva.has_value() ||
!state_getter_rva.has_value() || !eye_getter_rva.has_value() ||
!eye_initializer_rva.has_value() ||
!emulator_flag_storage_rva.has_value() ||
!device_create_framebuffer_vtable_offset.has_value()) {
return false;
}
*bridge_profile = VrDebugDeviceBridgeProfile{
*vtable_rva,
*constructor_rva,
*state_getter_rva,
*eye_getter_rva,
*eye_initializer_rva,
*emulator_flag_storage_rva,
*device_create_framebuffer_vtable_offset,
};
return true;
}
} // namespace
std::string_view BuildStatusName(BuildStatus status) noexcept {
@@ -211,6 +262,7 @@ ProfileLookupResult FindBuildProfile(const std::string& manifest_path,
const Json* reason = RequiredField(entry, "reason");
std::optional<std::uintptr_t> fullscreen_setter_rva;
std::optional<FmodOutputDeviceBridgeProfile> fmod_output_device_bridge;
std::optional<VrDebugDeviceBridgeProfile> vr_debug_device_bridge;
if (version_name == nullptr || !version_name->is_string() ||
version_code == nullptr || !version_code->is_number_integer() ||
profile_build_id == nullptr || !profile_build_id->is_string() ||
@@ -227,7 +279,8 @@ ProfileLookupResult FindBuildProfile(const std::string& manifest_path,
"user_game_settings_fullscreen_setter_rva",
&fullscreen_setter_rva) ||
!ParseOptionalFmodOutputDeviceBridge(
entry, &fmod_output_device_bridge)) {
entry, &fmod_output_device_bridge) ||
!ParseOptionalVrDebugDeviceBridge(entry, &vr_debug_device_bridge)) {
return Failure("compatibility profile has missing or invalid fields");
}
@@ -278,6 +331,7 @@ ProfileLookupResult FindBuildProfile(const std::string& manifest_path,
allow_host_constructor_replay->get<bool>(),
fullscreen_setter_rva,
fmod_output_device_bridge,
vr_debug_device_bridge,
reason->get_ref<const std::string&>(),
};
}
+30 -12
View File
@@ -6,6 +6,27 @@ namespace mocktail::compat {
namespace {
constexpr HostAbiProfile kProfiles[] = {
// Roblox 2.738.1397; promoted from the exact-build probation profile.
{"5f0704edd9064f566ee3d6df2bd2fabbcc709f03",
{{{0x1d73422, HostBridgeKind::kAllocate, "small-allocate"},
{0x1d7667a, HostBridgeKind::kUsableSize, "usable-size"},
{0x222f4c0, HostBridgeKind::kReallocate, "reallocate"},
{0x1d731c2, HostBridgeKind::kAllocate, "allocate"},
{0x1d8e7f9, HostBridgeKind::kAlignedAllocate, "aligned-allocate-direct"},
{0x1d76c19, HostBridgeKind::kFree, "free"}}},
6,
{0x76e2da0, 0x77efb98, 0x7c4b780, 0x400, 0x1d73838, 0x1dffad7, 0x7b87830,
0x7b878c0, 0x400000},
{0x1d731c2, 0x1d76c19},
0x70e1618,
3599,
{{{2, 3}, {5, 3599}}},
2,
{{{2, 3599}}},
1,
2,
{0x22308d3, 0x7b8b998},
HostAllocatorStrategy::kNativeMimalloc},
// Roblox 2.736.1408 arm64-v8a. Every RVA was derived from the AArch64
// payload of the same release as the x86-64 reference above, matched by
// allocator internals, init_array call graphs, and static state layout.
@@ -41,12 +62,11 @@ constexpr HostAbiProfile kProfiles[] = {
{0x1d2f43a, HostBridgeKind::kUsableSize, "usable-size"},
{0x21e68bf, HostBridgeKind::kReallocate, "reallocate"},
{0x1d2bf82, HostBridgeKind::kAllocate, "allocate"},
{0x1d47a63, HostBridgeKind::kAlignedAllocate,
"aligned-allocate-direct"},
{0x1d47a63, HostBridgeKind::kAlignedAllocate, "aligned-allocate-direct"},
{0x1d2f9d9, HostBridgeKind::kFree, "free"}}},
6,
{0x7652e70, 0x775fd28, 0x7bb4340, 0x400, 0x1d2c5f8, 0x1db7b57,
0x7af0430, 0x7af04c0, 0x400000},
{0x7652e70, 0x775fd28, 0x7bb4340, 0x400, 0x1d2c5f8, 0x1db7b57, 0x7af0430,
0x7af04c0, 0x400000},
{0x1d2bf82, 0x1d2f9d9},
0x7057818,
3570,
@@ -66,12 +86,11 @@ constexpr HostAbiProfile kProfiles[] = {
{0x1cf9c3a, HostBridgeKind::kUsableSize, "usable-size"},
{0x21af33f, HostBridgeKind::kReallocate, "reallocate"},
{0x1cf677e, HostBridgeKind::kAllocate, "allocate"},
{0x1d12197, HostBridgeKind::kAlignedAllocate,
"aligned-allocate-direct"},
{0x1d12197, HostBridgeKind::kAlignedAllocate, "aligned-allocate-direct"},
{0x1cfa1d9, HostBridgeKind::kFree, "free"}}},
6,
{0x7571570, 0x767e428, 0x7ad47e8, 0x400, 0x1cf6df8, 0x1d82be7,
0x7a10730, 0x7a107c0, 0x400000},
{0x7571570, 0x767e428, 0x7ad47e8, 0x400, 0x1cf6df8, 0x1d82be7, 0x7a10730,
0x7a107c0, 0x400000},
{0x1cf677e, 0x1cfa1d9},
0x6f78a18,
3556,
@@ -92,12 +111,11 @@ constexpr HostAbiProfile kProfiles[] = {
{0x1fe4c84, HostBridgeKind::kUsableSize, "usable-size"},
{0x203daae, HostBridgeKind::kReallocate, "reallocate"},
{0x1c39c3a, HostBridgeKind::kAllocate, "allocate"},
{0x1c8dbf8, HostBridgeKind::kAlignedAllocate,
"aligned-allocate-direct"},
{0x1c8dbf8, HostBridgeKind::kAlignedAllocate, "aligned-allocate-direct"},
{0x1c3eb72, HostBridgeKind::kFree, "free"}}},
6,
{0x7030ae0, 0x703a7e8, 0x72a16d0, 0x400, 0x1c39f1d, 0x2040817,
0x71e92f0, 0x71e9380, 0x400000},
{0x7030ae0, 0x703a7e8, 0x72a16d0, 0x400, 0x1c39f1d, 0x2040817, 0x71e92f0,
0x71e9380, 0x400000},
{0x1c39c3a, 0x1c3eb72},
0x6c2c9b0,
0x6cc8 / 8,
File diff suppressed because it is too large. Load diff
+6 -1
View File
@@ -37,9 +37,14 @@ PresentModePolicy ResolvePresentModePolicy(std::string_view vsync_value,
PresentModePolicy CachedPresentModePolicy() {
static const PresentModePolicy policy = [] {
// main() clears MOCKTAIL_VR_ENABLED when VR cannot start for this build,
// so the VR limit only applies while VR is actually active.
const std::string_view vr = EnvironmentOr("MOCKTAIL_VR_ENABLED", "0");
const bool vr_enabled = vr == "1" || vr == "true" || vr == "on";
return ResolvePresentModePolicy(
EnvironmentOr("MOCKTAIL_VSYNC", "auto"),
EnvironmentOr("MOCKTAIL_FRAME_RATE_LIMIT", "-1"));
vr_enabled ? EnvironmentOr("MOCKTAIL_VR_FRAME_RATE_LIMIT", "unlimited")
: EnvironmentOr("MOCKTAIL_FRAME_RATE_LIMIT", "-1"));
}();
return policy;
}
+100 -6
View File
@@ -61,6 +61,7 @@
#include "linker/linker.h"
#include "mocktail/graphics/bionic_egl_bridge.h"
#include "mocktail/graphics/chrome_trace_writer.h"
#include "mocktail/vr/roblox_vr_device_bridge.h"
#include "runtime/engine_pump_rest.h"
#include "runtime/device_memory_profile.h"
#include "runtime/display_size.h"
@@ -75,6 +76,7 @@
#include "runtime/roblox_capability_resolver.h"
#include "runtime/roblox_platform_web_symbols.h"
#include "runtime/roblox_experience_composition.h"
#include "runtime/roblox_haptic_protocol_bridge.h"
#include "runtime/roblox_game_session_native_adapter.h"
#include "runtime/roblox_window_input_runtime.h"
#include "runtime/roblox_text_input_jni_bridge.h"
@@ -214,6 +216,7 @@ using NativeSendGameLoadedFn = void (*)(JNIEnv*, jclass, jstring, jstring,
using NativePassSupportedRefreshRatesFn = void (*)(JNIEnv*, jclass,
jfloatArray);
using NativePassCurrentDisplayRefreshRateFn = void (*)(JNIEnv*, jclass, jfloat);
using NativeUpdateMobileHapticsStateFn = void (*)(JNIEnv*, jclass, jboolean, jstring);
using NativeSetStringParamFn = void (*)(JNIEnv*, jclass, jstring);
using NativeSetBaseUrlFn = void (*)(JNIEnv*, jclass, jstring, jstring);
using NativeSetTaskSchedulerBackgroundModeFn = void (*)(JNIEnv*, jclass,
@@ -381,6 +384,7 @@ struct EngineStartupContext {
NativePassSupportedRefreshRatesFn native_pass_supported_refresh_rates;
NativePassCurrentDisplayRefreshRateFn native_pass_current_display_refresh_rate;
NativeNoArgFn native_start_lua_app_dm;
NativeUpdateMobileHapticsStateFn native_update_mobile_haptics_state;
};
@@ -2242,8 +2246,12 @@ void MocktailSetAssetPath(JNIEnv* env, jstring asset_path) {
if (path.empty()) {
path = DefaultAssetPath();
}
setenv("MOCKTAIL_ASSET_ROOT", path.c_str(), 1);
std::cout << " [engine] asset root set to " << path << '\n' << std::flush;
std::filesystem::path root_path(path);
if (root_path.filename() == "content") {
root_path = root_path.parent_path();
}
setenv("MOCKTAIL_ASSET_ROOT", root_path.c_str(), 1);
std::cout << " [engine] asset root set to " << root_path.string() << '\n' << std::flush;
}
void MocktailAppBridgeInit(JNIEnv* env, jstring app_params) {
@@ -2682,7 +2690,8 @@ jobject BuildAppBridgeInitParams(JNIEnv* env, jstring client_settings,
SetStringField(env, params, "userAgent", user_agent.c_str());
SetBooleanField(env, params, "isTablet", JNI_FALSE);
SetBooleanField(env, params, "isPotato", JNI_FALSE);
SetBooleanField(env, params, "isVrDevice", JNI_FALSE);
SetBooleanField(env, params, "isVrDevice",
IsEnabled("MOCKTAIL_VR_ENABLED") ? JNI_TRUE : JNI_FALSE);
SetStringField(env, params, "buildVariant", "googleProdRelease");
SetObjectField(env, params, "vrContext", "Landroid/app/Activity;",
activity);
@@ -4309,6 +4318,23 @@ std::cerr << " [engine] nativeAppBridgeV2InitWithParams recovered\n"
std::cout << " [engine] updateAppUISizes returned\n" << std::flush;
}
if (context->native_update_mobile_haptics_state) {
env = ensure_env();
std::cout << " [engine] NativeGLInterface.nativeUpdateMobileHapticsState\n"
<< std::flush;
const std::string motors_json =
mocktail::runtime::RobloxHapticProtocolBridge::GetMotorsJson();
jstring motors_str = env->NewStringUTF(motors_json.c_str());
if (motors_str != nullptr) {
context->native_update_mobile_haptics_state(env, native_gl_class,
JNI_TRUE, motors_str);
env->DeleteLocalRef(motors_str);
}
std::cout
<< " [engine] NativeGLInterface.nativeUpdateMobileHapticsState returned\n"
<< std::flush;
}
int init_delay_ms = GetEnvInt("MOCKTAIL_APPBRIDGE_INIT_DELAY_MS", 0);
if (init_delay_ms > 0) {
std::cout << " [engine] wait after AppBridge init: " << init_delay_ms
@@ -4589,6 +4615,16 @@ std::cerr << " [engine] nativeAppBridgeV2InitWithParams recovered\n"
std::cout
<< " [engine] nativeAppBridgeV2SendAppEventOnGameLoaded returned\n"
<< std::flush;
if (context->native_update_mobile_haptics_state) {
const std::string motors_json =
mocktail::runtime::RobloxHapticProtocolBridge::GetMotorsJson();
jstring motors_str = env->NewStringUTF(motors_json.c_str());
if (motors_str != nullptr) {
context->native_update_mobile_haptics_state(env, native_gl_class,
JNI_TRUE, motors_str);
env->DeleteLocalRef(motors_str);
}
}
}
}
@@ -4685,6 +4721,11 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
"-1, display, unlimited, or a positive integer\n";
return EXIT_FAILURE;
}
if (!runtime_config.vr_frame_rate().valid()) {
std::cerr << "[FATAL] Invalid MOCKTAIL_VR_FRAME_RATE_LIMIT; expected "
"-1, display, unlimited, or a positive integer\n";
return EXIT_FAILURE;
}
if (!runtime_config.theme_mode_valid()) {
std::cerr
<< "[FATAL] Invalid MOCKTAIL_THEME; expected roblox, system, light, "
@@ -4746,8 +4787,14 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
return EXIT_FAILURE;
}
const mocktail::compat::BuildProfile& build_profile =
*profile_result.profile;
const mocktail::compat::BuildProfile& build_profile = *profile_result.profile;
if (IsEnabled("MOCKTAIL_VR_ENABLED") &&
mocktail::vr::FindRobloxVrDeviceProfile(build_profile.elf_build_id) ==
nullptr) {
std::cerr << "[FATAL] Experimental Roblox VR has no verified profile for "
<< build_profile.elf_build_id << "\n";
return EXIT_FAILURE;
}
const mocktail::compat::HostAbiProfile* host_abi_profile =
mocktail::compat::FindHostAbiProfile(build_profile.elf_build_id);
const bool experiment_allowed =
@@ -5897,6 +5944,20 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
std::cout << " [linker] Bionic unwind metadata validated for libroblox\n"
<< std::flush;
// The experimental VR bridge is armed only for --vr on an exact supported
// Build ID. It validates its machine contracts and interposes the
// DebugDeviceVR vtable slots after relocations and before guest startup
// can create the object. A no-op success while no bridge is armed.
if (const mocktail::Status vr_activation =
mocktail::vr::NotifyRobloxVrImageLoaded(
linker::FindLoadedAndroidLibraryBase("libroblox"));
!vr_activation.ok()) {
std::cerr << "\n[FATAL] Experimental Roblox VR bridge activation "
"failed: "
<< vr_activation.message() << '\n';
return EXIT_FAILURE;
}
// The pre-constructor hook owns installation; repeating arena setup here is
// not idempotent.
if (HostAbiExperimentRequested() &&
@@ -6073,6 +6134,8 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
eager_permissions_bridge;
std::unique_ptr<mocktail::runtime::RobloxCallProtocolBridge>
eager_call_protocol_bridge;
std::unique_ptr<mocktail::runtime::RobloxHapticProtocolBridge>
eager_haptic_protocol_bridge;
std::shared_ptr<ExperienceLifecycleTarget> experience_lifecycle_target;
std::shared_ptr<mocktail::runtime::RobloxWindowInputRuntime>
window_input_runtime;
@@ -6369,6 +6432,14 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
roblox_handle,
"Java_com_roblox_engine_jni_NativeGLInterface_"
"nativeAppBridgeV2SendAppEventOnGameLoaded"));
auto* native_update_mobile_haptics_state =
mocktail::runtime::RobloxHapticProtocolBridge::EnabledForThisSession()
? reinterpret_cast<NativeUpdateMobileHapticsStateFn>(
linker::ResolveSymbol(
roblox_handle,
"Java_com_roblox_engine_jni_NativeGLInterface_"
"nativeUpdateMobileHapticsState"))
: nullptr;
auto* native_set_asset_path =
reinterpret_cast<NativeSetStringParamFn>(linker::ResolveSymbol(
roblox_handle,
@@ -6447,6 +6518,10 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
<< reinterpret_cast<const void*>(native_send_app_ready) << '\n'
<< " native_send_game_loaded="
<< reinterpret_cast<const void*>(native_send_game_loaded) << '\n'
<< " native_update_mobile_haptics_state="
<< reinterpret_cast<const void*>(
native_update_mobile_haptics_state)
<< '\n'
<< " native_start_lua_app_dm="
<< reinterpret_cast<const void*>(native_start_lua_app_dm) << '\n'
<< " native_call_messages_from_main_thread="
@@ -6787,9 +6862,17 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
eager_call_protocol_bridge =
std::make_unique<mocktail::runtime::RobloxCallProtocolBridge>(
environment, symbols.permissions, objects);
if (mocktail::runtime::RobloxHapticProtocolBridge::
EnabledForThisSession()) {
eager_haptic_protocol_bridge =
std::make_unique<mocktail::runtime::RobloxHapticProtocolBridge>(
environment, symbols.permissions, objects);
}
status = eager_permissions_bridge->Initialize();
if (status.ok())
status = eager_call_protocol_bridge->Initialize();
if (status.ok() && eager_haptic_protocol_bridge != nullptr)
status = eager_haptic_protocol_bridge->Initialize();
env->DeleteLocalRef(bus);
if (!status.ok()) {
std::cerr << "[FATAL] GAME platform protocols did not initialize: "
@@ -6951,7 +7034,8 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
native_on_fragment_start,
native_pass_supported_refresh_rates,
native_pass_current_display_refresh_rate,
native_start_lua_app_dm};
native_start_lua_app_dm,
native_update_mobile_haptics_state};
if (game_session_runtime != nullptr) {
if (!game_present_observer.Register(
&mocktail::runtime::RobloxGameSessionRuntime::
@@ -7177,6 +7261,12 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
while (true) {
const uint64_t fps_tick_start_ns = fps_trace ? MonotonicNanos() : 0;
const bool keep_window_open = mocktail::window::PumpEvents();
// Deliver XR tracked-controller input on the same thread and through the
// same router as SDL input, once per loop iteration after the SDL pump.
// Inert in non-VR builds or while no OpenXR controller layer is active.
if (window_input_runtime != nullptr) {
window_input_runtime->DrainXrControllers();
}
game_surface_events_completed = drain_game_surface_events();
const uint64_t fps_after_sdl_ns = fps_trace ? MonotonicNanos() : 0;
if (!game_surface_events_completed || !keep_window_open) {
@@ -7292,6 +7382,10 @@ int mocktail::legacy::Run(const runtime::CommandLineOptions& options,
input_shutdown_completed = window_input_runtime->Shutdown().ok() &&
input_shutdown_completed;
}
if (eager_haptic_protocol_bridge != nullptr) {
(void)eager_haptic_protocol_bridge->Shutdown();
eager_haptic_protocol_bridge.reset();
}
if (eager_call_protocol_bridge != nullptr) {
(void)eager_call_protocol_bridge->Shutdown();
eager_call_protocol_bridge.reset();
+245 -17
View File
@@ -24,6 +24,11 @@
#include "mocktail/audio/fmod_jni_audio_bridge.h"
#include "mocktail/audio/webrtc_jni_audio_bridge.h"
#include "mocktail/audio/roblox_output_device_bridge.h"
#include "mocktail/vr/headset_detection.h"
#include "mocktail/vr/openxr_backend.h"
#include "mocktail/vr/openxr_probe.h"
#include "mocktail/vr/roblox_vr_device_bridge.h"
#include "mocktail/vr/virtual_keyboard.h"
#include "runtime/auth_runtime_composition.h"
#include "runtime/command_line.h"
#include "runtime/crash_report_policy.h"
@@ -43,6 +48,8 @@
#include "runtime/roblox_desktop_app_policy.h"
#include "runtime/roblox_experience_launch_bridge.h"
#include "runtime/roblox_fullscreen_runtime_bridge.h"
#include "runtime/roblox_text_display_state.h"
#include "runtime/roblox_text_editor.h"
#include "runtime/runtime_config_bootstrap.h"
#include "runtime/runtime_config_file.h"
#include "runtime/runtime_paths.h"
@@ -246,6 +253,11 @@ void ConfigureHostDriverEnvironment() {
int main(int argc, char* argv[]) {
ConfigureHostDriverEnvironment();
// vr.mode: auto asks SteamVR for a headset from a copy of this process.
if (const char* probe = std::getenv(mocktail::vr::kHeadsetProbeEnvironment);
probe != nullptr && std::string_view(probe) == "1") {
return mocktail::vr::RunHeadsetProbeChild();
}
const auto process_started_at = std::chrono::system_clock::now();
mocktail::runtime::CommandLineParseResult command_line =
mocktail::runtime::ParseCommandLine(argc, argv);
@@ -280,6 +292,7 @@ int main(int argc, char* argv[]) {
}
const mocktail::runtime::ProcessEnvironment environment;
const bool vr_requested = environment.HasNonEmpty("MOCKTAIL_VR_ENABLED");
const std::string built_in_compatibility_manifest = environment.GetOr(
"MOCKTAIL_UPDATE_COMPATIBILITY_PATH",
environment.GetOr("MOCKTAIL_COMPATIBILITY_MANIFEST",
@@ -323,6 +336,17 @@ int main(int argc, char* argv[]) {
}
mocktail::runtime::ScrubCommandLineLaunchArguments(&command_line.options,
argc, argv);
// Reject --vr in a build without OpenXR before touching any data or state
// directories. A vr.enabled config is checked once the config is loaded.
const std::string vr_request = environment.GetOr("MOCKTAIL_VR_ENABLED", "");
if ((vr_request == "1" || vr_request == "true" || vr_request == "on") &&
!mocktail::vr::IsOpenXrSupportCompiled()) {
std::cerr << " [vr] This build does not include OpenXR support. "
"Use a VR-enabled build of Mocktail; refusing to start "
"Roblox with --vr in a non-VR build.\n";
return EXIT_FAILURE;
}
mocktail::runtime::SessionLog session_log;
mocktail::runtime::FailureSupportBundleGuard support_bundle_guard(
environment, paths,
@@ -392,6 +416,41 @@ int main(int argc, char* argv[]) {
}
return EXIT_FAILURE;
}
if (command_line.options.mode == mocktail::runtime::CommandMode::kRun &&
runtime_config.config.vr_mode() == mocktail::runtime::VrMode::kAuto) {
std::error_code self_error;
const std::filesystem::path self =
std::filesystem::read_symlink("/proc/self/exe", self_error);
const mocktail::vr::HeadsetDetection detection =
mocktail::vr::DetectSteamVrHeadset(self_error ? std::string()
: self.string());
std::cerr << " [vr] vr.mode is auto: " << detection.reason
<< (detection.headset ? "; starting in VR\n"
: "; starting on the desktop\n");
if (detection.headset) {
// A detected headset is not an explicit --vr: a Roblox build without
// VR support still starts on the desktop.
if (setenv("MOCKTAIL_VR_ENABLED", "1", 1) != 0) {
std::cerr << "[FATAL] Cannot enable VR for the detected headset\n";
return EXIT_FAILURE;
}
runtime_config = mocktail::runtime::LoadRuntimeConfig(
environment, paths.config_file());
if (!runtime_config) {
std::cerr << "[FATAL] Cannot load " << paths.config_file() << ": "
<< runtime_config.error << '\n';
return EXIT_FAILURE;
}
}
}
if (runtime_config.config.vr_enabled() &&
!mocktail::vr::IsOpenXrSupportCompiled()) {
std::cerr << " [vr] This build does not include OpenXR support, but "
"VR is on; set vr.mode to off or auto in "
<< paths.config_file()
<< ".\n";
return EXIT_FAILURE;
}
if (runtime_config.config.use_system_proxy()) {
const mocktail::runtime::SystemProxyResult system_proxy =
mocktail::runtime::ResolveSystemProxy();
@@ -613,6 +672,10 @@ int main(int argc, char* argv[]) {
std::cerr << "[FATAL] Cannot bind assets to explicit Roblox library\n";
return EXIT_FAILURE;
}
const std::filesystem::path adjacent_root =
adjacent_assets.filename() == "content" ? adjacent_assets.parent_path()
: adjacent_assets;
setenv("MOCKTAIL_ASSET_ROOT", adjacent_root.c_str(), 1);
std::cout << " [runtime] explicit Roblox library uses adjacent assets: "
<< adjacent_assets << '\n';
}
@@ -704,8 +767,13 @@ int main(int argc, char* argv[]) {
if ((needs_active_library &&
setenv("ROBLOX_LIB_PATH", active.roblox_library.c_str(), 1) != 0) ||
(needs_active_assets &&
setenv("MOCKTAIL_ASSET_PATH", active.assets_content.c_str(), 1) !=
0)) {
(setenv("MOCKTAIL_ASSET_PATH", active.assets_content.c_str(), 1) != 0 ||
setenv("MOCKTAIL_ASSET_ROOT",
(active.assets_content.filename() == "content"
? active.assets_content.parent_path()
: active.assets_content)
.c_str(),
1) != 0))) {
std::cerr << "[FATAL] Cannot activate resolved Roblox payload\n";
return EXIT_FAILURE;
}
@@ -771,7 +839,72 @@ int main(int argc, char* argv[]) {
<< " model=\""
<< runtime_config.config.device_profile().display_name << "\"\n";
}
mocktail::compat::PayloadCompatibilityResult compatibility;
bool vr_active = false;
if (command_line.options.mode == mocktail::runtime::CommandMode::kRun) {
const std::string compatibility_manifest =
environment.GetOr("MOCKTAIL_COMPATIBILITY_MANIFEST",
MOCKTAIL_DEFAULT_COMPATIBILITY_MANIFEST);
compatibility = mocktail::compat::CheckPayloadCompatibility(
runtime_config.config.roblox_library_path().string(),
compatibility_manifest,
command_line.options.allow_unverified_build);
if (!compatibility) {
std::cerr << "[FATAL] " << compatibility.error << '\n';
return EXIT_FAILURE;
}
if (!compatibility.profile.vr_debug_device_bridge.has_value()) {
if (const auto* vr_profile =
mocktail::vr::FindRobloxVrDeviceProfile(compatibility.build_id)) {
compatibility.profile.vr_debug_device_bridge = *vr_profile;
}
}
const bool vr_build_supported =
compatibility.profile.vr_debug_device_bridge.has_value();
if (runtime_config.config.vr_enabled() && vr_requested &&
!vr_build_supported) {
// An explicit --vr on a build without VR support is a user error, not
// something to silently ignore; say which builds work and how to pick.
std::cerr << "[FATAL] VR is not available for Roblox "
<< compatibility.profile.version_name << " (version "
<< compatibility.profile.version_code << ", Build ID "
<< compatibility.build_id << ").\n"
<< " VR supports builds "
<< mocktail::vr::DescribeSupportedVrBuilds()
<< ". Point --roblox-lib at one of those payloads'\n"
<< " libroblox.so, or launch without --vr.\n";
return EXIT_FAILURE;
}
vr_active = runtime_config.config.vr_enabled() && vr_build_supported;
if (runtime_config.config.vr_enabled() && !vr_active) {
std::cerr << " [vr] Roblox " << compatibility.profile.version_name
<< " (Build ID " << compatibility.build_id
<< ") has no VR support (supported: "
<< mocktail::vr::DescribeSupportedVrBuilds()
<< "); continuing without VR\n";
if (setenv("MOCKTAIL_VR_ENABLED", "0", 1) != 0) {
std::cerr << "[FATAL] Cannot disable VR for this Roblox build\n";
return EXIT_FAILURE;
}
}
if (vr_active) {
const mocktail::vr::VrBackendMode vr_backend_mode =
mocktail::vr::ResolveVrBackendMode(true);
std::cerr << " [vr] experimental backend="
<< mocktail::vr::VrBackendModeName(vr_backend_mode)
<< ": Roblox renders both eyes through its DebugDeviceVR "
"debug device (Build ID "
<< compatibility.build_id << ")";
if (vr_backend_mode == mocktail::vr::VrBackendMode::kXrOutput) {
std::cerr << "; eye images are submitted to the active OpenXR runtime "
"as a projection layer. Start WiVRn or SteamVR/ALVR and "
"connect the headset "
"before launching.\n";
} else {
std::cerr << "; OpenXR output is disabled in this diagnostic mode "
"(MOCKTAIL_VR_BACKEND=native).\n";
}
}
const std::filesystem::path fflags_path =
paths.config_root() / "fflags.json";
const auto fflags = mocktail::services::LoadAndMergeFflagsFile(
@@ -789,7 +922,8 @@ int main(int argc, char* argv[]) {
std::string client_settings_overrides;
std::vector<std::string> kept_fflag_overrides;
if (!mocktail::runtime::MergeRuntimeClientSettingsOverrides(
runtime_config.config.frame_rate(),
vr_active ? runtime_config.config.vr_frame_rate()
: runtime_config.config.frame_rate(),
runtime_config.config.performance(), fflags.json,
&client_settings_overrides, &command_line_error,
&kept_fflag_overrides)) {
@@ -810,6 +944,12 @@ int main(int argc, char* argv[]) {
<< command_line_error << '\n';
return EXIT_FAILURE;
}
if (!mocktail::runtime::MergeVrClientSettingsOverrides(
vr_active, audio_capture_overrides,
&audio_capture_overrides, &command_line_error)) {
std::cerr << "[FATAL] Cannot apply VR policy: " << command_line_error << '\n';
return EXIT_FAILURE;
}
if (setenv("MOCKTAIL_CLIENT_SETTINGS_OVERRIDES_JSON",
audio_capture_overrides.c_str(), 1) != 0) {
std::cerr << "[FATAL] Cannot export runtime client-settings policy\n";
@@ -825,6 +965,12 @@ int main(int argc, char* argv[]) {
<< command_line_error << '\n';
return EXIT_FAILURE;
}
if (!mocktail::runtime::MergeVrClientSettingsOverrides(
vr_active, fast_flags_overrides,
&fast_flags_overrides, &command_line_error)) {
std::cerr << "[FATAL] Cannot apply VR policy: " << command_line_error << '\n';
return EXIT_FAILURE;
}
if (setenv("MOCKTAIL_FAST_FLAGS_JSON", fast_flags_overrides.c_str(), 1) !=
0) {
std::cerr << "[FATAL] Cannot export crash-report fast-flags policy\n";
@@ -865,10 +1011,14 @@ int main(int argc, char* argv[]) {
"counts under Roblox client-settings control\n";
}
}
const mocktail::runtime::FrameRatePolicy& active_frame_rate =
vr_active ? runtime_config.config.vr_frame_rate()
: runtime_config.config.frame_rate();
if (command_line.options.mode == mocktail::runtime::CommandMode::kRun &&
runtime_config.config.frame_rate().mode ==
active_frame_rate.mode ==
mocktail::runtime::FrameRateLimitMode::kUnlimited) {
std::cout << " [runtime] frame-rate mode=unlimited scheduler_target="
std::cout << " [runtime] " << (vr_active ? "vr " : "")
<< "frame-rate mode=unlimited scheduler_target="
<< mocktail::runtime::kMaximumSupportedRobloxSchedulerFps
<< " vsync=" << runtime_config.config.vsync_mode() << '\n';
}
@@ -883,19 +1033,15 @@ int main(int argc, char* argv[]) {
mocktail::runtime::RobloxFullscreenRuntimeBridge fullscreen_bridge;
mocktail::audio::RobloxOutputDeviceBridge output_device_bridge;
// Never destroyed: Shutdown() stops waiting after 2 s, and a guest thread
// still inside a hook then keeps using this object until the process ends.
static auto& vr_device_bridge = *new mocktail::vr::RobloxVrDeviceBridge;
// Never destroyed, for the same reason: the hooks call into the backend.
static auto& xr_backend = *new mocktail::vr::OpenXrBackend;
struct DisarmOnExit {
~DisarmOnExit() { xr_backend.Disarm(); }
} disarm_xr_backend_on_exit;
if (command_line.options.mode == mocktail::runtime::CommandMode::kRun) {
const std::string compatibility_manifest =
environment.GetOr("MOCKTAIL_COMPATIBILITY_MANIFEST",
MOCKTAIL_DEFAULT_COMPATIBILITY_MANIFEST);
const mocktail::compat::PayloadCompatibilityResult compatibility =
mocktail::compat::CheckPayloadCompatibility(
runtime_config.config.roblox_library_path().string(),
compatibility_manifest,
command_line.options.allow_unverified_build);
if (!compatibility) {
std::cerr << "[FATAL] " << compatibility.error << '\n';
return EXIT_FAILURE;
}
const mocktail::Status fullscreen_status =
fullscreen_bridge.Install(compatibility.profile);
if (!fullscreen_status.ok()) {
@@ -925,6 +1071,66 @@ int main(int argc, char* argv[]) {
<< output_device_status.message() << '\n';
return EXIT_FAILURE;
}
if (vr_active) {
const mocktail::Status vr_device_status =
vr_device_bridge.Install(compatibility.profile);
if (!vr_device_status.ok()) {
std::cerr << "[FATAL] Cannot install experimental Roblox VR device "
"bridge: "
<< vr_device_status.message() << '\n';
return EXIT_FAILURE;
}
if (mocktail::vr::ResolveVrBackendMode(true) ==
mocktail::vr::VrBackendMode::kXrOutput) {
// Fail closed and loudly when no OpenXR runtime is reachable; the
// native-stereo diagnostic mode must be requested explicitly.
const mocktail::Status xr_status =
xr_backend.Arm(runtime_config.config.graphics_backend() ==
mocktail::runtime::GraphicsBackend::kVulkan
? mocktail::vr::VrGraphicsApi::kVulkan
: mocktail::vr::VrGraphicsApi::kOpenGles);
if (!xr_status.ok()) {
std::cerr
<< "[FATAL] Cannot arm the OpenXR backend for --vr: "
<< xr_status.message() << '\n'
<< " Start WiVRn or SteamVR/ALVR, connect your headset, then "
"run mocktail -vr again.\n"
<< " For a custom runtime set XR_RUNTIME_JSON to its manifest.\n"
<< " Set MOCKTAIL_VR_BACKEND=native for the "
"native-stereo diagnostic mode without a runtime.\n";
return EXIT_FAILURE;
}
// The in-headset keyboard follows Roblox's TextBox keyboard requests
// and types through the same events as the desktop keyboard.
mocktail::vr::VirtualKeyboardHost keyboard_host;
keyboard_host.input_requested = [](void*) {
return mocktail::window::TextInputRequested();
};
keyboard_host.commit_text = [](void*, const char* utf8) {
mocktail::window::QueueVirtualTextInput(utf8);
};
keyboard_host.press_key = [](void*, std::uint32_t scancode) {
mocktail::window::QueueVirtualKeyPress(scancode);
};
mocktail::vr::SetVirtualKeyboardHost(&keyboard_host);
// TextBox text is drawn as a desktop window overlay that never reaches
// the eye images; mirror it into the keyboard's text field.
mocktail::runtime::SetRobloxTextDisplayMirror(
{nullptr, [](void*,
const mocktail::runtime::RobloxTextDisplayUpdate&
update) {
if (update.event ==
mocktail::runtime::RobloxTextDisplayEvent::kHide) {
mocktail::vr::ClearVirtualKeyboardText();
return;
}
mocktail::vr::SetVirtualKeyboardText(
update.utf8, update.utf8_size, update.cursor_utf16,
mocktail::runtime::IsRobloxPasswordTextInputType(
update.text_input_type));
}});
}
}
}
mocktail::legacy::RuntimeDependencies dependencies;
@@ -1060,6 +1266,26 @@ int main(int argc, char* argv[]) {
<< " updated=" << (desktop_policy.updated ? 1 : 0)
<< " runtime_override=1\n";
}
if (vr_active) {
std::string vr_client_settings;
std::string vr_fast_flags;
if (!mocktail::runtime::MergeVrAppPolicyClientSettingsOverrides(
vr_active,
environment.GetOr("MOCKTAIL_CLIENT_SETTINGS_OVERRIDES_JSON", "{}"),
environment.GetOr("MOCKTAIL_FAST_FLAGS_JSON", "{}"),
&vr_client_settings, &vr_fast_flags, &command_line_error)) {
std::cerr << "[FATAL] Cannot apply VR app-policy: "
<< command_line_error << '\n';
return EXIT_FAILURE;
}
if (setenv("MOCKTAIL_CLIENT_SETTINGS_OVERRIDES_JSON",
vr_client_settings.c_str(), 1) != 0 ||
setenv("MOCKTAIL_FAST_FLAGS_JSON", vr_fast_flags.c_str(), 1) != 0) {
std::cerr << "[FATAL] Cannot export VR app-policy\n";
return EXIT_FAILURE;
}
std::cout << " [vr] app-policy idle render throttling disabled\n";
}
auto android_window_context =
std::make_shared<AndroidWindowBridgeContext>();
jnivm::AndroidWindowCallbacks android_window_callbacks;
@@ -1132,6 +1358,8 @@ int main(int argc, char* argv[]) {
std::to_string(runtime_status) + ".");
}
support_bundle_guard.SetExitCode(runtime_status);
xr_backend.Disarm();
vr_device_bridge.Shutdown();
output_device_bridge.Shutdown();
fullscreen_bridge.Shutdown();
memory_limit_watchdog.Stop();
+11
View File
@@ -76,6 +76,8 @@ CommandLineParseResult ParseCommandLine(int argc, const char* const argv[]) {
&result.options.graphics_backend, &result.error)) {
return result;
}
} else if (argument == "--vr" || argument == "-vr" || argument == "--no-vr") {
result.options.vr_enabled = argument != "--no-vr";
} else if (argument == "--allow-unverified-build") {
result.options.allow_unverified_build = true;
} else if (argument == "--force-run-latest") {
@@ -149,6 +151,7 @@ CommandLineParseResult ParseCommandLine(int argc, const char* const argv[]) {
(!result.options.roblox_library_path.empty() ||
result.options.window_mode != WindowMode::kUnspecified ||
!result.options.graphics_backend.empty() ||
result.options.vr_enabled.has_value() ||
result.options.allow_unverified_build ||
!result.options.launch_request_json.empty())) {
result.error = "--force-run-latest must be used on its own";
@@ -226,6 +229,11 @@ void ScrubCommandLineLaunchArguments(CommandLineOptions* options, int argc,
bool ApplyCommandLineEnvironment(const CommandLineOptions& options,
std::string* error) {
if (options.vr_enabled.has_value() &&
!SetEnvironment("MOCKTAIL_VR_ENABLED", *options.vr_enabled ? "1" : "0",
error)) {
return false;
}
if (!options.roblox_library_path.empty() &&
!SetEnvironment("ROBLOX_LIB_PATH", options.roblox_library_path, error)) {
return false;
@@ -271,6 +279,9 @@ std::string CommandLineUsage(const std::string& program_name) {
<< compat::kGuestAbi << " libroblox.so\n"
<< " --headless Run without creating an SDL window\n"
<< " --windowed Force windowed startup (default)\n"
<< " -vr / --vr / --no-vr Launch Roblox via OpenXR (WiVRn) or "
"disable vr.enabled\n"
<< " VR builds enable VR by default; use --no-vr to opt out.\n"
<< " --graphics <backend> direct-vulkan | opengl | system | "
"angle-vulkan (default: direct-vulkan)\n"
<< " --allow-unverified-build Run a known but unverified Build-ID "
+1 -1
View File
@@ -93,7 +93,7 @@ bool UnthrottledPresentation(const RuntimeConfig& config) {
if (config.vsync_mode() == "off" || config.vsync_mode() == "0") {
return true;
}
return config.frame_rate().mode == FrameRateLimitMode::kUnlimited;
return config.active_frame_rate().mode == FrameRateLimitMode::kUnlimited;
}
bool ParsePciVendor(const std::string& raw, unsigned int* vendor) {
+41
View File
@@ -375,6 +375,47 @@ bool MergeAudioDeviceMenuClientSettingsOverrides(std::string_view base_json,
return true;
}
bool MergeVrClientSettingsOverrides(bool enabled, std::string_view base_json,
std::string* merged_json,
std::string* error) {
if (merged_json == nullptr) {
if (error != nullptr) {
*error = "VR client-settings output is required";
}
return false;
}
auto overrides = nlohmann::json::parse(base_json.empty() ? "{}" : base_json,
nullptr, false, true);
if (overrides.is_discarded() || !overrides.is_object()) {
if (error != nullptr) {
*error = "VR client-settings input must be a JSON object";
}
return false;
}
// Without VR the flags are left exactly as the user (fflags.json) set
// them; only an active VR session forces the emulator on.
if (enabled) {
overrides["FFlagDebugEnableVREmulator"] = "True";
// Roblox 2998's SurfaceController startup throttle renders at 1 Hz while
// waiting for AppStartRendering. Our XR frame loop is driven by guest
// presents, so that also stalls head/controller updates in the VR menu.
// Keep VR rendering continuous; leave the non-VR startup policy alone.
overrides["FFlagSlowDownRendering"] = "False";
// The eye textures already use the OpenXR extent; render their contents
// at full resolution as well, without Roblox's dynamic downscaling.
overrides["FFlagDebugDisableDRS"] = "True";
// Place the Avatar Editor character 6 studs in front of the VR camera
// instead of at the origin when IsVRAppBuild() is active.
overrides["FFlagFixVRAvatarPlacement"] = "True";
// Mobile suppresses vibration when a gamepad connects unless KeepHaptics
// is enabled; VR controllers connect as Gamepad1.
overrides["FFlagKeepHapticsOnMobileWhenGamepadConnected"] = "True";
overrides["FFlagVRHaptics"] = "True";
}
*merged_json = overrides.dump();
return true;
}
bool MergeAudioCaptureClientSettingsOverrides(bool microphone_enabled,
std::string_view base_json,
std::string* merged_json,
+65 -1
View File
@@ -23,6 +23,8 @@ namespace {
constexpr std::uintmax_t kMaximumJsonBytes = 16U * 1024U * 1024U;
constexpr std::string_view kAppPolicySuffix = ":app-policy";
constexpr const char* kFlyoutVariantFlag = "FIntFlyoutTestVariantRollout12";
constexpr const char* kDesktopFlyoutVariant = "2";
constexpr std::string_view kDesktopAppPolicyOverride =
"FStringAppConfigurationOverrideAppPolicy";
std::atomic<std::uint64_t> g_temporary_sequence{0};
@@ -368,10 +370,72 @@ bool MergeDesktopAppPolicyClientSettingsOverride(std::string_view policy_json,
// Roblox enrolls desktop clients in the flyout side panel through a
// server-side experiment that does not reach this client. Variant 2 is the
// desktop side panel; an explicit user override is preserved.
overrides.emplace("FIntFlyoutTestVariantRollout12", "2");
overrides.emplace(kFlyoutVariantFlag, kDesktopFlyoutVariant);
*merged_json = overrides.dump();
return true;
}
bool MergeVrAppPolicyClientSettingsOverrides(
bool enabled, std::string_view client_settings_json,
std::string_view fast_flags_json, std::string* merged_client_settings,
std::string* merged_fast_flags, std::string* error) {
if (merged_client_settings == nullptr || merged_fast_flags == nullptr) {
if (error != nullptr) {
*error = "VR app-policy outputs are required";
}
return false;
}
if (!enabled) {
*merged_client_settings = client_settings_json;
*merged_fast_flags = fast_flags_json;
return true;
}
auto client_settings = nlohmann::json::parse(
client_settings_json.empty() ? "{}" : client_settings_json, nullptr,
false, true);
auto fast_flags = nlohmann::json::parse(
fast_flags_json.empty() ? "{}" : fast_flags_json, nullptr, false, true);
if (!client_settings.is_object() || !fast_flags.is_object()) {
if (error != nullptr) {
*error = "VR client-settings and fast-flags overrides must be JSON objects";
}
return false;
}
nlohmann::json policy = nlohmann::json::object();
const nlohmann::json* channels[] = {&client_settings, &fast_flags};
for (const auto* channel : channels) {
const auto existing = channel->find(kDesktopAppPolicyOverride);
if (existing == channel->end()) {
continue;
}
// An empty FString means no app-policy override. The later fast-flags
// channel can clear/replace an earlier client-settings override.
if (existing->is_string() &&
existing->get_ref<const std::string&>().empty()) {
policy = nlohmann::json::object();
} else if (!DecodePolicy(*existing, &policy)) {
if (error != nullptr) {
*error = "VR app-policy override must be an encoded JSON object";
}
return false;
}
}
policy["ThrottleFramerate"] = false;
// The desktop side panel cannot be reached with the VR pointer. fflags.json
// is the later channel, so a user's own value there still decides.
const auto flyout = client_settings.find(kFlyoutVariantFlag);
if (flyout != client_settings.end() && *flyout == kDesktopFlyoutVariant) {
*flyout = "0";
}
const std::string encoded_policy = policy.dump();
client_settings[std::string(kDesktopAppPolicyOverride)] = encoded_policy;
fast_flags[std::string(kDesktopAppPolicyOverride)] = encoded_policy;
*merged_client_settings = client_settings.dump();
*merged_fast_flags = fast_flags.dump();
return true;
}
} // namespace runtime
} // namespace mocktail
+24 -2
View File
@@ -466,6 +466,20 @@ Status RobloxExperienceComposition::InitializePlatformProtocols() {
(void)ReleaseGlobalObjects();
return status;
}
std::unique_ptr<RobloxHapticProtocolBridge> haptic_protocol_bridge;
if (RobloxHapticProtocolBridge::EnabledForThisSession()) {
haptic_protocol_bridge = std::make_unique<RobloxHapticProtocolBridge>(
environment_, permissions_symbols_, permissions_objects);
status = haptic_protocol_bridge->Initialize();
if (!status.ok()) {
(void)call_protocol_bridge->Shutdown();
(void)permissions_bridge->Shutdown();
(void)browser_service_bridge->Shutdown();
(void)web_view_bridge->Shutdown();
(void)ReleaseGlobalObjects();
return status;
}
}
// AccountProtocol only shortens a device-integrity login challenge that
// cannot succeed here, so its absence or failure never blocks startup.
std::unique_ptr<RobloxAccountBridge> account_bridge;
@@ -493,6 +507,7 @@ Status RobloxExperienceComposition::InitializePlatformProtocols() {
browser_service_bridge_ = std::move(browser_service_bridge);
permissions_bridge_ = std::move(permissions_bridge);
call_protocol_bridge_ = std::move(call_protocol_bridge);
haptic_protocol_bridge_ = std::move(haptic_protocol_bridge);
account_bridge_ = std::move(account_bridge);
platform_protocols_initialized_ = true;
}
@@ -1694,6 +1709,7 @@ Status RobloxExperienceComposition::Shutdown() {
std::unique_ptr<RobloxBrowserServiceBridge> browser_service_bridge;
std::unique_ptr<RobloxPermissionsBridge> permissions_bridge;
std::unique_ptr<RobloxCallProtocolBridge> call_protocol_bridge;
std::unique_ptr<RobloxHapticProtocolBridge> haptic_protocol_bridge;
std::unique_ptr<RobloxAccountBridge> account_bridge;
std::shared_ptr<WebViewHelperProcess> web_surface_process;
jnivm::VM* late_lifecycle_vm = nullptr;
@@ -1720,6 +1736,7 @@ Status RobloxExperienceComposition::Shutdown() {
browser_service_bridge = std::move(browser_service_bridge_);
permissions_bridge = std::move(permissions_bridge_);
call_protocol_bridge = std::move(call_protocol_bridge_);
haptic_protocol_bridge = std::move(haptic_protocol_bridge_);
account_bridge = std::move(account_bridge_);
web_surface_process = std::move(web_surface_process_);
web_surface_logical_exit_observer_ = {};
@@ -1737,9 +1754,14 @@ Status RobloxExperienceComposition::Shutdown() {
if (web_surface_process != nullptr) {
(void)web_surface_process->RequestClose();
}
Status status = call_protocol_bridge != nullptr
? call_protocol_bridge->Shutdown()
Status status = haptic_protocol_bridge != nullptr
? haptic_protocol_bridge->Shutdown()
: Status::Ok();
const Status call_status = call_protocol_bridge != nullptr
? call_protocol_bridge->Shutdown()
: Status::Ok();
if (status.ok())
status = call_status;
const Status account_status =
account_bridge != nullptr ? account_bridge->Shutdown() : Status::Ok();
if (status.ok())
@@ -0,0 +1,497 @@
#include "runtime/roblox_haptic_protocol_bridge.h"
#include <dlfcn.h>
#include <algorithm>
#include <cctype>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <cstdlib>
#include <mutex>
#include <nlohmann/json.hpp>
#include <optional>
#include <string_view>
#include <thread>
#include <vector>
extern "C" {
void mocktail_vr_controller_request_haptics(int32_t hand, float amplitude,
uint64_t duration_ns,
float frequency_hz) __attribute__((weak));
void mocktail_vr_controller_stop_haptics(int32_t hand) __attribute__((weak));
}
namespace mocktail {
namespace runtime {
namespace {
constexpr std::size_t kMaximumJsonBytes = 64 * 1024;
constexpr uint64_t kMinimumDurationNs = 50000000ULL; // 50ms
constexpr uint64_t kMaximumDurationNs = 5000000000ULL; // 5 seconds
bool HasBoundedDepth(const std::string& json) {
int depth = 0;
bool quoted = false;
bool escaped = false;
for (const char ch : json) {
if (quoted) {
if (escaped)
escaped = false;
else if (ch == '\\')
escaped = true;
else if (ch == '"')
quoted = false;
} else if (ch == '"') {
quoted = true;
} else if (ch == '{' || ch == '[') {
if (++depth > 8)
return false;
} else if (ch == '}' || ch == ']') {
--depth;
}
}
return true;
}
std::optional<nlohmann::json> ParseJsonMessage(JNIEnv* env, jstring message) {
if (!message)
return std::nullopt;
const jsize size = env->GetStringUTFLength(message);
if (size <= 0 || static_cast<std::size_t>(size) > kMaximumJsonBytes)
return std::nullopt;
const char* bytes = env->GetStringUTFChars(message, nullptr);
if (!bytes)
return std::nullopt;
const std::string owned(bytes, static_cast<std::size_t>(size));
env->ReleaseStringUTFChars(message, bytes);
if (!HasBoundedDepth(owned))
return std::nullopt;
auto parsed = nlohmann::json::parse(owned, nullptr, false);
if (parsed.is_discarded() || !parsed.is_object())
return std::nullopt;
return parsed;
}
// 0 left, 1 right, -1 both (a motor that names neither hand).
int HandForMotor(const std::string& motor_name) {
std::string lower_name = motor_name;
for (char& c : lower_name) {
c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
}
if (lower_name.find("left") != std::string::npos ||
lower_name.find("large") != std::string::npos) {
return 0;
}
if (lower_name.find("right") != std::string::npos ||
lower_name.find("small") != std::string::npos) {
return 1;
}
return -1;
}
// One message's demand on each controller: a hand vibrates at the strongest
// motor that maps to it instead of at whichever one came last.
struct HandDemand {
bool present = false;
float amplitude = 0.0f;
uint64_t duration_ns = 0;
};
void AddMotorDemand(HandDemand (&hands)[2], const std::string& motor_name,
float amplitude, uint64_t duration_ns) {
if (!std::isfinite(amplitude))
amplitude = 0.0f;
const int hand = HandForMotor(motor_name);
for (int index = 0; index < 2; ++index) {
if (hand != -1 && hand != index)
continue;
auto& demand = hands[index];
demand.present = true;
if (amplitude > demand.amplitude) {
demand.amplitude = amplitude;
demand.duration_ns = duration_ns;
}
}
}
void DispatchHandDemands(const HandDemand (&hands)[2], const HapticSinkFn& sink,
const HapticStopFn& stop) {
for (int hand = 0; hand < 2; ++hand) {
const auto& demand = hands[hand];
if (!demand.present)
continue;
if (demand.amplitude <= 0.0f) {
if (stop)
stop(hand);
} else if (sink) {
sink(hand, std::clamp(demand.amplitude, 0.0f, 1.0f),
std::clamp(demand.duration_ns, kMinimumDurationNs,
kMaximumDurationNs),
0.0f);
}
}
}
} // namespace
// Plays SetMotorState buffers sample by sample. Roblox sends HapticEffect
// output as look-ahead waveforms (e.g. 100 samples x 25 ms) and replaces them
// as it mixes, so each hand follows its latest buffer in real time.
class HapticWaveformPlayer {
public:
using Clock = std::chrono::steady_clock;
HapticWaveformPlayer(HapticSinkFn sink, HapticStopFn stop)
: sink_(std::move(sink)), stop_(std::move(stop)) {}
~HapticWaveformPlayer() { Halt(); }
HapticWaveformPlayer(const HapticWaveformPlayer&) = delete;
HapticWaveformPlayer& operator=(const HapticWaveformPlayer&) = delete;
// Starts `samples` (already trimmed, at least one) on `hand` now.
void Play(int hand, std::vector<float> samples, uint64_t ns_per_sample) {
{
std::lock_guard<std::mutex> lock(mutex_);
if (halted_)
return;
tracks_[hand] = Track{std::move(samples), ns_per_sample, Clock::now(), 0,
true};
if (!thread_.joinable())
thread_ = std::thread([this] { Run(); });
}
wake_.notify_all();
}
void Stop(int hand) {
{
std::lock_guard<std::mutex> lock(mutex_);
tracks_[hand].active = false;
}
if (stop_)
stop_(hand);
}
// Accepts buffers again after Halt.
void Resume() {
std::lock_guard<std::mutex> lock(mutex_);
halted_ = false;
}
// Ends playback, stopping any hand still vibrating.
void Halt() {
bool was_active[2] = {};
{
std::lock_guard<std::mutex> lock(mutex_);
halted_ = true;
for (int hand = 0; hand < 2; ++hand) {
was_active[hand] = tracks_[hand].active;
tracks_[hand].active = false;
}
}
wake_.notify_all();
if (thread_.joinable())
thread_.join();
for (int hand = 0; hand < 2; ++hand) {
if (was_active[hand] && stop_)
stop_(hand);
}
}
private:
struct Track {
std::vector<float> samples;
uint64_t ns_per_sample = 0;
Clock::time_point start;
std::size_t next = 0;
bool active = false;
};
struct Step {
int hand;
float amplitude;
uint64_t duration_ns;
};
void Run() {
std::unique_lock<std::mutex> lock(mutex_);
while (!halted_) {
const auto now = Clock::now();
std::optional<Clock::time_point> wake_at;
std::vector<Step> steps;
for (int hand = 0; hand < 2; ++hand) {
Track& track = tracks_[hand];
if (!track.active)
continue;
const auto period = std::chrono::nanoseconds(track.ns_per_sample);
const auto index = static_cast<std::size_t>((now - track.start) / period);
if (index >= track.samples.size()) {
track.active = false;
steps.push_back({hand, 0.0f, 0});
continue;
}
// A late wake skips to the current sample rather than replaying.
if (index >= track.next) {
steps.push_back({hand, track.samples[index], track.ns_per_sample});
track.next = index + 1;
}
const auto due = track.start + period * track.next;
if (!wake_at || due < *wake_at)
wake_at = due;
}
lock.unlock();
for (const Step& step : steps) {
if (step.amplitude <= 0.0f) {
if (stop_)
stop_(step.hand);
} else if (sink_) {
// The lease outlives one sample a little so steps join seamlessly;
// the next sample, or the final stop, replaces it.
sink_(step.hand, step.amplitude,
std::clamp(step.duration_ns, kMinimumDurationNs,
kMaximumDurationNs),
0.0f);
}
}
lock.lock();
if (halted_)
break;
if (wake_at)
wake_.wait_until(lock, *wake_at);
else
wake_.wait(lock);
}
}
const HapticSinkFn sink_;
const HapticStopFn stop_;
std::mutex mutex_;
std::condition_variable wake_;
Track tracks_[2];
bool halted_ = false;
std::thread thread_;
};
namespace {
// One hand's share of a SetMotorState: the per-sample maximum of its motors,
// on the period of the first motor that reached it.
struct HandWaveform {
bool present = false;
uint64_t ns_per_sample = 0;
std::vector<float> samples;
};
void AddMotorWaveform(HandWaveform (&hands)[2], const std::string& motor_name,
const std::vector<float>& samples,
uint64_t ns_per_sample) {
const int hand = HandForMotor(motor_name);
for (int index = 0; index < 2; ++index) {
if (hand != -1 && hand != index)
continue;
HandWaveform& wave = hands[index];
if (!wave.present) {
wave.present = true;
wave.ns_per_sample = ns_per_sample;
}
// Resample by time onto the hand's period (Roblox uses one period for
// every motor, so this is normally index for index).
const double ratio = static_cast<double>(wave.ns_per_sample) /
static_cast<double>(ns_per_sample);
const auto length = static_cast<std::size_t>(
std::ceil(static_cast<double>(samples.size()) / ratio));
if (wave.samples.size() < length)
wave.samples.resize(length, 0.0f);
for (std::size_t at = 0; at < length; ++at) {
const auto source = static_cast<std::size_t>(at * ratio);
if (source < samples.size())
wave.samples[at] = std::max(wave.samples[at], samples[source]);
}
}
}
RobloxMessageBusReply Answer(JNIEnv* env, jstring message, std::size_t method,
const HapticSinkFn& sink,
const HapticStopFn& stop,
const HapticStateSyncFn& state_sync,
HapticWaveformPlayer& player) {
// Method 0: SupportsHaptics
if (method == 0) {
if (state_sync) {
state_sync(env);
}
return RobloxMessageBusReply{R"({"supportsHaptics":true})", 0, "supported"};
}
// Method 1: GetMotors
if (method == 1) {
if (state_sync) {
state_sync(env);
}
return RobloxMessageBusReply{RobloxHapticProtocolBridge::GetMotorsJson(), 0,
"ok"};
}
// Method 2: SetMotorState
if (method == 2) {
const auto parsed = ParseJsonMessage(env, message);
if (!parsed.has_value()) {
return RobloxMessageBusReply{"{}", 13, "invalid_request"};
}
const auto& root = *parsed;
HandWaveform hands[2];
for (auto it = root.begin(); it != root.end(); ++it) {
const auto& motor = it.value();
if (!motor.is_object())
continue;
double ms_per_sample = 20.0;
if (motor.contains("millisecondsPerSample") &&
motor["millisecondsPerSample"].is_number()) {
ms_per_sample = motor["millisecondsPerSample"].get<double>();
}
if (!std::isfinite(ms_per_sample) || ms_per_sample <= 0.0)
continue;
ms_per_sample = std::clamp(ms_per_sample, 1.0, 1000.0);
const auto ns_per_sample =
static_cast<uint64_t>(ms_per_sample * 1000000.0);
std::vector<float> samples;
if (motor.contains("intensities") && motor["intensities"].is_array()) {
for (const auto& item : motor["intensities"]) {
const float value = item.is_number() ? item.get<float>() : 0.0f;
samples.push_back(std::isfinite(value) ? std::clamp(value, 0.0f, 1.0f)
: 0.0f);
}
} else if (motor.contains("intensity") && motor["intensity"].is_number()) {
const float value = motor["intensity"].get<float>();
samples.push_back(
std::isfinite(value) ? std::clamp(value, 0.0f, 1.0f) : 0.0f);
}
AddMotorWaveform(hands, it.key(), samples, ns_per_sample);
}
for (int hand = 0; hand < 2; ++hand) {
HandWaveform& wave = hands[hand];
if (!wave.present)
continue;
// The buffer is padded with silence; play only up to its last bump.
while (!wave.samples.empty() && wave.samples.back() <= 0.0f)
wave.samples.pop_back();
const std::size_t longest =
static_cast<std::size_t>(kMaximumDurationNs / wave.ns_per_sample);
if (wave.samples.size() > longest)
wave.samples.resize(longest);
if (wave.samples.empty())
player.Stop(hand);
else
player.Play(hand, std::move(wave.samples), wave.ns_per_sample);
}
return RobloxMessageBusReply{"{}", 0, "ok"};
}
// Method 3: UpdateSingletonVibration
if (method == 3) {
const auto parsed = ParseJsonMessage(env, message);
if (!parsed.has_value()) {
return RobloxMessageBusReply{"{}", 13, "invalid_request"};
}
float intensity = 0.0f;
if (parsed->contains("intensity") && (*parsed)["intensity"].is_number()) {
intensity = (*parsed)["intensity"].get<float>();
}
HandDemand hands[2];
AddMotorDemand(hands, "defaultMotor", intensity, kMinimumDurationNs);
DispatchHandDemands(hands, sink, stop);
return RobloxMessageBusReply{"{}", 0, "ok"};
}
return RobloxMessageBusReply{"{}", 13, "unknown_method"};
}
} // namespace
bool RobloxHapticProtocolBridge::EnabledForThisSession() {
const char* value = std::getenv("MOCKTAIL_VR_ENABLED");
const std::string_view vr = value != nullptr ? value : "";
return vr == "1" || vr == "true" || vr == "on";
}
std::string RobloxHapticProtocolBridge::GetMotorsJson() {
// Positional HapticEffects place hands at x = -1 (left) and +1 (right):
// VOXEL DESTRUCT 2 aims its per-hand effects there with a 0.5 radius.
const nlohmann::json motors = {
{"leftHand", {{"x", -1.0}, {"y", 0.0}, {"z", 0.0}}},
{"rightHand", {{"x", 1.0}, {"y", 0.0}, {"z", 0.0}}},
{"LeftHand", {{"x", -1.0}, {"y", 0.0}, {"z", 0.0}}},
{"RightHand", {{"x", 1.0}, {"y", 0.0}, {"z", 0.0}}},
{"Large", {{"x", -1.0}, {"y", 0.0}, {"z", 0.0}}},
{"Small", {{"x", 1.0}, {"y", 0.0}, {"z", 0.0}}},
{"large", {{"x", -1.0}, {"y", 0.0}, {"z", 0.0}}},
{"small", {{"x", 1.0}, {"y", 0.0}, {"z", 0.0}}},
{"defaultMotor", {{"x", 0.0}, {"y", 0.0}, {"z", 0.0}}},
};
return motors.dump();
}
RobloxHapticProtocolBridge::RobloxHapticProtocolBridge(
JniEnvironmentProvider environment, RobloxMessageBusSymbols symbols,
RobloxMessageBusObjects objects, HapticSinkFn haptic_sink,
HapticStopFn haptic_stop, HapticStateSyncFn haptic_state_sync) {
if (!haptic_sink) {
haptic_sink = [](int hand, float amplitude, uint64_t duration_ns,
float frequency_hz) {
if (&mocktail_vr_controller_request_haptics != nullptr) {
mocktail_vr_controller_request_haptics(hand, amplitude, duration_ns,
frequency_hz);
} else {
using RequestFn = void (*)(int32_t, float, uint64_t, float);
auto fn = reinterpret_cast<RequestFn>(
dlsym(RTLD_DEFAULT, "mocktail_vr_controller_request_haptics"));
if (fn != nullptr) {
fn(hand, amplitude, duration_ns, frequency_hz);
}
}
};
}
if (!haptic_stop) {
haptic_stop = [](int hand) {
if (&mocktail_vr_controller_stop_haptics != nullptr) {
mocktail_vr_controller_stop_haptics(hand);
} else {
using StopFn = void (*)(int32_t);
auto fn = reinterpret_cast<StopFn>(
dlsym(RTLD_DEFAULT, "mocktail_vr_controller_stop_haptics"));
if (fn != nullptr) {
fn(hand);
}
}
};
}
player_ = std::make_shared<HapticWaveformPlayer>(haptic_sink, haptic_stop);
bridge_ = std::make_unique<RobloxMessageBusRequestBridge>(
environment, symbols, objects, "HapticProtocol",
std::vector<std::string>{"SupportsHaptics", "GetMotors", "SetMotorState",
"UpdateSingletonVibration"},
[sink = std::move(haptic_sink), stop = std::move(haptic_stop),
sync = std::move(haptic_state_sync), player = player_](
JNIEnv* env, jstring message, std::size_t method) {
return Answer(env, message, method, sink, stop, sync, *player);
});
}
RobloxHapticProtocolBridge::~RobloxHapticProtocolBridge() {
// A handler may outlive the bridge inside the message bus; never let its
// player keep a controller vibrating.
player_->Halt();
}
Status RobloxHapticProtocolBridge::Initialize() {
player_->Resume();
return bridge_->Initialize();
}
Status RobloxHapticProtocolBridge::Shutdown() {
const Status status = bridge_->Shutdown();
player_->Halt();
return status;
}
} // namespace runtime
} // namespace mocktail
@@ -350,6 +350,20 @@ Status RobloxTextSurfaceOverlay::UpdateViewport(
return Status::Ok();
}
namespace {
std::mutex g_display_mirror_mutex;
RobloxTextDisplaySink g_display_mirror;
// Lets a text update skip the lock while no mirror is installed.
std::atomic<bool> g_display_mirror_set{false};
} // namespace
void SetRobloxTextDisplayMirror(RobloxTextDisplaySink sink) {
std::lock_guard<std::mutex> lock(g_display_mirror_mutex);
g_display_mirror = sink;
g_display_mirror_set.store(sink.update != nullptr,
std::memory_order_release);
}
RobloxTextDisplaySink RobloxTextSurfaceOverlay::sink() {
return {this, &RobloxTextSurfaceOverlay::UpdateCallback};
}
@@ -359,6 +373,13 @@ void RobloxTextSurfaceOverlay::UpdateCallback(
if (context != nullptr) {
static_cast<RobloxTextSurfaceOverlay*>(context)->ApplyUpdate(update);
}
if (!g_display_mirror_set.load(std::memory_order_acquire)) {
return;
}
std::lock_guard<std::mutex> lock(g_display_mirror_mutex);
if (g_display_mirror.update != nullptr) {
g_display_mirror.update(g_display_mirror.context, update);
}
}
void RobloxTextSurfaceOverlay::ApplyUpdate(
@@ -80,6 +80,14 @@ Status RobloxWindowInputRuntime::Initialize() {
gamepad_status.message().c_str());
}
}
// XR tracked controllers join the same authoritative route. Initialize is a
// no-op (false) in non-VR builds or without an armed OpenXR backend; that is
// the normal state, not an error.
xr_controllers_initialized_ = xr_controllers_.Initialize(
&RobloxWindowInputRuntime::XrControllerEventCallback, this);
if (xr_controllers_initialized_) {
std::fprintf(stderr, " [vr-input] XR controller delivery armed\n");
}
return Status::Ok();
}
@@ -108,6 +116,7 @@ void RobloxWindowInputRuntime::PlatformEventCallback(
std::get_if<platform::WindowFocusEvent>(&event.payload)) {
if (focus->focused) {
self->gamepads_.ResendState();
self->xr_controllers_.ResendState();
}
}
}
@@ -124,6 +133,10 @@ Status RobloxWindowInputRuntime::Shutdown() {
window::ClearMouseLockQueryCallback();
mouse_lock_query_registered_ = false;
}
// Disconnect the synthetic XR controller while the router still processes
// events, so its release path runs through the normal route.
xr_controllers_.Shutdown();
xr_controllers_initialized_ = false;
gamepads_.Shutdown();
const RobloxInputSnapshot snapshot = runtime_.Snapshot();
Status status = runtime_.Deactivate();
@@ -203,6 +216,37 @@ RobloxInputSnapshot RobloxWindowInputRuntime::Snapshot() const {
return runtime_.Snapshot();
}
void RobloxWindowInputRuntime::InjectPlatformEvent(
const platform::PlatformEvent& event) {
// Same router and same thread as SDL delivery: the XR controller path is a
// producer of platform events, not a parallel input stack.
(void)runtime_.HandleEvent(event);
}
void RobloxWindowInputRuntime::XrControllerEventCallback(
void* context, const platform::PlatformEvent& event) {
auto* self = static_cast<RobloxWindowInputRuntime*>(context);
if (self != nullptr) {
self->InjectPlatformEvent(event);
}
}
void RobloxWindowInputRuntime::DrainXrControllers() {
if (!xr_controllers_initialized_) {
return;
}
xr_controllers_.Drain();
}
void RobloxWindowInputRuntime::RequestXrControllerHaptics(
int hand, float amplitude, std::uint64_t duration_ns, float frequency_hz) {
xr_controllers_.RequestHaptics(hand, amplitude, duration_ns, frequency_hz);
}
void RobloxWindowInputRuntime::StopXrControllerHaptics(int hand) {
xr_controllers_.StopHaptics(hand);
}
bool RobloxWindowInputRuntime::MouseLockQueryCallback(void* context,
bool* locked_center) {
if (context == nullptr || locked_center == nullptr) {
+272
View File
@@ -0,0 +1,272 @@
#include "runtime/roblox_xr_controller_input.h"
#include <dlfcn.h>
#include <algorithm>
#include <cmath>
#include <cstdio>
#include "mocktail/vr/xr_controller.h"
#include "mocktail/vr/xr_controller_abi.h"
namespace mocktail {
namespace runtime {
namespace {
// SDL3 gamepad ordinals for the merged synthetic controller. Both hands feed
// one device: left hand -> left stick/left shoulder/X/Y, right hand -> right
// stick/right shoulder/A/B; menu -> START.
constexpr std::int16_t kAxisFull = 32767;
std::int16_t FloatToRaw(float value) {
const float clamped =
std::clamp(std::isfinite(value) ? value : 0.f, -1.f, 1.f);
return static_cast<std::int16_t>(std::lround(clamped * kAxisFull));
}
std::int16_t TriggerToRaw(float value) {
const float clamped =
std::clamp(std::isfinite(value) ? value : 0.f, 0.f, 1.f);
return static_cast<std::int16_t>(std::lround(clamped * kAxisFull));
}
std::uint64_t NowNs() {
timespec ts{};
clock_gettime(CLOCK_MONOTONIC, &ts);
return static_cast<std::uint64_t>(ts.tv_sec) * 1000000000ULL +
static_cast<std::uint64_t>(ts.tv_nsec);
}
} // namespace
bool RobloxXrControllerInput::Initialize(EventSink sink, void *context) {
if (sink == nullptr) {
return false;
}
// Resolve the XR controller ABI through dlsym so this module links and runs
// in non-VR builds (symbols absent) and stays inert while no backend is
// armed. Never a hard dependency on OpenXR here.
auto pop = reinterpret_cast<MocktailVrControllerPopFn>(
dlsym(RTLD_DEFAULT, "mocktail_vr_controller_pop_delivery"));
auto haptics = reinterpret_cast<MocktailVrControllerHapticsFn>(
dlsym(RTLD_DEFAULT, "mocktail_vr_controller_request_haptics"));
auto stop_haptics = reinterpret_cast<MocktailVrControllerStopHapticsFn>(
dlsym(RTLD_DEFAULT, "mocktail_vr_controller_stop_haptics"));
return InitializeWithAbi(sink, context, pop, haptics, stop_haptics);
}
bool RobloxXrControllerInput::InitializeWithAbi(
EventSink sink, void *context, MocktailVrControllerPopFn pop,
MocktailVrControllerHapticsFn haptics,
MocktailVrControllerStopHapticsFn stop_haptics) {
if (sink == nullptr || pop == nullptr) {
// No XR controller layer (non-VR build or backend without controllers).
// This is the normal case for --no-vr and must not be treated as failure.
sink_ = nullptr;
context_ = nullptr;
pop_ = nullptr;
haptics_ = nullptr;
stop_haptics_ = nullptr;
initialized_ = false;
return false;
}
sink_ = sink;
context_ = context;
pop_ = reinterpret_cast<void *>(pop);
haptics_ = reinterpret_cast<void *>(haptics);
stop_haptics_ = reinterpret_cast<void *>(stop_haptics);
initialized_ = true;
return true;
}
void RobloxXrControllerInput::Shutdown() {
if (connected_) {
// Release the synthetic controller once so no button or stick stays live
// after teardown.
EmitConnection(false);
}
initialized_ = false;
generation_ = 0;
force_resync_ = false;
sink_ = nullptr;
context_ = nullptr;
pop_ = nullptr;
haptics_ = nullptr;
stop_haptics_ = nullptr;
}
void RobloxXrControllerInput::EmitConnection(bool connected) {
if (sink_ == nullptr) {
return;
}
platform::GamepadDescriptor descriptor;
descriptor.family = platform::GamepadFamily::kXbox;
descriptor.name = connected ? "Mocktail VR Controllers" : "";
if (connected) {
// Advertise exactly the buttons/axes this route can emit so the router
// accepts them: SOUTH/EAST/WEST/NORTH, START, both stick clicks, both
// shoulders.
for (const std::uint8_t button :
{static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
0, vr::ControllerButton::kFaceWest)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
0, vr::ControllerButton::kFaceNorth)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
1, vr::ControllerButton::kFaceSouth)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
1, vr::ControllerButton::kFaceEast)),
static_cast<std::uint8_t>(
vr::ControllerButtonToSdlOrdinal(0, vr::ControllerButton::kMenu)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
0, vr::ControllerButton::kThumbstick)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
1, vr::ControllerButton::kThumbstick)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
0, vr::ControllerButton::kSqueeze)),
static_cast<std::uint8_t>(vr::ControllerButtonToSdlOrdinal(
1, vr::ControllerButton::kSqueeze))}) {
descriptor.buttons |= (std::uint32_t{1} << button);
}
// Six axes: both sticks (x/y) and both triggers.
descriptor.axes = 0b111111;
}
platform::GamepadConnectionEvent event;
event.instance_id = kInstanceLeft;
event.connected = connected;
event.descriptor = std::move(descriptor);
event.remapped = false;
sink_(context_, platform::PlatformEvent{NowNs(), std::move(event)});
connected_ = connected;
std::fill(std::begin(last_buttons_), std::end(last_buttons_), false);
if (connected) {
std::fprintf(stderr, " [vr-input] XR controllers connected to Roblox as a "
"single synthetic gamepad\n");
} else {
std::fprintf(stderr,
" [vr-input] XR controllers disconnected from Roblox\n");
}
}
void RobloxXrControllerInput::EmitButton(int sdl_button, bool pressed) {
if (sink_ == nullptr || sdl_button < 0) {
return;
}
platform::GamepadButtonEvent event;
event.instance_id = kInstanceLeft;
event.button = static_cast<std::uint8_t>(sdl_button);
event.pressed = pressed;
sink_(context_, platform::PlatformEvent{NowNs(), std::move(event)});
++delivered_events_;
}
void RobloxXrControllerInput::EmitAxis(int sdl_axis, std::int16_t value) {
if (sink_ == nullptr || sdl_axis < 0) {
return;
}
platform::GamepadAxisEvent event;
event.instance_id = kInstanceLeft;
event.axis = static_cast<std::uint8_t>(sdl_axis);
event.value = value;
sink_(context_, platform::PlatformEvent{NowNs(), std::move(event)});
++delivered_events_;
}
void RobloxXrControllerInput::Drain() {
if (!initialized_ || pop_ == nullptr || sink_ == nullptr) {
return;
}
auto pop = reinterpret_cast<MocktailVrControllerPopFn>(pop_);
for (int batch = 0; batch < kMaxDrainPerPump; ++batch) {
MocktailVrControllerDelivery delivery{};
if (pop(&delivery) == 0) {
return;
}
if (delivery.session_generation < generation_)
continue;
if (delivery.session_generation != generation_) {
if (connected_)
EmitConnection(false);
generation_ = delivery.session_generation;
}
// Connection transitions first so subsequent button/axis events land on a
// registered device. Either hand being connected presents the merged
// controller as connected.
const bool connected = delivery.connected[0] || delivery.connected[1];
if (connected != connected_) {
EmitConnection(connected);
}
if (!connected_) {
// Nothing to deliver while disconnected; the disconnect already released
// every control through the router.
continue;
}
if (force_resync_) {
delivery.resync = 1;
force_resync_ = false;
}
if (delivery.resync)
++resyncs_;
// Levels belong to this delivery, not a separately read newer snapshot.
// Merge first so a release/unbound button on one hand cannot cancel the
// other.
bool merged[16] = {};
for (int hand = 0; hand < 2; ++hand) {
if (!delivery.connected[hand])
continue;
for (int index = 0; index < MOCKTAIL_VR_CONTROLLER_BUTTON_COUNT;
++index) {
const int button = vr::ControllerButtonToSdlOrdinal(
hand, static_cast<vr::ControllerButton>(index));
if (button >= 0 && button < 16)
merged[button] |= delivery.button_levels[hand][index] != 0;
}
}
for (int button = 0; button < 16; ++button) {
if (delivery.resync || merged[button] != last_buttons_[button])
EmitButton(button, merged[button]);
last_buttons_[button] = merged[button];
}
// Analog axes. Delivered as raw int16 so the router applies its single
// deadzone; we never pre-filter (avoids a double deadzone).
if (delivery.axes_valid[0] || delivery.axes_valid[1]) {
EmitAxis(
vr::ControllerAxisToSdlOrdinal(0, vr::ControllerAxis::kThumbstickX),
FloatToRaw(delivery.axes[0]));
EmitAxis(
vr::ControllerAxisToSdlOrdinal(0, vr::ControllerAxis::kThumbstickY),
FloatToRaw(delivery.axes[1]));
EmitAxis(
vr::ControllerAxisToSdlOrdinal(1, vr::ControllerAxis::kThumbstickX),
FloatToRaw(delivery.axes[2]));
EmitAxis(
vr::ControllerAxisToSdlOrdinal(1, vr::ControllerAxis::kThumbstickY),
FloatToRaw(delivery.axes[3]));
EmitAxis(vr::ControllerAxisToSdlOrdinal(0, vr::ControllerAxis::kTrigger),
TriggerToRaw(delivery.axes[4]));
EmitAxis(vr::ControllerAxisToSdlOrdinal(1, vr::ControllerAxis::kTrigger),
TriggerToRaw(delivery.axes[5]));
}
}
}
void RobloxXrControllerInput::RequestHaptics(int hand, float amplitude,
std::uint64_t duration_ns,
float frequency_hz) {
if (haptics_ == nullptr || (hand != 0 && hand != 1)) {
return;
}
reinterpret_cast<MocktailVrControllerHapticsFn>(haptics_)(
hand, amplitude, duration_ns, frequency_hz);
}
void RobloxXrControllerInput::StopHaptics(int hand) {
if (stop_haptics_ == nullptr || (hand != 0 && hand != 1)) {
return;
}
reinterpret_cast<MocktailVrControllerStopHapticsFn>(stop_haptics_)(hand);
}
} // namespace runtime
} // namespace mocktail
+30
View File
@@ -173,9 +173,35 @@ std::string BuildNetworkProxyUrl(const NetworkProxyConfig& proxy) {
(ipv6 ? "]" : "") + ":" + std::to_string(proxy.port);
}
const char* VrModeName(VrMode mode) {
switch (mode) {
case VrMode::kOn:
return "on";
case VrMode::kAuto:
return "auto";
case VrMode::kOff:
break;
}
return "off";
}
RuntimeConfig RuntimeConfig::FromEnvironment(const Environment& environment) {
RuntimeConfig config;
config.headless_ = LegacyEnabled(environment, "MOCKTAIL_HEADLESS");
if (environment.Get("MOCKTAIL_VR_ENABLED").has_value()) {
config.vr_enabled_ = ReadBoolean(environment, "MOCKTAIL_VR_ENABLED",
kDefaultVrEnabled, &config.vr_valid_);
config.vr_mode_ = config.vr_enabled_ ? VrMode::kOn : VrMode::kOff;
} else if (const auto mode = environment.Get("MOCKTAIL_VR_MODE");
mode.has_value()) {
if (*mode == "auto") {
config.vr_mode_ = VrMode::kAuto;
} else {
config.vr_enabled_ = ReadBoolean(environment, "MOCKTAIL_VR_MODE",
kDefaultVrEnabled, &config.vr_valid_);
config.vr_mode_ = config.vr_enabled_ ? VrMode::kOn : VrMode::kOff;
}
}
config.roblox_library_path_ = environment.GetOr(
"ROBLOX_LIB_PATH", config.roblox_library_path_.string());
config.graphics_backend_name_ = environment.GetOr(
@@ -281,6 +307,10 @@ RuntimeConfig RuntimeConfig::FromEnvironment(const Environment& environment) {
}
config.frame_rate_ = ParseFrameRatePolicy(
environment.GetOr("MOCKTAIL_FRAME_RATE_LIMIT", "-1"));
config.vr_frame_rate_ = ParseFrameRatePolicy(
environment.GetOr("MOCKTAIL_VR_FRAME_RATE_LIMIT", "unlimited"));
config.vr_anti_aliasing_ =
environment.GetOr("MOCKTAIL_VR_ANTI_ALIASING", config.vr_anti_aliasing_);
config.vsync_mode_ = environment.GetOr("MOCKTAIL_VSYNC", "auto");
config.performance_ = ParsePerformancePolicy(
environment.GetOr("MOCKTAIL_MULTITHREADED_RENDERING", "0"),
+22
View File
@@ -71,6 +71,28 @@ graphics:
# unless frame_rate_limit is unlimited.
# vsync: off
# vr:
# String (default: off): experimental VR through OpenXR on supported Roblox
# builds. off always starts on the desktop. on always starts in VR, like
# --vr. auto starts in VR when SteamVR is already running with a connected
# headset (ALVR included), and on the desktop otherwise. auto never starts
# SteamVR. --vr and --no-vr override this for one launch. The older
# enabled: true/false still works as on/off; set one or the other.
# Standalone OpenXR test scene: mocktail-vr-probe --scene.
# mode: off
# String (default: fxaa): smooth jagged edges in the headset. Roblox draws
# the VR eyes without anti-aliasing, so each eye is filtered on its way to
# the headset. fxaa is the cheapest. smaa finds edges more precisely and
# keeps text and fine detail sharper, at a little more GPU time. off sends
# the eyes as rendered.
# anti_aliasing: fxaa
# Optional string or integer: frame-rate limit used instead of
# graphics.frame_rate_limit while VR is active. Same values: -1 (Roblox's
# own cap, usually 60), display, unlimited, or any positive integer. The
# default, unlimited, lets the OpenXR runtime pace frames at the headset's
# refresh rate.
# frame_rate_limit: unlimited
performance:
# Boolean (default: false): size Roblox scheduler/render queues from every
# physical CPU core. A place's Lua/main thread can still remain serial.
+61 -1
View File
@@ -168,6 +168,10 @@ bool ValidateAndMap(const ValueMap& yaml, ValueMap* environment,
"device.keyboard",
"runtime.headless",
"runtime.roblox_library",
"vr.enabled",
"vr.mode",
"vr.anti_aliasing",
"vr.frame_rate_limit",
"appearance.theme",
"graphics.backend",
"graphics.frame_rate_limit",
@@ -318,6 +322,46 @@ bool ValidateAndMap(const ValueMap& yaml, ValueMap* environment,
}
(*environment)["MOCKTAIL_THEME"] = *theme;
}
// The file only sets MOCKTAIL_VR_MODE, so --vr and --no-vr
// (MOCKTAIL_VR_ENABLED) always win over it.
const auto enabled = value("vr.enabled");
const auto mode = value("vr.mode");
if (enabled.has_value() && mode.has_value()) {
*error = "set either vr.mode or vr.enabled, not both";
return false;
}
if (enabled.has_value()) {
bool parsed = false;
if (!ParseBoolean(*enabled, &parsed)) {
*error = "vr.enabled must be true or false";
return false;
}
(*environment)["MOCKTAIL_VR_MODE"] = parsed ? "on" : "off";
}
if (mode.has_value()) {
if (*mode != "on" && *mode != "off" && *mode != "auto") {
*error = "vr.mode must be on, off, or auto";
return false;
}
(*environment)["MOCKTAIL_VR_MODE"] = *mode;
}
if (const auto anti_aliasing = value("vr.anti_aliasing");
anti_aliasing.has_value()) {
if (*anti_aliasing != "fxaa" && *anti_aliasing != "smaa" &&
*anti_aliasing != "off") {
*error = "vr.anti_aliasing must be fxaa, smaa, or off";
return false;
}
(*environment)["MOCKTAIL_VR_ANTI_ALIASING"] = *anti_aliasing;
}
if (const auto frame_rate = value("vr.frame_rate_limit");
frame_rate.has_value()) {
if (!ParseFrameRatePolicy(*frame_rate).valid()) {
*error = "vr.frame_rate_limit is not supported: " + *frame_rate;
return false;
}
(*environment)["MOCKTAIL_VR_FRAME_RATE_LIMIT"] = *frame_rate;
}
if (const auto backend = value("graphics.backend"); backend.has_value()) {
if (RuntimeConfig::ParseGraphicsBackend(*backend) ==
GraphicsBackend::kUnknown) {
@@ -712,7 +756,7 @@ bool LoadYaml(const std::filesystem::path& path, ValueMap* values, bool* loaded,
*error = "device must be a preset scalar or detailed mapping";
valid = false;
}
} else if (key == "runtime" || key == "appearance" ||
} else if (key == "runtime" || key == "appearance" || key == "vr" ||
key == "graphics" || key == "performance" ||
key == "audio" || key == "window" || key == "input" ||
key == "compatibility" || key == "network") {
@@ -781,6 +825,12 @@ RuntimeConfigLoadResult LoadRuntimeConfig(
result.config = RuntimeConfig::FromEnvironment(layered);
if (!result.config.frame_rate().valid()) {
result.error = "frame-rate policy is invalid";
} else if (!result.config.vr_frame_rate().valid()) {
result.error = "VR frame-rate policy is invalid";
} else if (!result.config.vr_valid()) {
result.error = "VR configuration is invalid: MOCKTAIL_VR_ENABLED must be "
"1/0, true/false, or on/off, and MOCKTAIL_VR_MODE on, off, "
"or auto";
} else if (!result.config.device_profile_valid()) {
result.error = "device profile is invalid";
} else if (!result.config.input_capabilities().touch_enabled &&
@@ -821,6 +871,10 @@ RuntimeConfigLoadResult LoadRuntimeConfig(
bool ExportRuntimeConfigEnvironment(const RuntimeConfig& config,
std::string* error) {
if (!config.vr_valid()) {
if (error != nullptr) *error = "cannot export an invalid VR configuration";
return false;
}
if (!config.fleasion_valid()) {
if (error != nullptr) *error = "cannot export invalid Fleasion configuration";
return false;
@@ -877,6 +931,8 @@ bool ExportRuntimeConfigEnvironment(const RuntimeConfig& config,
const bool base_exported =
SetEnvironmentValue("MOCKTAIL_HEADLESS", config.headless() ? "1" : "0",
error) &&
SetEnvironmentValue("MOCKTAIL_VR_ENABLED", config.vr_enabled() ? "1" : "0",
error) &&
SetEnvironmentValue("MOCKTAIL_DEVICE_PROFILE", std::string(device.name),
error) &&
SetEnvironmentValue("MOCKTAIL_DEVICE_CLASS",
@@ -926,6 +982,10 @@ bool ExportRuntimeConfigEnvironment(const RuntimeConfig& config,
config.desktop_playability() ? "1" : "0", error) &&
SetEnvironmentValue("MOCKTAIL_FRAME_RATE_LIMIT",
FrameRateValue(config.frame_rate()), error) &&
SetEnvironmentValue("MOCKTAIL_VR_FRAME_RATE_LIMIT",
FrameRateValue(config.vr_frame_rate()), error) &&
SetEnvironmentValue("MOCKTAIL_VR_ANTI_ALIASING",
config.vr_anti_aliasing(), error) &&
SetEnvironmentValue("MOCKTAIL_VSYNC", config.vsync_mode(), error) &&
SetEnvironmentValue(
"MOCKTAIL_MULTITHREADED_RENDERING",
+75
View File
@@ -0,0 +1,75 @@
#include "mocktail/vr/openxr_probe.h"
#include "mocktail/vr/openxr_preview.h"
#include <sstream>
namespace mocktail::vr {
std::string FormatPreviewResult(const PreviewResult& result) {
std::ostringstream output;
output << "[VR] " << result.message << '\n'
<< "Frames: " << result.frames
<< ", rendered: " << result.rendered_frames
<< ", tracked: " << result.tracked_frames << '\n';
if (result.tracked_frames) {
output << "Head position (metres): " << result.head_position[0] << ", "
<< result.head_position[1] << ", " << result.head_position[2] << '\n'
<< "Head orientation (xyzw): " << result.head_orientation[0] << ", "
<< result.head_orientation[1] << ", " << result.head_orientation[2]
<< ", " << result.head_orientation[3] << '\n';
}
return output.str();
}
int ProbeExitCode(ProbeStatus status) {
switch (status) {
case ProbeStatus::kReady:
return 0;
case ProbeStatus::kBuildDisabled:
return 2;
case ProbeStatus::kRuntimeUnavailable:
return 3;
case ProbeStatus::kHeadsetUnavailable:
return 4;
case ProbeStatus::kIncompatibleRuntime:
return 5;
case ProbeStatus::kError:
return 1;
}
return 1;
}
std::string FormatProbeReport(const ProbeReport& report) {
std::ostringstream output;
output << "Mocktail OpenXR discovery\n";
if (!report.sdk_version.empty())
output << "SDK: " << report.sdk_version << '\n';
if (!report.runtime_name.empty()) {
output << "Runtime: " << report.runtime_name << " ("
<< report.runtime_version << ")\n";
}
if (!report.system_name.empty()) {
output << "System: " << report.system_name << '\n'
<< "Tracking: orientation=" << report.orientation_tracking
<< " position=" << report.position_tracking << '\n';
}
for (std::size_t eye = 0; eye < report.views.size(); ++eye) {
const auto& view = report.views[eye];
output << "Eye " << eye << ": " << view.width << 'x' << view.height
<< ", samples=" << view.samples << '\n';
}
if (!report.minimum_vulkan_version.empty()) {
output << "Vulkan: minimum=" << report.minimum_vulkan_version
<< ", highest runtime-tested=" << report.maximum_vulkan_version
<< '\n';
}
output << "Extensions (" << report.extensions.size() << "):\n";
for (const auto& extension : report.extensions)
output << " " << extension << '\n';
output << "Discovery: " << report.message << '\n'
<< "Roblox VR rendering: run mocktail --vr with a supported client.\n"
<< "This probe does not create an XR session or render frames.\n";
return output.str();
}
} // namespace mocktail::vr
+424
View File
@@ -0,0 +1,424 @@
#include "vr/gles_transport.h"
#include "mocktail/vr/openxr_backend.h"
#include <EGL/egl.h>
#include <GLES3/gl3.h>
#include <cstdio>
#include <cstring>
extern "C" void *mocktail_vr_initializing_object();
extern "C" bool mocktail_vr_current_eye(void **, int *, void **);
extern "C" void mocktail_vr_consume_eye();
namespace mocktail::vr {
namespace {
// The window owns a single EGL context. Function pointers are loaded from
// that exact SDL EGL/GLES implementation, never from libGL or the guest shim.
#define GL_PROCS(X) \
X(GetIntegerv, PFNGLGETINTEGERVPROC) \
X(IsEnabled, PFNGLISENABLEDPROC) \
X(Enable, PFNGLENABLEPROC) \
X(Disable, PFNGLDISABLEPROC) \
X(GenFramebuffers, PFNGLGENFRAMEBUFFERSPROC) X(DeleteFramebuffers, \
PFNGLDELETEFRAMEBUFFERSPROC) \
X(BindFramebuffer, PFNGLBINDFRAMEBUFFERPROC) \
X(CheckFramebufferStatus, PFNGLCHECKFRAMEBUFFERSTATUSPROC) X( \
GetFramebufferAttachmentParameteriv, \
PFNGLGETFRAMEBUFFERATTACHMENTPARAMETERIVPROC) \
X(FramebufferTexture2D, PFNGLFRAMEBUFFERTEXTURE2DPROC) X( \
FramebufferTextureLayer, PFNGLFRAMEBUFFERTEXTURELAYERPROC) \
X(FramebufferRenderbuffer, PFNGLFRAMEBUFFERRENDERBUFFERPROC) \
X(TexImage2D, PFNGLTEXIMAGE2DPROC) X( \
TexStorage2D, PFNGLTEXSTORAGE2DPROC) \
X(RenderbufferStorage, PFNGLRENDERBUFFERSTORAGEPROC) \
X(RenderbufferStorageMultisample, \
PFNGLRENDERBUFFERSTORAGEMULTISAMPLEPROC) \
X(DeleteTextures, PFNGLDELETETEXTURESPROC) \
X(DeleteRenderbuffers, \
PFNGLDELETERENDERBUFFERSPROC) \
X(BlitFramebuffer, \
PFNGLBLITFRAMEBUFFERPROC) \
X(Flush, PFNGLFLUSHPROC) \
X(BindTexture, \
PFNGLBINDTEXTUREPROC) \
X(TexSubImage2D, \
PFNGLTEXSUBIMAGE2DPROC)\
X(PixelStorei, \
PFNGLPIXELSTOREIPROC)
struct Procs {
#define DECLARE(name, type) type name = nullptr;
GL_PROCS(DECLARE)
#undef DECLARE
} gl;
thread_local GlesTransport *current = nullptr;
GlesTransport *Current() {
return current && current->IsCurrent() ? current : nullptr;
}
void GL_APIENTRY GenFramebuffers(GLsizei n, GLuint *ids) {
gl.GenFramebuffers(n, ids);
if (auto *t = Current())
t->Generated(n, ids);
}
void GL_APIENTRY DeleteFramebuffers(GLsizei n, const GLuint *ids) {
if (auto *t = Current())
t->Deleted(n, ids);
gl.DeleteFramebuffers(n, ids);
}
void GL_APIENTRY BindFramebuffer(GLenum target, GLuint id) {
gl.BindFramebuffer(target, id);
if (auto *t = Current())
t->Bind(target, id);
}
void GL_APIENTRY FramebufferTexture2D(GLenum target, GLenum attachment,
GLenum tex_target, GLuint texture,
GLint level) {
gl.FramebufferTexture2D(target, attachment, tex_target, texture, level);
if (auto *t = Current())
t->AttachmentChanged(target);
}
void GL_APIENTRY FramebufferTextureLayer(GLenum target, GLenum attachment,
GLuint texture, GLint level,
GLint layer) {
gl.FramebufferTextureLayer(target, attachment, texture, level, layer);
if (auto *t = Current())
t->AttachmentChanged(target);
}
void GL_APIENTRY FramebufferRenderbuffer(GLenum target, GLenum attachment,
GLenum rb_target, GLuint rb) {
gl.FramebufferRenderbuffer(target, attachment, rb_target, rb);
if (auto *t = Current())
t->AttachmentChanged(target);
}
void GL_APIENTRY TexImage2D(GLenum target, GLint level, GLint internal_format,
GLsizei w, GLsizei h, GLint border, GLenum format,
GLenum type, const void *data) {
gl.TexImage2D(target, level, internal_format, w, h, border, format, type,
data);
if (target == GL_TEXTURE_2D && level == 0)
if (auto *t = Current())
t->Storage(true, w, h, internal_format);
}
void GL_APIENTRY TexStorage2D(GLenum target, GLsizei levels, GLenum format,
GLsizei w, GLsizei h) {
gl.TexStorage2D(target, levels, format, w, h);
if (target == GL_TEXTURE_2D)
if (auto *t = Current())
t->Storage(true, w, h, format);
}
void GL_APIENTRY RenderbufferStorage(GLenum target, GLenum format, GLsizei w,
GLsizei h) {
gl.RenderbufferStorage(target, format, w, h);
if (target == GL_RENDERBUFFER)
if (auto *t = Current())
t->Storage(false, w, h, format);
}
void GL_APIENTRY RenderbufferStorageMultisample(GLenum target, GLsizei samples,
GLenum format, GLsizei w,
GLsizei h) {
gl.RenderbufferStorageMultisample(target, samples, format, w, h);
if (target == GL_RENDERBUFFER)
if (auto *t = Current())
t->Storage(false, w, h, format, samples);
}
void GL_APIENTRY DeleteTextures(GLsizei n, const GLuint *ids) {
if (auto *t = Current())
t->ObjectsDeleted(true, n, ids);
gl.DeleteTextures(n, ids);
}
void GL_APIENTRY DeleteRenderbuffers(GLsizei n, const GLuint *ids) {
if (auto *t = Current())
t->ObjectsDeleted(false, n, ids);
gl.DeleteRenderbuffers(n, ids);
}
} // namespace
bool GlesTransport::Initialize(void *display, void *config, void *context,
void *egl_get_proc,
void *(*resolve)(const char *),
std::string *error) {
if (!display || !config || !context || !egl_get_proc || !resolve) {
*error =
"OpenGL VR requires a real current EGL context, config and display";
return false;
}
#define LOAD(name, type) \
gl.name = reinterpret_cast<type>(resolve("gl" #name)); \
if (!gl.name) { \
*error = "OpenGL VR is missing gl" #name; \
return false; \
}
GL_PROCS(LOAD)
#undef LOAD
using GetProc = __eglMustCastToProperFunctionPointerType (*)(const char *);
get_current_context_ = reinterpret_cast<void *(*)()>(
reinterpret_cast<GetProc>(egl_get_proc)("eglGetCurrentContext"));
if (!get_current_context_ || get_current_context_() != context) {
*error = "OpenGL VR must initialize on the current EGL render context";
return false;
}
GLint major = 0, minor = 0;
gl.GetIntegerv(GL_MAJOR_VERSION, &major);
gl.GetIntegerv(GL_MINOR_VERSION, &minor);
if (major < 3) {
*error = "OpenGL VR requires OpenGL ES 3.0 or newer";
return false;
}
version_ = (std::uint64_t(major) << 48) | (std::uint64_t(minor) << 32);
display_ = display;
config_ = config;
context_ = context;
egl_get_proc_ = egl_get_proc;
gl.GenFramebuffers(2, copy_fbos_);
if (!copy_fbos_[0] || !copy_fbos_[1]) {
*error = "Cannot allocate OpenGL VR copy framebuffers";
return false;
}
MakeCurrent();
return true;
}
bool GlesTransport::IsCurrent() const {
return get_current_context_ && get_current_context_() == context_;
}
void GlesTransport::MakeCurrent() { current = this; }
void GlesTransport::ReleaseCurrent() { current = nullptr; }
void GlesTransport::Destroy() {
if (IsCurrent())
gl.DeleteFramebuffers(2, copy_fbos_);
copy_fbos_[0] = copy_fbos_[1] = 0;
if (current == this)
current = nullptr;
owners_.clear();
textures_.clear();
renderbuffers_.clear();
ResetEyes();
context_ = nullptr;
}
void *GlesTransport::Wrap(const char *name, void *raw) {
if (!raw || !name)
return raw;
#define WRAP(name_) \
if (std::strcmp(name, "gl" #name_) == 0) \
return reinterpret_cast<void *>(&name_);
WRAP(GenFramebuffers)
WRAP(DeleteFramebuffers)
WRAP(BindFramebuffer) WRAP(FramebufferTexture2D) WRAP(FramebufferRenderbuffer)
WRAP(FramebufferTextureLayer) WRAP(TexImage2D) WRAP(TexStorage2D)
WRAP(RenderbufferStorage) WRAP(RenderbufferStorageMultisample)
WRAP(DeleteTextures) WRAP(DeleteRenderbuffers)
#undef WRAP
return raw;
}
void GlesTransport::ResetEyes() {
eyes_[0] = {};
eyes_[1] = {};
}
void GlesTransport::Generated(int count, const unsigned *ids) {
void *owner = mocktail_vr_initializing_object();
if (!owner || !ids)
return;
for (int i = 0; i < count; ++i)
if (ids[i])
owners_[ids[i]] = owner;
}
void GlesTransport::Deleted(int count, const unsigned *ids) {
if (!ids)
return;
for (int i = 0; i < count; ++i) {
owners_.erase(ids[i]);
for (auto &eye : eyes_)
if (eye.framebuffer == ids[i])
eye = {};
}
}
void GlesTransport::ObjectsDeleted(bool texture, int count,
const unsigned *ids) {
if (!ids)
return;
for (int i = 0; i < count; ++i) {
(texture ? textures_ : renderbuffers_).erase(ids[i]);
for (auto &eye : eyes_)
if (eye.texture == texture && eye.object == ids[i])
eye = {};
}
}
void GlesTransport::Storage(bool texture, int w, int h, unsigned format,
int samples) {
GLint id = 0;
gl.GetIntegerv(texture ? GL_TEXTURE_BINDING_2D : GL_RENDERBUFFER_BINDING,
&id);
if (!id)
return;
// Storage redefinition invalidates previously rendered eye content.
const unsigned object = static_cast<unsigned>(id);
ObjectsDeleted(texture, 1, &object);
(texture ? textures_ : renderbuffers_)[object] = {w, h, format, samples};
}
void GlesTransport::AttachmentChanged(unsigned target) {
GLint id = 0;
gl.GetIntegerv(target == GL_READ_FRAMEBUFFER ? GL_READ_FRAMEBUFFER_BINDING
: GL_DRAW_FRAMEBUFFER_BINDING,
&id);
for (auto &eye : eyes_)
if (eye.framebuffer == static_cast<unsigned>(id))
eye = {};
}
void GlesTransport::Bind(unsigned target, unsigned framebuffer,
std::uint64_t frame) {
if (!frame)
return;
if (target != GL_FRAMEBUFFER && target != GL_DRAW_FRAMEBUFFER)
return;
auto found = owners_.find(framebuffer);
if (found == owners_.end())
return;
void *owner = nullptr;
void *guest_fb = nullptr;
int eye = -1;
if (!mocktail_vr_current_eye(&owner, &eye, &guest_fb) || eye < 0 || eye > 1 ||
found->second != owner)
return;
if (frame == UINT64_MAX) {
auto *backend = ActiveVrBackend();
const auto pose =
backend ? backend->PublishedHeadPose() : ScriptedPoseSample{};
if (!pose.valid)
return;
frame = pose.frame;
}
GLint bound = 0;
gl.GetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &bound);
if (static_cast<unsigned>(bound) != framebuffer)
return;
if (gl.CheckFramebufferStatus(GL_DRAW_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)
return;
GLint type = 0, object = 0, level = 0, face = 0;
gl.GetFramebufferAttachmentParameteriv(
GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE, &type);
if (type != GL_TEXTURE && type != GL_RENDERBUFFER)
return;
gl.GetFramebufferAttachmentParameteriv(
GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME, &object);
if (type == GL_TEXTURE) {
gl.GetFramebufferAttachmentParameteriv(
GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_LEVEL, &level);
gl.GetFramebufferAttachmentParameteriv(
GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_CUBE_MAP_FACE, &face);
if (level != 0 || face != 0)
return;
}
const auto &storage = type == GL_TEXTURE ? textures_ : renderbuffers_;
auto it = storage.find(object);
if (it == storage.end())
return;
const auto &size = it->second;
// Reject unsupported formats/extents instead of copying unrelated pixels.
if (size.samples != 0 ||
(size.format != GL_RGBA8 && size.format != GL_RGBA &&
size.format != GL_SRGB8_ALPHA8))
return;
if ((eyes_[0].owner && eyes_[0].owner != owner) ||
(eyes_[1].owner && eyes_[1].owner != owner))
ResetEyes();
const bool changed = eyes_[eye].framebuffer != framebuffer;
eyes_[eye] = {framebuffer,
static_cast<unsigned>(object),
type == GL_TEXTURE,
size.width,
size.height,
owner,
frame};
mocktail_vr_consume_eye();
if (changed)
std::fprintf(
stderr,
" [vr-backend] GLES eye-bound eye=%d owner=%p guest_fb=%p gl_fb=%u "
"color=%u extent=%dx%d provenance=guest_initializer_and_eye_getter\n",
eye, owner, guest_fb, framebuffer, object, size.width, size.height);
}
bool GlesTransport::Copy(const unsigned textures[2], unsigned width,
unsigned height) {
if (!IsCurrent() || !textures[0] || !textures[1] || !eyes_[0].object ||
!eyes_[1].object ||
(eyes_[0].texture == eyes_[1].texture &&
eyes_[0].object == eyes_[1].object))
return false;
GLint read_fb = 0, draw_fb = 0;
gl.GetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &read_fb);
gl.GetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &draw_fb);
const bool scissor = gl.IsEnabled(GL_SCISSOR_TEST);
gl.Disable(GL_SCISSOR_TEST);
bool copied = true;
for (int i = 0; i < 2; ++i) {
gl.BindFramebuffer(GL_READ_FRAMEBUFFER, copy_fbos_[0]);
if (eyes_[i].texture)
gl.FramebufferTexture2D(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D, eyes_[i].object, 0);
else
gl.FramebufferRenderbuffer(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_RENDERBUFFER, eyes_[i].object);
gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, copy_fbos_[1]);
gl.FramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D, textures[i], 0);
const auto read_status = gl.CheckFramebufferStatus(GL_READ_FRAMEBUFFER);
const auto draw_status = gl.CheckFramebufferStatus(GL_DRAW_FRAMEBUFFER);
if (read_status != GL_FRAMEBUFFER_COMPLETE ||
draw_status != GL_FRAMEBUFFER_COMPLETE) {
static bool reported = false;
if (!reported)
std::fprintf(stderr,
" [vr-backend] GLES copy framebuffer incomplete: "
"read=0x%x draw=0x%x XR texture=%u\n",
read_status, draw_status, textures[i]);
reported = true;
copied = false;
break;
}
gl.BlitFramebuffer(0, 0, eyes_[i].width, eyes_[i].height, 0, 0, width,
height, GL_COLOR_BUFFER_BIT, GL_NEAREST);
}
// Release references to runtime textures before releasing XR swapchain
// images.
gl.BindFramebuffer(GL_READ_FRAMEBUFFER, copy_fbos_[0]);
gl.FramebufferRenderbuffer(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_RENDERBUFFER, 0);
gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, copy_fbos_[1]);
gl.FramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D, 0, 0);
gl.BindFramebuffer(GL_READ_FRAMEBUFFER, read_fb);
gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, draw_fb);
if (scissor)
gl.Enable(GL_SCISSOR_TEST);
// OpenXR's GL binding synchronizes the flushed work at image release.
gl.Flush();
return copied;
}
bool GlesTransport::Upload(unsigned texture, int width, int height,
const void *rgba) {
if (!IsCurrent() || texture == 0 || rgba == nullptr)
return false;
// Restore the guest's bindings and unpack state around the upload.
GLint bound = 0, alignment = 4, row_length = 0, unpack_buffer = 0;
gl.GetIntegerv(GL_TEXTURE_BINDING_2D, &bound);
gl.GetIntegerv(GL_UNPACK_ALIGNMENT, &alignment);
gl.GetIntegerv(GL_UNPACK_ROW_LENGTH, &row_length);
gl.GetIntegerv(GL_PIXEL_UNPACK_BUFFER_BINDING, &unpack_buffer);
if (unpack_buffer != 0)
return false; // a bound PBO would reinterpret the pointer as an offset
gl.PixelStorei(GL_UNPACK_ALIGNMENT, 4);
gl.PixelStorei(GL_UNPACK_ROW_LENGTH, 0);
gl.BindTexture(GL_TEXTURE_2D, texture);
gl.TexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, width, height, GL_RGBA,
GL_UNSIGNED_BYTE, rgba);
gl.BindTexture(GL_TEXTURE_2D, static_cast<GLuint>(bound));
gl.PixelStorei(GL_UNPACK_ALIGNMENT, alignment);
gl.PixelStorei(GL_UNPACK_ROW_LENGTH, row_length);
gl.Flush();
return true;
}
} // namespace mocktail::vr
+71
View File
@@ -0,0 +1,71 @@
#ifndef MOCKTAIL_VR_GLES_TRANSPORT_H_
#define MOCKTAIL_VR_GLES_TRANSPORT_H_
#include <cstdint>
#include <string>
#include <unordered_map>
namespace mocktail::vr {
// One guest EGL context. GL callbacks and copies run on its current thread.
// Only framebuffers created inside the verified DebugDeviceVR initializer
// can become eyes; a live eye-getter request supplies the eye index.
class GlesTransport {
public:
struct Eye {
unsigned framebuffer = 0;
unsigned object = 0;
bool texture = false;
int width = 0, height = 0;
void *owner = nullptr;
std::uint64_t frame = 0;
};
bool Initialize(void *display, void *config, void *context,
void *egl_get_proc, void *(*resolve)(const char *),
std::string *error);
bool IsCurrent() const;
void Destroy(); // Requires this context current; call before EGL destruction.
void MakeCurrent();
static void ReleaseCurrent();
static void *Wrap(const char *name, void *raw);
void ResetEyes();
bool Copy(const unsigned textures[2], unsigned width, unsigned height);
// Uploads tightly packed RGBA8 pixels into a runtime swapchain texture.
bool Upload(unsigned texture, int width, int height, const void *rgba);
const Eye &eye(int index) const { return eyes_[index]; }
void *display() const { return display_; }
void *config() const { return config_; }
void *context() const { return context_; }
void *egl_get_proc() const { return egl_get_proc_; }
std::uint64_t version() const { return version_; }
// Called after the real GL operation, never while the backend mutex is held.
void Generated(int count, const unsigned *ids);
void Deleted(int count, const unsigned *ids);
void Storage(bool texture, int width, int height, unsigned format,
int samples = 0);
void ObjectsDeleted(bool texture, int count, const unsigned *ids);
void AttachmentChanged(unsigned target);
void Bind(unsigned target, unsigned framebuffer,
std::uint64_t frame = UINT64_MAX);
private:
friend struct GlesTransportTestAccess;
struct StorageInfo {
int width = 0, height = 0;
unsigned format = 0;
int samples = 0;
};
void *display_ = nullptr;
void *config_ = nullptr;
void *context_ = nullptr;
void *egl_get_proc_ = nullptr;
std::uint64_t version_ = 0;
void *(*get_current_context_)() = nullptr;
unsigned copy_fbos_[2] = {};
std::unordered_map<unsigned, void *> owners_;
std::unordered_map<unsigned, StorageInfo> textures_;
std::unordered_map<unsigned, StorageInfo> renderbuffers_;
Eye eyes_[2];
};
} // namespace mocktail::vr
#endif
+101
View File
@@ -0,0 +1,101 @@
#include "mocktail/vr/headset_detection.h"
#include <fcntl.h>
#include <signal.h>
#include <spawn.h>
#include <sys/wait.h>
#include <unistd.h>
#include <cstring>
#include <filesystem>
#include <thread>
#include <vector>
#include "mocktail/vr/openxr_backend.h"
#include "mocktail/vr/openxr_probe.h"
extern char** environ;
namespace mocktail::vr {
HeadsetDetection DetectSteamVrHeadset(const std::string& self_executable,
std::chrono::milliseconds timeout,
const std::string& proc_root) {
if (!IsOpenXrSupportCompiled()) {
return {false, "this build has no OpenXR support"};
}
const std::string manifest = RunningSteamVrManifest(proc_root);
if (manifest.empty()) {
return {false, "SteamVR is not running"};
}
std::error_code error;
if (!std::filesystem::is_regular_file(manifest, error)) {
return {false, "SteamVR's OpenXR runtime is missing at " + manifest};
}
// The child inherits this environment with the runtime and probe mode set.
const std::string runtime = "XR_RUNTIME_JSON=" + manifest;
const std::string probe = std::string(kHeadsetProbeEnvironment) + "=1";
std::vector<char*> environment;
for (char** entry = environ; entry != nullptr && *entry != nullptr; ++entry) {
if (std::strncmp(*entry, "XR_RUNTIME_JSON=", 16) != 0 &&
std::strncmp(*entry, probe.c_str(), probe.size() - 1) != 0) {
environment.push_back(*entry);
}
}
environment.push_back(const_cast<char*>(runtime.c_str()));
environment.push_back(const_cast<char*>(probe.c_str()));
environment.push_back(nullptr);
std::vector<char*> arguments = {const_cast<char*>(self_executable.c_str()),
nullptr};
posix_spawn_file_actions_t actions;
posix_spawn_file_actions_init(&actions);
posix_spawn_file_actions_addopen(&actions, STDOUT_FILENO, "/dev/null",
O_WRONLY, 0);
posix_spawn_file_actions_addopen(&actions, STDERR_FILENO, "/dev/null",
O_WRONLY, 0);
pid_t child = -1;
const int spawned =
posix_spawn(&child, self_executable.c_str(), &actions, nullptr,
arguments.data(), environment.data());
posix_spawn_file_actions_destroy(&actions);
if (spawned != 0) {
return {false, std::string("cannot start the headset check: ") +
std::strerror(spawned)};
}
const auto deadline = std::chrono::steady_clock::now() + timeout;
int status = 0;
while (true) {
const pid_t done = waitpid(child, &status, WNOHANG);
if (done == child) break;
if (done < 0) {
return {false, "the headset check was lost"};
}
if (std::chrono::steady_clock::now() >= deadline) {
kill(child, SIGKILL);
(void)waitpid(child, &status, 0);
return {false, "SteamVR did not answer the headset check in time"};
}
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
if (!WIFEXITED(status)) {
return {false, "the headset check crashed"};
}
const int code = WEXITSTATUS(status);
if (code == ProbeExitCode(ProbeStatus::kReady)) {
return {true, "SteamVR reports a connected headset"};
}
if (code == ProbeExitCode(ProbeStatus::kHeadsetUnavailable)) {
return {false, "SteamVR is running without a connected headset"};
}
// Without a headset SteamVR refuses the OpenXR client outright
// (VRInitError_Driver_*HmdNotConnected) rather than reporting no system.
return {false, "SteamVR refused the headset check, which usually means no "
"headset is connected"};
}
int RunHeadsetProbeChild() { return ProbeExitCode(ProbeOpenXr().status); }
} // namespace mocktail::vr
File diff suppressed because it is too large. Load diff
+292
View File
@@ -0,0 +1,292 @@
#include "mocktail/vr/openxr_backend.h"
#include <algorithm>
#include <cctype>
#include <cstdlib>
#include <filesystem>
#include <fstream>
#include <sstream>
#include <sys/stat.h>
#include <unistd.h>
#define JSON_NOEXCEPTION 1
#include <nlohmann/json.hpp>
namespace mocktail::vr {
std::string
SelectVrRuntimeManifest(const std::string &explicit_manifest,
const std::vector<std::string> &candidates) {
if (!explicit_manifest.empty())
return explicit_manifest;
for (const auto &path : candidates) {
std::error_code error;
if (std::filesystem::is_regular_file(path, error))
return path;
}
return {};
}
std::vector<std::string> VrRuntimeManifestCandidates(
const std::string &preference, const std::string &home,
const std::string &config_home, const std::string &config_dirs) {
if (preference == "steamvr" || preference == "alvr") {
if (home.empty())
return {};
std::vector<std::string> paths;
const auto config = config_home.empty() ? home + "/.config" : config_home;
const auto registry = config + "/openvr/openvrpaths.vrpath";
std::error_code error;
if (std::filesystem::file_size(registry, error) <= 1024 * 1024 && !error) {
std::ifstream input(registry);
const auto json = nlohmann::json::parse(input, nullptr, false);
if (json.is_object() && json.contains("runtime") &&
json["runtime"].is_array()) {
for (const auto &item : json["runtime"]) {
if (item.is_string()) {
const auto root = std::filesystem::path(item.get<std::string>());
if (root.is_absolute())
paths.push_back((root / "steamxr_linux64.json").string());
}
}
}
}
const std::vector<std::string> standard = {
home +
"/.local/share/Steam/steamapps/common/SteamVR/steamxr_linux64.json",
home + "/.steam/steam/steamapps/common/SteamVR/steamxr_linux64.json",
home + "/.steam/debian-installation/steamapps/common/SteamVR/"
"steamxr_linux64.json"};
paths.insert(paths.end(), standard.begin(), standard.end());
return paths;
}
const std::vector<std::string> wivrn = {
"/usr/local/share/openxr/1/openxr_wivrn.json",
"/usr/share/openxr/1/openxr_wivrn.json"};
if (preference == "wivrn")
return wivrn;
if (preference == "system")
return {}; // Use the loader's full discovery rules.
std::vector<std::string> candidates;
const auto add_config = [&](const std::string &path) {
if (path.empty() || path[0] != '/')
return;
candidates.push_back(path + "/openxr/1/active_runtime.x86_64.json");
candidates.push_back(path + "/openxr/1/active_runtime.json");
};
add_config(!config_home.empty() ? config_home
: (home.empty() ? "" : home + "/.config"));
std::istringstream directories(config_dirs.empty() ? "/etc/xdg"
: config_dirs);
std::string directory;
while (std::getline(directories, directory, ':'))
add_config(directory);
add_config("/etc");
candidates.insert(candidates.end(), wivrn.begin(), wivrn.end());
return candidates;
}
namespace {
// Restrict process discovery to this user. In particular, never select a
// root server for a desktop client or another user's SteamVR installation.
std::vector<std::pair<std::string, std::filesystem::path>>
UserVrServers(const std::string &proc_root) {
std::vector<std::pair<std::string, std::filesystem::path>> servers;
std::error_code error;
std::filesystem::directory_iterator processes(proc_root, error);
for (; !error && processes != std::filesystem::directory_iterator();
processes.increment(error)) {
const auto name = processes->path().filename().string();
if (name.empty() || !std::all_of(name.begin(), name.end(), [](char c) {
return c >= '0' && c <= '9';
}))
continue;
struct stat metadata{};
if (stat(processes->path().c_str(), &metadata) != 0 ||
metadata.st_uid != getuid())
continue;
std::ifstream comm_file(processes->path() / "comm");
std::string comm;
std::getline(comm_file, comm);
if (comm != "vrserver" && comm != "wivrn-server")
continue;
std::error_code exe_error;
const auto executable =
std::filesystem::read_symlink(processes->path() / "exe", exe_error);
if (exe_error || !executable.is_absolute() || executable.filename() != comm)
continue;
servers.emplace_back(comm, executable);
}
return servers;
}
// vrserver lives in <SteamVR>/bin/linux64/.
std::string SteamVrManifestForServer(const std::filesystem::path &vrserver) {
return (vrserver.parent_path().parent_path().parent_path() /
"steamxr_linux64.json")
.string();
}
} // namespace
std::string RunningSteamVrManifest(const std::string &proc_root) {
for (const auto &[comm, executable] : UserVrServers(proc_root))
if (comm == "vrserver")
return SteamVrManifestForServer(executable);
return {};
}
VrRuntimeDiscovery DiscoverVrRuntime(const std::string &preference,
const std::string &home,
const std::string &config_home,
const std::string &config_dirs,
const std::string &runtime_dir,
const std::string &proc_root) {
VrRuntimeDiscovery result;
auto steam =
VrRuntimeManifestCandidates("steamvr", home, config_home, config_dirs);
auto wivrn =
VrRuntimeManifestCandidates("wivrn", home, config_home, config_dirs);
bool steam_running = false;
bool wivrn_listening = false;
// Read the kernel's socket table: a leftover socket pathname alone is not
// evidence of a listening server. Do not connect and create a dummy XR
// client.
const auto ipc = (std::filesystem::path(runtime_dir) / "wivrn/comp_ipc")
.lexically_normal();
if (!runtime_dir.empty()) {
std::ifstream sockets(std::filesystem::path(proc_root) / "net/unix");
std::string line;
while (std::getline(sockets, line)) {
std::istringstream row(line);
std::string num, refs, protocol, flags, type, state, inode, path;
if (!(row >> num >> refs >> protocol >> flags >> type >> state >> inode))
continue;
std::getline(row >> std::ws, path);
if (flags == "00010000" && type == "0001" &&
std::filesystem::path(path).lexically_normal() == ipc) {
struct stat socket_info{};
if (stat(path.c_str(), &socket_info) == 0 &&
S_ISSOCK(socket_info.st_mode) && socket_info.st_uid == getuid())
wivrn_listening = true;
}
}
}
for (const auto &[comm, executable] : UserVrServers(proc_root)) {
if (comm == "vrserver") {
steam_running = true;
steam.insert(steam.begin(), SteamVrManifestForServer(executable));
} else {
// Native developer builds keep this manifest at their build root.
auto directory = executable.parent_path();
for (int i = 0; i < 3 && directory.has_relative_path();
++i, directory = directory.parent_path()) {
wivrn.insert(wivrn.begin(),
(directory / "openxr_wivrn-dev.json").string());
}
}
}
if (preference == "steamvr" || preference == "alvr") {
result.candidates = steam;
result.reason = "explicit SteamVR/ALVR preference";
} else if (preference == "wivrn") {
result.candidates = wivrn;
result.reason = "explicit WiVRn preference";
} else if (preference == "system") {
result.reason = "system OpenXR loader selection";
} else {
const auto registered =
VrRuntimeManifestCandidates("auto", home, config_home, config_dirs);
if (steam_running != wivrn_listening) {
const std::string runtime_tag = steam_running ? "steamvr" : "wivrn";
for (const auto &path : registered) {
// Only registered entries, not native fallback paths.
if (std::filesystem::path(path).filename().string().find(
"active_runtime") != 0)
continue;
std::error_code manifest_error;
if (std::filesystem::file_size(path, manifest_error) > 1024 * 1024 ||
manifest_error)
continue;
auto identity =
std::filesystem::weakly_canonical(path, manifest_error).string();
std::ifstream input(path);
const auto json = nlohmann::json::parse(input, nullptr, false);
if (json.is_object() && json.contains("runtime") &&
json["runtime"].is_object()) {
for (const auto *key : {"name", "library_path"}) {
const auto &runtime = json["runtime"];
if (runtime.contains(key) && runtime[key].is_string())
identity += " " + runtime[key].get<std::string>();
}
}
std::transform(identity.begin(), identity.end(), identity.begin(),
[](unsigned char c) { return std::tolower(c); });
if (identity.find(runtime_tag) != std::string::npos)
result.candidates.push_back(path);
}
const auto &active = steam_running ? steam : wivrn;
result.candidates.insert(result.candidates.end(), active.begin(),
active.end());
result.reason = steam_running
? "running SteamVR detected (including ALVR)"
: "listening WiVRn server detected";
} else {
result.reason =
steam_running
? "both servers detected; registered runtime has priority"
: "no single running server detected; registered runtime has "
"priority";
}
result.candidates.insert(result.candidates.end(), registered.begin(),
registered.end());
result.candidates.insert(result.candidates.end(), steam.begin(),
steam.end());
result.candidates.insert(result.candidates.end(), wivrn.begin(),
wivrn.end());
}
return result;
}
std::string VrRuntimeUnavailableHint(bool user_selected,
const std::string &manifest) {
if (user_selected) {
return "User-selected XR_RUNTIME_JSON '" + manifest +
"' is unavailable. Check the manifest/library and that its runtime "
"server "
"is running with a connected headset; this error alone does not "
"distinguish them.";
}
if (!manifest.empty()) {
return "Selected OpenXR runtime '" + manifest +
"' is unavailable. Start its server "
"(WiVRn, or SteamVR with ALVR) and connect the headset. Run the "
"server and "
"Mocktail as the same desktop user in the same runtime environment.";
}
return "The registered OpenXR runtime is unavailable. Start WiVRn or SteamVR "
"with ALVR and "
"connect the headset, or explicitly select an installed runtime with "
"XR_RUNTIME_JSON.";
}
VrBackendMode ResolveVrBackendMode(bool vr_enabled) {
if (!vr_enabled) {
return VrBackendMode::kDisabled;
}
const char *value = std::getenv("MOCKTAIL_VR_BACKEND");
const std::string mode = value ? value : "xr";
if (mode == "native" || mode == "native-stereo") {
return VrBackendMode::kNativeOnly;
}
return VrBackendMode::kXrOutput;
}
const char *VrBackendModeName(VrBackendMode mode) {
switch (mode) {
case VrBackendMode::kDisabled:
return "disabled";
case VrBackendMode::kNativeOnly:
return "native-stereo-diagnostic";
case VrBackendMode::kXrOutput:
return "openxr-output";
}
return "disabled";
}
} // namespace mocktail::vr
+43
View File
@@ -0,0 +1,43 @@
#include "mocktail/vr/openxr_probe.h"
#include "mocktail/vr/openxr_backend.h"
#include "mocktail/vr/openxr_preview.h"
namespace mocktail::vr {
OpenXrBackend::~OpenXrBackend() = default;
// The backend's keyboard_raster_ member needs this destructor. No rasterizer
// is ever created here, so SDL3_ttf and fontconfig stay out of this build.
VirtualKeyboardRasterizer::~VirtualKeyboardRasterizer() = default;
Status OpenXrBackend::Arm(VrGraphicsApi) {
return Status::Error(StatusCode::kUnavailable, "OpenXR support is disabled in this build");
}
void OpenXrBackend::Disarm() {}
void OpenXrBackend::RequestControllerHaptics(int, float, std::uint64_t, float) {}
void OpenXrBackend::StopControllerHaptics(int) {}
OpenXrBackend* ActiveVrBackend() { return nullptr; }
void OpenXrBackend::NotePoseApplied(void*, std::uint64_t) {}
ScriptedPoseSample OpenXrBackend::PublishedHeadPose() const { return {}; }
bool OpenXrBackend::RecommendedEyeExtent(std::uint32_t*, std::uint32_t*) const {
return false;
}
PreviewResult RunOpenXrPreview(const PreviewOptions&) {
PreviewResult result;
result.message =
"This build does not include OpenXR support. Use a VR-enabled build of "
"Mocktail.";
return result;
}
bool IsOpenXrSupportCompiled() { return false; }
ProbeReport ProbeOpenXr() {
ProbeReport report;
report.status = ProbeStatus::kBuildDisabled;
report.message =
"This build does not include OpenXR support. Use a "
"VR-enabled build of Mocktail.";
return report;
}
} // namespace mocktail::vr
+728
View File
@@ -0,0 +1,728 @@
#include "mocktail/vr/openxr_preview.h"
#include <vulkan/vulkan.h>
#include <openxr/openxr.h>
#include <openxr/openxr_platform.h>
#include <dlfcn.h>
#include <signal.h>
#include <algorithm>
#include <chrono>
#include <cstring>
#include <iostream>
#include <stdexcept>
#include <thread>
#include <type_traits>
#include <vector>
namespace mocktail::vr {
namespace {
using Clock = std::chrono::steady_clock;
volatile sig_atomic_t interrupted = 0;
void Interrupt(int) { interrupted = 1; }
class SignalScope {
public:
SignalScope() {
interrupted = 0;
struct sigaction action{};
action.sa_handler = Interrupt;
sigemptyset(&action.sa_mask);
have_int_ = sigaction(SIGINT, &action, &old_int_) == 0;
have_term_ = sigaction(SIGTERM, &action, &old_term_) == 0;
}
~SignalScope() {
if (have_int_) sigaction(SIGINT, &old_int_, nullptr);
if (have_term_) sigaction(SIGTERM, &old_term_, nullptr);
}
private:
struct sigaction old_int_{}, old_term_{};
bool have_int_ = false, have_term_ = false;
};
template <typename T>
T XrInfo(XrStructureType type) {
T value{};
value.type = type;
return value;
}
template <typename T>
T VkInfo(VkStructureType type) {
T value{};
value.sType = type;
return value;
}
void Require(bool condition, const std::string& message) {
if (!condition) throw std::runtime_error(message);
}
void CheckVk(VkResult result, const char* operation) {
Require(result == VK_SUCCESS,
std::string(operation) + " failed: " + std::to_string(result));
}
// Every Vulkan call uses the host loader explicitly. Linking global vk*
// symbols here could accidentally enter Mocktail's Android Vulkan adapter.
struct Vulkan {
void* library = nullptr;
PFN_vkGetInstanceProcAddr get_instance_proc = nullptr;
PFN_vkGetDeviceProcAddr get_device_proc = nullptr;
VkInstance instance = VK_NULL_HANDLE;
VkPhysicalDevice physical_device = VK_NULL_HANDLE;
VkDevice device = VK_NULL_HANDLE;
VkQueue queue = VK_NULL_HANDLE;
std::uint32_t queue_family = 0;
PFN_vkDestroyInstance DestroyInstance = nullptr;
#define MOCKTAIL_PREVIEW_VK_FUNCTIONS(X) \
X(DestroyDevice) \
X(GetDeviceQueue) \
X(DeviceWaitIdle) \
X(CreateImageView) \
X(DestroyImageView) \
X(CreateRenderPass) \
X(DestroyRenderPass) \
X(CreateFramebuffer) \
X(DestroyFramebuffer) \
X(CreateCommandPool) \
X(DestroyCommandPool) \
X(AllocateCommandBuffers) \
X(ResetCommandBuffer) \
X(BeginCommandBuffer) \
X(EndCommandBuffer) \
X(CmdBeginRenderPass) \
X(CmdEndRenderPass) \
X(CmdClearAttachments) \
X(CreateFence) \
X(DestroyFence) \
X(ResetFences) \
X(WaitForFences) \
X(QueueSubmit)
#define DECLARE(name) PFN_vk##name name = nullptr;
MOCKTAIL_PREVIEW_VK_FUNCTIONS(DECLARE)
#undef DECLARE
Vulkan() = default;
Vulkan(const Vulkan&) = delete;
Vulkan& operator=(const Vulkan&) = delete;
~Vulkan() {
if (device && DestroyDevice) DestroyDevice(device, nullptr);
if (instance && DestroyInstance) DestroyInstance(instance, nullptr);
if (library) dlclose(library);
}
void OpenLoader() {
library = dlopen("libvulkan.so.1", RTLD_NOW | RTLD_LOCAL);
Require(library != nullptr,
"Cannot load the host Vulkan loader libvulkan.so.1");
get_instance_proc = reinterpret_cast<PFN_vkGetInstanceProcAddr>(
dlsym(library, "vkGetInstanceProcAddr"));
Require(get_instance_proc != nullptr,
"Host vkGetInstanceProcAddr is missing");
}
template <typename T>
T InstanceFunction(const char* name) {
const auto function =
reinterpret_cast<T>(get_instance_proc(instance, name));
Require(function != nullptr,
std::string("Missing Vulkan function: ") + name);
return function;
}
void LoadDeviceFunctions() {
#define LOAD(name) \
name = reinterpret_cast<PFN_vk##name>(get_device_proc(device, "vk" #name)); \
Require(name != nullptr, "Missing Vulkan function: vk" #name);
MOCKTAIL_PREVIEW_VK_FUNCTIONS(LOAD)
#undef LOAD
GetDeviceQueue(device, queue_family, 0, &queue);
}
};
#undef MOCKTAIL_PREVIEW_VK_FUNCTIONS
class Preview {
public:
Preview() = default;
Preview(const Preview&) = delete;
Preview& operator=(const Preview&) = delete;
~Preview() {
// The runtime still needs our VkDevice when destroying its
// swapchain/session.
if (vk_.device && vk_.DeviceWaitIdle) (void)vk_.DeviceWaitIdle(vk_.device);
for (auto framebuffer : framebuffers_)
vk_.DestroyFramebuffer(vk_.device, framebuffer, nullptr);
for (auto view : image_views_)
vk_.DestroyImageView(vk_.device, view, nullptr);
if (render_pass_) vk_.DestroyRenderPass(vk_.device, render_pass_, nullptr);
if (fence_) vk_.DestroyFence(vk_.device, fence_, nullptr);
if (pool_) vk_.DestroyCommandPool(vk_.device, pool_, nullptr);
if (swapchain_) (void)xrDestroySwapchain(swapchain_);
if (view_space_) (void)xrDestroySpace(view_space_);
if (local_space_) (void)xrDestroySpace(local_space_);
if (session_) (void)xrDestroySession(session_);
if (instance_) (void)xrDestroyInstance(instance_);
}
void Initialize() {
const char* extension = XR_KHR_VULKAN_ENABLE2_EXTENSION_NAME;
auto info = XrInfo<XrInstanceCreateInfo>(XR_TYPE_INSTANCE_CREATE_INFO);
std::strcpy(info.applicationInfo.applicationName, "Mocktail VR preview");
std::strcpy(info.applicationInfo.engineName, "Mocktail");
info.applicationInfo.apiVersion = XR_MAKE_VERSION(1, 0, 0);
info.enabledExtensionCount = 1;
info.enabledExtensionNames = &extension;
Check(xrCreateInstance(&info, &instance_), "xrCreateInstance");
auto system = XrInfo<XrSystemGetInfo>(XR_TYPE_SYSTEM_GET_INFO);
system.formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY;
Check(xrGetSystem(instance_, &system, &system_), "xrGetSystem");
const auto views = Enumerate<XrViewConfigurationView>(
XR_TYPE_VIEW_CONFIGURATION_VIEW,
[&](auto capacity, auto* count, auto* data) {
return xrEnumerateViewConfigurationViews(
instance_, system_, XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO,
capacity, count, data);
});
Require(views.size() == 2,
"The VR preview requires PRIMARY_STEREO support");
const auto blends = Enumerate<XrEnvironmentBlendMode>(
XR_TYPE_UNKNOWN, [&](auto capacity, auto* count, auto* data) {
return xrEnumerateEnvironmentBlendModes(
instance_, system_, XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO,
capacity, count, data);
});
Require(!blends.empty(), "The runtime exposes no environment blend modes");
blend_ = std::find(blends.begin(), blends.end(),
XR_ENVIRONMENT_BLEND_MODE_OPAQUE) != blends.end()
? XR_ENVIRONMENT_BLEND_MODE_OPAQUE
: blends.front();
CreateGraphics();
auto binding = XrInfo<XrGraphicsBindingVulkan2KHR>(
XR_TYPE_GRAPHICS_BINDING_VULKAN2_KHR);
binding.instance = vk_.instance;
binding.physicalDevice = vk_.physical_device;
binding.device = vk_.device;
binding.queueFamilyIndex = vk_.queue_family;
auto session_info =
XrInfo<XrSessionCreateInfo>(XR_TYPE_SESSION_CREATE_INFO);
session_info.next = &binding;
session_info.systemId = system_;
Check(xrCreateSession(instance_, &session_info, &session_),
"xrCreateSession");
auto space_info =
XrInfo<XrReferenceSpaceCreateInfo>(XR_TYPE_REFERENCE_SPACE_CREATE_INFO);
space_info.poseInReferenceSpace.orientation.w = 1;
space_info.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_LOCAL;
Check(xrCreateReferenceSpace(session_, &space_info, &local_space_),
"xrCreateReferenceSpace LOCAL");
space_info.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW;
Check(xrCreateReferenceSpace(session_, &space_info, &view_space_),
"xrCreateReferenceSpace VIEW");
CreateImages();
}
void Run(const PreviewOptions& options, PreviewResult* report) {
const auto ready_deadline = Clock::now() + std::chrono::seconds(30);
auto exit_deadline = Clock::time_point::max();
bool running = false, exit_requested = false;
while (true) {
while (true) {
auto event = XrInfo<XrEventDataBuffer>(XR_TYPE_EVENT_DATA_BUFFER);
const auto result = xrPollEvent(instance_, &event);
if (result == XR_EVENT_UNAVAILABLE) break;
Check(result, "xrPollEvent");
if (event.type == XR_TYPE_EVENT_DATA_INSTANCE_LOSS_PENDING)
throw std::runtime_error("The OpenXR runtime was lost");
if (event.type != XR_TYPE_EVENT_DATA_SESSION_STATE_CHANGED) continue;
const auto& changed =
reinterpret_cast<const XrEventDataSessionStateChanged&>(event);
if (changed.session != session_) continue;
switch (changed.state) {
case XR_SESSION_STATE_READY: {
auto begin = XrInfo<XrSessionBeginInfo>(XR_TYPE_SESSION_BEGIN_INFO);
begin.primaryViewConfigurationType =
XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO;
Check(xrBeginSession(session_, &begin), "xrBeginSession");
running = true;
std::cerr << "[VR] Session started. A test panel is fixed two "
"metres ahead.\n";
break;
}
case XR_SESSION_STATE_STOPPING:
if (running) Check(xrEndSession(session_), "xrEndSession");
return;
case XR_SESSION_STATE_EXITING:
return;
case XR_SESSION_STATE_LOSS_PENDING:
throw std::runtime_error(
"The OpenXR session was lost; reconnect and launch again");
default:
break;
}
}
if (interrupted ||
(options.frame_limit && report->frames >= options.frame_limit)) {
if (!running) return;
if (!exit_requested) {
Check(xrRequestExitSession(session_), "xrRequestExitSession");
exit_requested = true;
exit_deadline = Clock::now() + std::chrono::seconds(5);
// Consume an immediate STOPPING event before starting another frame.
continue;
}
}
Require(Clock::now() < exit_deadline,
"The runtime did not stop the VR session after an exit request");
if (!running) {
Require(Clock::now() < ready_deadline,
"Timed out waiting for the VR session to become READY");
std::this_thread::sleep_for(std::chrono::milliseconds(10));
continue;
}
Frame(report);
}
}
private:
void Check(XrResult result, const char* operation) const {
if (XR_SUCCEEDED(result)) return;
char text[XR_MAX_RESULT_STRING_SIZE]{};
if (instance_) (void)xrResultToString(instance_, result, text);
if (!text[0] && result == XR_ERROR_RUNTIME_UNAVAILABLE)
std::strcpy(text, "XR_ERROR_RUNTIME_UNAVAILABLE");
std::string message =
std::string(operation) +
" failed: " + (text[0] ? std::string(text) : std::to_string(result));
if (result == XR_ERROR_RUNTIME_UNAVAILABLE)
message += ". No active OpenXR runtime was found. Start WiVRn for a real "
"headset, or select an installed runtime using XR_RUNTIME_JSON.";
else if (result == XR_ERROR_FORM_FACTOR_UNAVAILABLE)
message += ". Connect a headset or select a simulated runtime.";
else if (result == XR_ERROR_EXTENSION_NOT_PRESENT)
message += ". XR_KHR_vulkan_enable2 is required.";
throw std::runtime_error(message);
}
template <typename T>
T Function(const char* name) {
PFN_xrVoidFunction function = nullptr;
Check(xrGetInstanceProcAddr(instance_, name, &function), name);
Require(function != nullptr,
std::string("Missing OpenXR function: ") + name);
return reinterpret_cast<T>(function);
}
template <typename T, typename F>
std::vector<T> Enumerate(XrStructureType type, F function) {
for (int retry = 0; retry < 3; ++retry) {
std::uint32_t count = 0;
Check(function(0, &count, static_cast<T*>(nullptr)),
"OpenXR enumeration count");
Require(count <= 4096, "OpenXR enumeration exceeds the supported size");
std::vector<T> values(count);
if constexpr (std::is_class_v<T>)
for (auto& value : values) value.type = type;
if (!count) return values;
const auto result = function(count, &count, values.data());
if (result == XR_ERROR_SIZE_INSUFFICIENT) continue;
Check(result, "OpenXR enumeration data");
Require(count <= values.size(),
"OpenXR enumeration returned an invalid size");
values.resize(count);
return values;
}
throw std::runtime_error("OpenXR enumeration kept changing size");
}
void CreateGraphics() {
auto requirements = XrInfo<XrGraphicsRequirementsVulkan2KHR>(
XR_TYPE_GRAPHICS_REQUIREMENTS_VULKAN2_KHR);
Check(Function<PFN_xrGetVulkanGraphicsRequirements2KHR>(
"xrGetVulkanGraphicsRequirements2KHR")(instance_, system_,
&requirements),
"xrGetVulkanGraphicsRequirements2KHR");
vk_.OpenLoader();
auto application =
VkInfo<VkApplicationInfo>(VK_STRUCTURE_TYPE_APPLICATION_INFO);
application.pApplicationName = "Mocktail VR preview";
const auto version = std::max<XrVersion>(
requirements.minApiVersionSupported, XR_MAKE_VERSION(1, 0, 0));
application.apiVersion = VK_MAKE_API_VERSION(0, XR_VERSION_MAJOR(version),
XR_VERSION_MINOR(version), 0);
auto instance_info =
VkInfo<VkInstanceCreateInfo>(VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO);
instance_info.pApplicationInfo = &application;
auto xr_instance_info = XrInfo<XrVulkanInstanceCreateInfoKHR>(
XR_TYPE_VULKAN_INSTANCE_CREATE_INFO_KHR);
xr_instance_info.systemId = system_;
xr_instance_info.pfnGetInstanceProcAddr = vk_.get_instance_proc;
xr_instance_info.vulkanCreateInfo = &instance_info;
VkResult result = VK_ERROR_INITIALIZATION_FAILED;
Check(Function<PFN_xrCreateVulkanInstanceKHR>("xrCreateVulkanInstanceKHR")(
instance_, &xr_instance_info, &vk_.instance, &result),
"xrCreateVulkanInstanceKHR");
CheckVk(result, "vkCreateInstance");
vk_.DestroyInstance =
vk_.InstanceFunction<PFN_vkDestroyInstance>("vkDestroyInstance");
vk_.get_device_proc =
vk_.InstanceFunction<PFN_vkGetDeviceProcAddr>("vkGetDeviceProcAddr");
auto device_info = XrInfo<XrVulkanGraphicsDeviceGetInfoKHR>(
XR_TYPE_VULKAN_GRAPHICS_DEVICE_GET_INFO_KHR);
device_info.systemId = system_;
device_info.vulkanInstance = vk_.instance;
Check(Function<PFN_xrGetVulkanGraphicsDevice2KHR>(
"xrGetVulkanGraphicsDevice2KHR")(instance_, &device_info,
&vk_.physical_device),
"xrGetVulkanGraphicsDevice2KHR");
VkPhysicalDeviceProperties properties{};
vk_.InstanceFunction<PFN_vkGetPhysicalDeviceProperties>(
"vkGetPhysicalDeviceProperties")(vk_.physical_device, &properties);
Require(properties.apiVersion >= application.apiVersion,
"The runtime-selected GPU does not support the required Vulkan "
"version");
const auto get_families =
vk_.InstanceFunction<PFN_vkGetPhysicalDeviceQueueFamilyProperties>(
"vkGetPhysicalDeviceQueueFamilyProperties");
std::uint32_t count = 0;
get_families(vk_.physical_device, &count, nullptr);
Require(count > 0 && count <= 256,
"The runtime-selected GPU has no usable queue families");
std::vector<VkQueueFamilyProperties> families(count);
get_families(vk_.physical_device, &count, families.data());
const auto family =
std::find_if(families.begin(), families.end(), [](const auto& value) {
return value.queueCount && (value.queueFlags & VK_QUEUE_GRAPHICS_BIT);
});
Require(family != families.end(),
"The runtime-selected GPU has no graphics queue");
vk_.queue_family = static_cast<std::uint32_t>(family - families.begin());
const float priority = 1;
auto queue = VkInfo<VkDeviceQueueCreateInfo>(
VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO);
queue.queueFamilyIndex = vk_.queue_family;
queue.queueCount = 1;
queue.pQueuePriorities = &priority;
auto create =
VkInfo<VkDeviceCreateInfo>(VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO);
create.queueCreateInfoCount = 1;
create.pQueueCreateInfos = &queue;
auto xr_create = XrInfo<XrVulkanDeviceCreateInfoKHR>(
XR_TYPE_VULKAN_DEVICE_CREATE_INFO_KHR);
xr_create.systemId = system_;
xr_create.pfnGetInstanceProcAddr = vk_.get_instance_proc;
xr_create.vulkanPhysicalDevice = vk_.physical_device;
xr_create.vulkanCreateInfo = &create;
Check(Function<PFN_xrCreateVulkanDeviceKHR>("xrCreateVulkanDeviceKHR")(
instance_, &xr_create, &vk_.device, &result),
"xrCreateVulkanDeviceKHR");
CheckVk(result, "vkCreateDevice");
vk_.LoadDeviceFunctions();
std::cerr << "[VR] Vulkan device: " << properties.deviceName << '\n';
}
void CreateImages() {
const auto formats = Enumerate<std::int64_t>(
XR_TYPE_UNKNOWN, [&](auto capacity, auto* count, auto* data) {
return xrEnumerateSwapchainFormats(session_, capacity, count, data);
});
VkFormat format = VK_FORMAT_UNDEFINED;
for (auto candidate :
{VK_FORMAT_R8G8B8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB,
VK_FORMAT_R8G8B8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM}) {
if (std::find(formats.begin(), formats.end(), candidate) !=
formats.end()) {
format = candidate;
break;
}
}
Require(format != VK_FORMAT_UNDEFINED,
"The runtime exposes no supported RGBA swapchain format");
auto system_properties =
XrInfo<XrSystemProperties>(XR_TYPE_SYSTEM_PROPERTIES);
Check(xrGetSystemProperties(instance_, system_, &system_properties),
"xrGetSystemProperties");
width_ = std::min(
512u, system_properties.graphicsProperties.maxSwapchainImageWidth);
height_ = std::min(
512u, system_properties.graphicsProperties.maxSwapchainImageHeight);
Require(width_ >= 64 && height_ >= 64 &&
system_properties.graphicsProperties.maxLayerCount > 0,
"The runtime cannot display the preview panel");
auto swapchain =
XrInfo<XrSwapchainCreateInfo>(XR_TYPE_SWAPCHAIN_CREATE_INFO);
swapchain.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT;
swapchain.format = format;
swapchain.sampleCount = 1;
swapchain.width = width_;
swapchain.height = height_;
swapchain.faceCount = swapchain.arraySize = swapchain.mipCount = 1;
Check(xrCreateSwapchain(session_, &swapchain, &swapchain_),
"xrCreateSwapchain");
const auto images = Enumerate<XrSwapchainImageVulkanKHR>(
XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR,
[&](auto capacity, auto* count, auto* data) {
return xrEnumerateSwapchainImages(
swapchain_, capacity, count,
reinterpret_cast<XrSwapchainImageBaseHeader*>(data));
});
Require(!images.empty(), "The runtime returned an empty swapchain");
VkAttachmentDescription attachment{};
attachment.format = format;
attachment.samples = VK_SAMPLE_COUNT_1_BIT;
attachment.loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
attachment.storeOp = VK_ATTACHMENT_STORE_OP_STORE;
attachment.stencilLoadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;
attachment.stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
attachment.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
attachment.finalLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
VkAttachmentReference reference{0,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL};
VkSubpassDescription subpass{};
subpass.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
subpass.colorAttachmentCount = 1;
subpass.pColorAttachments = &reference;
VkSubpassDependency dependency{};
dependency.srcSubpass = VK_SUBPASS_EXTERNAL;
dependency.dstSubpass = 0;
dependency.srcStageMask = dependency.dstStageMask =
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
dependency.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
auto pass = VkInfo<VkRenderPassCreateInfo>(
VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO);
pass.attachmentCount = 1;
pass.pAttachments = &attachment;
pass.subpassCount = 1;
pass.pSubpasses = &subpass;
pass.dependencyCount = 1;
pass.pDependencies = &dependency;
CheckVk(vk_.CreateRenderPass(vk_.device, &pass, nullptr, &render_pass_),
"vkCreateRenderPass");
for (const auto& image : images) {
auto view_info = VkInfo<VkImageViewCreateInfo>(
VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO);
view_info.image = image.image;
view_info.viewType = VK_IMAGE_VIEW_TYPE_2D;
view_info.format = format;
view_info.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1};
VkImageView view = VK_NULL_HANDLE;
CheckVk(vk_.CreateImageView(vk_.device, &view_info, nullptr, &view),
"vkCreateImageView");
image_views_.push_back(view);
auto framebuffer_info = VkInfo<VkFramebufferCreateInfo>(
VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO);
framebuffer_info.renderPass = render_pass_;
framebuffer_info.attachmentCount = 1;
framebuffer_info.pAttachments = &image_views_.back();
framebuffer_info.width = width_;
framebuffer_info.height = height_;
framebuffer_info.layers = 1;
VkFramebuffer framebuffer = VK_NULL_HANDLE;
CheckVk(vk_.CreateFramebuffer(vk_.device, &framebuffer_info, nullptr,
&framebuffer),
"vkCreateFramebuffer");
framebuffers_.push_back(framebuffer);
}
auto pool = VkInfo<VkCommandPoolCreateInfo>(
VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO);
pool.flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT;
pool.queueFamilyIndex = vk_.queue_family;
CheckVk(vk_.CreateCommandPool(vk_.device, &pool, nullptr, &pool_),
"vkCreateCommandPool");
auto allocate = VkInfo<VkCommandBufferAllocateInfo>(
VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO);
allocate.commandPool = pool_;
allocate.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
allocate.commandBufferCount = 1;
CheckVk(vk_.AllocateCommandBuffers(vk_.device, &allocate, &command_),
"vkAllocateCommandBuffers");
auto fence = VkInfo<VkFenceCreateInfo>(VK_STRUCTURE_TYPE_FENCE_CREATE_INFO);
CheckVk(vk_.CreateFence(vk_.device, &fence, nullptr, &fence_),
"vkCreateFence");
}
void Render(std::uint32_t image, std::uint64_t frame) {
Require(image < framebuffers_.size(),
"The runtime acquired an invalid swapchain image index");
CheckVk(vk_.ResetCommandBuffer(command_, 0), "vkResetCommandBuffer");
auto begin = VkInfo<VkCommandBufferBeginInfo>(
VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO);
begin.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
CheckVk(vk_.BeginCommandBuffer(command_, &begin), "vkBeginCommandBuffer");
VkClearValue background{};
background.color = {{0.018f, 0.024f, 0.04f, 1.f}};
auto pass =
VkInfo<VkRenderPassBeginInfo>(VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO);
pass.renderPass = render_pass_;
pass.framebuffer = framebuffers_[image];
pass.renderArea.extent = {width_, height_};
pass.clearValueCount = 1;
pass.pClearValues = &background;
vk_.CmdBeginRenderPass(command_, &pass, VK_SUBPASS_CONTENTS_INLINE);
auto rectangle = [&](std::uint32_t x, std::uint32_t y, std::uint32_t w,
std::uint32_t h, VkClearColorValue color) {
VkClearAttachment attachment{};
attachment.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
attachment.clearValue.color = color;
VkClearRect rect{
{{static_cast<std::int32_t>(x), static_cast<std::int32_t>(y)},
{w, h}},
0,
1};
vk_.CmdClearAttachments(command_, 1, &attachment, 1, &rect);
};
for (std::uint32_t y = 0; y < 8; ++y)
for (std::uint32_t x = 0; x < 8; ++x)
if ((x + y) % 2 == 0)
rectangle(x * width_ / 8, y * height_ / 8, width_ / 8, height_ / 8,
x < 4 ? VkClearColorValue{{0.025f, 0.3f, 0.38f, 1.f}}
: VkClearColorValue{{0.23f, 0.08f, 0.4f, 1.f}});
rectangle(width_ / 2 - 1, height_ / 4, 2, height_ / 2,
{{0.9f, 0.9f, 0.9f, 1.f}});
rectangle(width_ / 4, height_ / 2 - 1, width_ / 2, 2,
{{0.9f, 0.9f, 0.9f, 1.f}});
rectangle(static_cast<std::uint32_t>(frame % (width_ - 16)), height_ - 12,
16, 6, {{0.8f, 0.55f, 0.08f, 1.f}});
vk_.CmdEndRenderPass(command_);
CheckVk(vk_.EndCommandBuffer(command_), "vkEndCommandBuffer");
CheckVk(vk_.ResetFences(vk_.device, 1, &fence_), "vkResetFences");
auto submit = VkInfo<VkSubmitInfo>(VK_STRUCTURE_TYPE_SUBMIT_INFO);
submit.commandBufferCount = 1;
submit.pCommandBuffers = &command_;
CheckVk(vk_.QueueSubmit(vk_.queue, 1, &submit, fence_), "vkQueueSubmit");
// One frame in flight for the first prototype: all GPU writes complete
// before release and command-buffer reuse. No runtime-owned image is freed.
CheckVk(
vk_.WaitForFences(vk_.device, 1, &fence_, VK_TRUE, 5'000'000'000ULL),
"vkWaitForFences");
}
void Frame(PreviewResult* report) {
auto wait = XrInfo<XrFrameWaitInfo>(XR_TYPE_FRAME_WAIT_INFO);
auto state = XrInfo<XrFrameState>(XR_TYPE_FRAME_STATE);
Check(xrWaitFrame(session_, &wait, &state), "xrWaitFrame");
auto begin = XrInfo<XrFrameBeginInfo>(XR_TYPE_FRAME_BEGIN_INFO);
Check(xrBeginFrame(session_, &begin), "xrBeginFrame");
auto end = XrInfo<XrFrameEndInfo>(XR_TYPE_FRAME_END_INFO);
end.displayTime = state.predictedDisplayTime;
end.environmentBlendMode = blend_;
bool frame_open = true, image_waited = false;
try {
auto head = XrInfo<XrSpaceLocation>(XR_TYPE_SPACE_LOCATION);
Check(xrLocateSpace(view_space_, local_space_, state.predictedDisplayTime,
&head),
"xrLocateSpace");
constexpr auto valid = XR_SPACE_LOCATION_ORIENTATION_VALID_BIT |
XR_SPACE_LOCATION_POSITION_VALID_BIT;
const bool pose_valid = (head.locationFlags & valid) == valid;
if (pose_valid) {
++report->tracked_frames;
report->head_position = {head.pose.position.x, head.pose.position.y,
head.pose.position.z};
report->head_orientation = {
head.pose.orientation.x, head.pose.orientation.y,
head.pose.orientation.z, head.pose.orientation.w};
}
auto layer =
XrInfo<XrCompositionLayerQuad>(XR_TYPE_COMPOSITION_LAYER_QUAD);
const XrCompositionLayerBaseHeader* layer_pointer =
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&layer);
if (state.shouldRender && pose_valid) {
auto acquire = XrInfo<XrSwapchainImageAcquireInfo>(
XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO);
std::uint32_t image = 0;
Check(xrAcquireSwapchainImage(swapchain_, &acquire, &image),
"xrAcquireSwapchainImage");
auto image_wait =
XrInfo<XrSwapchainImageWaitInfo>(XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO);
image_wait.timeout = 50'000'000;
const auto deadline = Clock::now() + std::chrono::seconds(5);
while (true) {
const auto result = xrWaitSwapchainImage(swapchain_, &image_wait);
if (result != XR_TIMEOUT_EXPIRED) {
Check(result, "xrWaitSwapchainImage");
break;
}
Require(!interrupted && Clock::now() < deadline,
"Timed out waiting for an OpenXR swapchain image");
}
image_waited = true;
Render(image, report->frames);
auto release = XrInfo<XrSwapchainImageReleaseInfo>(
XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO);
image_waited = false;
Check(xrReleaseSwapchainImage(swapchain_, &release),
"xrReleaseSwapchainImage");
layer.space = local_space_;
layer.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
layer.subImage.swapchain = swapchain_;
layer.subImage.imageRect.extent = {static_cast<std::int32_t>(width_),
static_cast<std::int32_t>(height_)};
layer.pose.orientation.w = 1;
layer.pose.position.z = -2;
layer.size = {1.4f, 1.4f};
end.layerCount = 1;
end.layers = &layer_pointer;
}
frame_open = false;
Check(xrEndFrame(session_, &end), "xrEndFrame");
++report->frames;
if (end.layerCount) ++report->rendered_frames;
} catch (...) {
if (image_waited) {
if (vk_.DeviceWaitIdle) (void)vk_.DeviceWaitIdle(vk_.device);
auto release = XrInfo<XrSwapchainImageReleaseInfo>(
XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO);
(void)xrReleaseSwapchainImage(swapchain_, &release);
}
if (frame_open) {
end.layerCount = 0;
end.layers = nullptr;
(void)xrEndFrame(session_, &end);
}
throw;
}
}
Vulkan vk_;
XrInstance instance_ = XR_NULL_HANDLE;
XrSystemId system_ = XR_NULL_SYSTEM_ID;
XrSession session_ = XR_NULL_HANDLE;
XrSpace local_space_ = XR_NULL_HANDLE, view_space_ = XR_NULL_HANDLE;
XrSwapchain swapchain_ = XR_NULL_HANDLE;
XrEnvironmentBlendMode blend_ = XR_ENVIRONMENT_BLEND_MODE_OPAQUE;
std::uint32_t width_ = 512, height_ = 512;
VkRenderPass render_pass_ = VK_NULL_HANDLE;
VkCommandPool pool_ = VK_NULL_HANDLE;
VkCommandBuffer command_ = VK_NULL_HANDLE;
VkFence fence_ = VK_NULL_HANDLE;
std::vector<VkImageView> image_views_;
std::vector<VkFramebuffer> framebuffers_;
};
} // namespace
PreviewResult RunOpenXrPreview(const PreviewOptions& options) {
PreviewResult result;
SignalScope signals;
try {
Preview preview;
preview.Initialize();
std::cerr << "[VR] Experimental test scene. Roblox rendering is not "
"connected. Ctrl+C exits.\n";
preview.Run(options, &result);
result.success = true;
result.message = "VR preview closed";
} catch (const std::exception& error) {
result.message = error.what();
}
return result;
}
} // namespace mocktail::vr
+211
View File
@@ -0,0 +1,211 @@
#include "mocktail/vr/openxr_probe.h"
#include <vulkan/vulkan.h>
#include <openxr/openxr.h>
#include <openxr/openxr_platform.h>
#include <algorithm>
#include <cstring>
#include <utility>
namespace mocktail::vr {
namespace {
std::string Version(XrVersion version) {
return std::to_string(XR_VERSION_MAJOR(version)) + "." +
std::to_string(XR_VERSION_MINOR(version)) + "." +
std::to_string(XR_VERSION_PATCH(version));
}
struct InstanceScope {
XrInstance instance = XR_NULL_HANDLE;
~InstanceScope() {
if (instance != XR_NULL_HANDLE) (void)xrDestroyInstance(instance);
}
};
void Failure(ProbeReport* report, XrInstance instance, const char* operation,
XrResult result) {
report->status = ProbeStatus::kError;
if (result == XR_ERROR_RUNTIME_UNAVAILABLE) {
report->status = ProbeStatus::kRuntimeUnavailable;
} else if (result == XR_ERROR_FORM_FACTOR_UNAVAILABLE) {
report->status = ProbeStatus::kHeadsetUnavailable;
} else if (result == XR_ERROR_FORM_FACTOR_UNSUPPORTED ||
result == XR_ERROR_VIEW_CONFIGURATION_TYPE_UNSUPPORTED) {
report->status = ProbeStatus::kIncompatibleRuntime;
}
char name[XR_MAX_RESULT_STRING_SIZE] = {};
if (instance != XR_NULL_HANDLE)
(void)xrResultToString(instance, result, name);
report->message = std::string(operation) + " failed: " +
(name[0] ? std::string(name) : std::to_string(result));
if (report->status == ProbeStatus::kRuntimeUnavailable) {
report->message += ". Start an OpenXR runtime such as WiVRn or Monado.";
} else if (report->status == ProbeStatus::kHeadsetUnavailable) {
report->message +=
". Connect a headset, or use a simulated runtime for development.";
}
}
// Counts can change between the two enumeration calls. Retry a bounded number
// of times and never consume an output array after XR_ERROR_SIZE_INSUFFICIENT.
template <typename T, typename Enumerate>
XrResult EnumerateProperties(XrStructureType type, Enumerate enumerate,
std::vector<T>* values) {
for (int attempt = 0; attempt < 3; ++attempt) {
std::uint32_t count = 0;
XrResult result = enumerate(0, &count, nullptr);
if (XR_FAILED(result)) return result;
if (count > 4096) return XR_ERROR_LIMIT_REACHED;
values->assign(count, T{});
for (auto& value : *values) value.type = type;
if (count == 0) return XR_SUCCESS;
result = enumerate(count, &count, values->data());
if (result == XR_ERROR_SIZE_INSUFFICIENT) continue;
if (XR_FAILED(result)) return result;
if (count > values->size()) return XR_ERROR_RUNTIME_FAILURE;
values->resize(count);
return XR_SUCCESS;
}
return XR_ERROR_SIZE_INSUFFICIENT;
}
} // namespace
bool IsOpenXrSupportCompiled() { return true; }
ProbeReport ProbeOpenXr() {
ProbeReport report;
report.sdk_version = Version(XR_CURRENT_API_VERSION);
std::vector<XrExtensionProperties> extensions;
XrResult result = EnumerateProperties<XrExtensionProperties>(
XR_TYPE_EXTENSION_PROPERTIES,
[](std::uint32_t capacity, std::uint32_t* count,
XrExtensionProperties* data) {
return xrEnumerateInstanceExtensionProperties(nullptr, capacity, count,
data);
},
&extensions);
if (XR_FAILED(result)) {
Failure(&report, XR_NULL_HANDLE, "xrEnumerateInstanceExtensionProperties",
result);
return report;
}
for (const auto& extension : extensions) {
report.extensions.emplace_back(extension.extensionName);
}
const bool vulkan =
std::find(report.extensions.begin(), report.extensions.end(),
XR_KHR_VULKAN_ENABLE2_EXTENSION_NAME) !=
report.extensions.end();
const char* enabled_extension = XR_KHR_VULKAN_ENABLE2_EXTENSION_NAME;
XrInstanceCreateInfo create_info{};
create_info.type = XR_TYPE_INSTANCE_CREATE_INFO;
std::strcpy(create_info.applicationInfo.applicationName, "Mocktail VR probe");
std::strcpy(create_info.applicationInfo.engineName, "Mocktail");
// Use the latest SDK while requiring only the core API this probe actually
// uses, so runtimes exposing OpenXR 1.0 can also be diagnosed.
create_info.applicationInfo.apiVersion = XR_MAKE_VERSION(1, 0, 0);
create_info.enabledExtensionCount = vulkan ? 1 : 0;
create_info.enabledExtensionNames = vulkan ? &enabled_extension : nullptr;
InstanceScope scope;
result = xrCreateInstance(&create_info, &scope.instance);
if (XR_FAILED(result)) {
Failure(&report, XR_NULL_HANDLE, "xrCreateInstance", result);
return report;
}
XrInstanceProperties properties{};
properties.type = XR_TYPE_INSTANCE_PROPERTIES;
result = xrGetInstanceProperties(scope.instance, &properties);
if (XR_FAILED(result)) {
Failure(&report, scope.instance, "xrGetInstanceProperties", result);
return report;
}
report.runtime_name = properties.runtimeName;
report.runtime_version = Version(properties.runtimeVersion);
XrSystemGetInfo system_info{};
system_info.type = XR_TYPE_SYSTEM_GET_INFO;
system_info.formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY;
XrSystemId system = XR_NULL_SYSTEM_ID;
result = xrGetSystem(scope.instance, &system_info, &system);
if (XR_FAILED(result)) {
Failure(&report, scope.instance, "xrGetSystem", result);
return report;
}
XrSystemProperties system_properties{};
system_properties.type = XR_TYPE_SYSTEM_PROPERTIES;
result = xrGetSystemProperties(scope.instance, system, &system_properties);
if (XR_FAILED(result)) {
Failure(&report, scope.instance, "xrGetSystemProperties", result);
return report;
}
report.system_name = system_properties.systemName;
report.orientation_tracking =
system_properties.trackingProperties.orientationTracking == XR_TRUE;
report.position_tracking =
system_properties.trackingProperties.positionTracking == XR_TRUE;
if (!vulkan) {
report.status = ProbeStatus::kIncompatibleRuntime;
report.message = "The runtime does not expose XR_KHR_vulkan_enable2.";
return report;
}
std::vector<XrViewConfigurationView> views;
result = EnumerateProperties<XrViewConfigurationView>(
XR_TYPE_VIEW_CONFIGURATION_VIEW,
[&](std::uint32_t capacity, std::uint32_t* count,
XrViewConfigurationView* data) {
return xrEnumerateViewConfigurationViews(
scope.instance, system, XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO,
capacity, count, data);
},
&views);
if (XR_FAILED(result)) {
Failure(&report, scope.instance, "xrEnumerateViewConfigurationViews",
result);
return report;
}
for (const auto& view : views) {
report.views.push_back({view.recommendedImageRectWidth,
view.recommendedImageRectHeight,
view.recommendedSwapchainSampleCount});
}
if (views.size() != 2 || std::any_of(report.views.begin(), report.views.end(),
[](const ViewConfiguration& view) {
return !view.width || !view.height ||
!view.samples;
})) {
report.status = ProbeStatus::kIncompatibleRuntime;
report.message = "The runtime did not provide two usable stereo views.";
return report;
}
PFN_xrVoidFunction function = nullptr;
result = xrGetInstanceProcAddr(
scope.instance, "xrGetVulkanGraphicsRequirements2KHR", &function);
if (XR_FAILED(result) || function == nullptr) {
Failure(&report, scope.instance,
"xrGetVulkanGraphicsRequirements2KHR lookup",
XR_FAILED(result) ? result : XR_ERROR_FUNCTION_UNSUPPORTED);
return report;
}
XrGraphicsRequirementsVulkan2KHR requirements{};
requirements.type = XR_TYPE_GRAPHICS_REQUIREMENTS_VULKAN2_KHR;
result = reinterpret_cast<PFN_xrGetVulkanGraphicsRequirements2KHR>(function)(
scope.instance, system, &requirements);
if (XR_FAILED(result)) {
Failure(&report, scope.instance, "xrGetVulkanGraphicsRequirements2KHR",
result);
return report;
}
report.minimum_vulkan_version = Version(requirements.minApiVersionSupported);
report.maximum_vulkan_version = Version(requirements.maxApiVersionSupported);
report.status = ProbeStatus::kReady;
report.message =
"OpenXR HMD and Vulkan requirements discovered successfully.";
return report;
}
} // namespace mocktail::vr
+50
View File
@@ -0,0 +1,50 @@
#include "mocktail/vr/openxr_probe.h"
#include "mocktail/vr/openxr_preview.h"
#include <charconv>
#include <iostream>
#include <string_view>
int main(int argc, char* argv[]) {
if (argc == 2 && (std::string_view(argv[1]) == "--help" ||
std::string_view(argv[1]) == "-h")) {
std::cout
<< "Usage: mocktail-vr-probe [--scene [--frames N]]\n"
"Inspect the host OpenXR runtime and HMD capabilities.\n"
"Uses the active runtime, or XR_RUNTIME_JSON if set.\n"
"Without arguments: discovery only, no XR session.\n"
"--scene: render a tracked test panel in a real Vulkan/XR session.\n"
"--frames N: request exit after N frames (positive integer).\n"
"Ctrl+C or the runtime closes the scene. Does not load Roblox.\n"
"Exit codes: 0=discovery ready, 1=API error, 2=build disabled,\n"
"3=runtime unavailable, 4=HMD unavailable, 5=incompatible "
"runtime.\n";
return 0;
}
if (argc >= 2 && std::string_view(argv[1]) == "--scene") {
mocktail::vr::PreviewOptions options;
if (argc == 4 && std::string_view(argv[2]) == "--frames") {
const std::string_view value(argv[3]);
const auto parsed = std::from_chars(
value.data(), value.data() + value.size(), options.frame_limit);
if (parsed.ec != std::errc() ||
parsed.ptr != value.data() + value.size() || !options.frame_limit) {
std::cerr << "--frames requires a positive integer.\n";
return 1;
}
} else if (argc != 2) {
std::cerr << "Use --scene [--frames N].\n";
return 1;
}
const auto result = mocktail::vr::RunOpenXrPreview(options);
std::cout << mocktail::vr::FormatPreviewResult(result);
return result.success ? 0 : 1;
}
if (argc != 1) {
std::cerr << "Use mocktail-vr-probe --help for supported options.\n";
return 1;
}
const auto report = mocktail::vr::ProbeOpenXr();
std::cout << mocktail::vr::FormatProbeReport(report);
return mocktail::vr::ProbeExitCode(report.status);
}
File diff suppressed because it is too large. Load diff
+200
View File
@@ -0,0 +1,200 @@
#ifndef MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_INTERNAL_H_
#define MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_INTERNAL_H_
#include <cstddef>
#include <cstdint>
#include <string_view>
#include "compat/build_profile.h"
#include "mocktail/vr/xr_controller.h"
namespace mocktail::vr { struct ScriptedPoseSample; }
namespace mocktail::vr::internal {
struct VrBuildContract {
std::string_view build_id;
compat::VrDebugDeviceBridgeProfile device;
std::uintptr_t state_memcpy_rva;
std::uintptr_t pointer_frame_rva;
std::uintptr_t workspace_getter_rva;
std::uintptr_t camera_vtable_offset;
std::uintptr_t hand_pitch_storage_rva;
std::uintptr_t haptic_sink_rva;
// Field offsets the pointer hook reads: the hand index in the pointer
// service and HeadScale in the workspace camera.
std::uintptr_t pointer_hand_selector_offset;
std::uintptr_t camera_head_scale_offset;
std::uintptr_t keep_haptics_flag_rva;
std::uintptr_t vr_haptics_flag_rva;
std::uintptr_t support_generic_haptics_flag_rva;
std::uintptr_t fix_register_motors_flag_rva;
// Roblox version code and name, for messages that list supported builds.
std::string_view version;
};
const VrBuildContract* FindVrBuildContract(std::string_view build_id);
// The hand the pointer service asks about: 1 left, 2 right.
int ReadPointerHandIndex(const void* service, const VrBuildContract& contract);
float ReadCameraHeadScale(const void* camera, const VrBuildContract& contract);
// Sets the fast flag at `rva` to true only when its name pointer (+8) points
// at `name` inside the image. Returns whether it wrote.
bool EnableNamedFastFlag(std::uintptr_t image_base, std::uintptr_t rva,
std::string_view name);
// HapticService:SetMotor arguments the Android client drops: its haptics
// backend only consumes Gamepad1 Large/Small, so LeftHand/RightHand need a
// VR sink of their own.
inline constexpr std::int32_t kUserInputTypeGamepad1 = 12;
inline constexpr std::int32_t kVibrationMotorLeftHand = 4;
inline constexpr std::int32_t kVibrationMotorRightHand = 5;
// How the engine's SetMotor intensity reader takes the value tuple.
enum class MotorIntensityAbi {
kNone,
// float(const shared_ptr<const Tuple>*), 3222.
kSharedTuple,
// float(const Variant* begin, const Variant* end), 2998.
kTupleSpan,
};
struct MotorIntensityReader {
std::uintptr_t rva = 0;
MotorIntensityAbi abi = MotorIntensityAbi::kNone;
};
// Both scan [scan_rva, scan_rva + scan_size) of a flat `image` view whose
// offsets are RVAs, and succeed only on exactly one match.
//
// The reader is the call in the Android backend's motor slot:
// cmp esi,Gamepad1; jne; cmp edx,1; je; test edx,edx; jne; ...; call reader;
// movss [r14+0x54],xmm0
MotorIntensityReader FindMotorIntensityReader(const std::uint8_t* image,
std::size_t image_size,
std::uintptr_t scan_rva,
std::size_t scan_size);
// The descriptor is the one registered as lea rdi,[descriptor];
// lea rdx,["SetMotor"]. Returns its RVA, or 0.
std::uintptr_t FindSetMotorDescriptor(const std::uint8_t* image,
std::size_t image_size,
std::uintptr_t scan_rva,
std::size_t scan_size);
// Offset of the member function pointer in that descriptor: the first of
// +0x80/+0x88 holding an even code address followed by a zero this-adjust.
// `is_code` says whether an RVA is executable. Returns 0 when neither fits.
std::size_t SetMotorDescriptorSlot(std::uintptr_t image_base,
std::uintptr_t descriptor_rva,
bool (*is_code)(std::uintptr_t image_base,
std::uintptr_t rva));
// Computes the record orientation for a hand pose: the aim pose (or, with
// MOCKTAIL_VR_HAND_POSE_SOURCE=grip, the grip pose) pitched by the engine's
// own hand pitch plus the configurable offset.
pose::Quat ComputeHandRecordOrientation(const VrHandPose& source,
float native_pitch_deg);
float GetHandPitchOffsetDeg();
// The engine's stored hand pitch (degrees), as last read from the guest;
// 20 until it has been read.
void NoteNativeHandPitchDeg(int degrees);
float NativeHandPitchDeg();
// Writes only a caller-owned state copy; preserves all resource readiness.
bool ApplyXrPose(void* state, const ScriptedPoseSample& pose);
// Adjust a world-space grip CoordinateFrame to OpenXR aim without changing
// the engine's hand pose. Native frame layout: row-major R[9], position[3].
bool ApplyAimToWorldFrame(float* frame, const VrHandPose& hand,
float head_scale, float native_hand_pitch_degrees);
// Guest ABI layout constants of RBX::Graphics::DebugDeviceVR. They are pinned
// per exact build by the machine-code contracts below; changing payload
// requires re-deriving both.
inline constexpr std::size_t kObjectNameOffset = 0x8;
inline constexpr std::size_t kObjectTypeOffset = 0x10;
inline constexpr std::uint32_t kExpectedObjectType = 6;
inline constexpr std::size_t kStateOffset = 0x14;
inline constexpr std::size_t kStateCopySize = 0x138;
inline constexpr std::size_t kStateReadyOffsetInCopy = 0x133;
// Per-record pose layout inside the 0x138 state copy. Verified against 2998
// consumer 0x28a6242 (docs/vr-agent-evidence/controller-2998): four 0x20-byte
// records are each converted by helper 0x37eacb8 (metres->studs x10/3, quat
// normalized) and stored via setUserCFrame 0x4d56906 at VRService+0x174+idx*0x30.
// copy+0x00 head -> UserCFrame index 0
// copy+0x20 extra -> index 3
// copy+0x40 left hand -> index 1
// copy+0x60 right hand-> index 2
// Record field layout (same for head and hands): valid byte +0, position
// (3 floats, metres) +4, quaternion xyzw (4 floats) +0x10. The guest applies
// the 10/3 conversion and a -20 deg hand pitch offset itself; neither is
// pre-applied here.
inline constexpr std::size_t kPoseRecordStride = 0x20;
inline constexpr std::size_t kHeadRecordOffset = 0x00;
inline constexpr std::size_t kExtraRecordOffset = 0x20;
inline constexpr std::size_t kLeftHandRecordOffset = 0x40;
inline constexpr std::size_t kRightHandRecordOffset = 0x60;
inline constexpr std::size_t kRecordValidOffset = 0x00;
inline constexpr std::size_t kRecordPositionOffset = 0x04;
inline constexpr std::size_t kRecordOrientationOffset = 0x10;
inline constexpr std::size_t kReadyByteOffset = 0x147;
inline constexpr std::size_t kFramebufferSlotOffset = 0x150;
inline constexpr std::size_t kTextureSlotOffset = 0x170;
inline constexpr std::size_t kEyeSlotStride = 0x10;
inline constexpr std::size_t kWidthOffset = 0x190;
inline constexpr std::size_t kHeightOffset = 0x194;
inline constexpr std::size_t kGraphicsDeviceOffset = 0x198;
// Per-eye camera parameters filled by the constructor: symmetric +-0.25 stud
// view offsets (half IPD), tan-space projection values and FOV degrees.
inline constexpr std::size_t kEye0ViewOffset = 0x10c;
inline constexpr std::size_t kEye1ViewOffset = 0x118;
inline constexpr std::size_t kEye0ProjectionOffset = 0x124;
inline constexpr std::size_t kEye1ProjectionOffset = 0x134;
inline constexpr std::size_t kFieldOfViewOffset = 0x1a0;
inline constexpr std::size_t kHalfInterpupillaryOffset = 0x1b8;
inline constexpr std::size_t kStateGetterVtableSlot = 0x28;
inline constexpr std::size_t kEyeGetterVtableSlot = 0x48;
inline constexpr std::size_t kEyeInitializerVtableSlot = 0xd0;
inline constexpr std::size_t kStateGetterContractSize = 33;
inline constexpr std::size_t kEyeGetterContractSize = 16;
inline constexpr std::size_t kConstructorContractSize = 664;
inline constexpr std::size_t kInitializerContractSize = 700;
enum class EyeResourceState { kEmpty, kReady, kInvalid };
// Reads only the current live object's ABI bytes. A ready byte alone is
// insufficient; dimensions and both distinct framebuffer/texture slots count.
EyeResourceState InspectEyeResources(const void *object);
// Exact 33-byte state getter: sret ABI (rdi = hidden 0x138-byte result
// buffer, rsi = device), copies state from self+0x14 and returns the buffer.
bool HasExpectedVrStateGetterContract(const std::uint8_t* code,
std::size_t size,
const VrBuildContract& contract);
// Exact 16-byte eye framebuffer getter: self + 0x150 + eyeIndex * 16.
bool HasExpectedVrEyeGetterContract(const std::uint8_t *code, std::size_t size);
// Constructor: captures rcx (graphics device) into self+0x198, width/height
// into +0x190/+0x194, type dword 6 into +0x10, and clears the state block.
bool HasExpectedVrConstructorContract(
const std::uint8_t* code, std::size_t size,
const compat::VrDebugDeviceBridgeProfile& profile);
bool HasExpectedVrPointerContract(const std::uint8_t* code, std::size_t size,
const VrBuildContract& contract);
// Eye-resource initializer: raises ready byte self+0x147 plus +0x94/+0x144
// and creates the per-eye resources through the graphics-device vtable slot
// taken from the profile (2998 uses +0x180; other builds differ).
bool HasExpectedVrEyeInitializerContract(
const std::uint8_t *code, std::size_t size,
const compat::VrDebugDeviceBridgeProfile &profile);
// DebugDeviceVR vtable must point the three interposed/used slots at the
// exact profile RVAs before any interposition.
bool HasExpectedVrDebugDeviceVtable(
const std::uintptr_t *vtable, std::uintptr_t image_base,
const compat::VrDebugDeviceBridgeProfile &profile);
} // namespace mocktail::vr::internal
#endif // MOCKTAIL_VR_ROBLOX_VR_DEVICE_BRIDGE_INTERNAL_H_
+393
View File
@@ -0,0 +1,393 @@
#include "mocktail/vr/virtual_keyboard.h"
#include <algorithm>
#include <cctype>
#include <cmath>
#include <mutex>
namespace mocktail::vr {
namespace {
using pose::kPi;
using pose::Pose;
using pose::Quat;
using pose::QuatConjugate;
using pose::QuatFromAxisAngle;
using pose::QuatMultiply;
using pose::QuatRotate;
using pose::Vec3;
// SDL scancodes, kept numeric so this pure module needs no SDL headers.
constexpr std::uint32_t kScancodeReturn = 40; // SDL_SCANCODE_RETURN
constexpr std::uint32_t kScancodeEscape = 41; // SDL_SCANCODE_ESCAPE
constexpr std::uint32_t kScancodeBackspace = 42; // SDL_SCANCODE_BACKSPACE
constexpr std::uint32_t kScancodeTab = 43; // SDL_SCANCODE_TAB
// Matches ControllerStateBuilder's analog click hysteresis.
constexpr float kTriggerPress = 0.75f;
constexpr float kTriggerRelease = 0.6f;
constexpr std::uint64_t kRepeatDelayNs = 450'000'000ULL;
constexpr std::uint64_t kRepeatIntervalNs = 60'000'000ULL;
// Cursor moves smaller than this (in UV) do not force a redraw.
constexpr float kCursorRedrawStep = 0.004f;
// A moving cursor redraws the panel at most this often; each redraw is a
// full texture upload on the frame thread.
constexpr std::uint64_t kCursorRedrawIntervalNs = 33'000'000ULL;
constexpr float kPanelDistanceMetres = 0.45f;
constexpr float kPanelDropMetres = 0.28f;
constexpr float kPanelTiltRadians = 35.f * kPi / 180.f;
// US QWERTY without the function row or Ctrl/Alt/Menu, in normal-key units.
// Widths follow the desktop shell OSK (Tab 1.6, Shift 2.5, wide keys fill).
constexpr float kColumns = 15.f;
constexpr int kRows = 5;
std::mutex g_host_mutex;
VirtualKeyboardHost g_host;
std::mutex g_text_mutex;
VirtualKeyboardText g_text;
void ScrubText(std::string* text) {
std::fill(text->begin(), text->end(), '\0');
text->clear();
}
struct KeySpec {
const char* label;
const char* text;
const char* shift_text;
VirtualKeyAction action;
float width;
};
KeySpec Char(const char* text, const char* shift_text) {
return {text, text, shift_text, VirtualKeyAction::kText, 1.f};
}
KeySpec Special(const char* label, VirtualKeyAction action, float width) {
return {label, "", "", action, width};
}
std::vector<std::vector<KeySpec>> Rows() {
using A = VirtualKeyAction;
return {
{Char("`", "~"), Char("1", "!"), Char("2", "@"), Char("3", "#"),
Char("4", "$"), Char("5", "%"), Char("6", "^"), Char("7", "&"),
Char("8", "*"), Char("9", "("), Char("0", ")"), Char("-", "_"),
Char("=", "+"), Special("Backspace", A::kBackspace, 2.f)},
{Special("Tab", A::kTab, 1.6f), Char("q", "Q"), Char("w", "W"),
Char("e", "E"), Char("r", "R"), Char("t", "T"), Char("y", "Y"),
Char("u", "U"), Char("i", "I"), Char("o", "O"), Char("p", "P"),
Char("[", "{"), Char("]", "}"), {"\\", "\\", "|", A::kText, 1.4f}},
{Special("Caps", A::kCapsLock, 2.f), Char("a", "A"), Char("s", "S"),
Char("d", "D"), Char("f", "F"), Char("g", "G"), Char("h", "H"),
Char("j", "J"), Char("k", "K"), Char("l", "L"), Char(";", ":"),
Char("'", "\""), Special("Enter", A::kEnter, 2.f)},
{Special("Shift", A::kShift, 2.5f), Char("z", "Z"), Char("x", "X"),
Char("c", "C"), Char("v", "V"), Char("b", "B"), Char("n", "N"),
Char("m", "M"), Char(",", "<"), Char(".", ">"), Char("/", "?"),
Special("Shift", A::kShift, 2.5f)},
{Special("Space", A::kSpace, 12.5f), Special("Close", A::kClose, 2.5f)},
};
}
bool IsLetter(const std::string& text) {
return text.size() == 1 &&
std::isalpha(static_cast<unsigned char>(text[0])) != 0;
}
float Dot(const Vec3& a, const Vec3& b) {
return a.x * b.x + a.y * b.y + a.z * b.z;
}
Vec3 Sub(const Vec3& a, const Vec3& b) {
return {a.x - b.x, a.y - b.y, a.z - b.z};
}
} // namespace
void SetVirtualKeyboardHost(const VirtualKeyboardHost* host) {
std::lock_guard<std::mutex> lock(g_host_mutex);
g_host = host != nullptr ? *host : VirtualKeyboardHost{};
}
VirtualKeyboardHost CurrentVirtualKeyboardHost() {
std::lock_guard<std::mutex> lock(g_host_mutex);
return g_host;
}
void SetVirtualKeyboardText(const char* utf8, std::size_t size,
int cursor_utf16, bool password) {
std::lock_guard<std::mutex> lock(g_text_mutex);
ScrubText(&g_text.utf8);
if (utf8 != nullptr) g_text.utf8.assign(utf8, size);
g_text.cursor_utf16 = std::max(0, cursor_utf16);
g_text.password = password;
++g_text.revision;
}
void ClearVirtualKeyboardText() {
std::lock_guard<std::mutex> lock(g_text_mutex);
ScrubText(&g_text.utf8);
g_text.cursor_utf16 = 0;
g_text.password = false;
++g_text.revision;
}
VirtualKeyboardText CurrentVirtualKeyboardText() {
std::lock_guard<std::mutex> lock(g_text_mutex);
return g_text;
}
bool IntersectVirtualKeyboardPanel(const Pose& quad, float width, float height,
const Pose& aim, float* u, float* v,
float* distance) {
const Vec3 normal = QuatRotate(quad.orientation, Vec3{0.f, 0.f, 1.f});
const Vec3 direction = QuatRotate(aim.orientation, Vec3{0.f, 0.f, -1.f});
const float denominator = Dot(direction, normal);
// Only the front face takes input; a ray from behind passes through.
if (denominator > -1e-4f) return false;
const float t = Dot(Sub(quad.position, aim.position), normal) / denominator;
if (!(t > 0.f)) return false;
const Vec3 hit{aim.position.x + direction.x * t,
aim.position.y + direction.y * t,
aim.position.z + direction.z * t};
const Vec3 local =
QuatRotate(QuatConjugate(quad.orientation), Sub(hit, quad.position));
const float hit_u = local.x / width + 0.5f;
const float hit_v = 0.5f - local.y / height;
if (hit_u < 0.f || hit_u > 1.f || hit_v < 0.f || hit_v > 1.f) return false;
if (u != nullptr) *u = hit_u;
if (v != nullptr) *v = hit_v;
if (distance != nullptr) *distance = t;
return true;
}
VirtualKeyboard::VirtualKeyboard() { RebuildLayout(); }
void VirtualKeyboard::RebuildLayout() {
const auto rows = Rows();
keys_.clear();
for (int row = 0; row < kRows; ++row) {
float column = 0.f;
for (const KeySpec& spec : rows[row]) {
VirtualKey key;
key.action = spec.action;
key.text = spec.text;
key.shift_text = spec.shift_text;
key.label = spec.action == VirtualKeyAction::kText ? Output(key)
: spec.label;
key.u0 = column / kColumns;
key.u1 = std::min(1.f, (column + spec.width) / kColumns);
constexpr float kField = VirtualKeyboard::kFieldFraction;
key.v0 = kField + (1.f - kField) * static_cast<float>(row) / kRows;
key.v1 = kField + (1.f - kField) * static_cast<float>(row + 1) / kRows;
keys_.push_back(std::move(key));
column += spec.width;
}
}
for (Hand& hand : hands_) {
hand.hovered = -1;
hand.pressed = -1;
}
Touch();
}
std::string VirtualKeyboard::Output(const VirtualKey& key) const {
// Caps Lock only affects letters; Shift inverts it for letters and selects
// the upper symbol on every other key.
if (IsLetter(key.text)) {
return (shift_ != caps_lock_) ? key.shift_text : key.text;
}
return shift_ ? key.shift_text : key.text;
}
const std::vector<VirtualKey>& VirtualKeyboard::keys() const { return keys_; }
void VirtualKeyboard::Show(const Pose& head) {
Vec3 forward = QuatRotate(head.orientation, Vec3{0.f, 0.f, -1.f});
forward.y = 0.f;
const float length = std::sqrt(forward.x * forward.x + forward.z * forward.z);
if (length < 1e-3f) {
forward = Vec3{0.f, 0.f, -1.f};
} else {
forward.x /= length;
forward.z /= length;
}
const float yaw = std::atan2(-forward.x, -forward.z);
panel_.orientation =
QuatMultiply(QuatFromAxisAngle(Vec3{0.f, 1.f, 0.f}, yaw),
QuatFromAxisAngle(Vec3{1.f, 0.f, 0.f}, -kPanelTiltRadians));
panel_.position =
Vec3{head.position.x + forward.x * kPanelDistanceMetres,
head.position.y - kPanelDropMetres,
head.position.z + forward.z * kPanelDistanceMetres};
visible_ = true;
shift_ = false;
for (Hand& hand : hands_) hand = Hand{};
RebuildLayout();
}
void VirtualKeyboard::SetText(const VirtualKeyboardText& text) {
if (text.revision == text_.revision) return;
const bool changed = text.utf8 != text_.utf8 ||
text.cursor_utf16 != text_.cursor_utf16 ||
text.password != text_.password;
ScrubText(&text_.utf8);
text_ = text;
if (changed) Touch();
}
void VirtualKeyboard::Hide() {
ScrubText(&text_.utf8);
text_.revision = UINT64_MAX; // the next SetText always reloads
if (!visible_) return;
visible_ = false;
for (int index = 0; index < 2; ++index) {
// Keeps a press already pending from an earlier hide of this same press.
release_pending_[index] =
release_pending_[index] ||
(hands_[index].trigger_down && hands_[index].captured);
hands_[index] = Hand{};
}
Touch();
}
int VirtualKeyboard::KeyAt(float u, float v) const {
for (std::size_t index = 0; index < keys_.size(); ++index) {
const VirtualKey& key = keys_[index];
if (u >= key.u0 && u < key.u1 && v >= key.v0 && v < key.v1) {
return static_cast<int>(index);
}
}
return -1;
}
bool VirtualKeyboard::CapturesTrigger(int hand) const {
const Hand& state = hands_[hand & 1];
return release_pending_[hand & 1] ||
(visible_ && (state.over_panel || state.captured));
}
bool VirtualKeyboard::cursor(int hand, float* u, float* v) const {
const Hand& state = hands_[hand & 1];
if (!visible_ || !state.over_panel) return false;
*u = state.u;
*v = state.v;
return true;
}
void VirtualKeyboard::Activate(const VirtualKey& key,
VirtualKeyboardOutput* output) {
switch (key.action) {
case VirtualKeyAction::kText:
output->text.push_back(Output(key));
if (shift_) {
shift_ = false; // one-shot, like a phone keyboard
RebuildLayout();
}
return;
case VirtualKeyAction::kSpace:
output->text.emplace_back(" ");
return;
case VirtualKeyAction::kBackspace:
output->scancodes.push_back(kScancodeBackspace);
return;
case VirtualKeyAction::kEnter:
output->scancodes.push_back(kScancodeReturn);
return;
case VirtualKeyAction::kTab:
output->scancodes.push_back(kScancodeTab);
return;
case VirtualKeyAction::kShift:
shift_ = !shift_;
RebuildLayout();
return;
case VirtualKeyAction::kCapsLock:
caps_lock_ = !caps_lock_;
RebuildLayout();
return;
case VirtualKeyAction::kClose:
output->scancodes.push_back(kScancodeEscape);
Hide();
return;
}
}
VirtualKeyboardOutput VirtualKeyboard::Update(
std::uint64_t now_ns, const VirtualKeyboardPointer (&pointers)[2]) {
VirtualKeyboardOutput output;
bool held_over[2] = {false, false};
for (int index = 0; index < 2; ++index) {
if (!release_pending_[index]) continue;
// Losing the aim pose is not a release: the trigger is read regardless.
held_over[index] = pointers[index].trigger > kTriggerRelease;
release_pending_[index] = held_over[index];
}
// Decided once per frame, or one moving hand starves the other's cursor.
const bool cursor_redraw_due =
now_ns - cursor_drawn_ns_ >= kCursorRedrawIntervalNs;
for (int index = 0; index < 2 && visible_; ++index) {
if (held_over[index]) continue;
Hand& hand = hands_[index];
const VirtualKeyboardPointer& pointer = pointers[index];
float u = 0.f, v = 0.f;
const bool over =
pointer.valid &&
IntersectVirtualKeyboardPanel(panel_, kPanelWidthMetres,
kPanelHeightMetres, pointer.aim, &u, &v,
nullptr);
const int hovered = over ? KeyAt(u, v) : -1;
const bool cursor_moved =
over && (std::abs(u - hand.u) > kCursorRedrawStep ||
std::abs(v - hand.v) > kCursorRedrawStep);
if (over != hand.over_panel || hovered != hand.hovered ||
(cursor_moved && cursor_redraw_due)) {
Touch();
hand.u = u;
hand.v = v;
cursor_drawn_ns_ = now_ns;
}
hand.over_panel = over;
hand.hovered = hovered;
const bool was_down = hand.trigger_down;
if (!pointer.valid) {
hand.trigger_down = false;
} else if (was_down) {
hand.trigger_down = pointer.trigger > kTriggerRelease;
} else {
hand.trigger_down = pointer.trigger >= kTriggerPress;
}
if (hand.trigger_down && !was_down) {
hand.captured = over;
if (hovered >= 0) {
hand.pressed = hovered;
hand.repeat_at_ns = now_ns + kRepeatDelayNs;
Touch();
// Activate may rebuild the layout and clear pressed/hovered.
const VirtualKey key = keys_[hovered];
Activate(key, &output);
}
} else if (hand.trigger_down) {
if (hand.pressed >= 0 && hand.pressed != hovered) {
hand.pressed = -1;
Touch();
} else if (hand.pressed >= 0 &&
keys_[hand.pressed].action == VirtualKeyAction::kBackspace &&
now_ns >= hand.repeat_at_ns) {
output.scancodes.push_back(kScancodeBackspace);
hand.repeat_at_ns = now_ns + kRepeatIntervalNs;
}
} else if (was_down) {
hand.captured = false;
if (hand.pressed >= 0) {
hand.pressed = -1;
Touch();
}
}
}
return output;
}
} // namespace mocktail::vr
+325
View File
@@ -0,0 +1,325 @@
#include "mocktail/vr/virtual_keyboard.h"
#include <SDL3/SDL_pixels.h>
#include <SDL3/SDL_surface.h>
#include <SDL3_ttf/SDL_ttf.h>
#include <fontconfig/fontconfig.h>
#include <algorithm>
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <map>
#include <string>
#include <utility>
namespace mocktail::vr {
namespace {
struct Rgb {
std::uint8_t r, g, b;
};
constexpr Rgb kBackground{24, 25, 30};
constexpr Rgb kKey{58, 60, 70};
constexpr Rgb kSpecialKey{42, 44, 52};
constexpr Rgb kHover{92, 96, 112};
constexpr Rgb kAccent{82, 132, 255};
constexpr Rgb kAccentHover{120, 160, 255};
constexpr Rgb kLabel{240, 242, 248};
constexpr Rgb kField{12, 13, 17};
constexpr int kFieldPadding = 24;
constexpr int kGap = 8;
constexpr int kRadius = 12;
constexpr float kLetterPoints = 40.f;
constexpr float kSpecialPoints = 28.f;
// fontconfig's best sans face; MOCKTAIL_TEXT_FONT wins like the TextBox
// overlay.
std::string ResolveSansFont() {
const char* configured = std::getenv("MOCKTAIL_TEXT_FONT");
if (configured != nullptr && configured[0] != '\0') return configured;
std::string file;
if (!FcInit()) return file;
FcPattern* pattern =
FcNameParse(reinterpret_cast<const FcChar8*>("sans:weight=medium"));
if (pattern == nullptr) return file;
FcConfigSubstitute(nullptr, pattern, FcMatchPattern);
FcDefaultSubstitute(pattern);
FcResult result = FcResultNoMatch;
FcPattern* match = FcFontMatch(nullptr, pattern, &result);
FcPatternDestroy(pattern);
if (match == nullptr) return file;
FcChar8* path = nullptr;
if (FcPatternGetString(match, FC_FILE, 0, &path) == FcResultMatch &&
path != nullptr) {
file = reinterpret_cast<const char*>(path);
}
FcPatternDestroy(match);
return file;
}
struct AlphaMask {
int width = 0;
int height = 0;
std::vector<std::uint8_t> alpha;
};
// Field text as drawn: password input shows one dot per code point.
std::string DisplayText(const VirtualKeyboard& keyboard) {
if (!keyboard.text_password()) return keyboard.text();
std::string dots;
for (const char c : keyboard.text())
if ((static_cast<unsigned char>(c) & 0xC0) != 0x80) dots += "\u2022";
return dots;
}
// Byte offset in `text` of the UTF-16 cursor. Password dots are one UTF-16
// unit each, like the BMP characters they stand for.
std::size_t CursorByte(const std::string& text, int cursor_utf16) {
std::size_t byte = 0;
int units = 0;
while (byte < text.size() && units < cursor_utf16) {
const auto lead = static_cast<unsigned char>(text[byte]);
const std::size_t length = lead < 0x80 ? 1 : lead < 0xE0 ? 2
: lead < 0xF0 ? 3 : 4;
units += length == 4 ? 2 : 1;
byte = std::min(text.size(), byte + length);
}
return byte;
}
void Scrub(std::vector<std::uint8_t>* bytes) {
std::fill(bytes->begin(), bytes->end(), 0);
bytes->clear();
}
bool InsideRoundedRect(int x, int y, int left, int top, int right, int bottom,
int radius) {
const int cx = std::clamp(x, left + radius, right - 1 - radius);
const int cy = std::clamp(y, top + radius, bottom - 1 - radius);
const int dx = x - cx;
const int dy = y - cy;
return dx * dx + dy * dy <= radius * radius;
}
} // namespace
struct VirtualKeyboardRasterizer::Impl {
bool font_attempted = false;
bool ttf_initialized = false;
TTF_Font* letter_font = nullptr;
TTF_Font* special_font = nullptr;
std::map<std::pair<std::string, bool>, AlphaMask> masks;
// Last rendered field text; not cached by content, since it is sensitive.
AlphaMask field;
~Impl() {
Scrub(&field.alpha);
if (letter_font != nullptr) TTF_CloseFont(letter_font);
if (special_font != nullptr) TTF_CloseFont(special_font);
if (ttf_initialized) TTF_Quit();
}
bool EnsureFonts() {
if (font_attempted) return letter_font != nullptr;
font_attempted = true;
if (!TTF_Init()) {
std::fprintf(stderr, " [vr-keyboard] SDL3_ttf unavailable: %s\n",
SDL_GetError());
return false;
}
ttf_initialized = true;
const std::string file = ResolveSansFont();
if (!file.empty()) {
letter_font = TTF_OpenFont(file.c_str(), kLetterPoints);
special_font = TTF_OpenFont(file.c_str(), kSpecialPoints);
}
if (letter_font == nullptr || special_font == nullptr) {
std::fprintf(stderr,
" [vr-keyboard] no usable font (%s); keys are unlabelled\n",
file.empty() ? "fontconfig found none" : file.c_str());
return false;
}
std::fprintf(stderr, " [vr-keyboard] labels use %s\n", file.c_str());
return true;
}
static AlphaMask Render(TTF_Font* font, const std::string& text) {
AlphaMask mask;
SDL_Surface* rendered = TTF_RenderText_Blended(
font, text.c_str(), text.size(), SDL_Color{255, 255, 255, 255});
if (rendered != nullptr) {
SDL_Surface* rgba = SDL_ConvertSurface(rendered, SDL_PIXELFORMAT_RGBA32);
if (rgba != nullptr && SDL_LockSurface(rgba)) {
mask.width = rgba->w;
mask.height = rgba->h;
mask.alpha.resize(static_cast<std::size_t>(mask.width) * mask.height);
for (int y = 0; y < mask.height; ++y) {
const auto* row = static_cast<const std::uint8_t*>(rgba->pixels) +
static_cast<std::size_t>(y) * rgba->pitch;
for (int x = 0; x < mask.width; ++x) {
mask.alpha[static_cast<std::size_t>(y) * mask.width + x] =
row[x * 4 + 3];
}
}
SDL_UnlockSurface(rgba);
}
if (rgba != nullptr) SDL_DestroySurface(rgba);
SDL_DestroySurface(rendered);
}
return mask;
}
const AlphaMask* Mask(const std::string& label, bool special) {
if (label.empty() || !EnsureFonts()) return nullptr;
const auto key = std::make_pair(label, special);
auto found = masks.find(key);
if (found != masks.end()) return &found->second;
return &masks
.emplace(key, Render(special ? special_font : letter_font,
label))
.first->second;
}
};
VirtualKeyboardRasterizer::VirtualKeyboardRasterizer() : impl_(new Impl) {}
VirtualKeyboardRasterizer::~VirtualKeyboardRasterizer() { delete impl_; }
bool VirtualKeyboardRasterizer::Draw(const VirtualKeyboard& keyboard,
std::vector<std::uint8_t>* rgba) {
constexpr int width = VirtualKeyboard::kTextureWidth;
constexpr int height = VirtualKeyboard::kTextureHeight;
rgba->resize(static_cast<std::size_t>(width) * height * 4);
std::uint8_t* pixels = rgba->data();
auto put = [&](int x, int y, Rgb color) {
std::uint8_t* p = pixels + (static_cast<std::size_t>(y) * width + x) * 4;
p[0] = color.r;
p[1] = color.g;
p[2] = color.b;
p[3] = 255;
};
auto blend = [&](int x, int y, Rgb color, std::uint8_t alpha) {
if (x < 0 || y < 0 || x >= width || y >= height || alpha == 0) return;
std::uint8_t* p = pixels + (static_cast<std::size_t>(y) * width + x) * 4;
const int a = alpha;
p[0] = static_cast<std::uint8_t>((color.r * a + p[0] * (255 - a)) / 255);
p[1] = static_cast<std::uint8_t>((color.g * a + p[1] * (255 - a)) / 255);
p[2] = static_cast<std::uint8_t>((color.b * a + p[2] * (255 - a)) / 255);
};
for (int y = 0; y < height; ++y)
for (int x = 0; x < width; ++x) put(x, y, kBackground);
// Text field: the focused TextBox's contents and caret, scrolled so the
// caret stays visible.
{
const int left = kGap / 2;
const int right = width - kGap / 2;
const int top = kGap / 2;
const int bottom = static_cast<int>(std::lround(
VirtualKeyboard::kFieldFraction * height)) -
kGap / 2;
for (int y = top; y < bottom; ++y)
for (int x = left; x < right; ++x)
if (InsideRoundedRect(x, y, left, top, right, bottom, kRadius))
put(x, y, kField);
const int clip_left = left + kFieldPadding;
const int clip_right = right - kFieldPadding;
const int inner = clip_right - clip_left;
const std::string shown = DisplayText(keyboard);
Scrub(&impl_->field.alpha);
impl_->field = AlphaMask{};
int caret_x = 0;
int line_height = static_cast<int>(kLetterPoints * 1.2f);
if (impl_->EnsureFonts()) {
line_height = TTF_GetFontHeight(impl_->letter_font);
if (!shown.empty()) {
impl_->field = Impl::Render(impl_->letter_font, shown);
const std::size_t cursor =
CursorByte(shown, keyboard.text_cursor_utf16());
int unused = 0;
if (cursor > 0)
(void)TTF_GetStringSize(impl_->letter_font, shown.c_str(), cursor,
&caret_x, &unused);
}
}
const int scroll = std::max(0, caret_x - inner + 4);
const int origin_y = (top + bottom - line_height) / 2;
const AlphaMask& text = impl_->field;
for (int y = 0; y < text.height; ++y)
for (int x = 0; x < text.width; ++x) {
const int px = clip_left + x - scroll;
if (px < clip_left || px >= clip_right) continue;
blend(px, origin_y + y, kLabel,
text.alpha[static_cast<std::size_t>(y) * text.width + x]);
}
const int caret = clip_left + caret_x - scroll;
const int caret_top = origin_y + line_height / 8;
const int caret_bottom = origin_y + line_height - line_height / 8;
for (int y = caret_top; y < caret_bottom; ++y)
for (int x = caret; x < caret + 3; ++x)
if (x >= clip_left - 2 && x < clip_right + 2) put(x, y, kAccent);
}
bool labelled = true;
const auto& keys = keyboard.keys();
for (std::size_t index = 0; index < keys.size(); ++index) {
const VirtualKey& key = keys[index];
const int left = static_cast<int>(std::lround(key.u0 * width)) + kGap / 2;
const int right = static_cast<int>(std::lround(key.u1 * width)) - kGap / 2;
const int top = static_cast<int>(std::lround(key.v0 * height)) + kGap / 2;
const int bottom =
static_cast<int>(std::lround(key.v1 * height)) - kGap / 2;
const int i = static_cast<int>(index);
const bool pressed =
keyboard.pressed_key(0) == i || keyboard.pressed_key(1) == i;
const bool hovered =
keyboard.hovered_key(0) == i || keyboard.hovered_key(1) == i;
const bool special = key.action != VirtualKeyAction::kText;
// Latched modifiers stay lit; hover still shows on top of them.
const bool latched =
(key.action == VirtualKeyAction::kShift && keyboard.shifted()) ||
(key.action == VirtualKeyAction::kCapsLock && keyboard.caps_lock());
Rgb fill = special ? kSpecialKey : kKey;
if (latched) fill = kAccent;
if (hovered) fill = latched ? kAccentHover : kHover;
if (pressed) fill = kAccent;
for (int y = top; y < bottom; ++y)
for (int x = left; x < right; ++x)
if (InsideRoundedRect(x, y, left, top, right, bottom, kRadius))
put(x, y, fill);
const AlphaMask* mask = impl_->Mask(key.label, special);
if (!key.label.empty() && (mask == nullptr || mask->alpha.empty())) {
labelled = false;
continue;
}
if (mask == nullptr) continue;
const int origin_x = (left + right - mask->width) / 2;
const int origin_y = (top + bottom - mask->height) / 2;
for (int y = 0; y < mask->height; ++y)
for (int x = 0; x < mask->width; ++x)
blend(origin_x + x, origin_y + y, kLabel,
mask->alpha[static_cast<std::size_t>(y) * mask->width + x]);
}
// Pointer dots, one per hand over the panel.
for (int hand = 0; hand < 2; ++hand) {
float u = 0.f, v = 0.f;
if (!keyboard.cursor(hand, &u, &v)) continue;
const int cx = static_cast<int>(std::lround(u * width));
const int cy = static_cast<int>(std::lround(v * height));
constexpr int kOuter = 11, kInner = 8;
for (int y = -kOuter; y <= kOuter; ++y)
for (int x = -kOuter; x <= kOuter; ++x) {
const int d2 = x * x + y * y;
if (d2 <= kInner * kInner)
blend(cx + x, cy + y, Rgb{255, 255, 255}, 255);
else if (d2 <= kOuter * kOuter)
blend(cx + x, cy + y, Rgb{0, 0, 0}, 200);
}
}
return labelled;
}
} // namespace mocktail::vr
+283
View File
@@ -0,0 +1,283 @@
#include "mocktail/vr/vr_mirror.h"
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>
#include <atomic>
#include <cerrno>
#include <cstdio>
#include <cstring>
namespace mocktail::vr {
namespace {
// Seqlock writers/reader fences: the header is plain shared memory, so
// ordering between the sequence counter and the payload is enforced here.
void StoreFence() { std::atomic_thread_fence(std::memory_order_release); }
std::uint32_t FindHostVisibleMemoryType(const MirrorVkProcs& vk,
VkPhysicalDevice physical,
std::uint32_t type_bits) {
VkPhysicalDeviceMemoryProperties properties{};
vk.GetPhysicalDeviceMemoryProperties(physical, &properties);
const auto required = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
// Prefer HOST_CACHED system memory: CPU reads of device-local BAR1 memory
// cost ~10ms per 2 MB pair on this GPU, while cached RAM readback is ~1ms.
for (std::uint32_t index = 0; index < properties.memoryTypeCount; ++index) {
if ((type_bits & (1u << index)) != 0 &&
(properties.memoryTypes[index].propertyFlags &
(required | VK_MEMORY_PROPERTY_HOST_CACHED_BIT)) ==
(required | VK_MEMORY_PROPERTY_HOST_CACHED_BIT)) {
return index;
}
}
for (std::uint32_t index = 0; index < properties.memoryTypeCount; ++index) {
if ((type_bits & (1u << index)) != 0 &&
(properties.memoryTypes[index].propertyFlags & required) == required) {
return index;
}
}
return UINT32_MAX;
}
} // namespace
VrMirror::~VrMirror() { Close(); }
bool VrMirror::Open(const std::string& shm_name, std::uint32_t width,
std::uint32_t height, std::string* error) {
if (mapping_ != nullptr) {
const auto* header = static_cast<const VrMirrorHeader*>(mapping_);
if (shm_name_ == shm_name && header->width == width && header->height == height) {
return true;
}
Close();
}
if (shm_name.empty() || shm_name[0] != '/' || width == 0 || height == 0) {
if (error != nullptr) {
*error = "invalid mirror segment name or extent";
}
return false;
}
const std::uint64_t bytes = VrMirrorBytes(width, height);
// A crashed run leaves its segment behind; remove our own stale one. In the
// sticky /dev/shm this cannot remove another user's segment, and O_EXCL
// then refuses to adopt it, so eye images never go into a segment someone
// else created or can read.
(void)shm_unlink(shm_name.c_str());
const int descriptor = shm_open(shm_name.c_str(),
O_CREAT | O_EXCL | O_RDWR | O_CLOEXEC, 0600);
if (descriptor < 0) {
if (error != nullptr) {
*error = "shm_open failed: " + std::string(std::strerror(errno)) +
(errno == EEXIST ? " (segment owned by another user)" : "");
}
return false;
}
struct stat status{};
if (fstat(descriptor, &status) != 0 || status.st_uid != getuid() ||
(status.st_mode & 077) != 0) {
if (error != nullptr) {
*error = "mirror segment is not private to this user";
}
close(descriptor);
return false;
}
if (ftruncate(descriptor, static_cast<off_t>(bytes)) != 0) {
if (error != nullptr) {
*error = "ftruncate failed: " + std::string(std::strerror(errno));
}
close(descriptor);
(void)shm_unlink(shm_name.c_str());
return false;
}
void* mapping = mmap(nullptr, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
descriptor, 0);
close(descriptor);
if (mapping == MAP_FAILED) {
if (error != nullptr) {
*error = "mmap failed: " + std::string(std::strerror(errno));
}
(void)shm_unlink(shm_name.c_str());
return false;
}
mapping_ = mapping;
mapping_bytes_ = bytes;
shm_name_ = shm_name;
auto* header = static_cast<VrMirrorHeader*>(mapping_);
std::memset(header, 0, sizeof(*header));
header->magic = kVrMirrorMagic;
header->header_bytes = kVrMirrorHeaderBytes;
header->width = width;
header->height = height;
StoreFence();
return true;
}
void VrMirror::Close() {
if (mapping_ != nullptr) {
munmap(mapping_, mapping_bytes_);
mapping_ = nullptr;
mapping_bytes_ = 0;
}
if (!shm_name_.empty()) {
(void)shm_unlink(shm_name_.c_str());
shm_name_.clear();
}
published_pairs_ = 0;
}
bool VrMirror::EnsureStaging(const MirrorVkProcs& vk, VkDevice device,
VkPhysicalDevice physical, std::uint32_t width,
std::uint32_t height, std::string* error) {
if (staging_device_ == device && staging_[0] != VK_NULL_HANDLE &&
staging_[1] != VK_NULL_HANDLE && staging_width_ == width &&
staging_height_ == height) {
return true;
}
DestroyStaging(vk);
staging_device_ = device; // Also owns partially constructed allocations.
const VkDeviceSize size =
static_cast<VkDeviceSize>(width) * height * 4u;
VkPhysicalDeviceMemoryProperties properties{};
vk.GetPhysicalDeviceMemoryProperties(physical, &properties);
for (int eye = 0; eye < 2; ++eye) {
VkBufferCreateInfo buffer_info{};
buffer_info.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
buffer_info.size = size;
buffer_info.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
if (vk.CreateBuffer(device, &buffer_info, nullptr, &staging_[eye]) !=
VK_SUCCESS) {
if (error != nullptr) {
*error = "mirror staging buffer creation failed";
}
DestroyStaging(vk);
return false;
}
VkMemoryRequirements requirements{};
vk.GetBufferMemoryRequirements(device, staging_[eye], &requirements);
const std::uint32_t type =
FindHostVisibleMemoryType(vk, physical, requirements.memoryTypeBits);
if (type == UINT32_MAX) {
if (error != nullptr) {
*error = "no host-visible coherent memory for the mirror";
}
DestroyStaging(vk);
return false;
}
VkMemoryAllocateInfo allocation{};
allocation.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
allocation.allocationSize = requirements.size;
allocation.memoryTypeIndex = type;
if (vk.AllocateMemory(device, &allocation, nullptr,
&staging_memory_[eye]) != VK_SUCCESS) {
if (error != nullptr) {
*error = "mirror staging allocation failed";
}
DestroyStaging(vk);
return false;
}
if (vk.BindBufferMemory(device, staging_[eye], staging_memory_[eye], 0) !=
VK_SUCCESS ||
vk.MapMemory(device, staging_memory_[eye], 0, size, 0,
&staging_map_[eye]) != VK_SUCCESS) {
if (error != nullptr) {
*error = "mirror staging bind/map failed";
}
DestroyStaging(vk);
return false;
}
}
staging_device_ = device;
staging_width_ = width;
staging_height_ = height;
return true;
}
void VrMirror::DestroyStaging(const MirrorVkProcs& vk) {
for (int eye = 0; eye < 2; ++eye) {
if (staging_map_[eye] != nullptr && staging_memory_[eye] != VK_NULL_HANDLE) {
vk.UnmapMemory(staging_device_, staging_memory_[eye]);
staging_map_[eye] = nullptr;
}
if (staging_memory_[eye] != VK_NULL_HANDLE) {
vk.FreeMemory(staging_device_, staging_memory_[eye], nullptr);
staging_memory_[eye] = VK_NULL_HANDLE;
}
if (staging_[eye] != VK_NULL_HANDLE) {
vk.DestroyBuffer(staging_device_, staging_[eye], nullptr);
staging_[eye] = VK_NULL_HANDLE;
}
}
staging_width_ = 0;
staging_height_ = 0;
staging_device_ = VK_NULL_HANDLE;
}
void VrMirror::RecordEyeCopy(const MirrorVkProcs& vk, VkCommandBuffer cmd,
int eye, VkImage source,
VkImageLayout /*final_layout*/,
std::uint32_t width, std::uint32_t height) {
// The caller's batched command buffer already holds the source in
// TRANSFER_SRC_OPTIMAL for the XR swapchain copy and restores its final
// layout afterwards; recording only the copy here keeps one transition pair
// per eye per frame.
if (eye < 0 || eye > 1 || staging_[eye] == VK_NULL_HANDLE ||
staging_width_ != width || staging_height_ != height) {
return;
}
VkBufferImageCopy region{};
region.imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1};
region.imageExtent = {width, height, 1};
vk.CmdCopyImageToBuffer(cmd, source, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
staging_[eye], 1, &region);
}
void VrMirror::Publish(std::uint64_t frame, VkFormat format,
std::uint32_t width, std::uint32_t height,
std::uint64_t now_ns) {
if (mapping_ == nullptr || staging_map_[0] == nullptr ||
staging_map_[1] == nullptr || staging_width_ != width ||
staging_height_ != height) {
return;
}
PublishFromBytesForTest(frame, format, width, height, now_ns, staging_map_[0],
staging_map_[1]);
}
void VrMirror::PublishFromBytesForTest(std::uint64_t frame, VkFormat format,
std::uint32_t width,
std::uint32_t height,
std::uint64_t now_ns, const void* eye0,
const void* eye1) {
if (mapping_ == nullptr || eye0 == nullptr || eye1 == nullptr) {
return;
}
auto* header = static_cast<VrMirrorHeader*>(mapping_);
auto* pixels = static_cast<std::uint8_t*>(mapping_) + kVrMirrorHeaderBytes;
const std::uint64_t eye_bytes =
static_cast<std::uint64_t>(width) * height * 4u;
if (kVrMirrorHeaderBytes + 2 * eye_bytes > mapping_bytes_) {
return;
}
// Seqlock: odd sequence marks the payload unstable for readers.
const std::uint64_t sequence = header->seq + 1;
header->seq = sequence | 1ull;
StoreFence();
header->width = width;
header->height = height;
header->format = static_cast<std::uint32_t>(format);
header->frame = frame;
header->producer_ns = now_ns;
std::memcpy(pixels, eye0, eye_bytes);
std::memcpy(pixels + eye_bytes, eye1, eye_bytes);
header->flags = kVrMirrorFlagValid;
header->published_pairs = ++published_pairs_;
StoreFence();
header->seq = sequence + 1;
}
} // namespace mocktail::vr
+225
View File
@@ -0,0 +1,225 @@
#include "mocktail/vr/vr_perf.h"
#include <time.h>
#include <algorithm>
#include <cstdio>
namespace mocktail::vr::perf {
namespace {
Collector& Global() {
static Collector collector;
return collector;
}
} // namespace
bool Enabled() {
static const bool enabled = [] {
const char* value = std::getenv("MOCKTAIL_VR_PERF");
return value != nullptr && value[0] != '\0' && *value != '0';
}();
if (!enabled) return false;
static const std::uint64_t start = [] {
const char* value = std::getenv("MOCKTAIL_VR_MEASURE_START_NS");
return value ? std::strtoull(value, nullptr, 10) : 0ULL;
}();
static const std::uint64_t end = [] {
const char* value = std::getenv("MOCKTAIL_VR_MEASURE_END_NS");
return value ? std::strtoull(value, nullptr, 10) : ~0ULL;
}();
if (start == 0 && end == ~0ULL) return true;
const auto now = NowNs();
return now >= start && now <= end;
}
std::uint64_t NowNs() {
timespec ts{};
if (clock_gettime(CLOCK_MONOTONIC, &ts) != 0) {
return 0;
}
return static_cast<std::uint64_t>(ts.tv_sec) * 1'000'000'000ULL +
static_cast<std::uint64_t>(ts.tv_nsec);
}
const char* StageName(Stage stage) {
switch (stage) {
case Stage::kFrameCycle: return "frame_cycle";
case Stage::kXrWaitFrame: return "xr_wait_frame";
case Stage::kXrBeginFrame: return "xr_begin_frame";
case Stage::kXrLocateViews: return "xr_locate_views";
case Stage::kSwapchainAcquire: return "swapchain_acquire";
case Stage::kSwapchainWait: return "swapchain_wait";
case Stage::kCopyRecord: return "copy_record";
case Stage::kCopySubmit: return "copy_submit";
case Stage::kCopyFenceWait: return "copy_fence_wait";
case Stage::kMirrorPublish: return "mirror_publish";
case Stage::kEvidence: return "evidence";
case Stage::kControllerSync: return "controller_sync";
case Stage::kDesktopMirror: return "desktop_mirror";
case Stage::kPresentPrepare: return "present_prepare";
case Stage::kPresentInterval: return "present_interval";
case Stage::kStageCount: break;
}
return "unknown";
}
const char* CounterName(Counter counter) {
switch (counter) {
case Counter::kPresents: return "presents";
case Counter::kFramesBegun: return "frames_begun";
case Counter::kFramesSubmitted: return "frames_submitted";
case Counter::kFramesSkipped: return "frames_skipped";
case Counter::kSkipShouldRender: return "skip_should_render";
case Counter::kSkipPoseInvalid: return "skip_pose_invalid";
case Counter::kSkipEyesUnbound: return "skip_eyes_unbound";
case Counter::kSkipEyeFrameMismatch: return "skip_eye_frame_mismatch";
case Counter::kSubmits: return "submits";
case Counter::kFenceWaits: return "fence_waits";
case Counter::kMirrorPairs: return "mirror_pairs";
case Counter::kMirrorDrops: return "mirror_drops";
case Counter::kSkipVisibilityLost: return "skip_visibility_lost";
case Counter::kReferenceSpaceChanges: return "reference_space_changes";
case Counter::kVisibilityLost: return "visibility_lost";
case Counter::kVisibilityGained: return "visibility_gained";
case Counter::kFramesInvalidated: return "frames_invalidated";
case Counter::kControllerSyncs: return "controller_syncs";
case Counter::kCounterCount: break;
}
return "unknown";
}
void Collector::Record(Stage stage, std::uint64_t nanos) {
if (!Enabled()) {
return;
}
const std::size_t index = static_cast<std::size_t>(stage);
if (index >= static_cast<std::size_t>(Stage::kStageCount)) {
return;
}
StageStat& stat = stages_[index];
if (stat.samples.size() < kReservoir) {
stat.samples.resize(kReservoir, 0);
}
if (stat.count == kReservoir) {
stat.total_ns -= stat.samples[stat.head];
}
stat.samples[stat.head] = nanos;
stat.head = (stat.head + 1) % kReservoir;
if (stat.count < kReservoir) {
++stat.count;
}
stat.total_ns += nanos;
}
void Collector::Count(Counter counter, std::uint64_t delta) {
if (!Enabled()) {
return;
}
const std::size_t index = static_cast<std::size_t>(counter);
if (index >= static_cast<std::size_t>(Counter::kCounterCount)) {
return;
}
counters_[index] += delta;
}
std::vector<std::uint64_t> Collector::Percentiles(Stage stage) const {
const std::size_t index = static_cast<std::size_t>(stage);
if (index >= static_cast<std::size_t>(Stage::kStageCount)) {
return {};
}
const StageStat& stat = stages_[index];
if (stat.count == 0) {
return {0, 0, 0, 0};
}
std::vector<std::uint64_t> sorted(stat.samples.begin(),
stat.samples.begin() + stat.count);
std::sort(sorted.begin(), sorted.end());
auto pick = [&sorted](double fraction) {
const std::size_t position = static_cast<std::size_t>(
fraction * static_cast<double>(sorted.size() - 1));
return sorted[position];
};
return {pick(0.50), pick(0.95), pick(0.99), sorted.back()};
}
std::size_t Collector::Samples(Stage stage) const {
const std::size_t index = static_cast<std::size_t>(stage);
if (index >= static_cast<std::size_t>(Stage::kStageCount)) {
return 0;
}
return stages_[index].count;
}
std::uint64_t Collector::counter(Counter counter_id) const {
const std::size_t index = static_cast<std::size_t>(counter_id);
if (index >= static_cast<std::size_t>(Counter::kCounterCount)) {
return 0;
}
return counters_[index];
}
void Collector::Reset() {
for (StageStat& stat : stages_) {
stat.head = 0;
stat.count = 0;
stat.total_ns = 0;
}
for (std::uint64_t& value : counters_) {
value = 0;
}
last_emit_ns_ = 0;
window_start_ns_ = 0;
}
void Collector::EmitIfDue(std::uint64_t now_ns, const char* context) {
if (!Enabled() || now_ns == 0) {
return;
}
if (window_start_ns_ == 0) {
window_start_ns_ = now_ns;
last_emit_ns_ = now_ns;
return;
}
if (now_ns - last_emit_ns_ < kIntervalNs) {
return;
}
const double window_seconds =
static_cast<double>(now_ns - window_start_ns_) / 1e9;
std::fprintf(stderr, " [vr-perf] window=%.1fs context=%s\n",
window_seconds, context != nullptr ? context : "");
for (std::size_t index = 0;
index < static_cast<std::size_t>(Stage::kStageCount); ++index) {
const StageStat& stat = stages_[index];
if (stat.count == 0) {
continue;
}
const std::vector<std::uint64_t> ranked =
Percentiles(static_cast<Stage>(index));
std::fprintf(stderr,
" [vr-perf] stage=%-16s n=%zu avg=%.2fms p50=%.2fms "
"p95=%.2fms p99=%.2fms max=%.2fms\n",
StageName(static_cast<Stage>(index)), stat.count,
static_cast<double>(stat.total_ns) /
static_cast<double>(stat.count) / 1e6,
ranked[0] / 1e6, ranked[1] / 1e6, ranked[2] / 1e6,
ranked[3] / 1e6);
}
for (std::size_t index = 0;
index < static_cast<std::size_t>(Counter::kCounterCount); ++index) {
const std::uint64_t value = counters_[index];
if (value == 0) {
continue;
}
std::fprintf(stderr, " [vr-perf] counter=%-22s total=%llu rate=%.2f/s\n",
CounterName(static_cast<Counter>(index)),
static_cast<unsigned long long>(value),
static_cast<double>(value) / std::max(window_seconds, 1e-9));
}
last_emit_ns_ = now_ns;
}
Collector& ProcessCollector() { return Global(); }
} // namespace mocktail::vr::perf
File diff suppressed because it is too large. Load diff
+937
View File
@@ -0,0 +1,937 @@
#include "mocktail/vr/xr_actions.h"
#include <openxr/openxr.h>
#include <algorithm>
#include <array>
#include <atomic>
#include <cinttypes>
#include <cmath>
#include <cstdarg>
#include <cstdio>
#include <cstring>
#include <stdexcept>
#include <utility>
#include <vector>
namespace mocktail::vr {
namespace {
void Log(const char *format, ...) {
va_list arguments;
va_start(arguments, format);
std::vfprintf(stderr, format, arguments);
va_end(arguments);
std::fflush(stderr);
}
template <typename T> T XrInfo(XrStructureType type) {
T value{};
value.type = type;
return value;
}
std::string XrResultText(XrInstance instance, XrResult result) {
char text[XR_MAX_RESULT_STRING_SIZE]{};
if (instance != XR_NULL_HANDLE) {
(void)xrResultToString(instance, result, text);
}
if (!text[0]) {
return std::to_string(static_cast<int>(result));
}
return text;
}
void CheckXr(XrInstance instance, XrResult result, const char *operation) {
if (XR_SUCCEEDED(result)) {
return;
}
throw std::runtime_error(std::string(operation) +
" failed: " + XrResultText(instance, result));
}
// Registry-verified interaction profiles
// (docs/vr-agent-evidence/controller-2998/openxr-interaction-profiles.md).
constexpr const char *kProfileTouch =
"/interaction_profiles/oculus/touch_controller";
constexpr const char *kProfileQuest2 =
"/interaction_profiles/meta/touch_controller_quest_2";
constexpr const char* kTouchProfiles[] = {
kProfileTouch,
kProfileQuest2,
"/interaction_profiles/facebook/touch_controller_pro",
"/interaction_profiles/meta/touch_controller_plus",
"/interaction_profiles/meta/touch_pro_controller",
"/interaction_profiles/meta/touch_plus_controller",
};
constexpr const char *kProfileIndex =
"/interaction_profiles/valve/index_controller";
constexpr const char *kProfileVive =
"/interaction_profiles/htc/vive_controller";
constexpr const char *kProfileSimple =
"/interaction_profiles/khr/simple_controller";
} // namespace
// All XR objects of one XrActions instance.
struct XrActions::Impl {
XrInstance instance = XR_NULL_HANDLE;
XrSession session = XR_NULL_HANDLE;
XrSpace local_space = XR_NULL_HANDLE; // borrowed from the backend
XrActionSet set = XR_NULL_HANDLE;
XrAction grip[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
XrAction aim[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
XrSpace grip_space[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
XrSpace aim_space[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
XrAction touches[static_cast<std::size_t>(ControllerTouch::kCount)] = {};
XrAction trigger = XR_NULL_HANDLE;
XrAction squeeze = XR_NULL_HANDLE;
XrAction stick = XR_NULL_HANDLE;
XrAction trigger_click = XR_NULL_HANDLE;
XrAction squeeze_click = XR_NULL_HANDLE;
XrAction stick_click = XR_NULL_HANDLE;
XrAction face_a = XR_NULL_HANDLE;
XrAction face_b = XR_NULL_HANDLE;
XrAction face_x = XR_NULL_HANDLE;
XrAction face_y = XR_NULL_HANDLE;
XrAction menu = XR_NULL_HANDLE;
XrAction haptic = XR_NULL_HANDLE;
XrPath subaction_left = XR_NULL_PATH;
XrPath subaction_right = XR_NULL_PATH;
std::mutex haptic_mutex;
struct HapticRequest {
bool pending = false;
bool stop = false;
float amplitude = 0.f;
std::uint64_t duration_ns = 0;
float frequency_hz = 0.f;
};
HapticRequest haptics[2];
XrPath active_profile[2] = {XR_NULL_PATH, XR_NULL_PATH};
std::string active_profile_name[2];
// Trigger-click actions that reported active this frame, per hand: the
// analog-derived click must not override a native click action.
bool native_trigger_click[2] = {false, false};
};
XrActions::~XrActions() { Destroy(); }
Status XrActions::Create(void *xr_instance) {
if (created_) {
return Status::Ok();
}
if (xr_instance == nullptr) {
return Status::Error(StatusCode::kInvalidArgument,
"XR actions require an instance");
}
const auto instance = static_cast<XrInstance>(xr_instance);
delete impl_;
impl_ = new Impl();
impl_->instance = instance;
try {
auto set_info =
XrInfo<XrActionSetCreateInfo>(XR_TYPE_ACTION_SET_CREATE_INFO);
std::strncpy(set_info.actionSetName, "mocktail_vr",
XR_MAX_ACTION_SET_NAME_SIZE - 1);
std::strncpy(set_info.localizedActionSetName, "Mocktail VR controls",
XR_MAX_LOCALIZED_ACTION_SET_NAME_SIZE - 1);
set_info.priority = 0;
CheckXr(instance, xrCreateActionSet(instance, &set_info, &impl_->set),
"xrCreateActionSet");
CheckXr(instance,
xrStringToPath(instance, "/user/hand/left", &impl_->subaction_left),
"xrStringToPath left");
CheckXr(
instance,
xrStringToPath(instance, "/user/hand/right", &impl_->subaction_right),
"xrStringToPath right");
const std::array<XrPath, 2> subactions{impl_->subaction_left,
impl_->subaction_right};
auto make = [&](XrActionType type, const char *name) {
auto info = XrInfo<XrActionCreateInfo>(XR_TYPE_ACTION_CREATE_INFO);
info.actionType = type;
info.countSubactionPaths = 2;
info.subactionPaths = subactions.data();
std::strncpy(info.actionName, name, XR_MAX_ACTION_NAME_SIZE - 1);
// A non-empty localized name is mandatory; runtimes reject an empty one
// with XR_ERROR_LOCALIZED_NAME_INVALID.
std::strncpy(info.localizedActionName, name,
XR_MAX_LOCALIZED_ACTION_NAME_SIZE - 1);
XrAction action = XR_NULL_HANDLE;
CheckXr(instance, xrCreateAction(impl_->set, &info, &action), name);
return action;
};
impl_->grip[0] = make(XR_ACTION_TYPE_POSE_INPUT, "grip_left");
impl_->grip[1] = make(XR_ACTION_TYPE_POSE_INPUT, "grip_right");
impl_->aim[0] = make(XR_ACTION_TYPE_POSE_INPUT, "aim_left");
impl_->aim[1] = make(XR_ACTION_TYPE_POSE_INPUT, "aim_right");
impl_->trigger = make(XR_ACTION_TYPE_FLOAT_INPUT, "trigger");
impl_->squeeze = make(XR_ACTION_TYPE_FLOAT_INPUT, "squeeze");
impl_->stick = make(XR_ACTION_TYPE_VECTOR2F_INPUT, "thumbstick");
impl_->trigger_click = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "trigger_click");
impl_->squeeze_click = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "squeeze_click");
impl_->stick_click = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "thumbstick_click");
impl_->face_a = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "face_a");
impl_->face_b = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "face_b");
impl_->face_x = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "face_x");
impl_->face_y = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "face_y");
impl_->menu = make(XR_ACTION_TYPE_BOOLEAN_INPUT, "menu");
const char *touch_names[] = {
"trigger_touch", "thumbstick_touch", "thumbrest_touch", "a_touch",
"b_touch", "x_touch", "y_touch"};
for (std::size_t i = 0; i < std::size(touch_names); ++i)
impl_->touches[i] = make(XR_ACTION_TYPE_BOOLEAN_INPUT, touch_names[i]);
impl_->haptic = make(XR_ACTION_TYPE_VIBRATION_OUTPUT, "haptic");
auto path = [&](const char *text) {
XrPath result = XR_NULL_PATH;
CheckXr(instance, xrStringToPath(instance, text, &result), text);
return result;
};
// Each profile is suggested independently: a runtime that does not know an
// optional profile rejects only that suggestion, never the primary Touch
// binding. The system-reserved button is not bound on any profile.
auto suggest = [&](const char *profile,
std::vector<std::pair<XrAction, XrPath>> binds) {
auto touch = [&](ControllerTouch kind, const char *component) {
binds.emplace_back(impl_->touches[static_cast<std::size_t>(kind)],
path(component));
};
const bool is_touch = std::any_of(
std::begin(kTouchProfiles), std::end(kTouchProfiles),
[&](const char* name) { return std::strcmp(profile, name) == 0; });
const bool is_index = std::strcmp(profile, kProfileIndex) == 0;
if (is_touch || is_index) {
touch(ControllerTouch::kTrigger, "/user/hand/left/input/trigger/touch");
touch(ControllerTouch::kTrigger,
"/user/hand/right/input/trigger/touch");
touch(ControllerTouch::kThumbstick,
"/user/hand/left/input/thumbstick/touch");
touch(ControllerTouch::kThumbstick,
"/user/hand/right/input/thumbstick/touch");
touch(ControllerTouch::kA, "/user/hand/right/input/a/touch");
touch(ControllerTouch::kB, "/user/hand/right/input/b/touch");
touch(ControllerTouch::kX, is_touch ? "/user/hand/left/input/x/touch"
: "/user/hand/left/input/a/touch");
touch(ControllerTouch::kY, is_touch ? "/user/hand/left/input/y/touch"
: "/user/hand/left/input/b/touch");
}
if (is_touch && std::strcmp(profile, kProfileQuest2) != 0) {
touch(ControllerTouch::kThumbrest,
"/user/hand/left/input/thumbrest/touch");
touch(ControllerTouch::kThumbrest,
"/user/hand/right/input/thumbrest/touch");
}
std::vector<XrActionSuggestedBinding> suggested;
suggested.reserve(binds.size());
for (const auto &bind : binds) {
XrActionSuggestedBinding entry{};
entry.action = bind.first;
entry.binding = bind.second;
suggested.push_back(entry);
}
auto info = XrInfo<XrInteractionProfileSuggestedBinding>(
XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING);
info.interactionProfile = path(profile);
info.countSuggestedBindings =
static_cast<std::uint32_t>(suggested.size());
info.suggestedBindings = suggested.data();
const XrResult result =
xrSuggestInteractionProfileBindings(instance, &info);
if (XR_FAILED(result)) {
Log(" [vr-input] bindings for %s not accepted (%s)\n", profile,
XrResultText(instance, result).c_str());
}
};
const XrPath l_grip = path("/user/hand/left/input/grip/pose");
const XrPath r_grip = path("/user/hand/right/input/grip/pose");
const XrPath l_aim = path("/user/hand/left/input/aim/pose");
const XrPath r_aim = path("/user/hand/right/input/aim/pose");
const XrPath l_haptic = path("/user/hand/left/output/haptic");
const XrPath r_haptic = path("/user/hand/right/output/haptic");
for (const char* profile : kTouchProfiles) {
suggest(
profile,
{
{impl_->grip[0], l_grip},
{impl_->grip[1], r_grip},
{impl_->aim[0], l_aim},
{impl_->aim[1], r_aim},
{impl_->trigger, path("/user/hand/left/input/trigger/value")},
{impl_->trigger, path("/user/hand/right/input/trigger/value")},
{impl_->squeeze, path("/user/hand/left/input/squeeze/value")},
{impl_->squeeze, path("/user/hand/right/input/squeeze/value")},
{impl_->stick, path("/user/hand/left/input/thumbstick")},
{impl_->stick, path("/user/hand/right/input/thumbstick")},
{impl_->stick_click,
path("/user/hand/left/input/thumbstick/click")},
{impl_->stick_click,
path("/user/hand/right/input/thumbstick/click")},
{impl_->face_x, path("/user/hand/left/input/x/click")},
{impl_->face_y, path("/user/hand/left/input/y/click")},
{impl_->face_a, path("/user/hand/right/input/a/click")},
{impl_->face_b, path("/user/hand/right/input/b/click")},
{impl_->menu, path("/user/hand/left/input/menu/click")},
{impl_->haptic, l_haptic},
{impl_->haptic, r_haptic},
});
}
suggest(
kProfileIndex,
{
{impl_->grip[0], l_grip},
{impl_->grip[1], r_grip},
{impl_->aim[0], l_aim},
{impl_->aim[1], r_aim},
{impl_->trigger, path("/user/hand/left/input/trigger/value")},
{impl_->trigger, path("/user/hand/right/input/trigger/value")},
{impl_->trigger_click, path("/user/hand/left/input/trigger/click")},
{impl_->trigger_click,
path("/user/hand/right/input/trigger/click")},
{impl_->squeeze, path("/user/hand/left/input/squeeze/force")},
{impl_->squeeze, path("/user/hand/right/input/squeeze/force")},
{impl_->stick, path("/user/hand/left/input/thumbstick")},
{impl_->stick, path("/user/hand/right/input/thumbstick")},
{impl_->stick_click,
path("/user/hand/left/input/thumbstick/click")},
{impl_->stick_click,
path("/user/hand/right/input/thumbstick/click")},
{impl_->face_x, path("/user/hand/left/input/a/click")},
{impl_->face_a, path("/user/hand/right/input/a/click")},
{impl_->face_y, path("/user/hand/left/input/b/click")},
{impl_->face_b, path("/user/hand/right/input/b/click")},
{impl_->haptic, l_haptic},
{impl_->haptic, r_haptic},
});
suggest(
kProfileVive,
{
{impl_->squeeze_click, path("/user/hand/left/input/squeeze/click")},
{impl_->squeeze_click,
path("/user/hand/right/input/squeeze/click")},
{impl_->grip[0], l_grip},
{impl_->grip[1], r_grip},
{impl_->aim[0], l_aim},
{impl_->aim[1], r_aim},
{impl_->trigger, path("/user/hand/left/input/trigger/value")},
{impl_->trigger, path("/user/hand/right/input/trigger/value")},
{impl_->stick, path("/user/hand/left/input/trackpad")},
{impl_->stick, path("/user/hand/right/input/trackpad")},
{impl_->stick_click, path("/user/hand/left/input/trackpad/click")},
{impl_->stick_click, path("/user/hand/right/input/trackpad/click")},
{impl_->menu, path("/user/hand/left/input/menu/click")},
{impl_->menu, path("/user/hand/right/input/menu/click")},
{impl_->haptic, l_haptic},
{impl_->haptic, r_haptic},
});
suggest(
kProfileSimple,
{
{impl_->grip[0], l_grip},
{impl_->grip[1], r_grip},
{impl_->aim[0], l_aim},
{impl_->aim[1], r_aim},
{impl_->trigger_click, path("/user/hand/left/input/select/click")},
{impl_->trigger_click, path("/user/hand/right/input/select/click")},
{impl_->menu, path("/user/hand/left/input/menu/click")},
{impl_->menu, path("/user/hand/right/input/menu/click")},
{impl_->haptic, l_haptic},
{impl_->haptic, r_haptic},
});
created_ = true;
Log(" [vr-input] action set created: grip+aim poses, trigger/squeeze "
"analog, thumbstick, face/menu buttons, haptic output; bindings "
"suggested for Touch/Quest2/Touch Pro/Touch Plus/Index/Vive/Simple "
"(system button never "
"bound)\n");
return Status::Ok();
} catch (const std::exception &error) {
Destroy();
return Status::Error(StatusCode::kUnavailable, error.what());
}
}
Status XrActions::Attach(void *xr_instance, void *xr_session,
void *xr_local_space) {
if (!created_ || impl_ == nullptr) {
return Status::Error(StatusCode::kFailedPrecondition,
"XR actions are not created");
}
if (xr_session == nullptr || xr_local_space == nullptr) {
return Status::Error(StatusCode::kInvalidArgument,
"XR actions attach requires a session and space");
}
const auto instance = static_cast<XrInstance>(xr_instance);
const auto session = static_cast<XrSession>(xr_session);
try {
auto attach_info = XrInfo<XrSessionActionSetsAttachInfo>(
XR_TYPE_SESSION_ACTION_SETS_ATTACH_INFO);
attach_info.countActionSets = 1;
attach_info.actionSets = &impl_->set;
CheckXr(instance, xrAttachSessionActionSets(session, &attach_info),
"xrAttachSessionActionSets");
for (int hand = 0; hand < 2; ++hand) {
const XrPath subaction =
hand == 0 ? impl_->subaction_left : impl_->subaction_right;
auto space_info =
XrInfo<XrActionSpaceCreateInfo>(XR_TYPE_ACTION_SPACE_CREATE_INFO);
space_info.subactionPath = subaction;
space_info.poseInActionSpace.orientation.w = 1.f;
space_info.action = impl_->grip[hand];
CheckXr(
instance,
xrCreateActionSpace(session, &space_info, &impl_->grip_space[hand]),
"xrCreateActionSpace grip");
space_info.action = impl_->aim[hand];
CheckXr(
instance,
xrCreateActionSpace(session, &space_info, &impl_->aim_space[hand]),
"xrCreateActionSpace aim");
}
impl_->session = session;
impl_->local_space = static_cast<XrSpace>(xr_local_space);
attached_ = true;
session_generation_++;
Log(" [vr-input] actions attached to session (generation %llu)\n",
static_cast<unsigned long long>(session_generation_));
return Status::Ok();
} catch (const std::exception &error) {
Detach();
return Status::Error(StatusCode::kUnavailable, error.what());
}
}
void XrActions::ResetInput() {
const bool had_input =
snapshot_.hands[0].connected || snapshot_.hands[1].connected;
const auto frame = snapshot_.frame;
builder_ = ControllerStateBuilder{};
snapshot_ = {};
hands_[0] = {};
hands_[1] = {};
if (had_input) {
delivery_queue_.Clear();
ControllerDelivery reset;
reset.frame = frame;
reset.session_generation = session_generation_;
reset.resync = true;
delivery_queue_.Push(reset);
}
last_delivered_connected_[0] = last_delivered_connected_[1] = false;
if (impl_) {
std::lock_guard<std::mutex> lock(impl_->haptic_mutex);
for (int hand = 0; hand < 2; ++hand) {
impl_->haptics[hand] = {};
if (attached_ && impl_->session) {
auto stop = XrInfo<XrHapticActionInfo>(XR_TYPE_HAPTIC_ACTION_INFO);
stop.action = impl_->haptic;
stop.subactionPath =
hand ? impl_->subaction_right : impl_->subaction_left;
(void)xrStopHapticFeedback(impl_->session, &stop);
}
}
}
}
void XrActions::Detach() {
ResetInput();
if (impl_ == nullptr) {
attached_ = false;
return;
}
// Haptics must not outlive the session.
if (attached_ && impl_->session != XR_NULL_HANDLE) {
for (int hand = 0; hand < 2; ++hand) {
auto stop = XrInfo<XrHapticActionInfo>(XR_TYPE_HAPTIC_ACTION_INFO);
stop.action = impl_->haptic;
stop.subactionPath =
hand == 0 ? impl_->subaction_left : impl_->subaction_right;
(void)xrStopHapticFeedback(impl_->session, &stop);
}
}
for (int hand = 0; hand < 2; ++hand) {
if (impl_->grip_space[hand] != XR_NULL_HANDLE) {
(void)xrDestroySpace(impl_->grip_space[hand]);
impl_->grip_space[hand] = XR_NULL_HANDLE;
}
if (impl_->aim_space[hand] != XR_NULL_HANDLE) {
(void)xrDestroySpace(impl_->aim_space[hand]);
impl_->aim_space[hand] = XR_NULL_HANDLE;
}
impl_->active_profile[hand] = XR_NULL_PATH;
impl_->active_profile_name[hand].clear();
impl_->native_trigger_click[hand] = false;
}
impl_->session = XR_NULL_HANDLE;
impl_->local_space = XR_NULL_HANDLE;
attached_ = false;
hands_[0] = VrHandPose{};
hands_[1] = VrHandPose{};
// Queued deliveries belong to the detached session; a replacement session
// must start from a clean slate with fresh connection levels.
// Preserve queued reset until the input thread drains it.
last_delivered_connected_[0] = false;
last_delivered_connected_[1] = false;
}
void XrActions::Destroy() {
Detach();
if (impl_ != nullptr) {
const auto instance = impl_->instance;
if (instance != XR_NULL_HANDLE) {
const std::array<XrAction, 16> actions{
impl_->grip[0], impl_->grip[1], impl_->aim[0],
impl_->aim[1], impl_->trigger, impl_->squeeze,
impl_->stick, impl_->trigger_click, impl_->squeeze_click,
impl_->stick_click, impl_->face_a, impl_->face_b,
impl_->face_x, impl_->face_y, impl_->menu,
impl_->haptic};
for (const XrAction action : actions) {
if (action != XR_NULL_HANDLE) {
(void)xrDestroyAction(action);
}
}
for (const auto action : impl_->touches)
if (action != XR_NULL_HANDLE)
(void)xrDestroyAction(action);
if (impl_->set != XR_NULL_HANDLE) {
(void)xrDestroyActionSet(impl_->set);
}
}
delete impl_;
impl_ = nullptr;
}
created_ = false;
attached_ = false;
snapshot_ = ControllerSnapshot{};
hands_[0] = VrHandPose{};
hands_[1] = VrHandPose{};
// Preserve queued reset until the input thread drains it.
last_delivered_connected_[0] = false;
last_delivered_connected_[1] = false;
session_generation_ = 0;
}
void XrActions::RequestHaptics(int hand, float amplitude,
std::uint64_t duration_ns, float frequency_hz) {
if (impl_ == nullptr || !attached_ || (hand != 0 && hand != 1)) {
return;
}
std::lock_guard<std::mutex> lock(impl_->haptic_mutex);
auto &request = impl_->haptics[hand];
request = {};
request.pending = true;
request.stop = false;
request.amplitude =
std::clamp(std::isfinite(amplitude) ? amplitude : 0.f, 0.f, 1.f);
request.duration_ns = duration_ns;
request.frequency_hz =
std::isfinite(frequency_hz) ? std::max(0.f, frequency_hz) : 0.f;
}
void XrActions::StopHaptics(int hand) {
if (impl_ == nullptr || !attached_ || (hand != 0 && hand != 1)) {
return;
}
std::lock_guard<std::mutex> lock(impl_->haptic_mutex);
impl_->haptics[hand] = Impl::HapticRequest{};
impl_->haptics[hand].pending = true;
impl_->haptics[hand].stop = true;
}
void XrActions::NoteProfileChanged() {
if (impl_ == nullptr) {
return;
}
for (int hand = 0; hand < 2; ++hand) {
impl_->active_profile[hand] = XR_NULL_PATH;
impl_->active_profile_name[hand].clear();
}
}
std::string XrActions::ActiveProfile(int hand) const {
if (impl_ == nullptr || (hand != 0 && hand != 1)) {
return {};
}
return impl_->active_profile_name[hand];
}
bool XrActions::TakeDelivery(ControllerDelivery *out) {
return delivery_queue_.Pop(out);
}
bool XrActions::Sync(void *xr_session, std::uint64_t predicted_display_time,
std::uint64_t frame, std::uint64_t /*generation*/) {
if (!attached_ || impl_ == nullptr) {
return false;
}
const auto session = static_cast<XrSession>(xr_session);
const auto instance = impl_->instance;
const auto time = static_cast<XrTime>(predicted_display_time);
try {
auto sync_info = XrInfo<XrActionsSyncInfo>(XR_TYPE_ACTIONS_SYNC_INFO);
const XrActiveActionSet active{impl_->set, XR_NULL_PATH};
sync_info.countActiveActionSets = 1;
sync_info.activeActionSets = &active;
const XrResult sync_result = xrSyncActions(session, &sync_info);
if (sync_result == XR_SESSION_NOT_FOCUSED) {
// Normal while the system compositor owns input: no fresh state exists
// this frame and nothing may be delivered.
ResetInput();
return false;
}
if (XR_FAILED(sync_result)) {
Log(" [vr-input] xrSyncActions failed: %s\n",
XrResultText(instance, sync_result).c_str());
ResetInput();
return false;
}
for (int hand = 0; hand < 2; ++hand) {
const XrPath subaction =
hand == 0 ? impl_->subaction_left : impl_->subaction_right;
// Inactive actions must neutralize previous state, even if a profile
// remains assigned after the device sleeps or the runtime changes
// bindings.
for (size_t i = 0; i < static_cast<size_t>(ControllerButton::kCount); ++i)
builder_.SetButton(hand, static_cast<ControllerButton>(i), false, true);
for (size_t i = 0; i < static_cast<size_t>(ControllerAxis::kCount); ++i)
builder_.SetAxis(hand, static_cast<ControllerAxis>(i), 0.f, true);
for (std::size_t i = 0;
i < static_cast<std::size_t>(ControllerTouch::kCount); ++i)
builder_.SetTouch(hand, static_cast<ControllerTouch>(i), false, false);
// Connection state comes from the runtime's current profile for the
// hand; profile changes refresh diagnostics here and on the event.
auto profile_info =
XrInfo<XrInteractionProfileState>(XR_TYPE_INTERACTION_PROFILE_STATE);
bool connected = false;
if (XR_SUCCEEDED(xrGetCurrentInteractionProfile(session, subaction,
&profile_info)) &&
profile_info.interactionProfile != XR_NULL_PATH) {
connected = true;
if (profile_info.interactionProfile != impl_->active_profile[hand]) {
impl_->active_profile[hand] = profile_info.interactionProfile;
char buffer[XR_MAX_PATH_LENGTH]{};
std::uint32_t length = 0;
if (XR_SUCCEEDED(xrPathToString(instance,
profile_info.interactionProfile,
sizeof(buffer), &length, buffer))) {
impl_->active_profile_name[hand] = buffer;
}
builder_.SetProfile(hand, impl_->active_profile_name[hand]);
Log(" [vr-input] hand %d interaction profile: %s\n", hand,
impl_->active_profile_name[hand].c_str());
}
} else {
if (impl_->active_profile[hand] != XR_NULL_PATH) {
Log(" [vr-input] hand %d controller disconnected\n", hand);
}
impl_->active_profile[hand] = XR_NULL_PATH;
impl_->active_profile_name[hand].clear();
builder_.SetProfile(hand, {});
}
if (connected && builder_.snapshot().hands[hand].profile !=
impl_->active_profile_name[hand])
builder_.SetProfile(hand, impl_->active_profile_name[hand]);
builder_.SetConnected(hand, connected);
if (!connected) {
// A disconnected hand must release anything held: force every button
// level to false so Publish emits at most one release edge per button
// and nothing can stay stuck down.
for (std::size_t index = 0;
index < static_cast<std::size_t>(ControllerButton::kCount);
++index) {
builder_.SetButton(hand, static_cast<ControllerButton>(index), false,
true);
}
builder_.SetAxis(hand, ControllerAxis::kTrigger, 0.f, true);
builder_.SetAxis(hand, ControllerAxis::kSqueeze, 0.f, true);
builder_.SetAxis(hand, ControllerAxis::kThumbstickX, 0.f, true);
builder_.SetAxis(hand, ControllerAxis::kThumbstickY, 0.f, true);
builder_.SetPoseValid(hand, false);
hands_[hand] = VrHandPose{};
continue;
}
// Grip and aim poses located at the very predicted display time the head
// and eyes used for this frame.
VrHandPose hand_pose;
bool hand_active = false;
for (int which = 0; which < 2; ++which) {
const XrSpace space =
which == 0 ? impl_->grip_space[hand] : impl_->aim_space[hand];
XrSpaceLocation location =
XrInfo<XrSpaceLocation>(XR_TYPE_SPACE_LOCATION);
const XrResult located =
xrLocateSpace(space, impl_->local_space, time, &location);
const bool orientation_valid =
XR_SUCCEEDED(located) &&
(location.locationFlags &
XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0;
const bool position_valid = XR_SUCCEEDED(located) &&
(location.locationFlags &
XR_SPACE_LOCATION_POSITION_VALID_BIT) != 0;
// The guest record has a single valid gate and the consumer converts
// position+quaternion together: orientation-only tracking is therefore
// reported untracked rather than paired with a stale position.
auto pose_info =
XrInfo<XrActionStateGetInfo>(XR_TYPE_ACTION_STATE_GET_INFO);
pose_info.action = which == 0 ? impl_->grip[hand] : impl_->aim[hand];
pose_info.subactionPath = subaction;
auto pose_state = XrInfo<XrActionStatePose>(XR_TYPE_ACTION_STATE_POSE);
const bool active = XR_SUCCEEDED(xrGetActionStatePose(
session, &pose_info, &pose_state)) &&
pose_state.isActive;
hand_active |= active;
const bool usable = active && orientation_valid && position_valid;
float *position =
which == 0 ? hand_pose.grip_position : hand_pose.aim_position;
float *orientation =
which == 0 ? hand_pose.grip_orientation : hand_pose.aim_orientation;
if (usable) {
position[0] = location.pose.position.x;
position[1] = location.pose.position.y;
position[2] = location.pose.position.z;
orientation[0] = location.pose.orientation.x;
orientation[1] = location.pose.orientation.y;
orientation[2] = location.pose.orientation.z;
orientation[3] = location.pose.orientation.w;
}
if (which == 0) {
hand_pose.grip_valid = usable;
} else {
hand_pose.aim_valid = usable;
}
}
hands_[hand] = hand_pose;
builder_.SetPoseValid(hand, hand_pose.grip_valid);
// Analog inputs. XrActionState{Float,Vector2f} carry the value in
// currentState and validity in isActive.
auto get_info =
XrInfo<XrActionStateGetInfo>(XR_TYPE_ACTION_STATE_GET_INFO);
get_info.subactionPath = subaction;
get_info.action = impl_->trigger;
XrActionStateFloat trigger_state =
XrInfo<XrActionStateFloat>(XR_TYPE_ACTION_STATE_FLOAT);
if (XR_SUCCEEDED(
xrGetActionStateFloat(session, &get_info, &trigger_state)) &&
trigger_state.isActive) {
hand_active = true;
builder_.SetAxis(hand, ControllerAxis::kTrigger,
trigger_state.currentState, true);
}
get_info.action = impl_->squeeze;
XrActionStateFloat squeeze_state =
XrInfo<XrActionStateFloat>(XR_TYPE_ACTION_STATE_FLOAT);
if (XR_SUCCEEDED(
xrGetActionStateFloat(session, &get_info, &squeeze_state)) &&
squeeze_state.isActive) {
hand_active = true;
builder_.SetAxis(hand, ControllerAxis::kSqueeze,
squeeze_state.currentState, true);
}
get_info.action = impl_->stick;
XrActionStateVector2f stick_state =
XrInfo<XrActionStateVector2f>(XR_TYPE_ACTION_STATE_VECTOR2F);
if (XR_SUCCEEDED(
xrGetActionStateVector2f(session, &get_info, &stick_state)) &&
stick_state.isActive) {
hand_active = true;
builder_.SetAxis(hand, ControllerAxis::kThumbstickX,
stick_state.currentState.x, true);
builder_.SetAxis(hand, ControllerAxis::kThumbstickY,
stick_state.currentState.y, true);
}
for (std::size_t i = 0;
i < static_cast<std::size_t>(ControllerTouch::kCount); ++i) {
get_info.action = impl_->touches[i];
auto state = XrInfo<XrActionStateBoolean>(XR_TYPE_ACTION_STATE_BOOLEAN);
const bool available =
XR_SUCCEEDED(xrGetActionStateBoolean(session, &get_info, &state)) &&
state.isActive;
builder_.SetTouch(hand, static_cast<ControllerTouch>(i),
state.currentState, available);
}
// Boolean inputs.
const std::pair<XrAction, ControllerButton> booleans[] = {
{impl_->trigger_click, ControllerButton::kTrigger},
{impl_->squeeze_click, ControllerButton::kSqueeze},
{impl_->stick_click, ControllerButton::kThumbstick},
{impl_->face_a, ControllerButton::kFaceSouth},
{impl_->face_b, ControllerButton::kFaceEast},
{impl_->face_x, ControllerButton::kFaceWest},
{impl_->face_y, ControllerButton::kFaceNorth},
{impl_->menu, ControllerButton::kMenu},
};
bool native_trigger_click = false;
bool native_squeeze_click = false;
for (const auto &entry : booleans) {
get_info.action = entry.first;
XrActionStateBoolean state =
XrInfo<XrActionStateBoolean>(XR_TYPE_ACTION_STATE_BOOLEAN);
if (!XR_SUCCEEDED(
xrGetActionStateBoolean(session, &get_info, &state)) ||
!state.isActive) {
continue;
}
hand_active = true;
const bool pressed = state.currentState != XR_FALSE;
builder_.SetButton(hand, entry.second, pressed, true);
if (entry.first == impl_->squeeze_click)
native_squeeze_click = true;
if (entry.first == impl_->trigger_click) {
native_trigger_click = true;
}
}
// Native VR has analog shoulder channels as well as trigger channels.
// Only click-only profiles (Vive) need a synthesized 0/1 squeeze value.
if (native_squeeze_click && !squeeze_state.isActive)
builder_.SetAxis(
hand, ControllerAxis::kSqueeze,
builder_.snapshot().hands[hand].buttons[static_cast<std::size_t>(
ControllerButton::kSqueeze)]
? 1.f
: 0.f,
true);
if (native_trigger_click && !trigger_state.isActive)
builder_.SetAxis(
hand, ControllerAxis::kTrigger,
builder_.snapshot()
.hands[hand]
.buttons[static_cast<size_t>(ControllerButton::kTrigger)]
? 1.f
: 0.f,
true);
if (!native_squeeze_click)
builder_.SetButton(hand, ControllerButton::kSqueeze,
builder_.AnalogClick(hand, ControllerAxis::kSqueeze),
true);
builder_.SetConnected(hand, hand_active);
impl_->native_trigger_click[hand] = native_trigger_click;
// Profiles without a trigger click action (Touch, Vive) derive the click
// from the analog value with hysteresis; the native analog value itself
// is always preserved.
if (!native_trigger_click) {
builder_.SetButton(hand, ControllerButton::kTrigger,
builder_.AnalogClick(hand, ControllerAxis::kTrigger),
true);
}
}
// One coherent snapshot for the frame; edges fire exactly once here.
std::vector<ControllerStateBuilder::Edge> edges;
snapshot_ = builder_.Publish(frame, session_generation_, &edges);
ControllerDelivery delivery;
delivery.frame = frame;
delivery.session_generation = session_generation_;
for (int hand = 0; hand < 2; ++hand) {
for (size_t i = 0; i < static_cast<size_t>(ControllerButton::kCount); ++i)
delivery.button_levels[hand][i] = snapshot_.hands[hand].buttons[i];
delivery.connected[hand] = snapshot_.hands[hand].connected;
delivery.connection_changed[hand] =
delivery.connected[hand] != last_delivered_connected_[hand];
last_delivered_connected_[hand] = delivery.connected[hand];
delivery.axes_valid[hand] = snapshot_.hands[hand].connected;
const auto &axes = snapshot_.hands[hand].axes;
const int stick_x =
ControllerAxisToSdlOrdinal(hand, ControllerAxis::kThumbstickX);
const int stick_y =
ControllerAxisToSdlOrdinal(hand, ControllerAxis::kThumbstickY);
const int trigger =
ControllerAxisToSdlOrdinal(hand, ControllerAxis::kTrigger);
delivery.axes[stick_x] =
axes[static_cast<std::size_t>(ControllerAxis::kThumbstickX)];
delivery.axes[stick_y] =
-axes[static_cast<std::size_t>(ControllerAxis::kThumbstickY)];
delivery.axes[trigger] =
axes[static_cast<std::size_t>(ControllerAxis::kTrigger)];
}
for (const auto &edge : edges) {
if (delivery.edges.size() >= ControllerDelivery::kMaxEdges) {
// Bounded growth: remaining edges are dropped, but a level resync is
// demanded so the consumer converges and nothing sticks down.
delivery.resync = true;
break;
}
delivery.edges.push_back(
ControllerDelivery::ButtonEdge{edge.hand, edge.button, edge.pressed});
}
delivery_queue_.Push(std::move(delivery));
// Apply queued haptic requests inside this XR call window.
{
std::lock_guard<std::mutex> lock(impl_->haptic_mutex);
for (int hand = 0; hand < 2; ++hand) {
auto &request = impl_->haptics[hand];
if (!snapshot_.hands[hand].connected) {
request = {};
request.pending = true;
request.stop = true;
}
if (!request.pending) {
continue;
}
request.pending = false;
auto haptic_info =
XrInfo<XrHapticActionInfo>(XR_TYPE_HAPTIC_ACTION_INFO);
haptic_info.action = impl_->haptic;
haptic_info.subactionPath =
hand == 0 ? impl_->subaction_left : impl_->subaction_right;
if (request.stop || !snapshot_.hands[hand].connected) {
// Never vibrate a disconnected controller.
(void)xrStopHapticFeedback(session, &haptic_info);
continue;
}
auto vibration = XrInfo<XrHapticVibration>(XR_TYPE_HAPTIC_VIBRATION);
vibration.amplitude = request.amplitude;
vibration.duration = request.duration_ns == 0
? XR_MIN_HAPTIC_DURATION
: static_cast<XrDuration>(std::min<uint64_t>(
request.duration_ns, INT64_MAX));
// 0 lets the runtime choose its resonant frequency; the runtime
// clamps unsupported frequencies itself.
vibration.frequency = request.frequency_hz;
const XrResult result = xrApplyHapticFeedback(
session, &haptic_info,
reinterpret_cast<const XrHapticBaseHeader *>(&vibration));
// Rumble requests repeat every few frames, so each outcome is logged
// once per hand.
static std::atomic<bool> logged_failure[2]{};
static std::atomic<bool> logged[2]{};
if (XR_FAILED(result)) {
if (!logged_failure[hand].exchange(true)) {
Log(" [vr-input] xrApplyHapticFeedback failed for hand %d: %d\n",
hand, static_cast<int>(result));
}
} else {
if (!logged[hand].exchange(true)) {
Log(" [vr-input] xrApplyHapticFeedback active for hand %d: amp=%.2f dur_ns=%" PRIu64 "\n",
hand, request.amplitude,
static_cast<uint64_t>(vibration.duration));
}
}
}
}
return true;
} catch (const std::exception &error) {
Log(" [vr-input] action sync failed: %s\n", error.what());
ResetInput();
return false;
}
}
} // namespace mocktail::vr
+53
View File
@@ -0,0 +1,53 @@
// Inert XrActions implementation for non-VR builds. The public header is
// OpenXR-free (opaque Impl pointer), so these definitions let OpenXrBackend
// hold an XrActions member and link without the OpenXR SDK. Every entry point
// is a no-op: a non-VR build has no session, no runtime and no controllers.
#include "mocktail/vr/xr_actions.h"
namespace mocktail::vr {
struct XrActions::Impl {};
XrActions::~XrActions() = default;
Status XrActions::Create(void *) {
return Status::Error(StatusCode::kUnavailable,
"OpenXR support is disabled in this build");
}
Status XrActions::Attach(void *, void *, void *) {
return Status::Error(StatusCode::kUnavailable,
"OpenXR support is disabled in this build");
}
void XrActions::ResetInput() {}
void XrActions::Detach() { attached_ = false; }
void XrActions::Destroy() {
attached_ = false;
created_ = false;
snapshot_ = ControllerSnapshot{};
hands_[0] = VrHandPose{};
hands_[1] = VrHandPose{};
delivery_queue_.Clear();
last_delivered_connected_[0] = false;
last_delivered_connected_[1] = false;
session_generation_ = 0;
}
bool XrActions::Sync(void *, std::uint64_t, std::uint64_t, std::uint64_t) {
return false;
}
bool XrActions::TakeDelivery(ControllerDelivery *) { return false; }
void XrActions::RequestHaptics(int, float, std::uint64_t, float) {}
void XrActions::StopHaptics(int) {}
void XrActions::NoteProfileChanged() {}
std::string XrActions::ActiveProfile(int) const { return {}; }
} // namespace mocktail::vr
+376
View File
@@ -0,0 +1,376 @@
#include "mocktail/vr/xr_controller.h"
#include <algorithm>
#include <cmath>
namespace mocktail::vr {
namespace {
constexpr std::size_t kButtons =
static_cast<std::size_t>(ControllerButton::kCount);
constexpr std::size_t kAxes = static_cast<std::size_t>(ControllerAxis::kCount);
bool IsValidHand(int hand) { return hand == 0 || hand == 1; }
} // namespace
ControllerStateBuilder::ControllerStateBuilder(AnalogClickThresholds thresholds)
: thresholds_(thresholds) {}
void ControllerStateBuilder::SetTouch(int hand, ControllerTouch touch,
bool value, bool active) {
const auto i = static_cast<std::size_t>(touch);
if (!IsValidHand(hand) ||
i >= static_cast<std::size_t>(ControllerTouch::kCount))
return;
snapshot_.hands[hand].touches[i] = active && value;
snapshot_.hands[hand].touch_active[i] = active;
}
void ControllerStateBuilder::SetButton(int hand, ControllerButton button,
bool value, bool active) {
if (!IsValidHand(hand)) {
return;
}
const std::size_t index = static_cast<std::size_t>(button);
if (index >= kButtons) {
return;
}
snapshot_.hands[hand].buttons[index] = active && value;
}
void ControllerStateBuilder::SetAxis(int hand, ControllerAxis axis, float value,
bool active) {
if (!IsValidHand(hand)) {
return;
}
const std::size_t index = static_cast<std::size_t>(axis);
if (index >= kAxes) {
return;
}
if (!active || !std::isfinite(value)) {
snapshot_.hands[hand].axes[index] = 0.f;
return;
}
// Clamp to the contract range; runtimes may overshoot slightly on analog
// controls and the guest expects normalized values.
if (axis == ControllerAxis::kTrigger || axis == ControllerAxis::kSqueeze) {
snapshot_.hands[hand].axes[index] = std::clamp(value, 0.f, 1.f);
} else {
snapshot_.hands[hand].axes[index] = std::clamp(value, -1.f, 1.f);
}
}
void ControllerStateBuilder::SetConnected(int hand, bool connected) {
if (!IsValidHand(hand)) {
return;
}
snapshot_.hands[hand].connected = connected;
if (!connected) {
snapshot_.hands[hand].touches = {};
snapshot_.hands[hand].touch_active = {};
analog_latch_[hand * 2] = false;
analog_latch_[hand * 2 + 1] = false;
}
}
void ControllerStateBuilder::SetProfile(int hand, std::string profile) {
if (!IsValidHand(hand)) {
return;
}
snapshot_.hands[hand].profile = std::move(profile);
}
void ControllerStateBuilder::SetPoseValid(int hand, bool valid) {
if (!IsValidHand(hand)) {
return;
}
snapshot_.hands[hand].pose_valid = valid;
}
bool ControllerStateBuilder::AnalogClick(int hand, ControllerAxis axis) {
if (!IsValidHand(hand) ||
(axis != ControllerAxis::kTrigger && axis != ControllerAxis::kSqueeze)) {
return false;
}
const float value =
snapshot_.hands[hand].axes[static_cast<std::size_t>(axis)];
const std::size_t axis_index = static_cast<std::size_t>(axis);
// Only two analog controls are click-derivable; index them 0/1.
const std::size_t latch_index = static_cast<std::size_t>(hand) * 2 +
(axis == ControllerAxis::kTrigger ? 0u : 1u);
const bool latched = analog_latch_[latch_index];
const bool next =
latched ? value > thresholds_.release : value >= thresholds_.press;
analog_latch_[latch_index] = next;
static_cast<void>(axis_index);
return next;
}
ControllerSnapshot ControllerStateBuilder::Publish(std::uint64_t frame,
std::uint64_t generation,
std::vector<Edge> *edges) {
snapshot_.frame = frame;
snapshot_.session_generation = generation;
// Derive click levels for analog controls whose profile has no click action,
// then emit exactly one edge per transition. published_buttons_ starts
// all-false, so the first publish legitimately reports already-held buttons
// as press edges; later publishes report only real transitions, so repeated
// native getters reading the snapshot never see duplicate edges.
for (int hand = 0; hand < 2; ++hand) {
for (std::size_t index = 0; index < kButtons; ++index) {
const bool level = snapshot_.hands[hand].buttons[index];
const bool previous = published_buttons_[hand][index];
if (level != previous && edges != nullptr) {
edges->push_back(
Edge{hand, static_cast<ControllerButton>(index), level});
}
published_buttons_[hand][index] = level;
}
}
have_published_ = true;
return snapshot_;
}
void ControllerStateBuilder::ReleaseAll(std::uint64_t frame,
std::uint64_t generation,
std::vector<Edge> *edges) {
snapshot_.frame = frame;
snapshot_.session_generation = generation;
for (int hand = 0; hand < 2; ++hand) {
snapshot_.hands[hand].touches = {};
snapshot_.hands[hand].touch_active = {};
for (std::size_t index = 0; index < kAxes; ++index) {
snapshot_.hands[hand].axes[index] = 0.f;
}
for (std::size_t index = 0; index < kButtons; ++index) {
if (snapshot_.hands[hand].buttons[index] && edges != nullptr) {
edges->push_back(
Edge{hand, static_cast<ControllerButton>(index), false});
}
snapshot_.hands[hand].buttons[index] = false;
published_buttons_[hand][index] = false;
}
// Connection and pose validity are properties of the runtime state, not of
// the press levels; callers set them explicitly around this call.
}
analog_latch_.fill(false);
have_published_ = true;
}
int32_t ControllerButtonToAndroidKey(int hand, ControllerButton button) {
// Android KEYCODE_BUTTON_* values already verified end-to-end by the
// existing physical-gamepad route (MapSdlGamepadButtonToAndroid). The
// per-hand squeeze/thumbstick map onto the shoulder/thumb identifiers the
// guest already accepts; the right hand uses the R-side variants.
const bool right = hand == 1;
switch (button) {
case ControllerButton::kFaceSouth:
return 96; // BUTTON_A
case ControllerButton::kFaceEast:
return 97; // BUTTON_B
case ControllerButton::kFaceWest:
return 99; // BUTTON_X
case ControllerButton::kFaceNorth:
return 100; // BUTTON_Y
case ControllerButton::kSqueeze:
return right ? 103 : 102; // R1 / L1
case ControllerButton::kThumbstick:
return right ? 107 : 106; // THUMB_R / L
case ControllerButton::kMenu:
return 108; // BUTTON_START
case ControllerButton::kTrigger:
return -1; // analog click, no keycode
case ControllerButton::kCount:
break;
}
return -1;
}
int32_t ControllerAxisToAndroidAxis(int hand, ControllerAxis axis) {
// Android MotionEvent AXIS_* values from kAndroidAxes in the verified
// gamepad route: {AXIS_X=0, AXIS_Y=1, AXIS_Z=11, AXIS_RZ=14,
// AXIS_LTRIGGER=17, AXIS_RTRIGGER=18}.
switch (axis) {
case ControllerAxis::kThumbstickX:
return hand == 1 ? 11 : 0; // Z / X
case ControllerAxis::kThumbstickY:
return hand == 1 ? 14 : 1; // RZ / Y
case ControllerAxis::kTrigger:
return hand == 1 ? 18 : 17; // R / L trigger
case ControllerAxis::kSqueeze:
return -1; // squeeze has no analog axis id
case ControllerAxis::kCount:
break;
}
return -1;
}
int32_t ControllerAxisComponentCount(ControllerAxis axis) {
switch (axis) {
case ControllerAxis::kThumbstickX:
case ControllerAxis::kThumbstickY:
return 2; // sticks are delivered as an (x, y) motion pair
case ControllerAxis::kTrigger:
return 1;
default:
return 0;
}
}
int32_t ControllerButtonToSdlOrdinal(int hand, ControllerButton button) {
// SDL_GamepadButton ordinals (SDL3): SOUTH=0, EAST=1, WEST=2, NORTH=3,
// BACK=4, START=6, LEFT_STICK=7, RIGHT_STICK=8, LEFT_SHOULDER=9,
// RIGHT_SHOULDER=10. The router converts these to the verified Android
// keycodes; XR delivery must not bypass that conversion.
switch (button) {
case ControllerButton::kFaceSouth:
return 0;
case ControllerButton::kFaceEast:
return 1;
case ControllerButton::kFaceWest:
return 2;
case ControllerButton::kFaceNorth:
return 3;
case ControllerButton::kThumbstick:
return hand == 1 ? 8 : 7;
case ControllerButton::kSqueeze:
return hand == 1 ? 10 : 9;
case ControllerButton::kMenu:
return 6;
case ControllerButton::kTrigger:
return -1; // analog click, no button
case ControllerButton::kCount:
break;
}
return -1;
}
int32_t ControllerAxisToSdlOrdinal(int hand, ControllerAxis axis) {
// SDL_GamepadAxis ordinals (SDL3): LEFTX=0, LEFTY=1, RIGHTX=2, RIGHTY=3,
// TRIGGERLEFT=4, TRIGGERRIGHT=5.
switch (axis) {
case ControllerAxis::kThumbstickX:
return hand == 1 ? 2 : 0;
case ControllerAxis::kThumbstickY:
return hand == 1 ? 3 : 1;
case ControllerAxis::kTrigger:
return hand == 1 ? 5 : 4;
case ControllerAxis::kSqueeze:
return -1; // delivered as a button only
case ControllerAxis::kCount:
break;
}
return -1;
}
bool EncodeNativeControllerChannels(const ControllerSnapshot &snapshot,
float (&channels)[28]) {
std::fill(std::begin(channels), std::end(channels), 0.f);
for (int hand = 0; hand < 2; ++hand) {
const auto &input = snapshot.hands[hand];
if (!input.connected)
continue;
auto button = [&](ControllerButton b) {
return input.buttons[static_cast<std::size_t>(b)] ? 1.f : 0.f;
};
const int base = hand * 9;
channels[base] = button(ControllerButton::kThumbstick);
channels[base + 1] =
input.axes[static_cast<std::size_t>(ControllerAxis::kThumbstickX)];
channels[base + 2] =
input.axes[static_cast<std::size_t>(ControllerAxis::kThumbstickY)];
channels[base + 3] =
input.axes[static_cast<std::size_t>(ControllerAxis::kTrigger)];
channels[base + 4] =
input.axes[static_cast<std::size_t>(ControllerAxis::kSqueeze)];
channels[hand == 0 ? 24 : 22] = button(
hand == 0 ? ControllerButton::kFaceWest : ControllerButton::kFaceSouth);
channels[hand == 0 ? 25 : 23] = button(
hand == 0 ? ControllerButton::kFaceNorth : ControllerButton::kFaceEast);
channels[27] = std::max(channels[27], button(ControllerButton::kMenu));
}
return snapshot.hands[0].connected || snapshot.hands[1].connected;
}
void EncodeControllerDelivery(const ControllerDelivery &delivery,
MocktailVrControllerDelivery *out) {
if (!out)
return;
*out = MocktailVrControllerDelivery{};
out->frame = delivery.frame;
out->resync = delivery.resync ? 1 : 0;
out->session_generation = delivery.session_generation;
const std::size_t button_count =
static_cast<std::size_t>(mocktail::vr::ControllerButton::kCount);
for (int hand = 0; hand < 2; ++hand) {
out->connected[hand] = delivery.connected[hand] ? 1 : 0;
out->connection_changed[hand] = delivery.connection_changed[hand] ? 1 : 0;
out->axes_valid[hand] = delivery.axes_valid[hand] ? 1 : 0;
for (std::size_t index = 0;
index < button_count &&
index < static_cast<std::size_t>(MOCKTAIL_VR_CONTROLLER_BUTTON_COUNT);
++index) {
out->button_levels[hand][index] =
delivery.button_levels[hand][index] ? 1 : 0;
}
}
for (int axis = 0; axis < MOCKTAIL_VR_CONTROLLER_AXIS_COUNT; ++axis) {
out->axes[axis] = delivery.axes[axis];
}
const std::size_t edges = std::min<std::size_t>(
delivery.edges.size(), MOCKTAIL_VR_CONTROLLER_MAX_EDGES);
out->edge_count = static_cast<int32_t>(edges);
for (std::size_t index = 0; index < edges; ++index) {
out->edges[index].hand = delivery.edges[index].hand;
out->edges[index].button =
static_cast<int32_t>(delivery.edges[index].button);
out->edges[index].pressed = delivery.edges[index].pressed ? 1 : 0;
}
}
void ControllerDeliveryQueue::Push(ControllerDelivery delivery) {
std::lock_guard<std::mutex> lock(mutex_);
if (overflowed_) {
// The next delivered frame must re-sync full levels because intermediate
// edges were dropped with the discarded frames.
delivery.resync = true;
overflowed_ = false;
}
if (queue_.size() >= kCapacity) {
queue_.pop_front();
++dropped_frames_;
overflowed_ = true;
// Edges inside the dropped frame are lost; converge through resync on the
// newest frame so nothing can stick down.
delivery.resync = true;
}
queue_.push_back(std::move(delivery));
}
bool ControllerDeliveryQueue::Pop(ControllerDelivery *out) {
if (out == nullptr) {
return false;
}
std::lock_guard<std::mutex> lock(mutex_);
if (queue_.empty()) {
return false;
}
*out = std::move(queue_.front());
queue_.pop_front();
return true;
}
std::size_t ControllerDeliveryQueue::size() const {
std::lock_guard<std::mutex> lock(mutex_);
return queue_.size();
}
void ControllerDeliveryQueue::Clear() {
std::lock_guard<std::mutex> lock(mutex_);
queue_.clear();
overflowed_ = false;
}
} // namespace mocktail::vr
+140 -8
View File
@@ -11,8 +11,10 @@
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <deque>
#include <filesystem>
#include <memory>
#include <mutex>
#include <string>
#include <utility>
#include <vector>
@@ -41,6 +43,9 @@
namespace mocktail {
namespace window {
namespace {
bool AttachVrEgl();
}
// Local EGL declarations avoid conflicts with the stub headers.
@@ -165,6 +170,17 @@ static WindowFullscreenStateSync g_fullscreen_state_sync;
static std::unique_ptr<WindowResizeReadinessGate> g_resize_readiness_gate;
static std::unique_ptr<SdlTextInputBackend> g_text_input_backend;
static std::unique_ptr<WindowTextInputOwner> g_text_input_owner;
// Generation of the showKeyboard request still awaiting its hideKeyboard;
// 0 when Roblox wants no keyboard.
static std::atomic<uint64_t> g_text_input_requested_generation{0};
struct VirtualInput {
std::string text; // committed text when non-empty
uint32_t scancode = 0; // otherwise one key press
};
static std::mutex g_virtual_input_mutex;
static std::deque<VirtualInput> g_virtual_input;
// Set while the queue may hold input, so a pump with none takes no lock.
static std::atomic<bool> g_virtual_input_pending{false};
static std::unique_ptr<SdlPointerCaptureBackend> g_pointer_capture_backend;
// The owner is created and destroyed on the SDL thread, but the experience
// composition reaches it from its own thread. The mutex guards the pointer
@@ -966,6 +982,9 @@ bool ActivateWindowEventLifecycles() {
std::lock_guard<std::mutex> lock(g_pointer_capture_owner_mutex);
g_pointer_capture_owner = std::make_shared<WindowPointerCaptureOwner>(
g_pointer_capture_backend.get());
// main() exports the resolved VR state before the window exists.
const char* vr = GetEnvNonEmpty("MOCKTAIL_VR_ENABLED");
g_pointer_capture_owner->SetVrActive(StringEquals(vr, "1"));
}
fprintf(stderr,
" [input] SDL event pump target=%llu Hz raw-motion, no smoothing\n",
@@ -1119,8 +1138,7 @@ bool Init(int width, int height, const char* title) {
return false;
}
if (StringEquals(GetEnvNonEmpty("MOCKTAIL_GRAPHICS_BACKEND"),
"direct-vulkan")) {
if (ShouldUseNativeVulkanBackend()) {
if (!SDL_Init(SDL_INIT_VIDEO)) {
fprintf(stderr, " [window] SDL_Init failed: %s\n", SDL_GetError());
return false;
@@ -1415,6 +1433,16 @@ bool Init(int width, int height, const char* title) {
}
return CreateSoftwareWaitingWindow(width, height, title);
}
if (!AttachVrEgl()) {
SDL_GL_DestroyContext(sdl_context);
SDL_DestroyWindow(g_state.sdl_window);
g_state.sdl_window = nullptr;
g_state.egl_context = nullptr;
g_state.egl_display = nullptr;
g_state.egl_config = nullptr;
g_state.egl_surface = nullptr;
return false;
}
fprintf(stderr, " [window] EGL context and surface initialized via SDL3\n");
if (WindowTraceEnabled()) {
fprintf(
@@ -1448,6 +1476,23 @@ bool Init(int width, int height, const char* title) {
return true;
}
namespace {
bool AttachVrEgl() {
using Attach =
bool (*)(void *, void *, void *, void *, void *(*)(const char *));
auto attach =
reinterpret_cast<Attach>(dlsym(RTLD_DEFAULT, "mocktail_vr_attach_egl"));
if (!attach)
return true;
return attach(
g_state.egl_display, g_state.egl_config, g_state.egl_context,
reinterpret_cast<void *>(SDL_EGL_GetProcAddress("eglGetProcAddress")),
+[](const char *name) -> void * {
return reinterpret_cast<void *>(SDL_GL_GetProcAddress(name));
});
}
} // namespace
bool MakeCurrentOnThread() {
if (!g_state.initialised || !g_state.egl_context) {
if (WindowTraceEnabled()) {
@@ -1465,7 +1510,7 @@ bool MakeCurrentOnThread() {
return false;
}
fprintf(stderr, " [window] EGL context now current on engine thread\n");
return true;
return AttachVrEgl();
}
bool ReleaseCurrentOnThread() {
@@ -1480,6 +1525,11 @@ bool ReleaseCurrentOnThread() {
SDL_GetError());
return false;
}
using Release = void (*)();
auto release =
reinterpret_cast<Release>(dlsym(RTLD_DEFAULT, "mocktail_vr_release_egl"));
if (release)
release();
if (WindowTraceEnabled()) {
fprintf(stderr, " [window] EGL context released on current thread\n");
}
@@ -1616,24 +1666,63 @@ bool SetPreTextInputPumpCallback(PreTextInputPumpCallback callback,
}
void ClearPreTextInputPumpCallback() { g_pre_text_input_pump_gate.Clear(); }
void SetWindowTextInputOwnerEnabled(bool enabled) {
if (!enabled) {
g_text_input_requested_generation.store(0, std::memory_order_release);
}
if (g_text_input_owner != nullptr) {
g_text_input_owner->SetEnabled(enabled);
}
}
bool RequestShowTextInput(uint64_t generation, const TextInputArea& area,
const TextInputOptions& options) {
return g_text_input_owner != nullptr &&
g_text_input_owner->RequestShowTextInput(generation, area, options);
const bool accepted =
g_text_input_owner != nullptr &&
g_text_input_owner->RequestShowTextInput(generation, area, options);
if (accepted) {
g_text_input_requested_generation.store(generation,
std::memory_order_release);
}
return accepted;
}
bool RequestHideTextInput(uint64_t generation) {
return g_text_input_owner != nullptr &&
g_text_input_owner->RequestHideTextInput(generation);
const bool accepted = g_text_input_owner != nullptr &&
g_text_input_owner->RequestHideTextInput(generation);
if (accepted) {
uint64_t expected = generation;
g_text_input_requested_generation.compare_exchange_strong(
expected, 0, std::memory_order_acq_rel);
}
return accepted;
}
bool TextInputRequested() {
return g_text_input_requested_generation.load(std::memory_order_acquire) !=
0;
}
void QueueVirtualTextInput(const char* utf8) {
if (utf8 == nullptr || utf8[0] == '\0') {
return;
}
std::lock_guard<std::mutex> lock(g_virtual_input_mutex);
g_virtual_input.push_back(VirtualInput{utf8, 0});
g_virtual_input_pending.store(true, std::memory_order_release);
}
void QueueVirtualKeyPress(uint32_t scancode) {
if (scancode == 0) {
return;
}
std::lock_guard<std::mutex> lock(g_virtual_input_mutex);
g_virtual_input.push_back(VirtualInput{{}, scancode});
g_virtual_input_pending.store(true, std::memory_order_release);
}
void* GetGLProcAddress(const char* name) {
if (!g_state.initialised || name == nullptr || name[0] == '\0') {
return nullptr;
}
return reinterpret_cast<void*>(SDL_GL_GetProcAddress(name));
void *raw = reinterpret_cast<void *>(SDL_GL_GetProcAddress(name));
using Wrap = void *(*)(const char *, void *);
static auto wrap =
reinterpret_cast<Wrap>(dlsym(RTLD_DEFAULT, "mocktail_vr_wrap_gles_proc"));
return wrap ? wrap(name, raw) : raw;
}
void ShowIfHidden() {
@@ -1713,6 +1802,16 @@ bool SwapBuffers() {
return false;
}
using Notify = void (*)();
static auto vr_begin = reinterpret_cast<Notify>(
dlsym(RTLD_DEFAULT, "mocktail_vr_note_host_present_begin"));
static auto vr_present =
reinterpret_cast<Notify>(dlsym(RTLD_DEFAULT, "mocktail_vr_gles_present"));
if (vr_begin)
vr_begin();
if (vr_present)
vr_present();
if (g_gles_text_overlay == nullptr) {
graphics::GlesTextOverlaySource source;
source.may_present = reinterpret_cast<decltype(source.may_present)>(
@@ -2228,6 +2327,31 @@ bool PumpEvents() {
g_text_input_owner->active())) {
fprintf(stderr, " [input] SDL pointer capture state update failed\n");
}
if (g_virtual_input_pending.exchange(false, std::memory_order_acq_rel)) {
std::deque<VirtualInput> virtual_input;
{
std::lock_guard<std::mutex> lock(g_virtual_input_mutex);
virtual_input.swap(g_virtual_input);
}
for (const VirtualInput& input : virtual_input) {
platform::PlatformEvent event{};
event.timestamp_ns = SDL_GetTicksNS();
if (!input.text.empty()) {
event.payload = platform::TextInputEvent{input.text};
g_platform_event_observer.Notify(event);
continue;
}
const auto scancode = static_cast<SDL_Scancode>(input.scancode);
const auto keycode = static_cast<uint32_t>(
SDL_GetKeyFromScancode(scancode, SDL_KMOD_NONE, false));
event.payload = platform::KeyEvent{true, false, input.scancode, keycode,
0};
g_platform_event_observer.Notify(event);
event.payload = platform::KeyEvent{false, false, input.scancode, keycode,
0};
g_platform_event_observer.Notify(event);
}
}
platform::PlatformEvent pending_motion{};
bool has_pending_motion = false;
@@ -2502,6 +2626,14 @@ void Shutdown() {
// resources below.
g_gles_text_overlay.reset();
if (!g_state.direct_vulkan && g_state.egl_context != nullptr) {
// Drain the XR session while its EGL context still exists and is current.
SDL_GL_MakeCurrent(g_state.sdl_window,
static_cast<SDL_GLContext>(g_state.egl_context));
using Detach = void (*)(void *);
auto detach =
reinterpret_cast<Detach>(dlsym(RTLD_DEFAULT, "mocktail_vr_detach_egl"));
if (detach)
detach(g_state.egl_context);
SDL_GL_MakeCurrent(g_state.sdl_window, nullptr);
}
if (!g_state.direct_vulkan && g_state.egl_context) {
+9
View File
@@ -98,6 +98,15 @@ bool RequestShowTextInput(uint64_t generation, const TextInputArea& area,
const TextInputOptions& options);
bool RequestHideTextInput(uint64_t generation);
// In-headset keyboard support. TextInputRequested reports whether Roblox
// currently wants an on-screen keyboard (between showKeyboard and the matching
// hideKeyboard). The Queue calls are safe from any thread; PumpEvents delivers
// them on the main thread as the same platform events a physical keyboard
// produces, so the focused TextBox edits them identically.
bool TextInputRequested();
void QueueVirtualTextInput(const char* utf8);
void QueueVirtualKeyPress(uint32_t scancode);
void* GetEGLDisplay();
void* GetEGLSurface();
+5 -1
View File
@@ -58,7 +58,7 @@ void WindowPointerCaptureOwner::ClearQuery() {
}
bool WindowPointerCaptureOwner::Pump(bool text_input_active) {
if (!focused_) {
if (!focused_ || vr_active_) {
return Apply(false, true);
}
MouseLockQueryCallback callback = nullptr;
@@ -126,6 +126,10 @@ void WindowPointerCaptureOwner::SetGameSessionActive(bool active) {
game_session_active_ = active;
}
void WindowPointerCaptureOwner::SetVrActive(bool active) {
vr_active_ = active;
}
bool WindowPointerCaptureOwner::OnShiftKeyPressed(bool text_input_active) {
focused_ = true;
return Pump(text_input_active);
@@ -36,6 +36,9 @@ class WindowPointerCaptureOwner final {
// is none, and capturing there freezes the cursor over Roblox's own UI.
// Starts false: the app surface is what the client shows first.
void SetGameSessionActive(bool active);
// In VR the controllers aim and the desktop window only mirrors the left
// eye, so neither Roblox's mouse lock nor RMB drag may trap the pointer.
void SetVrActive(bool active);
// Shift can enable Roblox's persistent mouse lock while a transient RMB
// camera drag is active. Refresh the native state after dispatching that
// key so the subsequent RMB release preserves the intentional lock.
@@ -71,6 +74,7 @@ class WindowPointerCaptureOwner final {
bool right_button_held_ = false;
// Written by the experience composition, read on the SDL thread.
std::atomic<bool> game_session_active_{false};
std::atomic<bool> vr_active_{false};
bool native_lock_observed_ = false;
bool native_lock_was_active_before_right_drag_ = false;
bool shift_key_pressed_during_right_drag_ = false;
+29 -2
View File
@@ -26,6 +26,7 @@
#include <cstdlib>
#include <cstring>
#include <deque>
#include <filesystem>
#include <limits>
#include <mutex>
#include <new>
@@ -159,7 +160,13 @@ ResolvedAsset ResolveAssetPath(const char* filename) {
if (filename == nullptr) {
return {};
}
const std::string_view requested_view = StripAssetUriPrefix(filename);
std::string_view requested_view = StripAssetUriPrefix(filename);
std::string normalized_storage;
if (requested_view.find("..") != std::string_view::npos) {
std::filesystem::path p(requested_view);
normalized_storage = p.lexically_normal().string();
requested_view = normalized_storage;
}
if (HasUnsafePathSegment(requested_view)) {
return {};
}
@@ -167,6 +174,13 @@ ResolvedAsset ResolveAssetPath(const char* filename) {
const std::string requested(requested_view);
const char* root_env = GetEnvNonEmpty("MOCKTAIL_ASSET_ROOT");
const std::string root = root_env != nullptr ? root_env : "rbx_bin/assets";
std::string root_parent;
{
std::filesystem::path rp(root);
if (rp.filename() == "content") {
root_parent = rp.parent_path().string();
}
}
const char* overlay_env = GetEnvNonEmpty("MOCKTAIL_ASSET_OVERLAY_DIR");
const std::string overlay =
overlay_env != nullptr ? overlay_env : "rbx_bin/asset_overlay";
@@ -201,7 +215,7 @@ ResolvedAsset ResolveAssetPath(const char* filename) {
}
}
const std::vector<std::string> root_candidates = {
std::vector<std::string> root_candidates = {
root + "/" + requested,
root + "/" + stripped,
root + "/content/" + requested,
@@ -209,6 +223,14 @@ ResolvedAsset ResolveAssetPath(const char* filename) {
root + "/ExtraContent/" + requested,
root + "/ExtraContent/" + stripped,
};
if (!root_parent.empty()) {
root_candidates.push_back(root_parent + "/" + requested);
root_candidates.push_back(root_parent + "/" + stripped);
root_candidates.push_back(root_parent + "/content/" + requested);
root_candidates.push_back(root_parent + "/content/" + stripped);
root_candidates.push_back(root_parent + "/ExtraContent/" + requested);
root_candidates.push_back(root_parent + "/ExtraContent/" + stripped);
}
for (const std::string& candidate : root_candidates) {
if (FileExists(candidate)) {
return CacheResolvedAsset(cache_key, {candidate, false});
@@ -510,6 +532,11 @@ int AAsset_openFileDescriptor(AAsset* asset, off_t* outStart,
fd = ::open(asset->path.c_str(), O_RDONLY | O_CLOEXEC);
}
if (fd < 0) {
if (AssetTraceEnabled()) {
std::fprintf(stderr,
"[asset] openFileDescriptor failed for %s (errno=%d)\n",
asset->path.c_str(), errno);
}
return -1;
}
if (outStart != nullptr) {
+154
View File
@@ -3,6 +3,129 @@
#
# Licensed under the Apache License, Version 2.0 (the "License")
find_package(Python3 COMPONENTS Interpreter REQUIRED)
# Test-only implicit Vulkan layer that captures GPU-level native stereo
# evidence (eye render-pass counts and per-eye pixel readback).
# Never linked into production code.
add_library(mocktail_vr_evidence_layer MODULE vr/vr_evidence_layer.cc)
target_link_libraries(mocktail_vr_evidence_layer PRIVATE Vulkan::Headers
${CMAKE_DL_LIBS})
mocktail_apply_compile_options(mocktail_vr_evidence_layer)
set_target_properties(mocktail_vr_evidence_layer PROPERTIES
PREFIX "lib" OUTPUT_NAME "mocktail_vr_evidence_layer")
file(GENERATE
OUTPUT "${CMAKE_BINARY_DIR}/tests/vr/VkLayer_mocktail_vr_evidence.json"
CONTENT [=[{
"file_format_version": "1.2.0",
"layer": {
"name": "VK_LAYER_MOCKTAIL_vr_evidence",
"type": "GLOBAL",
"library_path": "$<TARGET_FILE:mocktail_vr_evidence_layer>",
"api_version": "1.3.280",
"implementation_version": "1",
"description": "Mocktail VR native stereo evidence capture (test-only)",
"disable_environment": {"MOCKTAIL_VR_EVIDENCE_LAYER_DISABLE": "1"}
}
}
]=])
add_executable(vr_mirror_test vr/vr_mirror_test.cc vr/runtime_detection_test.cc)
target_link_libraries(vr_mirror_test PRIVATE Mocktail::VR GTest::gtest_main)
mocktail_apply_compile_options(vr_mirror_test)
add_executable(vr_pose_math_test vr/vr_pose_math_test.cc)
target_include_directories(vr_pose_math_test PRIVATE "${PROJECT_SOURCE_DIR}/include")
target_link_libraries(vr_pose_math_test PRIVATE GTest::gtest_main)
mocktail_apply_compile_options(vr_pose_math_test)
add_executable(xr_controller_test vr/xr_controller_test.cc)
target_link_libraries(xr_controller_test PRIVATE Mocktail::VR GTest::gtest_main)
mocktail_apply_compile_options(xr_controller_test)
add_executable(virtual_keyboard_test vr/virtual_keyboard_test.cc)
target_link_libraries(virtual_keyboard_test PRIVATE Mocktail::VR GTest::gtest_main)
mocktail_apply_compile_options(virtual_keyboard_test)
add_executable(vr_evidence_layer_test vr/vr_evidence_layer_test.cc)
target_link_libraries(vr_evidence_layer_test PRIVATE Vulkan::Headers
${CMAKE_DL_LIBS} GTest::gtest_main)
mocktail_apply_compile_options(vr_evidence_layer_test)
add_executable(roblox_vr_device_bridge_test vr/roblox_vr_device_bridge_test.cc)
target_include_directories(roblox_vr_device_bridge_test PRIVATE
"${PROJECT_SOURCE_DIR}/src")
target_compile_definitions(roblox_vr_device_bridge_test PRIVATE
"MOCKTAIL_TEST_BINARY_DIRECTORY=\"${CMAKE_CURRENT_BINARY_DIR}\"")
target_link_libraries(roblox_vr_device_bridge_test PRIVATE
Mocktail::VR
Mocktail::Compat
GTest::gtest_main
)
mocktail_apply_compile_options(roblox_vr_device_bridge_test)
if(MOCKTAIL_ENABLE_VR)
add_executable(openxr_backend_test vr/openxr_backend_test.cc)
target_include_directories(openxr_backend_test PRIVATE "${PROJECT_SOURCE_DIR}/third_party/OpenXR-SDK/include")
target_link_libraries(openxr_backend_test PRIVATE Mocktail::VR GTest::gtest_main)
gtest_discover_tests(openxr_backend_test)
add_executable(virtual_keyboard_raster_test
vr/virtual_keyboard_raster_test.cc)
target_link_libraries(virtual_keyboard_raster_test PRIVATE
Mocktail::VR GTest::gtest_main)
mocktail_apply_compile_options(virtual_keyboard_raster_test)
gtest_discover_tests(virtual_keyboard_raster_test)
add_executable(vr_queue_submission_test vr/queue_submission_test.cc)
target_link_libraries(vr_queue_submission_test PRIVATE
Mocktail::VR stub_vulkan GTest::gtest_main)
# The adapter looks up the backend's exported symbols at run time.
set_target_properties(vr_queue_submission_test PROPERTIES ENABLE_EXPORTS ON)
gtest_discover_tests(vr_queue_submission_test)
# Loads the adapter with dlopen() like the runtime does, so it must not
# link it; the adapter finds the backend through the exported symbols.
add_executable(vr_queue_sharing_load_test vr/queue_sharing_load_test.cc)
target_link_libraries(vr_queue_sharing_load_test PRIVATE
Mocktail::VR ${CMAKE_DL_LIBS} GTest::gtest_main)
target_compile_definitions(vr_queue_sharing_load_test PRIVATE
"MOCKTAIL_TEST_VULKAN_ADAPTER=\"$<TARGET_FILE:stub_vulkan>\"")
set_target_properties(vr_queue_sharing_load_test PROPERTIES
ENABLE_EXPORTS ON)
add_dependencies(vr_queue_sharing_load_test stub_vulkan)
gtest_discover_tests(vr_queue_sharing_load_test)
add_executable(gles_vr_transport_test vr/gles_transport_test.cc)
target_include_directories(gles_vr_transport_test PRIVATE "${PROJECT_SOURCE_DIR}/src" "${PROJECT_SOURCE_DIR}/third_party/OpenXR-SDK/include" ${MOCKTAIL_GLES3_INCLUDE_DIR})
target_link_libraries(gles_vr_transport_test PRIVATE Mocktail::VR SDL3::SDL3 GTest::gtest_main)
gtest_discover_tests(gles_vr_transport_test PROPERTIES ENVIRONMENT "SDL_VIDEODRIVER=offscreen")
add_library(mocktail_test_openxr_runtime MODULE
vr/fake_openxr_runtime.cc vr/fake_openxr_preview.cc)
target_link_libraries(mocktail_test_openxr_runtime PRIVATE
OpenXR::headers Vulkan::Headers)
target_compile_definitions(mocktail_test_openxr_runtime PRIVATE
XR_USE_GRAPHICS_API_VULKAN)
set_target_properties(mocktail_test_openxr_runtime PROPERTIES
CXX_VISIBILITY_PRESET hidden)
mocktail_apply_compile_options(mocktail_test_openxr_runtime)
add_test(NAME OpenXrProbeContract
COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/vr/probe_contract_test.py"
"$<TARGET_FILE:mocktail_vr_probe>" "$<TARGET_FILE:mocktail_test_openxr_runtime>")
add_test(NAME OpenXrPreviewContract
COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/vr/preview_contract_test.py"
"$<TARGET_FILE:mocktail_vr_probe>" "$<TARGET_FILE:mocktail_test_openxr_runtime>"
"$<TARGET_FILE:mocktail>")
set_tests_properties(OpenXrPreviewContract PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 90)
# The checked-in SPIR-V must match shaders/vr. Needs glslc to check.
find_program(MOCKTAIL_GLSLC glslc)
if(MOCKTAIL_GLSLC)
add_test(NAME VrShaders.SpirvMatchesSources
COMMAND "${Python3_EXECUTABLE}"
"${PROJECT_SOURCE_DIR}/scripts/generate_vr_shaders.py" --check
--glslc "${MOCKTAIL_GLSLC}")
endif()
target_compile_definitions(openxr_backend_test PRIVATE
"MOCKTAIL_TEST_OPENXR_RUNTIME=\"$<TARGET_FILE:mocktail_test_openxr_runtime>\"")
add_dependencies(openxr_backend_test mocktail_test_openxr_runtime)
else()
add_test(NAME OpenXrProbeDisabled
COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/vr/probe_contract_test.py"
--disabled "$<TARGET_FILE:mocktail>")
endif()
add_executable(jnivm_test
jnivm_test.cc
)
@@ -411,6 +534,15 @@ target_link_libraries(roblox_permissions_bridge_test PRIVATE
)
mocktail_apply_compile_options(roblox_permissions_bridge_test)
add_executable(roblox_haptic_protocol_bridge_test roblox_haptic_protocol_bridge_test.cc)
target_link_libraries(roblox_haptic_protocol_bridge_test PRIVATE
Mocktail::LegacyJni
Mocktail::Runtime
GTest::gtest_main
Threads::Threads
)
mocktail_apply_compile_options(roblox_haptic_protocol_bridge_test)
add_executable(roblox_account_bridge_test roblox_account_bridge_test.cc)
target_link_libraries(roblox_account_bridge_test PRIVATE
Mocktail::LegacyJni
@@ -722,6 +854,14 @@ target_include_directories(roblox_output_device_bridge_test PRIVATE
)
mocktail_apply_compile_options(roblox_output_device_bridge_test)
add_executable(native_input_capture_test native_input_capture_test.cc)
target_include_directories(native_input_capture_test PRIVATE
"${PROJECT_SOURCE_DIR}/src")
target_link_libraries(native_input_capture_test PRIVATE
mocktail_audio_sdl GTest::gtest_main)
mocktail_apply_compile_options(native_input_capture_test)
gtest_discover_tests(native_input_capture_test)
add_executable(window_text_input_owner_test
window_text_input_owner_test.cc
)
@@ -1039,6 +1179,12 @@ target_link_libraries(headless_graphics_test PRIVATE
)
include(GoogleTest)
gtest_discover_tests(roblox_vr_device_bridge_test)
gtest_discover_tests(vr_evidence_layer_test)
gtest_discover_tests(vr_mirror_test)
gtest_discover_tests(vr_pose_math_test)
gtest_discover_tests(xr_controller_test)
gtest_discover_tests(virtual_keyboard_test)
gtest_discover_tests(jnivm_test)
gtest_discover_tests(jnivm_fields_test)
gtest_discover_tests(jnivm_fmod_write_allocation_test)
@@ -1090,6 +1236,7 @@ gtest_discover_tests(roblox_game_session_native_adapter_test)
gtest_discover_tests(roblox_experience_launch_bridge_test)
gtest_discover_tests(roblox_web_view_bridge_test)
gtest_discover_tests(roblox_permissions_bridge_test)
gtest_discover_tests(roblox_haptic_protocol_bridge_test)
gtest_discover_tests(roblox_account_bridge_test)
gtest_discover_tests(roblox_browser_service_bridge_test)
gtest_discover_tests(roblox_fresh_game_launch_controller_test)
@@ -1259,3 +1406,10 @@ add_test(
${CMAKE_BINARY_DIR}
)
endif()
if(MOCKTAIL_ENABLE_VR)
add_executable(xr_actions_test vr/xr_actions_test.cc)
target_link_libraries(xr_actions_test PRIVATE Mocktail::InputRuntime GTest::gtest_main)
target_include_directories(xr_actions_test PRIVATE "${CMAKE_SOURCE_DIR}/third_party/OpenXR-SDK/include")
gtest_discover_tests(xr_actions_test)
endif()
+84
View File
@@ -411,4 +411,88 @@ TEST_F(AssetManagerTest, CacheSeparatesOverlayDirectoryChanges) {
AAsset_close(second);
}
TEST_F(AssetManagerTest, ResolvesShadersWhenAssetRootPointsToContentOrAssets) {
WriteAsset("shaders/shaders_vulkan_mobile.pack", "shader-pack-data");
WriteAsset("content/test.txt", "content-data");
AAssetManager* manager = AAssetManager_fromJava(nullptr, nullptr);
// 1. Standard asset root pointing to parent assets directory:
AAsset* asset = AAssetManager_open(manager, "shaders/shaders_vulkan_mobile.pack", 0);
ASSERT_NE(asset, nullptr);
off_t start = -1;
off_t length = -1;
int fd = AAsset_openFileDescriptor(asset, &start, &length);
ASSERT_GE(fd, 0);
EXPECT_EQ(length, 16);
close(fd);
AAsset_close(asset);
// 2. Relative traversal inside asset folder should normalize safely:
AAsset* relative_asset = AAssetManager_open(
manager, "content/../shaders/shaders_vulkan_mobile.pack", 0);
ASSERT_NE(relative_asset, nullptr);
AAsset_close(relative_asset);
// 3. Fallback when MOCKTAIL_ASSET_ROOT accidentally points to /content directory:
const std::string content_root = root_ + "/content";
ASSERT_EQ(setenv("MOCKTAIL_ASSET_ROOT", content_root.c_str(), 1), 0);
AAsset* fallback_asset = AAssetManager_open(
manager, "shaders/shaders_vulkan_mobile.pack", 0);
ASSERT_NE(fallback_asset, nullptr);
fd = AAsset_openFileDescriptor(fallback_asset, &start, &length);
ASSERT_GE(fd, 0);
EXPECT_EQ(length, 16);
close(fd);
AAsset_close(fallback_asset);
}
TEST_F(AssetManagerTest, OverlayStillWinsWhenTheRootIsTheContentFolder) {
WriteAsset("content/textures/ArrowCursor.png", "stock-arrow");
const std::string overlay_root = MakeTempRoot();
ASSERT_FALSE(overlay_root.empty());
WriteAssetAt(overlay_root, "content/textures/ArrowCursor.png",
"custom-overlay-arrow");
ASSERT_EQ(setenv("MOCKTAIL_ASSET_OVERLAY_DIR", overlay_root.c_str(), 1), 0);
const std::string content_root = root_ + "/content";
ASSERT_EQ(setenv("MOCKTAIL_ASSET_ROOT", content_root.c_str(), 1), 0);
AAssetManager* manager = AAssetManager_fromJava(nullptr, nullptr);
for (const char* name :
{"content/textures/ArrowCursor.png", "textures/ArrowCursor.png",
"content/fonts/../textures/ArrowCursor.png"}) {
AAsset* asset = AAssetManager_open(manager, name, 0);
ASSERT_NE(asset, nullptr) << name;
EXPECT_EQ(std::string(static_cast<const char*>(AAsset_getBuffer(asset)),
static_cast<std::size_t>(AAsset_getLength(asset))),
"custom-overlay-arrow")
<< name;
AAsset_close(asset);
}
}
TEST_F(AssetManagerTest, DotDotPathsCannotLeaveTheAssetFolders) {
// The file sits beside the asset root, where only an escape can reach it.
const std::string outside = root_ + "/../outside-" +
std::to_string(static_cast<long>(getpid())) +
".txt";
{ std::ofstream(outside) << "secret"; }
const std::string outside_name =
outside.substr(outside.find_last_of('/') + 1);
const std::string content_root = root_ + "/content";
WriteAsset("content/inside.txt", "inside");
AAssetManager* manager = AAssetManager_fromJava(nullptr, nullptr);
for (const std::string& root : {root_, content_root}) {
ASSERT_EQ(setenv("MOCKTAIL_ASSET_ROOT", root.c_str(), 1), 0);
for (const std::string& name :
{"../" + outside_name, "content/../../" + outside_name,
"content/../../../" + outside_name, "a/../../" + outside_name,
std::string("content/.."), std::string("content/../")}) {
EXPECT_EQ(AAssetManager_open(manager, name.c_str(), 0), nullptr)
<< root << " " << name;
}
}
std::remove(outside.c_str());
}
} // namespace
+36 -1
View File
@@ -1,5 +1,6 @@
#include "compat/build_profile.h"
#include <algorithm>
#include <gtest/gtest.h>
#include <nlohmann/json.hpp>
@@ -19,6 +20,23 @@ namespace {
const std::string kManifestPath =
std::string(MOCKTAIL_TEST_SOURCE_DIR) + "/config/roblox_compatibility.json";
TEST(BuildProfileTest, Payload3092IncludesVrAndSecondAudioLayout) {
const auto result = FindBuildProfile(
kManifestPath, "5f0704edd9064f566ee3d6df2bd2fabbcc709f03");
ASSERT_TRUE(result) << result.error;
ASSERT_TRUE(result.profile.has_value());
EXPECT_EQ(result.profile->version_name, "2.738.1397");
EXPECT_EQ(result.profile->version_code, 3092);
EXPECT_TRUE(result.profile->default_allowed);
EXPECT_TRUE(result.profile->allow_host_abi_bridges);
EXPECT_FALSE(result.profile->allow_legacy_binary_patches);
ASSERT_TRUE(result.profile->vr_debug_device_bridge.has_value());
EXPECT_EQ(result.profile->vr_debug_device_bridge->vtable_rva, 0x6d08108u);
ASSERT_TRUE(result.profile->fmod_output_device_bridge.has_value());
EXPECT_EQ(result.profile->fmod_output_device_bridge->vtable_layout_version,
2u);
}
TEST(BuildProfileTest, FindsCurrentPayloadAsSupported) {
const ProfileLookupResult result = FindBuildProfile(
kManifestPath, "d0cb1fa0deb3d9161b4cd77530cbcd2e50de3a21");
@@ -160,7 +178,24 @@ TEST(BuildProfileTest, RejectsPartialInputProfilesAndUpgradesCachedLayouts) {
std::ifstream input(kManifestPath);
input >> document;
}
auto &entry = document["profiles"].back();
// Start from build 2998, whose input RVAs the assertions below expect, and
// later re-identify it as 3092 to test the cached-layout upgrade. Drop the
// shipped 3092 entry first so the Build ID stays unique.
auto &profiles = document["profiles"];
for (auto it = profiles.begin(); it != profiles.end();) {
if (it->value("elf_build_id", "") ==
"5f0704edd9064f566ee3d6df2bd2fabbcc709f03")
it = profiles.erase(it);
else
++it;
}
const auto found = std::find_if(
profiles.begin(), profiles.end(), [](const nlohmann::json &profile) {
return profile.value("elf_build_id", "") ==
"ade08266c67aee88ec9c1d00902150e1684dad3a";
});
ASSERT_NE(found, profiles.end());
auto &entry = *found;
auto &bridge = entry["fmod_output_device_bridge"];
char path[] = "/tmp/mocktail-input-profile-XXXXXX";
int fd = mkstemp(path);
+60
View File
@@ -15,6 +15,66 @@ namespace mocktail {
namespace runtime {
namespace {
TEST(CommandLineTest, ShortVrAliasSurvivesReexecAndLastOverrideWins) {
const char* arguments[] = {"mocktail", "-vr"};
const auto parsed = ParseCommandLine(2, arguments);
ASSERT_TRUE(parsed);
ASSERT_TRUE(parsed.options.vr_enabled.has_value());
EXPECT_TRUE(*parsed.options.vr_enabled);
std::vector<std::string> reexec;
std::string error;
ASSERT_TRUE(BuildCommandLineReexecArguments(parsed.options, 2, arguments, &reexec, &error));
EXPECT_EQ(reexec, (std::vector<std::string>{"-vr"}));
const char* disabled[] = {"mocktail", "-vr", "--no-vr"};
EXPECT_FALSE(*ParseCommandLine(3, disabled).options.vr_enabled);
}
TEST(CommandLineTest, VrOverrideSurvivesReexecAndCanDisableEnvironmentDefault) {
const std::array<const char*, 4> arguments = {"mocktail", "--vr",
"--headless", "--no-vr"};
const auto parsed = ParseCommandLine(arguments.size(), arguments.data());
ASSERT_TRUE(parsed) << parsed.error;
ASSERT_TRUE(parsed.options.vr_enabled.has_value());
EXPECT_FALSE(*parsed.options.vr_enabled);
std::vector<std::string> reexec;
std::string error;
ASSERT_TRUE(BuildCommandLineReexecArguments(
parsed.options, arguments.size(), arguments.data(), &reexec, &error))
<< error;
EXPECT_EQ(reexec,
(std::vector<std::string>{"--vr", "--headless", "--no-vr"}));
const char* previous = std::getenv("MOCKTAIL_VR_ENABLED");
const std::optional<std::string> saved =
previous ? std::optional<std::string>(previous) : std::nullopt;
const char* previous_headless = std::getenv("MOCKTAIL_HEADLESS");
const std::optional<std::string> saved_headless =
previous_headless ? std::optional<std::string>(previous_headless)
: std::nullopt;
setenv("MOCKTAIL_VR_ENABLED", "true", 1);
EXPECT_TRUE(ApplyCommandLineEnvironment(parsed.options, &error)) << error;
EXPECT_STREQ(std::getenv("MOCKTAIL_VR_ENABLED"), "0");
if (saved)
setenv("MOCKTAIL_VR_ENABLED", saved->c_str(), 1);
else
unsetenv("MOCKTAIL_VR_ENABLED");
if (saved_headless)
setenv("MOCKTAIL_HEADLESS", saved_headless->c_str(), 1);
else
unsetenv("MOCKTAIL_HEADLESS");
}
TEST(CommandLineTest, ParsesExplicitVrAndKeepsUnspecifiedDistinctFromFalse) {
const char* enabled[] = {"mocktail", "--vr"};
const auto parsed = ParseCommandLine(2, enabled);
ASSERT_TRUE(parsed) << parsed.error;
EXPECT_EQ(parsed.options.vr_enabled, std::optional<bool>(true));
const char* normal[] = {"mocktail"};
EXPECT_FALSE(ParseCommandLine(1, normal).options.vr_enabled.has_value());
const char* incompatible[] = {"mocktail", "--force-run-latest", "--vr"};
EXPECT_FALSE(ParseCommandLine(3, incompatible));
}
TEST(CommandLineTest, ParsesCompleteRunContractOnce) {
const std::array<const char*, 11> arguments = {
"mocktail",
+22 -1
View File
@@ -109,5 +109,26 @@
},
"schema_version": 1,
"version_code": 2998,
"version_name": "2.736.1408"
"version_name": "2.736.1408",
"vr_device_bridge": {
"camera_head_scale_offset": "0x140",
"camera_vtable_offset": "0x3a8",
"constructor_rva": "0x3810614",
"device_create_framebuffer_vtable_offset": "0x180",
"emulator_flag_storage_rva": "0x7369920",
"eye_getter_rva": "0x38109e6",
"eye_initializer_rva": "0x3810a18",
"fix_register_motors_flag_rva": "0x7367da0",
"hand_pitch_storage_rva": "0x73677a8",
"haptic_sink_rva": "0x1d48b70",
"keep_haptics_flag_rva": "0x72fcd18",
"pointer_frame_rva": "0x4d573ee",
"pointer_hand_selector_offset": "0x118",
"state_getter_rva": "0x38109f6",
"state_memcpy_rva": "0x6b7bfe0",
"support_generic_haptics_flag_rva": "0x73677d0",
"vr_haptics_flag_rva": "0x7367588",
"vtable_rva": "0x6c87b38",
"workspace_getter_rva": "0x263778c"
}
}
+22 -1
View File
@@ -110,5 +110,26 @@
},
"schema_version": 1,
"version_code": 3092,
"version_name": "2.738.1397"
"version_name": "2.738.1397",
"vr_device_bridge": {
"camera_head_scale_offset": "0x140",
"camera_vtable_offset": "0x3a0",
"constructor_rva": "0x37f9fe4",
"device_create_framebuffer_vtable_offset": "0x188",
"emulator_flag_storage_rva": "0x73f4690",
"eye_getter_rva": "0x37fa3b6",
"eye_initializer_rva": "0x37fa3e8",
"fix_register_motors_flag_rva": "0x73f2ac0",
"hand_pitch_storage_rva": "0x73f24c8",
"haptic_sink_rva": "0x1d8f8c0",
"keep_haptics_flag_rva": "0x7387118",
"pointer_frame_rva": "0x4d97b98",
"pointer_hand_selector_offset": "0x118",
"state_getter_rva": "0x37fa3c6",
"state_memcpy_rva": "0x6bfb180",
"support_generic_haptics_flag_rva": "0x73f24f0",
"vr_haptics_flag_rva": "0x73f2268",
"vtable_rva": "0x6d08108",
"workspace_getter_rva": "0x265cebc"
}
}
+22 -1
View File
@@ -110,5 +110,26 @@
},
"schema_version": 1,
"version_code": 3222,
"version_name": "2.742.1059"
"version_name": "2.742.1059",
"vr_device_bridge": {
"camera_head_scale_offset": "0x130",
"camera_vtable_offset": "0x3a0",
"constructor_rva": "0x549c884",
"device_create_framebuffer_vtable_offset": "0x190",
"emulator_flag_storage_rva": "0x74d1640",
"eye_getter_rva": "0x549cc56",
"eye_initializer_rva": "0x549cc88",
"fix_register_motors_flag_rva": "0x74cfa10",
"hand_pitch_storage_rva": "0x74cf430",
"haptic_sink_rva": "0x1e2fcf0",
"keep_haptics_flag_rva": "0x763fef8",
"pointer_frame_rva": "0x4fd8cb0",
"pointer_hand_selector_offset": "0x108",
"state_getter_rva": "0x549cc66",
"state_memcpy_rva": "0x6daad30",
"support_generic_haptics_flag_rva": "0x74cf458",
"vr_haptics_flag_rva": "0x74cf1e0",
"vtable_rva": "0x7083a48",
"workspace_getter_rva": "0x264bb76"
}
}
+22
View File
@@ -28,6 +28,28 @@
namespace mocktail::compat {
namespace {
TEST(HostAbiProfileTest, Payload3092KeepsVerifiedConstructorBoundaries) {
const auto* profile =
FindHostAbiProfile("5f0704edd9064f566ee3d6df2bd2fabbcc709f03");
ASSERT_NE(profile, nullptr);
EXPECT_EQ(profile->bridge_entry_count, 6u);
EXPECT_EQ(profile->init_array_offset, 0x70e1618u);
EXPECT_EQ(profile->init_array_count, 3599u);
EXPECT_TRUE(profile->HasValidConstructorRanges());
EXPECT_TRUE(profile->AllowsConstructor(2));
EXPECT_FALSE(profile->AllowsConstructor(3));
EXPECT_FALSE(profile->AllowsConstructor(4));
EXPECT_TRUE(profile->AllowsConstructor(5));
EXPECT_TRUE(profile->AllowsConstructor(3598));
EXPECT_FALSE(profile->AllowsConstructor(3599));
EXPECT_TRUE(profile->HasValidNativeMimallocConstructorRanges());
EXPECT_FALSE(profile->AllowsNativeMimallocConstructor(1));
EXPECT_TRUE(profile->AllowsNativeMimallocConstructor(2));
EXPECT_TRUE(profile->AllowsNativeMimallocConstructor(3598));
EXPECT_FALSE(profile->AllowsNativeMimallocConstructor(3599));
EXPECT_TRUE(profile->HasValidNativePreJniBootstrap());
}
TEST(HostAbiProfileTest, Supported2908HasExactDerivedBoundaries) {
const HostAbiProfile* profile =
FindHostAbiProfile("63c5109637b7d7b2bdb8ed8f858023ff5ef49326");
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff. Show more