diff --git a/README-QUEST-VR.md b/README-QUEST-VR.md index 1516af5..9d9b506 100644 --- a/README-QUEST-VR.md +++ b/README-QUEST-VR.md @@ -91,6 +91,30 @@ it has the field-of-view menu and the fixes of 0.14 to 0.17 above, and headset no longer shows the question, allow it in the headset's Settings, under the app permissions for the microphone, for "Astro VR Host". core.log says what the game hears ("Microphone: the loudest of the last 10 seconds was ..."). +- **more controllers than the DualSense.** The app was made for a DualSense and took + everything else for a lesser one. Now (all of it under "Controllers" below): + - **the headset's own controllers play the game**, for whoever has no gamepad. The app + reads them through the headset's runtime, with where they are and how they point; the + right one is the controller in the game. + - with a gamepad and the headset's controllers both at hand, **the game is played with the + one used last**. + - **the start-up menu takes whatever is in the player's hands**: a stick to a side chooses, + ✕ / A or a trigger starts the game. + - **a gamepad without motion sensors is turned and tilted in the game by the hands that + hold it.** Before, it followed the hands to where they were and kept pointing straight + ahead, which left the game's gadgets nothing to aim with. + - **the right stick as the touchpad's finger lifts where the stick is let go**, as on the + PC, which is what the catapult at the end of a level needs: pull the stick towards you + and let go. A button that presses the touchpad touches it as well. + - smaller things for gamepads that are not PlayStation's: a right stick that reports on + other axes, triggers that are only buttons, a Back button the system would take for its + own "back" and close the app with. + + Tried on the headset without wearing it: the runtime accepts the controllers' bindings, + the app starts and plays with a made-up gamepad, and the self-test reports a controller + the way a session does and shows it in the game's picture where and how it should be (see + "Testing on the headset without wearing it"). **Not tried: any of it worn**, and no + gamepad other than a DualSense. What else is new is for the PC (`README-PC-VR.md`): SteamVR and other PC headsets, a view for spectators on the monitor, a gamepad in the game that can be moved where nothing tracks @@ -125,10 +149,12 @@ Quest 3 ├─ shows the stereo frames as a projection layer; the compositor reprojects them to the │ current head pose every refresh, like PSVR's own reprojection did ├─ sends the head pose (and lens distance) to the core every display refresh - ├─ maps a Bluetooth DualSense to the DualShock 4 the game expects: buttons, sticks, - │ triggers, touchpad and motion sensors in; rumble and light bar colour out - ├─ places that controller in the game: from the hands the headset sees holding it when it - │ can, otherwise at a fixed spot in front of the player, turned by its motion sensors + ├─ makes the DualShock 4 the game expects of what the player holds: a gamepad paired with + │ the headset (of a DualSense everything: buttons, sticks, triggers, touchpad and motion + │ sensors in; rumble and light bar colour out) or the headset's own controllers + ├─ places that controller in the game: one of the headset's own is where it is; a gamepad + │ is where the hands are that the headset sees holding it (otherwise at a fixed spot in + │ front of the player), turned by its motion sensors or, where it has none, by the hands ├─ plays the sound (one audio stream per port the game uses) └─ starts the emulator core as a child process ──► shadps4 (aarch64 Linux, glibc) ├─ FEX runs the game's x86-64 code @@ -162,15 +188,13 @@ adb install -r build/quest/astro-vr-host.apk adb push games/CUSA12392 /data/local/tmp/astro/games/ # 13 GB, a few minutes ``` -Pair the DualSense with the headset (Settings ▸ Bluetooth; hold Create + PS until the light bar -flashes), then start **Astro VR Host** from the app library (Unknown Sources). A floating panel -reports what the app is doing, and which controller it found, until the game's first frame arrives. -(The headset's own controllers count as gamepads too, and were what the app took for the -controller in the third session: the game then gets buttons and sticks from the DualSense but -neither its motion sensors nor rumble or light bar. From 0.7 on a controller of Sony's make goes -first whenever it shows up, and the controller is the one input actually comes from.) -Put the Touch controllers aside: with them out of the way the headset tracks your hands, which is -what places the DualSense in the game. +To play with a gamepad, pair it with the headset (Settings ▸ Bluetooth; on a DualSense, hold +Create + PS until the light bar flashes); the headset's own controllers need nothing. Then start +**Astro VR Host** from the app library (Unknown Sources). A floating panel reports what the app +is doing, and what the game will be played with, until the game's first frame arrives. +With a gamepad, put the Touch controllers aside: with them out of the way the headset tracks +your hands, which is what places the gamepad in the game. See "Controllers" below for what +plays how. In the game: @@ -178,15 +202,16 @@ In the game: front of you, where the outline is. (If the headset does not see your hands, the controller is assumed to be there and the screen passes on its own.) - The world map is selected by **looking** at a planet and pressing ✕. -- Touchpad: the DualSense's own. If the system does not hand it over, the right stick stands in - for the finger (flick it) and Create for the click. +- Touchpad: the DualSense's own. Without one, the right stick is the finger and Create / Back / + View the press (see "Controllers"). - **Resetting the view**: hold OPTIONS for a second (as on a PlayStation VR), or press the PS - button. Where your head is then is where you sit as far as the game goes, and the way you face - is straight ahead; the controller's heading is taken anew as well. The app does it by itself - when ✕ is pressed for the first time, and when the headset's own "reset view" is used (hold the - Meta button). Do it again whenever you have settled differently and things are too close, too - far or off to a side: the game takes its bearings only when it is told to (see "Where the - player sits"). + button; with the headset's own controllers, hold the left one's menu button for a second or + press both sticks in. Where your head is then is where you sit as far as the game goes, and + the way you face is straight ahead; the controller's heading is taken anew as well. The app + does it by itself when ✕ is pressed for the first time, and when the headset's own "reset + view" is used (hold the Meta button). Do it again whenever you have settled differently and + things are too close, too far or off to a side: the game takes its bearings only when it is + told to (see "Where the player sits"). - Taking the headset off pauses the game; putting it back on resumes it. - If the headset's boundary lights up, you are near its edge: besides being in the way, the boundary costs the GPU time the game needs. A boundary with more room around the seat avoids @@ -202,11 +227,14 @@ Optional settings go in `/sdcard/Android/data/com.astrobotquest.vrhost/files/vrh | `resolution=960` | the size the game draws its scene at, per eye: `816` (x870), `960` (x1080), `1200` (x1280), `1440` (x1536), or `game` for the game's own choice. Default: the emulator chooses, as large as the GPU manages at the pace, see "Speed" | | `antialias=0` | leave edges as jagged as the game draws them without multisampling. Default: the emulator smooths them, see "The picture" | | `real_time=0` | let the game count time in frames as on the console: slow motion whenever a frame takes longer than 1/60 s. Default: the emulator has it run by the clock, see "Speed" | -| `hands=0` | do not use hand tracking; the controller stays at its fixed spot | +| `hands=0` | do not use hand tracking; a gamepad stays at its fixed spot | +| `own_controllers=0` | never play with the headset's own controllers, whoever holds them | +| `pad_hand=left` | with the headset's own controllers, the left one is the controller in the game instead of the right one | +| `pad_tilt=-15` | for a gamepad without motion sensors, which the hands holding it turn and tilt in the game: how many degrees higher (lower, if negative) its front points than the hands do (-60 to 60). For a gamepad that points too high or too low in the game | | `sharpen=0` | no sharpening of the picture. Default 1: the emulator sharpens every frame of the game once. 2: the headset's Super Resolution filter on every refresh instead (finer; about a twentieth of the GPU's time). 3: that filter whenever the system finds the GPU has time for it (in a level: hardly ever). 4: the headset's plainer filter. See "The picture" | | `cubic=1` | have the headset enlarge the picture with a cubic filter instead of a linear one (sharper, for some GPU time on every refresh) | | `stick_touchpad=0` | right stick no longer doubles as a finger on the touchpad | -| `motion=0`, `rumble=0` | ignore the motion sensors / no vibration | +| `motion=0`, `rumble=0` | ignore the gamepad's motion sensors (the hands holding it then say how it is turned, as for a gamepad that has none) / no vibration | | `msaa=4` | let the game multisample as on the console (default 1 = off, much faster) | | `stats=1` | keep a small panel in view with the game's frame rate, the display's refresh rate and whether the hands holding the controller are seen | | `predict_ms=25` | how far beyond the next refresh the head pose given to the game is predicted (0–80). More if the picture's edges show when turning the head, less if the world wobbles | @@ -229,6 +257,62 @@ What a session leaves behind, in the same folder as `vrhost.txt`: what to look at (or send along) when something did not work. `adb logcat -s AstroVR` shows the host's log live. The emulator's own folder (saves, `sys_modules`) is `.../files/data/shadPS4/`. +## Controllers + +The game is played with one DualShock 4, which the app makes of what the player holds. + +**A DualSense** (or a DualShock 4, which the headset's system knows the same way; untested) has +everything the game uses: its touchpad, its motion sensors for how it is turned, rumble and +light bar. Where it is in the game is where the headset sees the hands that hold it. + +**Other gamepads** (the headset's system has drivers and key layouts for Xbox controllers, +8BitDo, Logitech, Razer, PDP, Hori and others; nothing but a DualSense has been tried) lack +two things the game is built around, which are made up for: + +- *the touchpad.* The right stick, for which this game has no use, moves a finger over it: the + finger comes down in the middle of the pad, goes where the stick goes, and lifts where the + stick is let go (not back in the middle, where a stick flies to). That is what the catapult + at the end of a level needs: pull the stick towards you and let go. The Create / Back / View + button presses the pad. (`stick_touchpad=0` leaves the stick a stick.) +- *the motion sensors*, by which the game knows how the controller is turned: its gadgets aim + with it. The hands holding the gamepad stand in: the line from one palm to the other is the + gamepad's sideways axis, which gives how it is turned and how far it is rolled, and the way + the hands point is how far its front is tilted up or down. That is as steady as the + headset's view of two hands around a gamepad, and off by however a gamepad's shape has the + hands point: `pad_tilt=` moves it if it points too high or too low in the game. Motion + sensors of gamepads that are not PlayStation's are not used even where the system offers + them: which way is up for a motion sensor is each maker's own affair. (`motion=0` has a + DualSense go by the hands as well. For a gamepad that has both, `host.log` says every ten + seconds how far the two are apart: "hands against the gamepad's motion sensors".) + +**The headset's own controllers**, for whoever has no gamepad. The app reads them through the +headset's runtime, which says where they are and whether anybody holds them (the gamepad the +system makes of them for other apps says neither, and is ignored): + +| On the controllers | In the game | +| --- | --- | +| the right controller, where it is and the way it points (`pad_hand=left`: the left one) | the controller | +| left stick, pressed in | left stick, L3 | +| A, B (right) | ✕, □ | +| X, Y (left) | ○, △ | +| X and Y together | blowing into the microphone, for as long as they are held | +| grips | L1, R1 | +| triggers | L2, R2 | +| right stick | the finger on the touchpad, as on a gamepad without one | +| right stick pressed in | the touchpad pressed | +| menu button (left) | OPTIONS; held for a second, it resets the view | +| both sticks pressed in | resets the view | + +Both controllers shake with the game's rumble, the left one more with the heavy motor and +the right one more with the light one. + +**With a gamepad and the headset's controllers both at hand, the game is played with the one +that was used last**: a button on the other takes over, and `host.log` says so. From a +gamepad, only what a hand cannot do without a controller in it takes over (a button under a +thumb, a stick): while a gamepad is held it is the hands the headset sees, and their pinching +must never count. The start-up menu takes either: a stick to a side chooses, ✕ / A or a +trigger starts the game, and what started it is not pressed in the game. + ## Where the player sits The game takes the place of the player's head once, from the first position the tracker gives @@ -604,6 +688,8 @@ Settings of `quest-selftest.sh` that make it more like one (given like the core' | `HOST_COMPOSITOR=[,[,]]` | a stand-in for the compositor: so many times a second a pass the size of the compositor's own (9 megapixels) and n - 1 small ones (n = 5), on a GPU context that comes before the emulator's. With it the GPU is the limit as in a session | | `HOST_DISPLAY=` | the test's loop goes round once per refresh of a display of that rate, tells the emulator of each as the app does, and counts for how many refreshes each frame of the game stayed the newest | | `HOST_LOOK=":,;..."` | where the head looks from which second on (degrees); `8:85,22;110:0,-12` finds the first level from the world map and then looks along it | +| `HOST_PAD=":,,,,,,;..."` | what the app says of the controller from which second on, the way a session does: `c` one of the headset's own controllers (it says everything of itself), `h` hands around a gamepad without motion sensors, `p` hands around one that has them (only where it is), `n` nothing seen; where it is in metres from the head, how it is turned in degrees. With `HOST_LOOK=0:0,-18` the controller is in the picture | +| `HOST_SHOTS=` | a picture every so many seconds instead of every ten | | `XDG_DATA_HOME=` | where the emulator keeps its data, saves among them. The self-test starts from nothing each time, so the game plays its prologue (four minutes); with a folder that holds a save it is in the first level after 75 seconds | | `SHADPS4_VR_FOLLOW_DISPLAY=0` | the emulated headset keeps to its own clock although the host tells of its display's refreshes | @@ -640,7 +726,14 @@ it: everything the app does when it is launched runs (settings, log files, OpenX session, unpacking the runtime, the emulator with the game, sound), a controller that is not there presses ✕ now and then (through the same code a real one's events go through), and the `host.log` and `core.log` the app writes are pulled to `build/quest/drystart/` together with -pictures of what its status panel showed. +pictures of what its status panel showed and of the start-up menu. + +`quest-host/build.sh --test-package` builds the same app under another name +(`com.astrobotquest.vrhost.test`, "Astro VR Host (test)", `build/quest/astro-vr-host-test.apk`). +It installs next to the app proper and touches neither it nor its saves: for trying a build on +a headset whose owner plays with the other one. `ASTRO_PACKAGE=com.astrobotquest.vrhost.test` +before `quest-xrprobe.sh`, `quest-drystart.sh`, `quest-selftest.sh` or `quest-sandbox-test.sh` +runs them in it; `adb uninstall com.astrobotquest.vrhost.test` removes it again. ## Watching a real session diff --git a/README.md b/README.md index daba968..85fc845 100644 --- a/README.md +++ b/README.md @@ -42,9 +42,12 @@ over Virtual Desktop and fixed since. Expect rough edges, and please report what - **A Meta Quest 3, or a Valve Index connected to a Windows PC running SteamVR.** (The Quest 3S has the same chip and should work, but nobody has tried.) -- **A PS5 DualSense controller.** It stands in for the PS4 controller the game expects: - buttons, sticks, touchpad, motion sensors, rumble and light bar. (A DualShock 4 may work - too; untested.) On the PC the Quest's own Touch controllers can stand in for it. +- **Something to play with.** A PS5 DualSense controller is best: it stands in for the PS4 + controller the game expects with everything that one has (buttons, sticks, touchpad, motion + sensors, rumble and light bar; a DualShock 4 should do the same, untested). Without one, + **the Quest's own Touch controllers play the game**, on the headset alone as on the PC. + Other gamepads play too (Xbox, 8BitDo and the like): what they lack of a PS4 controller, + the touchpad and the motion sensors, is made up for, see "Playing" below. - **ASTRO BOT Rescue Mission, European release CUSA12392, version 1.00 or 1.04** (the game as on its disc, or with its last update), dumped from your own console and game: either as the game's folder (the one with `eboot.bin`, `sce_sys`, `sce_module` in it, about 13 GB) or @@ -106,21 +109,36 @@ Download `AstroQuest--Quest3.apk` from the Either way, only a package made from a dump of the game can be unpacked; one downloaded from the PlayStation Store is encrypted. -3. **Pair the DualSense with the headset**: Settings > Bluetooth > Pair, and on the - controller hold Create and the PS button until the light bar flashes. +3. **If you play with a gamepad, pair it with the headset**: Settings > Bluetooth > Pair (on + a DualSense, hold Create and the PS button until the light bar flashes). The Touch + controllers need nothing. 4. **Start "Astro VR Host"** from the App Library, under *Unknown Sources*. The first start takes a few seconds longer (it unpacks the emulator) and asks for the microphone: the game listens to it for blowing at things, as it did with PlayStation VR's. Playing: -- Put the Touch controllers aside and hold the DualSense: the headset tracks your hands around - it, and that is where the controller is in the game. +- The app begins with a menu for the field of view: a stick to a side chooses, ✕ / A or a + trigger starts the game. +- **With a gamepad**: put the Touch controllers aside and hold it. The headset tracks your + hands around it, and that is where the controller is in the game. A gamepad without a + touchpad has its right stick for the finger (for the catapult at the end of a level, pull + the stick towards you and let go) and its Create / Back / View button to press the pad. A + gamepad without motion sensors, which is every one that is not a PlayStation's, is turned + and tilted in the game by your hands, as far as the headset sees them. +- **With the Touch controllers**: the right one is the controller in the game, where it is + and the way it points. Left stick to move; A is ✕, B is □, X is ○, Y is △; the grips are L1 + and R1, the triggers L2 and R2; the right stick is the finger on the touchpad, and pressed + in it presses the pad; the left controller's menu button is OPTIONS. X and Y together + blow, where the game wants you to blow into the microphone. +- With both at hand, the game is played with the one you used last: a button on the other + changes over. - The first screen asks to move the controller into a floating outline: hold it up in front of you where the outline is. - On the world map, **look** at a planet and press ✕. -- **Reset the view** by holding OPTIONS for a second (or pressing the PS button) whenever you - sit differently and things are too close, too far or off to one side. +- **Reset the view** by holding OPTIONS for a second (or pressing the PS button; with the + Touch controllers, hold the left menu button for a second or press both sticks in) + whenever you sit differently and things are too close, too far or off to one side. - Taking the headset off pauses the game. - Settings (refresh rate, resolution, field of view, sharpening, microphone gain...) go in `/sdcard/Android/data/com.astrobotquest.vrhost/files/vrhost.txt`; all of them are explained in diff --git a/quest-host/AndroidManifest.xml b/quest-host/AndroidManifest.xml index 73fec2e..fe16d57 100644 --- a/quest-host/AndroidManifest.xml +++ b/quest-host/AndroidManifest.xml @@ -24,7 +24,8 @@ android:required="true" /> diff --git a/quest-host/build.sh b/quest-host/build.sh index 92dc31e..f496f67 100644 --- a/quest-host/build.sh +++ b/quest-host/build.sh @@ -7,10 +7,16 @@ # build/arm64 (tools/build-arm64.sh). # # quest-host/build.sh [--driver ] [--trade-cpu-for-gpu] +# [--test-package] # # --trade-cpu-for-gpu builds the app that gives up one processor clock level (1.92 -> 1.65 GHz # on a Quest 3) for one more GPU clock level (up to 640 MHz instead of 599). The system only # takes that choice from the manifest, so it is a build of its own. +# +# --test-package builds the same app under another name (com.astrobotquest.vrhost.test, +# "Astro VR Host (test)", build/quest/astro-vr-host-test.apk, debug key): it installs next to +# the app proper and touches neither it nor its saves, for trying a build on a headset whose +# owner plays with the other one (the tools take ASTRO_PACKAGE for which of the two they run). set -euo pipefail root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) @@ -32,10 +38,12 @@ driver="turnip-vauzi-7xx-EMULATOR.zip" prefix_map="-ffile-prefix-map=$(cygpath -m "$root")/=" trade="" +test_package="" while [[ $# -gt 0 ]]; do case "$1" in --driver) driver=$2; shift 2 ;; --trade-cpu-for-gpu) trade=1; shift ;; + --test-package) test_package=com.astrobotquest.vrhost.test; shift ;; *) echo "unknown option $1" >&2; exit 64 ;; esac done @@ -100,6 +108,7 @@ echo "== native" -Wno-missing-field-initializers \ -I "$(cygpath -m "$OPENXR/prefab/modules/headers/include")" \ -I "$(cygpath -m "$root/shadps4-arm64-main/src/core/vr")" \ + -I "$(cygpath -m "$root/shadps4-arm64-main/src/input")" \ "$(cygpath -m "$host/cpp/main.cpp")" "$(cygpath -m "$host/cpp/core_process.cpp")" \ "$(cygpath -m "$host/cpp/xr_host.cpp")" "$(cygpath -m "$host/cpp/gl_frames.cpp")" \ "$(cygpath -m "$host/cpp/self_test.cpp")" "$(cygpath -m "$host/cpp/log.cpp")" \ @@ -133,10 +142,25 @@ if [[ -n "$trade" ]]; then grep -q trade_cpu_for_gpu_amount "$manifest" || { echo "could not add the trade to the manifest" >&2; exit 1; } echo " with one processor level traded for one GPU level" fi -"$BUILD_TOOLS/aapt2.exe" compile --dir "$(cygpath -m "$host/res")" -o "$(cygpath -m "$out/res.zip")" +res="$host/res" +rename=() +apk="$out/astro-vr-host.apk" +if [[ -n "$test_package" ]]; then + # Another name for the system and for the eye; the classes keep theirs. + res="$out/res-test" + rm -rf "$res" + cp -r "$host/res" "$res" + sed -i 's|>Astro VR Host<|>Astro VR Host (test)<|' "$res/values/strings.xml" + grep -q "(test)" "$res/values/strings.xml" || { echo "could not rename the test app" >&2; exit 1; } + rename=(--rename-manifest-package "$test_package" + --rename-instrumentation-target-package "$test_package") + apk="$out/astro-vr-host-test.apk" + echo " as $test_package" +fi +"$BUILD_TOOLS/aapt2.exe" compile --dir "$(cygpath -m "$res")" -o "$(cygpath -m "$out/res.zip")" "$BUILD_TOOLS/aapt2.exe" link -o "$(cygpath -m "$out/base.apk")" \ -I "$(cygpath -m "$PLATFORM_JAR")" --manifest "$(cygpath -m "$manifest")" \ - -A "$(cygpath -m "$out/apk/assets")" --debug-mode \ + -A "$(cygpath -m "$out/apk/assets")" --debug-mode "${rename[@]}" \ --min-sdk-version 32 --target-sdk-version 32 "$(cygpath -m "$out/res.zip")" "$PYTHON" - "$out/base.apk" "$out/apk" <<'PY' import os, sys, zipfile @@ -151,7 +175,7 @@ PY # An update only installs over an app signed with the same key. Releases are signed with the # key named in tools/signing.local (two lines: the keystore, then a file holding its password; # neither belongs in the repository), other builds with the Android SDK's debug key. -if [[ -f "$root/tools/signing.local" ]]; then +if [[ -f "$root/tools/signing.local" && -z "$test_package" ]]; then { read -r keystore; read -r password_file; } < <(tr -d '\r' < "$root/tools/signing.local") # (Its key has the keystore's password: apksigner then needs no --key-pass.) signing=(--ks "$keystore" --ks-pass "file:$password_file") @@ -162,6 +186,6 @@ else echo " signed with the debug key" fi java -jar "$(cygpath -m "$BUILD_TOOLS/lib/apksigner.jar")" sign "${signing[@]}" \ - --out "$(cygpath -m "$out/astro-vr-host.apk")" "$(cygpath -m "$out/aligned.apk")" + --out "$(cygpath -m "$apk")" "$(cygpath -m "$out/aligned.apk")" -ls -la "$out/astro-vr-host.apk" +ls -la "$apk" diff --git a/quest-host/cpp/core_process.cpp b/quest-host/cpp/core_process.cpp index 3469940..363010d 100644 --- a/quest-host/cpp/core_process.cpp +++ b/quest-host/cpp/core_process.cpp @@ -393,6 +393,8 @@ void ServeMicrophone(int fd, uint32_t sample_rate, const MicrophoneSettings& set : settings.test_signal ? ": a test signal stands in for it" : ""); const auto started = Clock::now(); uint64_t generated = 0; + bool was_made = false; + bool blow_reported = false; uint32_t noise = 0x2545f491; while (true) { // The guest says nothing more on this connection; it ends by closing it. @@ -402,8 +404,16 @@ void ServeMicrophone(int fd, uint32_t sample_rate, const MicrophoneSettings& set errno != EINTR)) { break; } - if (!settings.enabled) { - ::usleep(100'000); + // Blowing with buttons is heard whatever the microphone is, or is allowed to be. + const bool blown = + settings.blowing != nullptr && settings.blowing->load(std::memory_order_relaxed); + if (blown && !blow_reported) { + blow_reported = true; + LOGI("microphone: the player blows with buttons, the game hears that instead"); + } + if (!settings.enabled && !blown) { + was_made = false; + ::usleep(50'000); continue; } @@ -432,17 +442,26 @@ void ServeMicrophone(int fd, uint32_t sample_rate, const MicrophoneSettings& set reported = now; } int32_t frames = 0; - if (settings.test_signal) { + const bool made = settings.test_signal || blown; + if (!made) { + was_made = false; + } + if (made) { // By the clock, a fiftieth of a second at a time. const auto due = static_cast( std::chrono::duration(now - started).count() * sample_rate); + if (!was_made) { + // (Nothing is owed for the time the microphone spoke for itself.) + was_made = true; + generated = due; + } if (due - generated < samples.size() / 2) { ::usleep(2'000); continue; } frames = static_cast(samples.size() / 2); // Evenly distributed noise of this amplitude has a root mean square of 0.35. - const bool blowing = generated % (5ull * sample_rate) < sample_rate; + const bool blowing = blown || generated % (5ull * sample_rate) < sample_rate; for (size_t i = 0; i < samples.size(); ++i) { noise ^= noise << 13; noise ^= noise >> 17; @@ -472,7 +491,7 @@ void ServeMicrophone(int fd, uint32_t sample_rate, const MicrophoneSettings& set reported_failure = false; } - if (!settings.test_signal) { + if (!made) { frames = input.Read(samples.data(), static_cast(samples.size() / 2)); } if (frames < 0) { @@ -486,8 +505,10 @@ void ServeMicrophone(int fd, uint32_t sample_rate, const MicrophoneSettings& set } const size_t count = static_cast(frames) * 2; + // (What buttons blow is as loud as blowing gets, whatever the microphone needs.) + const float gain = blown ? 1.0f : settings.gain; for (size_t i = 0; i < count; ++i) { - const float amplified = static_cast(samples[i]) * settings.gain; + const float amplified = static_cast(samples[i]) * gain; samples[i] = static_cast(std::clamp(amplified, -32768.0f, 32767.0f)); const double sample = samples[i] / 32768.0; block_squares += sample * sample; @@ -1015,7 +1036,9 @@ void CoreProcess::AudioLoop() { } // The guest opens and closes ports as it likes; each lives until its socket closes, // which happens at the latest when the core exits. - std::thread{ServeAudio, fd, microphone}.detach(); + MicrophoneSettings settings = microphone; + settings.blowing = blowing; + std::thread{ServeAudio, fd, settings}.detach(); } } diff --git a/quest-host/cpp/core_process.h b/quest-host/cpp/core_process.h index 4a3135d..deb405f 100644 --- a/quest-host/cpp/core_process.h +++ b/quest-host/cpp/core_process.h @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -12,22 +13,9 @@ #include +#include "pad_state.h" #include "vr_protocol.h" -/// What the emulated DualShock 4 looks like right now. Buttons use the PS4 pad bit layout. -struct PadState { - uint32_t buttons{}; - uint8_t left_x{128}, left_y{128}, right_x{128}, right_y{128}; - uint8_t left_trigger{}, right_trigger{}; - bool touch_down{}; - uint16_t touch_x{}, touch_y{}; - bool has_motion{}; - float gyro[3]{}; - float accel[3]{0.0f, 9.81f, 0.0f}; - - bool operator==(const PadState&) const = default; -}; - /// Whether the game may listen to the headset's microphone, and how much louder than it is /// what it hears should be made. struct MicrophoneSettings { @@ -36,6 +24,9 @@ struct MicrophoneSettings { /// For tests nobody is there to blow for: instead of the microphone, a second of noise /// every five, as loud as the game takes blowing hard to be. bool test_signal{}; + /// Set while the player blows with buttons instead of with their breath: the game then + /// hears that same noise, whatever the microphone has to say (or is allowed to). + std::shared_ptr> blowing; }; /// Everything needed to start the emulator core. @@ -103,6 +94,10 @@ public: /// Applies to microphone ports the game opens from now on. void SetMicrophone(bool enabled, float gain, bool test_signal = false); + /// The player blows with buttons, or has stopped: what the game hears follows at once. + void SetBlowing(bool blowing_) { + blowing->store(blowing_, std::memory_order_relaxed); + } /// Thread id of the core's thread whose name starts with `prefix`, or 0. pid_t FindThread(const std::string& prefix) const; @@ -136,6 +131,8 @@ private: std::thread wait_thread; MicrophoneSettings microphone; + /// Shared with the threads that serve the game's microphone ports, which may outlast this. + std::shared_ptr> blowing{std::make_shared>(false)}; std::mutex mutex; std::string message; diff --git a/quest-host/cpp/held_pad.h b/quest-host/cpp/held_pad.h new file mode 100644 index 0000000..7588f6c --- /dev/null +++ b/quest-host/cpp/held_pad.h @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#pragma once + +#include +#include + +/// Where a gamepad is and how it is held, told by the two hands around it. A gamepad cannot +/// be tracked, the hands that hold it can: the palms a controller's width apart give away +/// where it is, the line from one to the other is its sideways axis (which way it is turned, +/// and how far it is rolled to a side), and the way the hands point is how far its front is +/// tilted up or down. +/// +/// Everything is in the headset's space: metres, +X right, +Y up, -Z forward. +namespace HeldPad { + +struct Vec { + float x{}, y{}, z{}; +}; + +struct Quat { + float x{}, y{}, z{}, w{1.0f}; +}; + +struct Hands { + Vec left_palm; + Vec right_palm; + /// The way each hand points, from the wrist to the knuckles (along the middle finger's + /// bone in the palm), each of length one: if the headset can tell. + bool pointing_known{}; + Vec left_pointing; + Vec right_pointing; +}; + +struct Pose { + /// The middle of the gamepad. + Vec position; + /// The hands are level enough for the gamepad's heading and attitude to be told. + bool turned_known{}; + /// Heading: 0 when the gamepad points straight ahead (-Z), positive to the left. + float yaw{}; + /// How far the hands point above level, in radians: measured, or `assumed_pitch` where + /// the headset cannot tell. + float pitch{}; + bool pitch_measured{}; + /// The complete attitude: -Z is the way the gamepad's front points, +Y out of its face. + Quat orientation; +}; + +inline Vec Sub(const Vec& a, const Vec& b) { + return {a.x - b.x, a.y - b.y, a.z - b.z}; +} + +inline float Dot(const Vec& a, const Vec& b) { + return a.x * b.x + a.y * b.y + a.z * b.z; +} + +inline Vec Cross(const Vec& a, const Vec& b) { + return {a.y * b.z - a.z * b.y, a.z * b.x - a.x * b.z, a.x * b.y - a.y * b.x}; +} + +inline float Length(const Vec& a) { + return std::sqrt(Dot(a, a)); +} + +inline Vec Scale(const Vec& a, float by) { + return {a.x * by, a.y * by, a.z * by}; +} + +inline Vec Rotate(const Quat& q, const Vec& v) { + const Vec u{q.x, q.y, q.z}; + const Vec t = Scale(Cross(u, v), 2.0f); + const Vec s = Cross(u, t); + return {v.x + q.w * t.x + s.x, v.y + q.w * t.y + s.y, v.z + q.w * t.z + s.z}; +} + +/// The rotation that takes +X to `right`, +Y to `up` and +Z to `back`: three directions of +/// length one, each at a right angle to the others. +inline Quat FromAxes(const Vec& right, const Vec& up, const Vec& back) { + const float trace = right.x + up.y + back.z; + Quat q; + if (trace > 0.0f) { + const float s = std::sqrt(trace + 1.0f) * 2.0f; + q = {(up.z - back.y) / s, (back.x - right.z) / s, (right.y - up.x) / s, 0.25f * s}; + } else if (right.x > up.y && right.x > back.z) { + const float s = std::sqrt(1.0f + right.x - up.y - back.z) * 2.0f; + q = {0.25f * s, (up.x + right.y) / s, (back.x + right.z) / s, (up.z - back.y) / s}; + } else if (up.y > back.z) { + const float s = std::sqrt(1.0f + up.y - right.x - back.z) * 2.0f; + q = {(up.x + right.y) / s, 0.25f * s, (back.y + up.z) / s, (back.x - right.z) / s}; + } else { + const float s = std::sqrt(1.0f + back.z - right.x - up.y) * 2.0f; + q = {(back.x + right.z) / s, (back.y + up.z) / s, 0.25f * s, (right.y - up.x) / s}; + } + const float length = std::sqrt(q.x * q.x + q.y * q.y + q.z * q.z + q.w * q.w); + return {q.x / length, q.y / length, q.z / length, q.w / length}; +} + +/// Part of the way from one attitude to another: `blend` of it, 0 to 1. +inline Quat Towards(const Quat& from, const Quat& to, float blend) { + // (The same attitude has two quaternions: the nearer one.) + const float side = from.x * to.x + from.y * to.y + from.z * to.z + from.w * to.w < 0.0f + ? -1.0f + : 1.0f; + Quat q{from.x + (to.x * side - from.x) * blend, from.y + (to.y * side - from.y) * blend, + from.z + (to.z * side - from.z) * blend, from.w + (to.w * side - from.w) * blend}; + const float length = std::sqrt(q.x * q.x + q.y * q.y + q.z * q.z + q.w * q.w); + if (!(length > 1e-6f)) { + return to; + } + return {q.x / length, q.y / length, q.z / length, q.w / length}; +} + +/// Hands further apart than a gamepad is wide, or closer together, are not holding one: +/// nothing then. `tilt` is added to how far the hands point up, for where a gamepad's front +/// points by more or less than the hands around it do; `assumed_pitch` is how far up the +/// hands are taken to point while the headset cannot tell. +inline std::optional Locate(const Hands& hands, float tilt, float assumed_pitch) { + const Vec across = Sub(hands.right_palm, hands.left_palm); + const float distance = Length(across); + if (!(distance > 0.05f && distance < 0.32f)) { + return std::nullopt; + } + Pose pose; + pose.position = {(hands.left_palm.x + hands.right_palm.x) * 0.5f, + (hands.left_palm.y + hands.right_palm.y) * 0.5f, + (hands.left_palm.z + hands.right_palm.z) * 0.5f}; + pose.pitch = assumed_pitch; + const float level = std::hypot(across.x, across.z); + if (level > 0.6f * distance) { + // The line from the left to the right palm is the gamepad's sideways axis; its + // heading follows from that. + pose.turned_known = true; + pose.yaw = std::atan2(-across.z, across.x); + // The palms close around the grips, which sit behind and below the middle of the + // gamepad. + pose.position.x += -std::sin(pose.yaw) * 0.035f; + pose.position.z += -std::cos(pose.yaw) * 0.035f; + + const Vec right = Scale(across, 1.0f / distance); + // Level and at a right angle to that line: where the front points if it is neither + // tilted up nor down. What is up for a gamepad held like that goes with it. + const Vec level_front_long = Cross({0.0f, 1.0f, 0.0f}, right); + const Vec level_front = Scale(level_front_long, 1.0f / Length(level_front_long)); + const Vec level_up = Cross(right, level_front); + if (hands.pointing_known) { + // Both hands point inwards as much as forwards: what they agree on, without + // what goes along the line between them, is the way the gamepad points. + Vec pointing{hands.left_pointing.x + hands.right_pointing.x, + hands.left_pointing.y + hands.right_pointing.y, + hands.left_pointing.z + hands.right_pointing.z}; + pointing = Sub(pointing, Scale(right, Dot(pointing, right))); + if (Length(pointing) > 0.5f) { + pose.pitch = std::atan2(Dot(pointing, level_up), Dot(pointing, level_front)); + pose.pitch_measured = true; + } + } + const float pitch = pose.pitch + tilt; + const Vec front{level_front.x * std::cos(pitch) + level_up.x * std::sin(pitch), + level_front.y * std::cos(pitch) + level_up.y * std::sin(pitch), + level_front.z * std::cos(pitch) + level_up.z * std::sin(pitch)}; + pose.orientation = FromAxes(right, Cross(right, front), Scale(front, -1.0f)); + } + pose.position.y += 0.015f; + return pose; +} + +} // namespace HeldPad diff --git a/quest-host/cpp/main.cpp b/quest-host/cpp/main.cpp index 4150d69..02f0d08 100644 --- a/quest-host/cpp/main.cpp +++ b/quest-host/cpp/main.cpp @@ -18,6 +18,7 @@ #include "core_process.h" #include "log.h" +#include "pad_router.h" #include "xr_host.h" namespace { @@ -31,8 +32,9 @@ struct App { std::thread xr_thread; std::atomic quit{}; std::atomic recenter_requests{}; - std::mutex pad_mutex; - PadState pad; + /// What the game is played with: a gamepad the activity hears of, the headset's own + /// controllers the session reads, or both. + PadRouter pads; }; std::unique_ptr g_app; @@ -65,13 +67,20 @@ extern "C" { JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeStartXr( JNIEnv* env, jobject activity, jfloat refresh_rate, jint eye_width, jint eye_height, jboolean track_hands, jint sharpen, jboolean cubic, jboolean show_stats, jfloat predict_ms, - jint dynamic_resolution, jboolean cpu_boost) { + jint dynamic_resolution, jboolean cpu_boost, jboolean own_controllers, jint pad_hand, + jfloat pad_tilt, jboolean rumble, jboolean stick_touchpad) { if (g_app) { return; } g_app = std::make_unique(); env->GetJavaVM(&g_app->vm); g_app->activity = env->NewGlobalRef(activity); + g_app->pads.SetStickTouchpad(stick_touchpad == JNI_TRUE); + g_app->pads.SetNotice([](const char* text) { LOGI("%s", text); }); + g_app->pads.SetSink([app = g_app.get()](const PadRouter::Output& output) { + app->core.SetPad(output.pad); + app->core.SetBlowing(output.blowing); + }); XrHostOptions options; options.refresh_rate = refresh_rate; @@ -85,6 +94,10 @@ JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeStartXr( options.dynamic_resolution = dynamic_resolution != 0; options.follow_layer_size = dynamic_resolution == 2; options.cpu_boost = cpu_boost == JNI_TRUE; + options.own_controllers = own_controllers == JNI_TRUE; + options.pad_hand = pad_hand == 0 ? 0 : 1; + options.pad_tilt = pad_tilt; + options.rumble = rumble == JNI_TRUE; g_app->xr_thread = std::thread{[options] { App& app = *g_app; JNIEnv* thread_env = nullptr; @@ -92,8 +105,8 @@ JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeStartXr( // This thread has little to do, but it has to do it every refresh: it must not wait // for the emulator's threads, which outnumber the processor cores. ::setpriority(PRIO_PROCESS, 0, -16); - RunXrHost(app.vm, app.activity, app.core, app.status, options, app.xr_status, app.quit, - app.recenter_requests); + RunXrHost(app.vm, app.activity, app.core, app.pads, app.status, options, app.xr_status, + app.quit, app.recenter_requests); LOGI("XR thread finished"); if (!app.quit) { // The system ended the session (the user quit from its menu), or there never was @@ -204,25 +217,28 @@ JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeSetStatu ++g_app->status.version; } +/// What the gamepad says: buttons in the PS4 pad's bit layout, sticks from -1 to 1 (to the +/// right and towards the player), triggers from 0 to 1, and the finger on its own touchpad +/// if it has one (0 to 1 from the pad's left and from its far edge). JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeInput( - JNIEnv*, jobject, jint buttons, jint left_x, jint left_y, jint right_x, jint right_y, - jint left_trigger, jint right_trigger, jboolean touch_down, jint touch_x, jint touch_y) { + JNIEnv*, jobject, jint buttons, jfloat left_x, jfloat left_y, jfloat right_x, jfloat right_y, + jfloat left_trigger, jfloat right_trigger, jboolean touch_down, jfloat touch_x, + jfloat touch_y) { if (!g_app) { return; } - std::scoped_lock lock{g_app->pad_mutex}; - PadState& pad = g_app->pad; - pad.buttons = static_cast(buttons); - pad.left_x = static_cast(left_x); - pad.left_y = static_cast(left_y); - pad.right_x = static_cast(right_x); - pad.right_y = static_cast(right_y); - pad.left_trigger = static_cast(left_trigger); - pad.right_trigger = static_cast(right_trigger); - pad.touch_down = touch_down == JNI_TRUE; - pad.touch_x = static_cast(touch_x); - pad.touch_y = static_cast(touch_y); - g_app->core.SetPad(pad); + GamepadState state; + state.buttons = static_cast(buttons); + state.left_x = left_x; + state.left_y = left_y; + state.right_x = right_x; + state.right_y = right_y; + state.left_trigger = left_trigger; + state.right_trigger = right_trigger; + state.touch_down = touch_down == JNI_TRUE; + state.touch_x = touch_x; + state.touch_y = touch_y; + g_app->pads.SetGamepad(state, PadRouter::Now()); } /// Gyroscope in rad/s and accelerometer in m/s², in the controller's frame. @@ -231,16 +247,52 @@ JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeMotion( if (!g_app) { return; } - std::scoped_lock lock{g_app->pad_mutex}; - PadState& pad = g_app->pad; - pad.has_motion = true; - pad.gyro[0] = gx; - pad.gyro[1] = gy; - pad.gyro[2] = gz; - pad.accel[0] = ax; - pad.accel[1] = ay; - pad.accel[2] = az; - g_app->core.SetPad(pad); + const float gyro[3]{gx, gy, gz}; + const float accel[3]{ax, ay, az}; + g_app->pads.SetGamepadMotion(gyro, accel, PadRouter::Now()); +} + +/// A gamepad is there to play with (again, or another one than before), or none is any more. +JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeSetGamepad( + JNIEnv*, jobject, jboolean connected) { + if (g_app) { + g_app->pads.SetGamepadConnected(connected == JNI_TRUE, PadRouter::Now()); + } +} + +/// The host's own menu is over (or not yet): what the player presses is the game's. +JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeSetPlaying( + JNIEnv*, jobject, jboolean playing) { + if (g_app) { + g_app->pads.SetPlaying(playing == JNI_TRUE, PadRouter::Now()); + } +} + +/// What the game is played with: 0 nothing, 1 the gamepad, 2 the headset's own controllers. +/// 4 is added while those are in the player's hands. +JNIEXPORT jint JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeController(JNIEnv*, + jobject) { + if (!g_app) { + return 0; + } + const PadRouter::View view = g_app->pads.Look(); + return static_cast(view.source) | (view.touch_present ? 4 : 0); +} + +/// The player chose with it in the host's menu: 1 the gamepad, 2 the headset's controllers. +JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeUseController( + JNIEnv*, jobject, jint which) { + if (g_app) { + g_app->pads.Use(which == 2 ? PadRouter::Source::Touch : PadRouter::Source::Gamepad, + PadRouter::Now()); + } +} + +/// What the headset's own controllers say to the host's menu: 1 a stick held to the left, 2 +/// to the right, 4 what confirms (A or a trigger). +JNIEXPORT jint JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeTouchMenu(JNIEnv*, + jobject) { + return g_app ? static_cast(g_app->pads.TouchMenu()) : 0; } /// What the game asks of the controller: see CoreProcess::GetPadFeedback for the layout. @@ -277,6 +329,7 @@ JNIEXPORT void JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeLog(JNIE } /// Bit 0: the headset session is running. Bit 1: the hands holding the controller are seen. +/// Bit 2: the one of the headset's own controllers that is the controller in the game is. /// Bits 8 to 15: the display's refresh rate in Hz. JNIEXPORT jint JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeXrStatus(JNIEnv*, jobject) { @@ -286,7 +339,7 @@ JNIEXPORT jint JNICALL Java_com_astrobotquest_vrhost_MainActivity_nativeXrStatus const int refresh_rate = std::clamp(static_cast(g_app->xr_status.refresh_rate.load() + 0.5f), 0, 255); return (g_app->xr_status.session_running ? 1 : 0) | (g_app->xr_status.hands_tracked ? 2 : 0) | - (refresh_rate << 8); + (g_app->xr_status.controller_tracked ? 4 : 0) | (refresh_rate << 8); } } // extern "C" diff --git a/quest-host/cpp/pad_router.h b/quest-host/cpp/pad_router.h new file mode 100644 index 0000000..f773aa8 --- /dev/null +++ b/quest-host/cpp/pad_router.h @@ -0,0 +1,487 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include "pad_state.h" +// From the emulator (src/input): the right stick as a finger on the touchpad. +#include "stick_finger.h" + +/// PS4 pad button bits (ORBIS_PAD_BUTTON_*). +namespace PadButton { +inline constexpr uint32_t L3 = 0x2, R3 = 0x4, Options = 0x8; +inline constexpr uint32_t Up = 0x10, Right = 0x20, Down = 0x40, Left = 0x80; +inline constexpr uint32_t L2 = 0x100, R2 = 0x200, L1 = 0x400, R1 = 0x800; +inline constexpr uint32_t Triangle = 0x1000, Circle = 0x2000, Cross = 0x4000, Square = 0x8000; +inline constexpr uint32_t TouchPad = 0x100000; +} // namespace PadButton + +/// What a gamepad says, as the system reports it. Buttons use the PS4 pad bit layout; sticks +/// go from -1 to 1, to the right and towards the player; triggers from 0 to 1. +struct GamepadState { + uint32_t buttons{}; + float left_x{}, left_y{}, right_x{}, right_y{}; + float left_trigger{}, right_trigger{}; + /// A finger on the gamepad's own touchpad, 0 to 1 from its left and from its far edge. + bool touch_down{}; + float touch_x{0.5f}, touch_y{0.5f}; +}; + +/// What the headset's own two controllers say, as its runtime reports them: sticks from -1 to +/// 1, to the right and away from the player (the other way up than a gamepad's). +struct TouchState { + /// They are in the player's hands, and the game is what the player has in front of them. + bool present{}; + bool a{}, b{}, x{}, y{}, menu{}; + bool left_stick_in{}, right_stick_in{}; + float left_grip{}, right_grip{}; + float left_trigger{}, right_trigger{}; + float left_x{}, left_y{}, right_x{}, right_y{}; +}; + +/// Makes the one DualShock 4 the game is played with out of what the player has: a gamepad +/// paired with the headset, the headset's own two controllers, or both. +/// +/// The headset's controllers are played like this: the left stick is the left stick; the +/// right one is a finger on the touchpad (as it is for gamepads without one) and pressing it +/// in presses the touchpad; A and B, under the right thumb, are cross and square, X and Y +/// circle and triangle, and X and Y together blow into the microphone; the triggers are L2 +/// and R2, the grips L1 and R1, the left controller's menu button is OPTIONS. Both sticks +/// pressed in reset the view. +/// +/// With both a gamepad and the headset's controllers at hand, the game is played with the +/// one that was used last: a button on the other takes over. A gamepad is taken over from +/// only by what a hand cannot do without holding a controller (a button under a thumb, a +/// stick), since it is hands the headset sees while a gamepad is held. +class PadRouter { +public: + enum class Source { None, Gamepad, Touch }; + + /// Bits of what TakeRecenter returns (those of XrRecenter). + static constexpr uint32_t RecenterPad = 1; + static constexpr uint32_t RecenterSeat = 2; + /// Bits of what TouchMenu returns. + static constexpr uint32_t MenuLeft = 1; + static constexpr uint32_t MenuRight = 2; + static constexpr uint32_t MenuConfirm = 4; + + /// How long OPTIONS is held for the view to be reset, in seconds: as on a PlayStation VR. + static constexpr double OptionsHold = 1.0; + + struct Output { + PadState pad; + /// The player blows with buttons: the game is to hear it. + bool blowing{}; + + bool operator==(const Output&) const = default; + }; + + /// What the rest of the host needs to know to place the controller and to answer it. + struct View { + Source source{Source::None}; + bool playing{}; + bool gamepad_connected{}; + /// The gamepad's own motion sensors say how it is held. + bool gamepad_motion{}; + float gamepad_accel[3]{}; + bool touch_present{}; + }; + + /// The time everything here is told, in seconds: the same clock for every caller. + static double Now() { + return std::chrono::duration( + std::chrono::steady_clock::now().time_since_epoch()) + .count(); + } + + /// Both are called with the router's lock held: they must not call back into it. + void SetSink(std::function sink_) { + std::scoped_lock lock{mutex}; + sink = std::move(sink_); + } + void SetNotice(std::function notice_) { + std::scoped_lock lock{mutex}; + notice = std::move(notice_); + } + + /// Whether the right stick moves a finger over the touchpad. + void SetStickTouchpad(bool enabled) { + std::scoped_lock lock{mutex}; + stick_touchpad = enabled; + } + + /// The game runs, and what the player presses is meant for it. Until then the controls + /// belong to the host's own menu, and nothing of them reaches the game: neither does + /// what is still held down from there, until it is let go. + void SetPlaying(bool playing_, double now) { + std::scoped_lock lock{mutex}; + if (playing_ && !playing) { + playing = true; + held_back = 0; + held_back = Compose(now, false).pad.buttons; + } + playing = playing_; + Publish(now); + } + + /// A gamepad is there (again, or another one than before), or the last one is gone. + void SetGamepadConnected(bool connected, double now) { + std::scoped_lock lock{mutex}; + gamepad_connected = connected; + // Its motion sensors, if it has any, speak up within milliseconds. + gamepad_motion = false; + if (!connected) { + gamepad = {}; + } + Arbitrate(false, false); + Publish(now); + } + + void SetGamepad(const GamepadState& state, double now) { + std::scoped_lock lock{mutex}; + const bool used = (state.buttons & ~gamepad.buttons) != 0 || + (state.touch_down && !gamepad.touch_down) || + Pushed(state.left_x, state.left_y, gamepad.left_x, gamepad.left_y) || + Pushed(state.right_x, state.right_y, gamepad.right_x, gamepad.right_y) || + Pulled(state.left_trigger, gamepad.left_trigger) || + Pulled(state.right_trigger, gamepad.right_trigger); + gamepad = state; + Arbitrate(used, false); + Publish(now); + } + + /// Gyroscope in rad/s and accelerometer in m/s², in the gamepad's own frame. + void SetGamepadMotion(const float gyro[3], const float accel[3], double now) { + std::scoped_lock lock{mutex}; + gamepad_motion = true; + std::copy(gyro, gyro + 3, gamepad_gyro); + std::copy(accel, accel + 3, gamepad_accel); + Publish(now); + } + + /// To be called for every picture the headset shows: it is also what keeps time for + /// buttons that are held and for the stick's finger. + void SetTouch(const TouchState& state, double now) { + std::scoped_lock lock{mutex}; + const TouchState next = state.present ? state : TouchState{}; + // (Triggers and grips are left out: a pinch of the fingers must never count.) + const bool used = + (next.a && !touch.a) || (next.b && !touch.b) || (next.x && !touch.x) || + (next.y && !touch.y) || (next.left_stick_in && !touch.left_stick_in) || + (next.right_stick_in && !touch.right_stick_in) || + Pushed(next.left_x, next.left_y, touch.left_x, touch.left_y) || + Pushed(next.right_x, next.right_y, touch.right_x, touch.right_y); + touch = next; + Arbitrate(false, used); + Publish(now); + } + + /// Without the headset's controllers: time has passed. + void Tick(double now) { + std::scoped_lock lock{mutex}; + Publish(now); + } + + /// The player chose with this one in the host's menu: it is what they hold. + void Use(Source wanted, double now) { + std::scoped_lock lock{mutex}; + Arbitrate(wanted == Source::Gamepad, wanted == Source::Touch); + Publish(now); + } + + View Look() { + std::scoped_lock lock{mutex}; + View view; + view.source = source; + view.playing = playing; + view.gamepad_connected = gamepad_connected; + view.gamepad_motion = gamepad_motion; + std::copy(gamepad_accel, gamepad_accel + 3, view.gamepad_accel); + view.touch_present = touch.present; + return view; + } + + /// The resets of the view the player has asked for with buttons since the last call. + uint32_t TakeRecenter() { + std::scoped_lock lock{mutex}; + return std::exchange(recenter, 0u); + } + + /// What the headset's controllers say to the host's own menu: a stick held to a side, or + /// what confirms (A, or a trigger as everywhere else in the headset). + uint32_t TouchMenu() { + std::scoped_lock lock{mutex}; + if (!touch.present) { + return 0; + } + const float x = std::abs(touch.left_x) >= std::abs(touch.right_x) ? touch.left_x + : touch.right_x; + return (x < -0.55f ? MenuLeft : 0) | (x > 0.55f ? MenuRight : 0) | + (touch.a || touch.left_trigger > 0.6f || touch.right_trigger > 0.6f ? MenuConfirm + : 0); + } + +private: + /// A stick that was near its centre and is well away from it now. + static bool Pushed(float x, float y, float before_x, float before_y) { + return std::hypot(x, y) > 0.6f && std::hypot(before_x, before_y) <= 0.6f; + } + + static bool Pulled(float value, float before) { + return value > 0.5f && before <= 0.5f; + } + + void Note(const char* text) { + if (notice) { + notice(text); + } + } + + /// Decides which of the two the game is played with. + void Arbitrate(bool gamepad_used, bool touch_used) { + Source wanted = source; + if (gamepad_used && gamepad_connected) { + wanted = Source::Gamepad; + } + if (touch_used && touch.present) { + wanted = Source::Touch; + } + if (wanted == Source::Gamepad && !gamepad_connected) { + wanted = Source::None; + } + if (wanted == Source::None) { + // Nothing was chosen: a gamepad if there is one, as before the headset's own + // controllers could play; else those, as soon as they are picked up. + wanted = gamepad_connected ? Source::Gamepad + : touch.present ? Source::Touch : Source::None; + } + if (wanted == source) { + return; + } + source = wanted; + // What the other one's stick was doing on the touchpad is over. + stick_finger.Reset(); + options_since = -1.0; + options_fired = false; + view_reset_held = false; + Note(source == Source::Touch + ? "the game is played with the headset's own controllers now" + : source == Source::Gamepad ? "the game is played with the gamepad now" + : "there is no controller to play the game with"); + } + + static uint8_t Stick(float value) { + return static_cast( + std::clamp(static_cast(std::lround(128.0f + value * 127.0f)), 0, 255)); + } + + static uint8_t Trigger(float value) { + return static_cast( + std::lround(std::clamp(value, 0.0f, 1.0f) * 255.0f)); + } + + /// What the game is given now. With `act`, what the buttons mean besides (resets of the + /// view) is acted on as well. + Output Compose(double now, bool act) { + Output out; + if (!playing) { + return out; + } + uint32_t buttons = 0; + float left_x = 0.0f, left_y = 0.0f, right_x = 0.0f, right_y = 0.0f; + float left_trigger = 0.0f, right_trigger = 0.0f; + bool real_finger = false; + float touch_x = 0.5f, touch_y = 0.5f; + bool view_reset = false; + bool blow = false; + + if (source == Source::Touch) { + const TouchState& t = touch; + const auto add = [&](bool down, uint32_t button) { + if (down) { + buttons |= button; + } + }; + add(t.a, PadButton::Cross); + add(t.b, PadButton::Square); + // The left controller's two buttons together are not circle and triangle: they + // blow into the microphone, for as long as they are held. + blow = t.x && t.y; + add(t.x && !blow, PadButton::Circle); + add(t.y && !blow, PadButton::Triangle); + add(t.menu, PadButton::Options); + add(t.left_grip > 0.5f, PadButton::L1); + add(t.right_grip > 0.5f, PadButton::R1); + // Both sticks pressed in is not something a game is played with: it resets the + // view, for which a gamepad has its PS button. + view_reset = t.left_stick_in && t.right_stick_in; + add(t.left_stick_in && !view_reset, PadButton::L3); + add(t.right_stick_in && !view_reset, PadButton::TouchPad); + // (A trigger that is not quite at rest is not pulled: any pull counts as the + // button.) + left_trigger = t.left_trigger < 0.12f ? 0.0f : t.left_trigger; + right_trigger = t.right_trigger < 0.12f ? 0.0f : t.right_trigger; + add(left_trigger > 0.0f, PadButton::L2); + add(right_trigger > 0.0f, PadButton::R2); + left_x = t.left_x; + left_y = -t.left_y; + right_x = t.right_x; + right_y = -t.right_y; + } else { + const GamepadState& g = gamepad; + buttons = g.buttons; + left_x = g.left_x; + left_y = g.left_y; + right_x = g.right_x; + right_y = g.right_y; + left_trigger = g.left_trigger; + right_trigger = g.right_trigger; + real_finger = g.touch_down; + touch_x = g.touch_x; + touch_y = g.touch_y; + if (gamepad_motion) { + out.pad.has_motion = true; + std::copy(gamepad_gyro, gamepad_gyro + 3, out.pad.gyro); + std::copy(gamepad_accel, gamepad_accel + 3, out.pad.accel); + } + } + + // What was held down when the game got the controls is not the game's. + held_back &= buttons; + buttons &= ~held_back; + if ((held_back & PadButton::L2) != 0) { + left_trigger = 0.0f; + } + if ((held_back & PadButton::R2) != 0) { + right_trigger = 0.0f; + } + + // This title has no use for the right stick, while several of its gadgets want + // swipes: the stick moves a finger over the touchpad (see Input::StickFinger: a + // stick let go lifts the finger where it was, which is what lets a pull on the stick + // shoot the game's catapults). + bool touching = real_finger; + if (real_finger) { + // A real finger is on the pad: the stick's has nothing to say, and is not back + // on the pad the moment the real one lifts either. + stick_finger.Reset(); + } else if (stick_touchpad) { + const Input::StickFinger::Touch dragged = stick_finger.Update(now, right_x, right_y); + if (dragged.down) { + touching = true; + touch_x = dragged.x; + touch_y = dragged.y; + } + } + if (source == Source::Touch && stick_touchpad) { + right_x = 0.0f; + right_y = 0.0f; + } + if (!touching && (buttons & PadButton::TouchPad) != 0) { + // A touchpad cannot be pressed without touching it: in its middle, where a + // button or a stick pressed in stands in for the press. + touching = true; + touch_x = 0.5f; + touch_y = 0.5f; + } + if (touching) { + sent_touch_x = static_cast( + std::clamp(static_cast(std::lround(touch_x * (TouchWidth - 1))), 0, + TouchWidth - 1)); + sent_touch_y = static_cast( + std::clamp(static_cast(std::lround(touch_y * (TouchHeight - 1))), 0, + TouchHeight - 1)); + } + + out.pad.buttons = buttons; + out.pad.left_x = Stick(left_x); + out.pad.left_y = Stick(left_y); + out.pad.right_x = Stick(right_x); + out.pad.right_y = Stick(right_y); + out.pad.left_trigger = Trigger(left_trigger); + out.pad.right_trigger = Trigger(right_trigger); + out.pad.touch_down = touching; + // (A finger that lifts does so from where it was.) + out.pad.touch_x = sent_touch_x; + out.pad.touch_y = sent_touch_y; + out.blowing = blow; + + if (!act) { + return out; + } + if ((buttons & PadButton::Cross) != 0 && !seat_taken) { + // The first press of X is the player settled in, controller in hand, looking at + // the game: that, and not where they were when the app started, is their seat. + seat_taken = true; + recenter |= RecenterSeat; + Note("first press of X: the player's seat is where they are now"); + } + if ((buttons & PadButton::Options) == 0) { + options_since = -1.0; + options_fired = false; + } else if (options_since < 0.0) { + options_since = now; + } else if (!options_fired && now - options_since >= OptionsHold) { + // What a PlayStation VR does when OPTIONS is held, and what the game tells its + // players to do when the view is off. The game still sees the button. + options_fired = true; + recenter |= RecenterPad | RecenterSeat; + Note("OPTIONS held: view reset"); + } + if (view_reset && !view_reset_held) { + recenter |= RecenterPad | RecenterSeat; + Note("both sticks pressed in: view reset"); + } + view_reset_held = view_reset; + return out; + } + + void Publish(double now) { + const Output out = Compose(now, true); + if (published && out == last) { + return; + } + published = true; + last = out; + if (sink) { + sink(out); + } + } + + // The touchpad as the emulator takes it: the resolution of a DualSense's. + static constexpr int TouchWidth = 1920; + static constexpr int TouchHeight = 1080; + + std::mutex mutex; + std::function sink; + std::function notice; + bool stick_touchpad{true}; + + bool playing{}; + Source source{Source::None}; + bool gamepad_connected{}; + GamepadState gamepad; + bool gamepad_motion{}; + float gamepad_gyro[3]{}; + float gamepad_accel[3]{0.0f, 9.81f, 0.0f}; + TouchState touch; + + Input::StickFinger stick_finger; + uint32_t held_back{}; + uint16_t sent_touch_x{TouchWidth / 2}; + uint16_t sent_touch_y{TouchHeight / 2}; + bool seat_taken{}; + double options_since{-1.0}; + bool options_fired{}; + bool view_reset_held{}; + uint32_t recenter{}; + + bool published{}; + Output last; +}; diff --git a/quest-host/cpp/pad_state.h b/quest-host/cpp/pad_state.h new file mode 100644 index 0000000..fe3146e --- /dev/null +++ b/quest-host/cpp/pad_state.h @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#pragma once + +#include + +/// What the emulated DualShock 4 looks like right now. Buttons use the PS4 pad bit layout. +struct PadState { + uint32_t buttons{}; + uint8_t left_x{128}, left_y{128}, right_x{128}, right_y{128}; + uint8_t left_trigger{}, right_trigger{}; + bool touch_down{}; + uint16_t touch_x{}, touch_y{}; + bool has_motion{}; + float gyro[3]{}; + float accel[3]{0.0f, 9.81f, 0.0f}; + + bool operator==(const PadState&) const = default; +}; diff --git a/quest-host/cpp/self_test.cpp b/quest-host/cpp/self_test.cpp index 7a4e0db..729a89e 100644 --- a/quest-host/cpp/self_test.cpp +++ b/quest-host/cpp/self_test.cpp @@ -21,6 +21,7 @@ #include "core_process.h" #include "gl_frames.h" +#include "held_pad.h" #include "log.h" namespace Protocol = Core::Vr::Protocol; @@ -352,6 +353,48 @@ std::string RunSelfTest(const CoreLaunch& launch, const std::string& out_dir, in note("the head follows %zu looks", looks.size()); } + // HOST_PAD=":,,,,,,;..." among the + // settings: what the host says of the controller from when on, the way a session does. + // Where it is in metres from the head (which this test keeps where it starts), how it is + // turned in degrees (to the left, its front up, its right side up). The kinds: + // c one of the headset's own controllers: it says everything of itself + // h hands seen around a gamepad without motion sensors: they say where it is and, as + // far as hands can, how it is turned (HeldPad) + // p hands seen around a gamepad that has motion sensors: only where it is, and its + // heading for the sensors to hold on to + // n nothing is seen of it + struct PadStep { + float from; + char kind; + float right, up, ahead; + float yaw, pitch, roll; + }; + std::vector pad_steps; + float shot_interval = 10.0f; + for (const auto& [name, value] : launch.extra_env) { + if (name == "HOST_SHOTS") { + // A picture every so many seconds instead of every ten. + shot_interval = std::max(1.0f, std::strtof(value.c_str(), nullptr)); + } + if (name != "HOST_PAD") { + continue; + } + size_t at = 0; + while (at < value.size()) { + const size_t end = std::min(value.find(';', at), value.size()); + PadStep step{}; + if (std::sscanf(value.substr(at, end - at).c_str(), "%f:%c,%f,%f,%f,%f,%f,%f", + &step.from, &step.kind, &step.right, &step.up, &step.ahead, &step.yaw, + &step.pitch, &step.roll) == 8) { + pad_steps.push_back(step); + } + at = end + 1; + } + note("the controller follows %zu steps", pad_steps.size()); + } + const PadStep* pad_step = nullptr; + bool pad_seen = false; + const auto start = Clock::now(); const auto seconds_since_start = [&] { return std::chrono::duration(Clock::now() - start).count(); @@ -416,6 +459,76 @@ std::string RunSelfTest(const CoreLaunch& launch, const std::string& out_dir, in note("view reset asked for with the head turned %.0f degrees", yaw * 57.29578f); } + // The controller, as a session would report it. + for (const PadStep& step : pad_steps) { + if (now >= step.from && (pad_step == nullptr || step.from > pad_step->from)) { + pad_step = &step; + note("%.0f s: the controller is '%c', %.2f right, %.2f up, %.2f ahead, turned " + "%.0f left, %.0f up, rolled %.0f", + now, step.kind, step.right, step.up, step.ahead, step.yaw, step.pitch, + step.roll); + } + } + if (pad_step != nullptr) { + const PadStep& step = *pad_step; + const float turn = step.yaw / 57.29578f; + const float tilt = step.pitch / 57.29578f; + const float roll = step.roll / 57.29578f; + const HeldPad::Vec right{std::cos(turn) * std::cos(roll), std::sin(roll), + -std::sin(turn) * std::cos(roll)}; + const HeldPad::Vec level_front{-std::sin(turn), 0.0f, -std::cos(turn)}; + const HeldPad::Vec level_up = HeldPad::Cross(right, level_front); + const HeldPad::Vec front{ + level_front.x * std::cos(tilt) + level_up.x * std::sin(tilt), + level_front.y * std::cos(tilt) + level_up.y * std::sin(tilt), + level_front.z * std::cos(tilt) + level_up.z * std::sin(tilt)}; + const HeldPad::Vec middle{step.right, step.up, -step.ahead}; + Protocol::PadPose message; + const auto place = [&](const HeldPad::Vec& at) { + message.flags |= Protocol::PadPose::PositionValid; + message.position[0] = at.x; + message.position[1] = at.y; + message.position[2] = at.z; + }; + const auto turned = [&](const HeldPad::Quat& q) { + message.flags |= Protocol::PadPose::OrientationValid; + message.orientation[0] = q.x; + message.orientation[1] = q.y; + message.orientation[2] = q.z; + message.orientation[3] = q.w; + }; + if (step.kind == 'c') { + place(middle); + turned(HeldPad::FromAxes(right, HeldPad::Cross(right, front), + HeldPad::Scale(front, -1.0f))); + } else if (step.kind == 'h' || step.kind == 'p') { + // Two palms a gamepad's width apart, both hands pointing the way it does. + HeldPad::Hands hands; + hands.left_palm = HeldPad::Sub(middle, HeldPad::Scale(right, 0.1f)); + hands.right_palm = HeldPad::Sub(middle, HeldPad::Scale(right, -0.1f)); + hands.pointing_known = true; + hands.left_pointing = front; + hands.right_pointing = front; + if (const auto held = HeldPad::Locate(hands, 0.0f, 0.35f)) { + place(held->position); + if (held->turned_known) { + message.yaw = held->yaw; + message.flags |= Protocol::PadPose::YawValid; + if (step.kind == 'h') { + turned(held->orientation); + message.flags |= Protocol::PadPose::HeldInHands; + } + } + } + } + // While it is seen, every time; once more when it no longer is. + const bool seen = message.flags != 0; + if (seen || pad_seen) { + core.SendPadPose(message); + } + pad_seen = seen; + } + // Cross for a quarter of every five seconds gets a title past its menus. PadState pad; pad.buttons = std::fmod(now, 5.0f) < 0.25f && now > 20.0f ? 0x4000u : 0u; @@ -471,7 +584,7 @@ std::string RunSelfTest(const CoreLaunch& launch, const std::string& out_dir, in WritePng(path, copy, width, height); }}; ++saved; - next_save_at = now + 10.0f; + next_save_at = now + shot_interval; } } if (now - interval_start >= 10.0f) { diff --git a/quest-host/cpp/xr_host.cpp b/quest-host/cpp/xr_host.cpp index 005826e..c1c51df 100644 --- a/quest-host/cpp/xr_host.cpp +++ b/quest-host/cpp/xr_host.cpp @@ -26,10 +26,17 @@ #include #include "core_process.h" +#include "held_pad.h" #include "log.h" +#include "pad_router.h" namespace Protocol = Core::Vr::Protocol; +static_assert(PadRouter::RecenterPad == XrRecenter::Pad && + PadRouter::RecenterSeat == XrRecenter::Seat); +static_assert(static_cast(PadRouter::Source::Gamepad) == 1 && + static_cast(PadRouter::Source::Touch) == 2); + namespace { constexpr int64_t FormatSrgb8Alpha8 = 0x8C43; // GL_SRGB8_ALPHA8 @@ -97,11 +104,12 @@ struct Swapchain { class Host { public: - Host(JavaVM* vm_, jobject activity_, CoreProcess& core_, StatusImage& status_, - const XrHostOptions& options_, XrHostStatus& host_status_, std::atomic& quit_, - std::atomic& recenter_requests_) - : vm{vm_}, activity{activity_}, core{core_}, status{status_}, options{options_}, - host_status{host_status_}, quit{quit_}, recenter_requests{recenter_requests_} {} + Host(JavaVM* vm_, jobject activity_, CoreProcess& core_, PadRouter& pads_, + StatusImage& status_, const XrHostOptions& options_, XrHostStatus& host_status_, + std::atomic& quit_, std::atomic& recenter_requests_) + : vm{vm_}, activity{activity_}, core{core_}, pads{pads_}, status{status_}, + options{options_}, host_status{host_status_}, quit{quit_}, + recenter_requests{recenter_requests_} {} void Run() { if (gl.Create() && InitOpenXr() && InitRendering()) { @@ -248,6 +256,9 @@ private: if (has_hand_tracking) { CreateHandTrackers(); } + if (options.own_controllers) { + CreateActions(); + } if (has_metrics) { FindMetrics(); } @@ -346,6 +357,256 @@ private: LOGI("hand trackers created"); } + /// The headset's own controllers as the game's gamepad, for a player who holds them + /// instead of one (PadRouter says what does what). Where one of them is and how it points + /// (the right one, unless the settings say the left) is where the controller in the game + /// is. + void CreateActions() { + XrActionSetCreateInfo set_info{XR_TYPE_ACTION_SET_CREATE_INFO}; + std::strcpy(set_info.actionSetName, "gamepad"); + std::strcpy(set_info.localizedActionSetName, "Gamepad"); + if (!Check(xrCreateActionSet(instance, &set_info, &action_set), "xrCreateActionSet")) { + action_set = XR_NULL_HANDLE; + return; + } + xrStringToPath(instance, "/user/hand/left", &hand_paths[0]); + xrStringToPath(instance, "/user/hand/right", &hand_paths[1]); + const auto make = [&](XrAction& action, const char* name, const char* shown, + XrActionType type, bool per_hand = false) { + XrActionCreateInfo info{XR_TYPE_ACTION_CREATE_INFO}; + std::strcpy(info.actionName, name); + std::strcpy(info.localizedActionName, shown); + info.actionType = type; + if (per_hand) { + info.countSubactionPaths = 2; + info.subactionPaths = hand_paths; + } + return XR_SUCCEEDED(xrCreateAction(action_set, &info, &action)); + }; + bool made = make(act_move, "left_stick", "Left stick", XR_ACTION_TYPE_VECTOR2F_INPUT); + made = made && make(act_finger, "touchpad_finger", "Finger on the touchpad", + XR_ACTION_TYPE_VECTOR2F_INPUT); + made = made && make(act_left_in, "l3", "L3", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_right_in, "touchpad_press", "Press the touchpad", + XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_a, "cross", "Cross", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_b, "square", "Square", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_x, "circle", "Circle", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_y, "triangle", "Triangle", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_menu, "options", "Options", XR_ACTION_TYPE_BOOLEAN_INPUT); + made = made && make(act_left_grip, "l1", "L1", XR_ACTION_TYPE_FLOAT_INPUT); + made = made && make(act_right_grip, "r1", "R1", XR_ACTION_TYPE_FLOAT_INPUT); + made = made && make(act_left_trigger, "l2", "L2", XR_ACTION_TYPE_FLOAT_INPUT); + made = made && make(act_right_trigger, "r2", "R2", XR_ACTION_TYPE_FLOAT_INPUT); + made = made && make(act_pose, "controller", "Controller", XR_ACTION_TYPE_POSE_INPUT); + made = made && + make(act_rumble, "rumble", "Rumble", XR_ACTION_TYPE_VIBRATION_OUTPUT, true); + if (!made) { + LOGW("the headset's own controllers could not be set up as a gamepad"); + return; + } + + std::vector bindings; + const auto bind = [&](XrAction action, const char* path) { + XrPath binding = XR_NULL_PATH; + if (XR_SUCCEEDED(xrStringToPath(instance, path, &binding))) { + bindings.push_back({action, binding}); + } + }; + bind(act_move, "/user/hand/left/input/thumbstick"); + bind(act_finger, "/user/hand/right/input/thumbstick"); + bind(act_left_in, "/user/hand/left/input/thumbstick/click"); + bind(act_right_in, "/user/hand/right/input/thumbstick/click"); + bind(act_a, "/user/hand/right/input/a/click"); + bind(act_b, "/user/hand/right/input/b/click"); + bind(act_x, "/user/hand/left/input/x/click"); + bind(act_y, "/user/hand/left/input/y/click"); + bind(act_menu, "/user/hand/left/input/menu/click"); + bind(act_left_grip, "/user/hand/left/input/squeeze/value"); + bind(act_right_grip, "/user/hand/right/input/squeeze/value"); + bind(act_left_trigger, "/user/hand/left/input/trigger/value"); + bind(act_right_trigger, "/user/hand/right/input/trigger/value"); + bind(act_pose, options.pad_hand == 0 ? "/user/hand/left/input/aim/pose" + : "/user/hand/right/input/aim/pose"); + bind(act_rumble, "/user/hand/left/output/haptic"); + bind(act_rumble, "/user/hand/right/output/haptic"); + // (The profile every controller of this make answers to, whatever its generation.) + XrInteractionProfileSuggestedBinding suggested{ + XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING}; + xrStringToPath(instance, "/interaction_profiles/oculus/touch_controller", + &suggested.interactionProfile); + suggested.countSuggestedBindings = static_cast(bindings.size()); + suggested.suggestedBindings = bindings.data(); + if (!Check(xrSuggestInteractionProfileBindings(instance, &suggested), + "xrSuggestInteractionProfileBindings")) { + return; + } + XrSessionActionSetsAttachInfo attach{XR_TYPE_SESSION_ACTION_SETS_ATTACH_INFO}; + attach.countActionSets = 1; + attach.actionSets = &action_set; + if (!Check(xrAttachSessionActionSets(session, &attach), "xrAttachSessionActionSets")) { + return; + } + XrActionSpaceCreateInfo space_info{XR_TYPE_ACTION_SPACE_CREATE_INFO}; + space_info.action = act_pose; + space_info.poseInActionSpace.orientation.w = 1.0f; + if (!Check(xrCreateActionSpace(session, &space_info, &controller_space), + "xrCreateActionSpace")) { + controller_space = XR_NULL_HANDLE; + return; + } + actions_ready = true; + LOGI("the headset's own controllers can play the game: %zu controls bound, the %s one " + "is the controller in the game", + bindings.size(), options.pad_hand == 0 ? "left" : "right"); + } + + /// What the headset's own controllers say right now. Nothing of it counts unless at + /// least one control is live: lying somewhere asleep, they say nothing at all. + void ReadControllers(TouchState& touch) { + XrActionStateGetInfo get{XR_TYPE_ACTION_STATE_GET_INFO}; + bool any_active = false; + const auto pressed = [&](XrAction action) { + get.action = action; + XrActionStateBoolean state{XR_TYPE_ACTION_STATE_BOOLEAN}; + if (XR_FAILED(xrGetActionStateBoolean(session, &get, &state)) || + state.isActive != XR_TRUE) { + return false; + } + any_active = true; + return state.currentState == XR_TRUE; + }; + const auto pulled = [&](XrAction action) { + get.action = action; + XrActionStateFloat state{XR_TYPE_ACTION_STATE_FLOAT}; + if (XR_FAILED(xrGetActionStateFloat(session, &get, &state)) || + state.isActive != XR_TRUE) { + return 0.0f; + } + any_active = true; + return std::clamp(state.currentState, 0.0f, 1.0f); + }; + const auto stick = [&](XrAction action) { + get.action = action; + XrActionStateVector2f state{XR_TYPE_ACTION_STATE_VECTOR2F}; + if (XR_FAILED(xrGetActionStateVector2f(session, &get, &state)) || + state.isActive != XR_TRUE) { + return XrVector2f{}; + } + any_active = true; + return state.currentState; + }; + touch.a = pressed(act_a); + touch.b = pressed(act_b); + touch.x = pressed(act_x); + touch.y = pressed(act_y); + touch.menu = pressed(act_menu); + touch.left_stick_in = pressed(act_left_in); + touch.right_stick_in = pressed(act_right_in); + touch.left_grip = pulled(act_left_grip); + touch.right_grip = pulled(act_right_grip); + touch.left_trigger = pulled(act_left_trigger); + touch.right_trigger = pulled(act_right_trigger); + const XrVector2f move = stick(act_move); + const XrVector2f finger = stick(act_finger); + touch.left_x = move.x; + touch.left_y = move.y; + touch.right_x = finger.x; + touch.right_y = finger.y; + touch.present = any_active; + } + + /// Reads the headset's own controllers for the router, which decides whether the game is + /// played with them, and finds the one that is the controller in the game if it is. + void UpdateControllers(XrTime time) { + TouchState touch; + if (actions_ready) { + const XrActiveActionSet active{action_set, XR_NULL_PATH}; + XrActionsSyncInfo sync{XR_TYPE_ACTIONS_SYNC_INFO}; + sync.countActiveActionSets = 1; + sync.activeActionSets = &active; + // (Anything but plain success: the game is not what the player has in front of + // them, and what they press is not meant for it.) + if (xrSyncActions(session, &sync) == XR_SUCCESS) { + ReadControllers(touch); + } + } + // (With or without controllers: this is also what keeps the router's time, for + // buttons that are held and for a stick that moves a finger over the touchpad.) + pads.SetTouch(touch, PadRouter::Now()); + pad_view = pads.Look(); + + controllers_used = pad_view.source == PadRouter::Source::Touch; + if (controllers_used && !layout_logged) { + layout_logged = true; + LOGI("the headset's own controllers play the game: left stick to move, A = cross, " + "B = square, X = circle, Y = triangle (X and Y together blow), right stick = " + "finger on the touchpad, pressed in = touchpad pressed, grips = L1 and R1, " + "triggers = L2 and R2, left menu button = OPTIONS (held for a second: view " + "reset), both sticks pressed in = view reset"); + } + controller_located = false; + if (controllers_used && touch.present) { + XrSpaceVelocity velocity{XR_TYPE_SPACE_VELOCITY}; + XrSpaceLocation location{XR_TYPE_SPACE_LOCATION}; + location.next = &velocity; + if (XR_SUCCEEDED(xrLocateSpace(controller_space, local_space, time, &location)) && + (location.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0 && + (location.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) != 0) { + controller_located = true; + controller_pose = location.pose; + controller_linear = + (velocity.velocityFlags & XR_SPACE_VELOCITY_LINEAR_VALID_BIT) != 0 + ? velocity.linearVelocity + : XrVector3f{}; + controller_angular = + (velocity.velocityFlags & XR_SPACE_VELOCITY_ANGULAR_VALID_BIT) != 0 + ? velocity.angularVelocity + : XrVector3f{}; + } + } + ApplyRumble(controllers_used && touch.present); + } + + /// What the game asks of the gamepad's two motors, on the two controllers: the heavy + /// motor is in the gamepad's left grip, the light one in its right, and each is felt a + /// little in the other hand too. + void ApplyRumble(bool on) { + if (!actions_ready) { + return; + } + // (The small motor in the lowest byte, the large one above it.) + const uint32_t wanted = + on && options.rumble ? static_cast(core.GetPadFeedback() & 0xffff) : 0; + const auto now = std::chrono::steady_clock::now(); + // A vibration is asked for by its length: one that goes on is asked for again. + if (wanted == rumble_applied && + (wanted == 0 || now - rumble_time < std::chrono::milliseconds{250})) { + return; + } + rumble_applied = wanted; + rumble_time = now; + const float light = static_cast(wanted & 0xff) / 255.0f; + const float heavy = static_cast((wanted >> 8) & 0xff) / 255.0f; + for (int hand = 0; hand < 2; ++hand) { + const float amplitude = + hand == 0 ? std::max(heavy, light * 0.4f) : std::max(light, heavy * 0.4f); + XrHapticActionInfo info{XR_TYPE_HAPTIC_ACTION_INFO}; + info.action = act_rumble; + info.subactionPath = hand_paths[hand]; + if (amplitude > 0.0f) { + XrHapticVibration vibration{XR_TYPE_HAPTIC_VIBRATION}; + vibration.duration = 500'000'000; + vibration.frequency = XR_FREQUENCY_UNSPECIFIED; + vibration.amplitude = std::min(amplitude, 1.0f); + xrApplyHapticFeedback(session, &info, + reinterpret_cast(&vibration)); + } else { + xrStopHapticFeedback(session, &info); + } + } + } + bool CreateSwapchain(Swapchain& swapchain, uint32_t width, uint32_t height) { XrSwapchainCreateInfo info{XR_TYPE_SWAPCHAIN_CREATE_INFO}; info.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT; @@ -904,6 +1165,9 @@ private: probed = true; ProbeIdleSession(); } + // (Nobody wears the headset, and nobody plays: but a button held on a + // gamepad still takes its time.) + pads.Tick(PadRouter::Now()); usleep(50'000); continue; } @@ -929,6 +1193,7 @@ private: frame_state.predictedDisplayTime + static_cast(std::clamp(options.predict_ms, 0.0f, 80.0f) * 1e6f); SendHeadPose(pose_time); + UpdateControllers(pose_time); SendPadPose(pose_time); UpdateLayerSize(frame_state.predictedDisplayTime); @@ -1071,6 +1336,40 @@ private: 100.0f * static_cast(pad_samples) / static_cast(display_frames), pad_from_head.x * held, pad_from_head.y * held, -pad_from_head.z * held); } + if (held_samples != 0) { + const HeldPad::Vec front = HeldPad::Rotate(held_orientation, {0.0f, 0.0f, -1.0f}); + LOGI("gamepad: no motion sensors say how it is held, so the hands holding it do: " + "%.0f%% of the time, now pointing %.0f degrees left and %.0f up", + 100.0f * static_cast(held_samples) / static_cast(display_frames), + std::atan2(-front.x, -front.z) * Degrees, + std::asin(std::clamp(front.y, -1.0f, 1.0f)) * Degrees); + } + if (compare_samples >= 50) { + // What tells how well the hands do as motion sensors: for a gamepad that has + // both, how far the two are apart and how much that wanders. + const double count = compare_samples; + const auto spread = [&](double sum, double squares) { + return std::sqrt(std::max(0.0, squares / count - (sum / count) * (sum / count))); + }; + LOGI("hands against the gamepad's motion sensors: by the hands its front points " + "%.0f degrees higher (give or take %.0f) and its right side is %.0f degrees " + "higher (give or take %.0f) than by the sensors; %u samples", + compare_pitch / count, spread(compare_pitch, compare_pitch_squares), + compare_roll / count, spread(compare_roll, compare_roll_squares), + compare_samples); + } + if (controllers_used) { + const float seen = + controller_samples != 0 ? 1.0f / static_cast(controller_samples) : 0.0f; + LOGI("controllers: the headset's own play the game; the %s one, which is the " + "controller in the game, was seen %.0f%% of the time, on average %.2f right, " + "%.2f up, %.2f ahead of the head", + options.pad_hand == 0 ? "left" : "right", + 100.0f * static_cast(controller_samples) / + static_cast(display_frames), + controller_from_head.x * seen, controller_from_head.y * seen, + -controller_from_head.z * seen); + } ReportMetrics(); reported_frames = frame_count; reported_cpu_seconds = usage.cpu_seconds; @@ -1080,6 +1379,11 @@ private: palm_distance = 0.0f; pad_samples = 0; pad_from_head = {}; + held_samples = 0; + compare_samples = 0; + compare_pitch = compare_pitch_squares = compare_roll = compare_roll_squares = 0.0; + controller_samples = 0; + controller_from_head = {}; last_report = now; } @@ -1169,8 +1473,15 @@ private: } } + struct Palm { + XrVector3f position; + /// The way the hand points, from the wrist to the knuckles: if the headset says. + bool pointing_known; + XrVector3f pointing; + }; + /// Where the palm of a hand is, if the headset sees it. - std::optional LocatePalm(int hand, XrTime time) { + std::optional LocatePalm(int hand, XrTime time) { XrHandJointLocationEXT joints[XR_HAND_JOINT_COUNT_EXT]; XrHandJointLocationsEXT locations{XR_TYPE_HAND_JOINT_LOCATIONS_EXT}; locations.jointCount = XR_HAND_JOINT_COUNT_EXT; @@ -1186,14 +1497,47 @@ private: if ((palm.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) == 0) { return std::nullopt; } - return palm.pose.position; + Palm result{palm.pose.position, false, {}}; + if ((palm.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0) { + // (A palm's -Z runs along the bone of its middle finger, towards the fingers.) + result.pointing_known = true; + result.pointing = Rotate(palm.pose.orientation, {0.0f, 0.0f, -1.0f}); + } + return result; } - /// A gamepad cannot be tracked, the hands that hold it can: both palms a controller's width - /// apart give away where it is and which way it points. + /// For a gamepad that has motion sensors and is held in hands the headset sees: how far + /// what the hands say of its attitude is from what its own sensors say, while it is held + /// still enough for those to know which way is up. Only counted, for the log. + void CompareWithSensors(const HeldPad::Pose& held) { + const float* accel = pad_view.gamepad_accel; + const float gravity = std::hypot(accel[0], accel[1], accel[2]); + if (!held.pitch_measured || !(gravity > 0.8f * 9.81f && gravity < 1.2f * 9.81f)) { + return; + } + // What is up, seen from the gamepad (+X its right, +Y out of its face, +Z towards + // the player), tells how far its front and its right side are above level. + const HeldPad::Vec front = HeldPad::Rotate(held.orientation, {0.0f, 0.0f, -1.0f}); + const HeldPad::Vec right = HeldPad::Rotate(held.orientation, {1.0f, 0.0f, 0.0f}); + const auto angle = [](float sine) { + return std::asin(std::clamp(sine, -1.0f, 1.0f)) * Degrees; + }; + const double pitch = angle(front.y) - angle(-accel[2] / gravity); + const double roll = angle(right.y) - angle(accel[0] / gravity); + ++compare_samples; + compare_pitch += pitch; + compare_pitch_squares += pitch * pitch; + compare_roll += roll; + compare_roll_squares += roll * roll; + } + + /// Says where the controller the game is played with is. One of the headset's own says + /// that of itself. A gamepad cannot be tracked, but the hands that hold it can: both + /// palms a controller's width apart give away where it is and which way it points, and + /// for a gamepad without motion sensors also how it is tilted (see HeldPad). void SendPadPose(XrTime display_time) { Protocol::PadPose message; - const uint32_t requests = recenter_requests.exchange(0); + const uint32_t requests = recenter_requests.exchange(0) | pads.TakeRecenter(); if ((requests & XrRecenter::Pad) != 0) { message.flags |= Protocol::PadPose::RecenterYaw; } @@ -1208,36 +1552,65 @@ private: LOGI("view reset: %s", system_reset ? "the headset's own" : "asked for by the player"); } + // For the log: a place as seen from the head. + const auto from_head = [&](const XrVector3f& place) { + const XrQuaternionf to_head{-head_pose.orientation.x, -head_pose.orientation.y, + -head_pose.orientation.z, head_pose.orientation.w}; + return Rotate(to_head, {place.x - head_pose.position.x, place.y - head_pose.position.y, + place.z - head_pose.position.z}); + }; + bool seen = false; - if (has_hand_tracking) { + bool hands_seen = false; + if (controllers_used) { + // (Hands that hold the headset's own controllers hold no gamepad.) + if (controller_located) { + message.flags |= + Protocol::PadPose::PositionValid | Protocol::PadPose::OrientationValid; + message.position[0] = controller_pose.position.x; + message.position[1] = controller_pose.position.y; + message.position[2] = controller_pose.position.z; + message.orientation[0] = controller_pose.orientation.x; + message.orientation[1] = controller_pose.orientation.y; + message.orientation[2] = controller_pose.orientation.z; + message.orientation[3] = controller_pose.orientation.w; + message.linear_velocity[0] = controller_linear.x; + message.linear_velocity[1] = controller_linear.y; + message.linear_velocity[2] = controller_linear.z; + message.angular_velocity[0] = controller_angular.x; + message.angular_velocity[1] = controller_angular.y; + message.angular_velocity[2] = controller_angular.z; + const XrVector3f offset = from_head(controller_pose.position); + controller_from_head.x += offset.x; + controller_from_head.y += offset.y; + controller_from_head.z += offset.z; + ++controller_samples; + seen = true; + } + } else if (has_hand_tracking) { const auto left = LocatePalm(0, display_time); const auto right = LocatePalm(1, display_time); if (left && right) { - const XrVector3f across{right->x - left->x, right->y - left->y, - right->z - left->z}; - const float distance = - std::sqrt(across.x * across.x + across.y * across.y + across.z * across.z); + HeldPad::Hands hands; + hands.left_palm = {left->position.x, left->position.y, left->position.z}; + hands.right_palm = {right->position.x, right->position.y, right->position.z}; + hands.pointing_known = left->pointing_known && right->pointing_known; + hands.left_pointing = {left->pointing.x, left->pointing.y, left->pointing.z}; + hands.right_pointing = {right->pointing.x, right->pointing.y, right->pointing.z}; ++palm_samples; - palm_distance += distance; - // Hands further apart, or closer together, are not holding a controller. - if (distance > 0.05f && distance < 0.32f) { - XrVector3f centre{(left->x + right->x) * 0.5f, (left->y + right->y) * 0.5f, - (left->z + right->z) * 0.5f}; - const float level = std::sqrt(across.x * across.x + across.z * across.z); - if (level > 0.6f * distance) { - // The line from the left to the right palm is the controller's - // sideways axis; its heading follows from that. - const float yaw = std::atan2(-across.z, across.x); - message.yaw = yaw; + palm_distance += HeldPad::Length(HeldPad::Sub(hands.right_palm, hands.left_palm)); + // Hands further apart than a controller is wide, or closer together, are not + // holding one. + if (const auto held = + HeldPad::Locate(hands, options.pad_tilt / Degrees, held_pitch)) { + const XrVector3f centre{held->position.x, held->position.y, + held->position.z}; + if (held->turned_known) { + message.yaw = held->yaw; message.flags |= Protocol::PadPose::YawValid; - // The palms close around the grips, which sit behind and below the - // middle of the controller. - centre.x += -std::sin(yaw) * 0.035f; - centre.z += -std::cos(yaw) * 0.035f; } - centre.y += 0.015f; - if (pad_seen && display_time > pad_time) { + if (hands_held && display_time > pad_time) { const float elapsed = static_cast(display_time - pad_time) * 1e-9f; if (elapsed < 0.1f) { @@ -1255,13 +1628,7 @@ private: pad_position = centre; pad_time = display_time; - // For the log: the same, as seen from the head. - const XrQuaternionf to_head{-head_pose.orientation.x, -head_pose.orientation.y, - -head_pose.orientation.z, head_pose.orientation.w}; - const XrVector3f offset = - Rotate(to_head, {centre.x - head_pose.position.x, - centre.y - head_pose.position.y, - centre.z - head_pose.position.z}); + const XrVector3f offset = from_head(centre); pad_from_head.x += offset.x; pad_from_head.y += offset.y; pad_from_head.z += offset.z; @@ -1274,18 +1641,53 @@ private: message.linear_velocity[0] = pad_velocity.x; message.linear_velocity[1] = pad_velocity.y; message.linear_velocity[2] = pad_velocity.z; + + if (held->turned_known) { + if (held->pitch_measured) { + held_pitch = held->pitch; + } + // Hands tremble, and what is seen of them more so: what is passed + // on follows them a little behind. After a gap it starts afresh. + const float elapsed = + held_shown && display_time > held_time + ? static_cast(display_time - held_time) * 1e-9f + : 1.0f; + held_orientation = + elapsed < 0.25f + ? HeldPad::Towards(held_orientation, held->orientation, + 1.0f - std::exp(-elapsed / HeldFollowTime)) + : held->orientation; + held_shown = true; + held_time = display_time; + if (!pad_view.gamepad_motion) { + // No motion sensors say how the gamepad is held (it has none, + // or there is no gamepad at all yet): the hands do. + message.flags |= Protocol::PadPose::OrientationValid | + Protocol::PadPose::HeldInHands; + message.orientation[0] = held_orientation.x; + message.orientation[1] = held_orientation.y; + message.orientation[2] = held_orientation.z; + message.orientation[3] = held_orientation.w; + ++held_samples; + } else { + CompareWithSensors(*held); + } + } seen = true; + hands_seen = true; } } } - // While the hands are seen, every frame; once more when they are lost, so that the + // While the controller is seen, every frame; once more when it is lost, so that the // emulator stops trusting the last position. if (seen || pad_seen || message.flags != 0) { core.SendPadPose(message); } pad_seen = seen; - host_status.hands_tracked = seen; + hands_held = hands_seen; + host_status.hands_tracked = hands_seen; + host_status.controller_tracked = controllers_used && seen; } /// Describes the shown frame to the compositor: where the head was when the emulator drew @@ -1352,6 +1754,13 @@ private: xrDestroySwapchain(swapchain->handle); } } + if (controller_space != XR_NULL_HANDLE) { + xrDestroySpace(controller_space); + } + if (action_set != XR_NULL_HANDLE) { + // (Its actions go with it.) + xrDestroyActionSet(action_set); + } if (local_space != XR_NULL_HANDLE) { xrDestroySpace(local_space); } @@ -1370,6 +1779,7 @@ private: JavaVM* vm; jobject activity; CoreProcess& core; + PadRouter& pads; StatusImage& status; XrHostOptions options; XrHostStatus& host_status; @@ -1409,10 +1819,56 @@ private: /// or 0. XrTime space_change_time{}; bool pad_seen{}; + /// The hands were seen holding a gamepad in the frame before. + bool hands_held{}; XrVector3f pad_position{}; XrVector3f pad_velocity{}; XrTime pad_time{}; + static constexpr float Degrees = 57.29578f; + + // The headset's own controllers, which play the game where the player holds them + // instead of a gamepad. + XrActionSet action_set{XR_NULL_HANDLE}; + XrAction act_move{XR_NULL_HANDLE}; + XrAction act_finger{XR_NULL_HANDLE}; + XrAction act_left_in{XR_NULL_HANDLE}; + XrAction act_right_in{XR_NULL_HANDLE}; + XrAction act_a{XR_NULL_HANDLE}; + XrAction act_b{XR_NULL_HANDLE}; + XrAction act_x{XR_NULL_HANDLE}; + XrAction act_y{XR_NULL_HANDLE}; + XrAction act_menu{XR_NULL_HANDLE}; + XrAction act_left_grip{XR_NULL_HANDLE}; + XrAction act_right_grip{XR_NULL_HANDLE}; + XrAction act_left_trigger{XR_NULL_HANDLE}; + XrAction act_right_trigger{XR_NULL_HANDLE}; + XrAction act_pose{XR_NULL_HANDLE}; + XrAction act_rumble{XR_NULL_HANDLE}; + XrPath hand_paths[2]{XR_NULL_PATH, XR_NULL_PATH}; + XrSpace controller_space{XR_NULL_HANDLE}; + bool actions_ready{}; + /// What the game is played with, as of this frame. + PadRouter::View pad_view; + bool controllers_used{}; + bool layout_logged{}; + bool controller_located{}; + XrPosef controller_pose{{0.0f, 0.0f, 0.0f, 1.0f}, {}}; + XrVector3f controller_linear{}; + XrVector3f controller_angular{}; + uint32_t rumble_applied{}; + std::chrono::steady_clock::time_point rumble_time{}; + + // A gamepad as the hands holding it turn and tilt it. + /// How quickly what is passed on follows what the hands say, in seconds. + static constexpr float HeldFollowTime = 0.06f; + HeldPad::Quat held_orientation; + bool held_shown{}; + XrTime held_time{}; + /// How far up the hands pointed when the headset could last tell: about how a gamepad + /// is held, before it ever could. + float held_pitch{0.35f}; + struct Metric { XrPath path; std::string name; @@ -1456,6 +1912,14 @@ private: float palm_distance{}; uint32_t pad_samples{}; XrVector3f pad_from_head{}; + uint32_t held_samples{}; + uint32_t compare_samples{}; + double compare_pitch{}; + double compare_pitch_squares{}; + double compare_roll{}; + double compare_roll_squares{}; + uint32_t controller_samples{}; + XrVector3f controller_from_head{}; uint64_t status_version{}; bool status_ready{}; float ipd{0.063f}; @@ -1464,9 +1928,9 @@ private: } // namespace -void RunXrHost(JavaVM* vm, jobject activity, CoreProcess& core, StatusImage& status, - const XrHostOptions& options, XrHostStatus& host_status, std::atomic& quit, - std::atomic& recenter_requests) { - Host host{vm, activity, core, status, options, host_status, quit, recenter_requests}; +void RunXrHost(JavaVM* vm, jobject activity, CoreProcess& core, PadRouter& pads, + StatusImage& status, const XrHostOptions& options, XrHostStatus& host_status, + std::atomic& quit, std::atomic& recenter_requests) { + Host host{vm, activity, core, pads, status, options, host_status, quit, recenter_requests}; host.Run(); } diff --git a/quest-host/cpp/xr_host.h b/quest-host/cpp/xr_host.h index 6593cd1..ce3549e 100644 --- a/quest-host/cpp/xr_host.h +++ b/quest-host/cpp/xr_host.h @@ -10,6 +10,7 @@ #include class CoreProcess; +class PadRouter; /// A picture the Java side draws (text is far easier there) and the headset shows on a floating /// panel whenever the game itself has nothing on screen. @@ -65,6 +66,17 @@ struct XrHostOptions { /// read the pose it drew it for; aiming at that moment leaves the compositor less to /// correct, which is what shows as black edges when the head turns. float predict_ms{25.0f}; + /// Let the headset's own two controllers play the game where the player holds them + /// instead of a gamepad (see PadRouter for what does what). + bool own_controllers{true}; + /// Which of the two is the controller in the game then: 1 the right one, 0 the left. + int pad_hand{1}; + /// Let them shake when the game shakes the gamepad. + bool rumble{true}; + /// For a gamepad without motion sensors, which the hands holding it turn and tilt in the + /// game: how far its front is taken to point above (or, negative, below) the way the + /// hands point, in degrees. + float pad_tilt{0.0f}; }; /// What a request to reset the view asks for (bits of the counter RunXrHost is given). @@ -79,6 +91,8 @@ inline constexpr uint32_t Seat = 2; struct XrHostStatus { std::atomic session_running{}; std::atomic hands_tracked{}; + /// The one of the headset's own controllers that is the controller in the game is seen. + std::atomic controller_tracked{}; std::atomic refresh_rate{}; /// Set when the startup choice is confirmed, before the game can deliver any frames. std::atomic reduced_fov{}; @@ -89,7 +103,8 @@ struct XrHostStatus { }; /// Owns the OpenXR session: feeds the head pose to the core and shows the stereo frames it -/// delivers. Blocks until `quit` is set or the system ends the session. -void RunXrHost(JavaVM* vm, jobject activity, CoreProcess& core, StatusImage& status, - const XrHostOptions& options, XrHostStatus& host_status, std::atomic& quit, - std::atomic& recenter_requests); +/// delivers, reads the headset's own controllers for `pads` and places the controller the +/// game is played with. Blocks until `quit` is set or the system ends the session. +void RunXrHost(JavaVM* vm, jobject activity, CoreProcess& core, PadRouter& pads, + StatusImage& status, const XrHostOptions& options, XrHostStatus& host_status, + std::atomic& quit, std::atomic& recenter_requests); diff --git a/quest-host/cpp/xr_probe.cpp b/quest-host/cpp/xr_probe.cpp index 0c5c209..4a7f356 100644 --- a/quest-host/cpp/xr_probe.cpp +++ b/quest-host/cpp/xr_probe.cpp @@ -22,6 +22,7 @@ #include "core_process.h" #include "gl_frames.h" +#include "pad_router.h" #include "xr_host.h" #define XR_USE_PLATFORM_ANDROID @@ -77,6 +78,7 @@ extern "C" JNIEXPORT jstring JNICALL Java_com_astrobotquest_vrhost_SandboxShell_ // The host as the app runs it, with no emulator behind it and nobody to show a session // to: it sets itself up, waits for a session that does not come, and is told to stop. CoreProcess core; + PadRouter pads; StatusImage status; XrHostOptions options; options.probe = true; @@ -87,7 +89,8 @@ extern "C" JNIEXPORT jstring JNICALL Java_com_astrobotquest_vrhost_SandboxShell_ std::this_thread::sleep_for(std::chrono::seconds{5}); quit = true; }}; - RunXrHost(vm, context, core, status, options, host_status, quit, recenter_requests); + RunXrHost(vm, context, core, pads, status, options, host_status, quit, + recenter_requests); stopper.join(); note("the host ran and stopped: session %s, refresh rate %.0f Hz", host_status.session_running ? "running" : "not running", diff --git a/quest-host/java/com/astrobotquest/vrhost/MainActivity.java b/quest-host/java/com/astrobotquest/vrhost/MainActivity.java index 4707ab8..a1e8150 100644 --- a/quest-host/java/com/astrobotquest/vrhost/MainActivity.java +++ b/quest-host/java/com/astrobotquest/vrhost/MainActivity.java @@ -47,9 +47,11 @@ import java.util.Locale; /** * Hosts the PS4 emulator core on a Meta Quest: installs its Linux runtime, starts it, feeds it a - * paired DualSense as the DualShock 4 the game expects (buttons, sticks, touchpad, motion - * sensors in; rumble and light bar colour out) and lets the native side show its frames in the - * headset. + * gamepad paired with the headset as the DualShock 4 the game expects (of a DualSense + * everything: buttons, sticks, touchpad, motion sensors in; rumble and light bar colour out; + * of other gamepads what they have) and lets the native side show its frames in the headset. + * The headset's own controllers play as well; those the native side reads itself, and it + * decides which of the two the game is played with (PadRouter). * * Files the user provides live in the app's external folder * (/sdcard/Android/data/com.astrobotquest.vrhost/files): @@ -71,9 +73,6 @@ public class MainActivity extends Activity private static final int TRIANGLE = 0x1000, CIRCLE = 0x2000, CROSS = 0x4000, SQUARE = 0x8000; private static final int TOUCHPAD = 0x100000; - // The touchpad as the emulator takes it: the resolution of a DualSense's. - private static final int TOUCH_WIDTH = 1920, TOUCH_HEIGHT = 1080; - private static final int STATUS_WIDTH = 1024, STATUS_HEIGHT = 512; /** Second place games are looked for; every app may read what adb puts there. */ @@ -94,7 +93,8 @@ public class MainActivity extends Activity private native void nativeStartXr(float refreshRate, int eyeWidth, int eyeHeight, boolean trackHands, int sharpen, boolean cubic, boolean showStats, float predictMs, - int dynamicResolution, boolean cpuBoost); + int dynamicResolution, boolean cpuBoost, boolean ownControllers, int padHand, + float padTilt, boolean rumble, boolean stickTouchpad); private native boolean nativeStartCore(String loader, String runtimeRoot, String storageRoot, String game, String logFile, String[] extraArgs, String[] extraEnv); @@ -109,13 +109,38 @@ public class MainActivity extends Activity private native void nativeSetStatusImage(ByteBuffer pixels); - private native void nativeInput(int buttons, int leftX, int leftY, int rightX, int rightY, - int leftTrigger, int rightTrigger, boolean touchDown, int touchX, int touchY); + /** + * What the gamepad says: sticks from -1 to 1, triggers from 0 to 1, the finger on its own + * touchpad from 0 to 1 across and from its far edge. + */ + private native void nativeInput(int buttons, float leftX, float leftY, float rightX, + float rightY, float leftTrigger, float rightTrigger, boolean touchDown, float touchX, + float touchY); private native void nativeMotion(float gx, float gy, float gz, float ax, float ay, float az); private native long nativePadFeedback(); + /** A gamepad is there to play with (again, or another one), or none is any more. */ + private native void nativeSetGamepad(boolean connected); + + /** The start-up menu is over: what the player presses is the game's. */ + private native void nativeSetPlaying(boolean playing); + + /** What the game is played with, see the CONTROLLER_ constants. */ + private native int nativeController(); + + /** The player chose with it in the start-up menu. */ + private native void nativeUseController(int which); + + /** What the headset's own controllers say to the start-up menu, see the MENU_ constants. */ + private native int nativeTouchMenu(); + + private static final int CONTROLLER_WHICH = 3, CONTROLLER_GAMEPAD = 1, CONTROLLER_OWN = 2; + /** Added to it while the headset's own controllers are in the player's hands. */ + private static final int CONTROLLER_OWN_HELD = 4; + private static final int MENU_LEFT = 1, MENU_RIGHT = 2, MENU_CONFIRM = 4; + private static final int VENDOR_SONY = 0x054c; private static final int VENDOR_META = 0x2833; @@ -151,6 +176,11 @@ public class MainActivity extends Activity private float padTouchX, padTouchY; private boolean padTouchClick; private final float[] acceleration = {0.0f, 9.81f, 0.0f}; + /** Whose sticks and triggers the fields below describe. */ + private int axesDevice = -1; + private int rightXAxis = MotionEvent.AXIS_Z, rightYAxis = MotionEvent.AXIS_RZ; + /** The gamepad says how far its triggers are pulled, and not only that they are. */ + private boolean analogTriggers = true; // The controller itself: what the game's feedback goes to. private InputDevice gamepad; @@ -166,6 +196,9 @@ public class MainActivity extends Activity private boolean motion = true; private boolean rumble = true; private boolean trackHands = true; + private boolean ownControllers = true; + private int padHand = 1; + private float padTilt; private int sharpen = 1; private boolean cubic; private boolean antialias = true; @@ -186,6 +219,12 @@ public class MainActivity extends Activity private long nextFovMoveAt; private float[] headsetFov; private static final int FOV_MIN = 50, FOV_MAX = 100, FOV_STEP = 5; + private long nextOwnFovMoveAt; + /** + * What confirms the menu on the headset's own controllers has been seen let go while the + * menu showed: the trigger that started the app from the headset's library is not it. + */ + private boolean ownConfirmArmed; /** * Set by the test that runs the activity without showing it (SandboxShell "drystart"): * nothing is asked of the user or of the system's window manager then. @@ -193,13 +232,6 @@ public class MainActivity extends Activity private boolean dryRun; private int statusPictures; private float microphoneGain = 1.0f; - private boolean seatTaken; - /** How long OPTIONS is held for the view to be reset, as on a PlayStation VR. */ - private static final long OPTIONS_HOLD_MS = 1000; - private final Runnable optionsHeld = () -> { - logInfo("OPTIONS held: view reset"); - nativeRecenter(RECENTER_PAD | RECENTER_SEAT); - }; private long quietFrames = -1; private long quietSince; private long statsFrames = -1; @@ -242,14 +274,23 @@ public class MainActivity extends Activity readSettings(); choosingFov = !dryRun; nativeStartXr(refreshRate, 1440, 1536, trackHands, sharpen, cubic, showStats, predictMs, - dynamicResolution, cpuBoost); + dynamicResolution, cpuBoost, ownControllers, padHand, padTilt, rumble, + stickTouchpad); InputManager inputManager = getSystemService(InputManager.class); inputManager.registerInputDeviceListener(this, handler); findGamepad(); if (dryRun) { + // Nobody sees the menu in a test: it is drawn for the pictures the test keeps, + // once the way it waits for the headset and once with a headset's view. + drawFovMenu(); + headsetFov = new float[] {1.376f, 0.839f, 0.966f, 1.428f}; + drawFovMenu(); + headsetFov = null; startGame(); + } else { + handler.post(this::pollOwnControllers); } handler.post(this::refreshStatus); handler.post(this::applyFeedback); @@ -349,6 +390,12 @@ public class MainActivity extends Activity * emulator has it run by the clock * hands=0 do not use hand tracking to place the controller; it is then held * at a fixed spot in front of the player + * own_controllers=0 never play with the headset's own controllers, whoever holds them + * pad_hand=left with those, the left one is the controller in the game instead of + * the right one + * pad_tilt=-15 for a gamepad without motion sensors, which the hands holding it + * turn and tilt in the game: how many degrees higher (lower, if + * negative) its front points than the hands do, -60 to 60 * sharpen=0 leave the picture as soft as it gets from being shown larger * than it was drawn. By default (1) the emulator sharpens it once * per frame of the game. 2 has the headset do it instead, with its @@ -359,7 +406,8 @@ public class MainActivity extends Activity * cubic=1 have the headset enlarge the picture with a cubic filter instead * of a linear one: sharper, for some GPU time on every refresh * stick_touchpad=0 stop using the right stick as a finger on the touchpad - * motion=0 ignore the controller's motion sensors + * motion=0 ignore the controller's motion sensors (the hands holding it + * then say how it is turned, as for a gamepad that has none) * rumble=0 no vibration * msaa=4 let the game multisample as it does on the console (1, 2 or 4). * The default of 1 turns that off: it is by far the most expensive @@ -439,6 +487,15 @@ public class MainActivity extends Activity case "hands": trackHands = !value.equals("0"); break; + case "own_controllers": + ownControllers = !value.equals("0"); + break; + case "pad_hand": + padHand = value.equalsIgnoreCase("left") || value.equals("0") ? 0 : 1; + break; + case "pad_tilt": + padTilt = Math.max(-60.0f, Math.min(60.0f, Float.parseFloat(value))); + break; case "sharpen": sharpen = Math.max(0, Math.min(4, Integer.parseInt(value))); break; @@ -510,6 +567,9 @@ public class MainActivity extends Activity } choosingFov = false; shownStatus = ""; + // From here on, what the player presses is the game's; what is still held down from + // confirming the menu is not, until it is let go. + nativeSetPlaying(true); nativeSetFieldOfView(fieldOfView); if (!dryRun) { getPreferences(MODE_PRIVATE).edit().putInt("fov", fieldOfView).apply(); @@ -671,7 +731,8 @@ public class MainActivity extends Activity private void refreshStatus() { if (choosingFov) { headsetFov = nativeHeadsetFov(); - String status = "fov:" + fieldOfView + ":" + (gamepad != null) + String status = "fov:" + fieldOfView + ":" + (gamepad != null) + ":" + + ((nativeController() & CONTROLLER_OWN_HELD) != 0) + ":" + Arrays.toString(headsetFov); if (!status.equals(shownStatus)) { shownStatus = status; @@ -715,15 +776,24 @@ public class MainActivity extends Activity break; } // Waiting is the moment to say what the game will be played with. - if (gamepad == null) { - status += "\n\nNo controller found. Switch the DualSense on (PS button); pair it " - + "first under Settings > Bluetooth if it never was."; - } else { + int controller = nativeController(); + if ((controller & CONTROLLER_WHICH) == CONTROLLER_OWN) { + status += "\n\nController: the headset's own (A cross, B square, X circle, Y " + + "triangle, right stick = touchpad, menu button = OPTIONS)." + + "\nTo reset the view, hold the menu button for a second or press both " + + "sticks in."; + } else if (gamepad != null) { status += "\n\nController: " + gamepad.getName() - + (controllerSensors != null ? "" : " (no motion sensors)") - + "\nTo reset the view, hold OPTIONS for a second or press the PS button: " - + "where you are then is your seat, and the way you face is straight " + + (controllerSensors != null ? "" + : " (no motion sensors: your hands turn it in the game)") + + "\nTo reset the view, hold OPTIONS for a second" + + (gamepad.getVendorId() == VENDOR_SONY ? " or press the PS button" : "") + + ": where you are then is your seat, and the way you face is straight " + "ahead."; + } else { + status += "\n\nNo controller found. Pick up the headset's controllers, or switch " + + "a gamepad on (pair it first under Settings > Bluetooth if it never " + + "was)."; } } if (!status.equals(shownStatus)) { @@ -750,7 +820,9 @@ public class MainActivity extends Activity String rate = statsRate; int xr = nativeXrStatus(); return "Game: " + rate + " frames a second\nDisplay: " + ((xr >> 8) & 0xff) + " Hz\n" - + "Hands holding the controller: " + ((xr & 2) != 0 ? "seen" : "not seen"); + + ((nativeController() & CONTROLLER_WHICH) == CONTROLLER_OWN + ? "The headset's controller: " + ((xr & 4) != 0 ? "seen" : "not seen") + : "Hands holding the controller: " + ((xr & 2) != 0 ? "seen" : "not seen")); } /** Visible coverage is the symmetric render envelope clipped by the actual headset. */ @@ -814,17 +886,22 @@ public class MainActivity extends Activity paint.setTextAlign(Paint.Align.CENTER); canvas.drawText(fieldOfView < 100 ? "Soft squircle border / Default: 85%" : "No added border / Default: 85%", STATUS_WIDTH / 2.0f, 385, paint); - paint.setColor(gamepad != null ? Color.WHITE : Color.rgb(255, 200, 110)); + // Whatever the player has in their hands chooses: a gamepad, or the headset's own + // controllers. + boolean controller = gamepad != null + || (nativeController() & CONTROLLER_OWN_HELD) != 0; + paint.setColor(controller ? Color.WHITE : Color.rgb(255, 200, 110)); paint.setTextSize(25); - canvas.drawText(gamepad != null ? "Left / Right on D-pad or left stick: choose" - : "Connect your gamepad to choose and start", STATUS_WIDTH / 2.0f, 422, paint); + canvas.drawText(controller ? "Left / Right on a stick or the D-pad: choose" + : "Pick up the controllers, or switch a gamepad on, to choose and start", + STATUS_WIDTH / 2.0f, 422, paint); paint.setColor(headsetFov != null ? Color.rgb(45, 104, 182) : Color.rgb(48, 56, 76)); - canvas.drawRoundRect(new RectF(372, 443, 652, 495), 18, 18, paint); + canvas.drawRoundRect(new RectF(322, 443, 702, 495), 18, 18, paint); paint.setColor(Color.WHITE); paint.setTextSize(28); paint.setTypeface(Typeface.create(Typeface.DEFAULT, Typeface.BOLD)); - canvas.drawText(headsetFov != null ? "X / A: Play" : "Detecting FOV...", + canvas.drawText(headsetFov != null ? "X / A or trigger: Play" : "Detecting FOV...", STATUS_WIDTH / 2.0f, 478, paint); publishStatusBitmap(bitmap); } @@ -834,6 +911,36 @@ public class MainActivity extends Activity drawFovMenu(); } + /** + * The start-up menu, worked with the headset's own controllers: a stick to a side + * chooses, A or a trigger starts the game. (The native side reads them; nothing of them + * comes this way by itself.) + */ + private void pollOwnControllers() { + if (!choosingFov) { + return; + } + int menu = nativeTouchMenu(); + long now = SystemClock.uptimeMillis(); + int direction = (menu & MENU_LEFT) != 0 ? -1 : (menu & MENU_RIGHT) != 0 ? 1 : 0; + if (direction == 0) { + nextOwnFovMoveAt = 0; + } else if (now >= nextOwnFovMoveAt) { + nativeUseController(CONTROLLER_OWN); + adjustFov(direction); + nextOwnFovMoveAt = now + 250; + } + if ((menu & MENU_CONFIRM) == 0) { + ownConfirmArmed = true; + } else if (ownConfirmArmed && headsetFov != null) { + logInfo("the menu was confirmed with the headset's own controllers"); + nativeUseController(CONTROLLER_OWN); + startGame(); + return; + } + handler.postDelayed(this::pollOwnControllers, 50); + } + private void drawStatus(String status) { Bitmap bitmap = Bitmap.createBitmap(STATUS_WIDTH, STATUS_HEIGHT, Bitmap.Config.ARGB_8888); Canvas canvas = new Canvas(bitmap); @@ -935,6 +1042,24 @@ public class MainActivity extends Activity @Override public boolean dispatchKeyEvent(KeyEvent event) { + if (isOwnController(event.getDevice())) { + // The headset's own controllers are read by the native side, through the + // headset's runtime. Should the system pass their buttons this way as well, they + // are not counted twice, nor left to what the system makes of a button nobody + // took: it takes B for "back", and may pass the menu button on as "back" itself, + // either of which would close the app in the middle of the game. + int code = event.getKeyCode(); + boolean back = code == KeyEvent.KEYCODE_BACK || code == KeyEvent.KEYCODE_MENU; + if (back && choosingFov) { + // (Before the game, backing out of the app is what it has always done.) + if (code == KeyEvent.KEYCODE_BACK && event.getAction() == KeyEvent.ACTION_DOWN) { + finish(); + } + return true; + } + return back || buttonFor(code) != 0 || code == KeyEvent.KEYCODE_BUTTON_MODE + || super.dispatchKeyEvent(event); + } // Consume the entire confirming press, including repeats and its release. It must // not press X in the game, or move the player's seat before they have settled in. if (fovConfirmKey != KeyEvent.KEYCODE_UNKNOWN && event.getKeyCode() == fovConfirmKey) { @@ -946,6 +1071,9 @@ public class MainActivity extends Activity if (choosingFov) { if (event.getAction() == KeyEvent.ACTION_DOWN && event.getRepeatCount() == 0) { noteInputFrom(event.getDevice()); + if (isGamepad(event.getDevice())) { + nativeUseController(CONTROLLER_GAMEPAD); + } switch (event.getKeyCode()) { case KeyEvent.KEYCODE_DPAD_LEFT: adjustFov(-1); @@ -980,31 +1108,24 @@ public class MainActivity extends Activity } return true; } - int button = buttonFor(event.getKeyCode()); + int keyCode = event.getKeyCode(); + if (keyCode == KeyEvent.KEYCODE_BACK && isGamepad(event.getDevice())) { + // Some gamepads have the system take their Back / View button for its own "back", + // which would close the app in the middle of the game: it is the button that + // presses the touchpad, as on every other gamepad. + keyCode = KeyEvent.KEYCODE_BUTTON_SELECT; + } + int button = buttonFor(keyCode); if (button == 0) { return super.dispatchKeyEvent(event); } + // (What the buttons mean about the view besides - the first press of X takes the + // player's seat, OPTIONS held for a second resets the view - the native side sees + // to, for whichever controller the game is played with.) if (event.getAction() == KeyEvent.ACTION_DOWN) { noteInputFrom(event.getDevice()); - if (button == CROSS && !seatTaken) { - // The first press of X is the player settled in, controller in hand, looking - // at the game: that, and not where they were when the app started, is their - // seat. - seatTaken = true; - logInfo("first press of X: the player's seat is where they are now"); - nativeRecenter(RECENTER_SEAT); - } - if (button == OPTIONS && event.getRepeatCount() == 0) { - // What a PlayStation VR does when OPTIONS is held, and what the game tells its - // players to do when the view is off. The game still sees the button. - handler.removeCallbacks(optionsHeld); - handler.postDelayed(optionsHeld, OPTIONS_HOLD_MS); - } keyButtons |= button; } else if (event.getAction() == KeyEvent.ACTION_UP) { - if (button == OPTIONS) { - handler.removeCallbacks(optionsHeld); - } keyButtons &= ~button; } sendInput(); @@ -1019,8 +1140,42 @@ public class MainActivity extends Activity return range != null && Math.abs(value) <= range.getFlat() ? 0.0f : value; } + /** + * Where a gamepad has its right stick and whether its triggers say how far they are + * pulled: most gamepads agree on that, the others are met half way. + */ + private void learnAxes(InputDevice device) { + if (device == null || device.getId() == axesDevice) { + return; + } + axesDevice = device.getId(); + boolean standard = hasAxis(device, MotionEvent.AXIS_Z) + && hasAxis(device, MotionEvent.AXIS_RZ); + boolean other = hasAxis(device, MotionEvent.AXIS_RX) + && hasAxis(device, MotionEvent.AXIS_RY); + rightXAxis = standard || !other ? MotionEvent.AXIS_Z : MotionEvent.AXIS_RX; + rightYAxis = standard || !other ? MotionEvent.AXIS_RZ : MotionEvent.AXIS_RY; + analogTriggers = hasAxis(device, MotionEvent.AXIS_LTRIGGER) + || hasAxis(device, MotionEvent.AXIS_BRAKE) + || hasAxis(device, MotionEvent.AXIS_RTRIGGER) + || hasAxis(device, MotionEvent.AXIS_GAS); + if (!standard || !analogTriggers) { + logInfo("this gamepad: right stick on " + MotionEvent.axisToString(rightXAxis) + " and " + + MotionEvent.axisToString(rightYAxis) + ", triggers " + + (analogTriggers ? "by how far they are pulled" : "as buttons only")); + } + } + + private static boolean hasAxis(InputDevice device, int axis) { + return device.getMotionRange(axis, InputDevice.SOURCE_JOYSTICK) != null; + } + @Override public boolean dispatchGenericMotionEvent(MotionEvent event) { + if (isOwnController(event.getDevice())) { + // Read by the native side, see dispatchKeyEvent. + return true; + } if (choosingFov) { if (event.isFromSource(InputDevice.SOURCE_JOYSTICK) && event.getAction() == MotionEvent.ACTION_MOVE) { @@ -1033,6 +1188,7 @@ public class MainActivity extends Activity if (direction == 0) { nextFovMoveAt = 0; } else if (now >= nextFovMoveAt) { + nativeUseController(CONTROLLER_GAMEPAD); adjustFov(direction); nextFovMoveAt = now + 250; } @@ -1041,10 +1197,11 @@ public class MainActivity extends Activity } if (event.isFromSource(InputDevice.SOURCE_JOYSTICK) && event.getAction() == MotionEvent.ACTION_MOVE) { + learnAxes(event.getDevice()); leftX = axis(event, MotionEvent.AXIS_X); leftY = axis(event, MotionEvent.AXIS_Y); - rightX = axis(event, MotionEvent.AXIS_Z); - rightY = axis(event, MotionEvent.AXIS_RZ); + rightX = axis(event, rightXAxis); + rightY = axis(event, rightYAxis); leftTrigger = Math.max(event.getAxisValue(MotionEvent.AXIS_LTRIGGER), event.getAxisValue(MotionEvent.AXIS_BRAKE)); rightTrigger = Math.max(event.getAxisValue(MotionEvent.AXIS_RTRIGGER), @@ -1096,56 +1253,55 @@ public class MainActivity extends Activity return true; } - private static int stick(float value) { - return Math.max(0, Math.min(255, Math.round(128.0f + value * 127.0f))); - } - + /** + * Hands what the gamepad says to the native side as it is. What a gamepad lacks is made + * up for there, the same way for every controller: the right stick as a finger on the + * touchpad (unless stick_touchpad=0), a finger in the pad's middle for a button that + * presses it. + */ private void sendInput() { if (choosingFov) { return; } int buttons = keyButtons | hatButtons | triggerButtons | (padTouchClick ? TOUCHPAD : 0); - - boolean touchDown = padTouchDown; - float touchX = padTouchX, touchY = padTouchY; - if (!touchDown && stickTouchpad) { - // This title has no use for the right stick, while several of its gadgets want - // swipes: the stick moves a finger that touches down at the pad's centre. - float deflection = (float) Math.hypot(rightX, rightY); - if (deflection > 0.25f) { - touchDown = true; - touchX = 0.5f + rightX * 0.45f; - touchY = 0.5f + rightY * 0.45f; - } + float left = leftTrigger, right = rightTrigger; + if (!analogTriggers) { + // Triggers that are only buttons are pulled all the way or not at all. + left = (keyButtons & L2) != 0 ? 1.0f : 0.0f; + right = (keyButtons & R2) != 0 ? 1.0f : 0.0f; } - - nativeInput(buttons, stick(leftX), stick(leftY), stick(rightX), stick(rightY), - Math.round(Math.max(0.0f, Math.min(1.0f, leftTrigger)) * 255.0f), - Math.round(Math.max(0.0f, Math.min(1.0f, rightTrigger)) * 255.0f), touchDown, - Math.max(0, Math.min(TOUCH_WIDTH - 1, Math.round(touchX * (TOUCH_WIDTH - 1)))), - Math.max(0, Math.min(TOUCH_HEIGHT - 1, Math.round(touchY * (TOUCH_HEIGHT - 1))))); + nativeInput(buttons, leftX, leftY, rightX, rightY, left, right, padTouchDown, padTouchX, + padTouchY); } // --- controller: the device, its motion sensors, rumble and light ---------------------------- + /** + * The headset's own controllers, which the system lists as gamepads too (and first, when + * the app was started with one of them). They are no gamepad here: this way they come + * without where they are, without rumble and without telling whether anybody holds them. + * The native side reads them through the headset's runtime, which has all of that. + */ + private static boolean isOwnController(InputDevice device) { + return device != null && device.getVendorId() == VENDOR_META + && device.supportsSource(InputDevice.SOURCE_GAMEPAD); + } + private static boolean isGamepad(InputDevice device) { - return device != null && !device.isVirtual() + return device != null && !device.isVirtual() && !isOwnController(device) && device.supportsSource(InputDevice.SOURCE_GAMEPAD) && device.supportsSource(InputDevice.SOURCE_JOYSTICK); } /** - * How much a device looks like the controller the game is played with. The headset's own - * controllers also show up as gamepads (and come first in the list when the app was started - * with one of them): a controller from the console's maker goes before anything else, the - * headset's own after everything else, and motion sensors and motors count for the rest. + * How much a device looks like the controller the game was made for, where more than one + * gamepad is there: a controller from the console's maker goes before anything else, and + * motion sensors and motors count for the rest. */ private int rank(InputDevice device) { int rank = 0; if (device.getVendorId() == VENDOR_SONY) { rank += 8; - } else if (device.getVendorId() == VENDOR_META) { - rank -= 8; } try { if (findSensors(device) != null) { @@ -1204,9 +1360,20 @@ public class MainActivity extends Activity + ", product " + device.getProductId() + ", sources 0x" + Integer.toHexString(device.getSources()) + ")"); + learnAxes(device); + nativeSetGamepad(true); + // Every part is optional: what a controller offers depends on it and on the system. try { SensorManager sensors = motion ? findSensors(device) : null; + if (sensors != null && device.getVendorId() != VENDOR_SONY) { + // Which way is up for a motion sensor is each maker's own affair, and only + // this one's is known here. A gamepad turned the wrong way round in the game + // is worse than one the hands holding it turn. + logInfo("this gamepad has motion sensors, which are not used: only for " + + "PlayStation controllers is it known which way theirs point"); + sensors = null; + } if (sensors != null) { sensors.registerListener(this, sensors.getDefaultSensor(Sensor.TYPE_ACCELEROMETER), SensorManager.SENSOR_DELAY_FASTEST); @@ -1239,7 +1406,8 @@ public class MainActivity extends Activity } logInfo("controller offers: motion " + (controllerSensors != null) + ", rumble " + (controllerVibrators != null) + ", light bar " + (lightBar != null) - + ", touchpad " + device.supportsSource(InputDevice.SOURCE_TOUCHPAD)); + + ", touchpad " + device.supportsSource(InputDevice.SOURCE_TOUCHPAD) + + ", analog triggers " + analogTriggers); shownFeedback = -1; } @@ -1298,6 +1466,11 @@ public class MainActivity extends Activity /** Carries out what the game last asked of the controller: rumble and light bar colour. */ private void applyFeedback() { long feedback = nativePadFeedback(); + if ((nativeController() & CONTROLLER_WHICH) != CONTROLLER_GAMEPAD) { + // The game is played with the headset's own controllers, which the native side + // shakes: a gamepad lying about has no business rattling. + feedback &= ~0xffffL; + } long now = android.os.SystemClock.uptimeMillis(); int small = (int) (feedback & 0xff); int large = (int) ((feedback >> 8) & 0xff); @@ -1377,7 +1550,8 @@ public class MainActivity extends Activity /** The motion sensors of a controller may be listed a moment after the controller. */ private void adoptLateSensors() { - if (gamepad == null || controllerSensors != null || !motion) { + if (gamepad == null || controllerSensors != null || !motion + || gamepad.getVendorId() != VENDOR_SONY) { return; } InputDevice device = getSystemService(InputManager.class).getInputDevice(gamepad.getId()); @@ -1401,6 +1575,10 @@ public class MainActivity extends Activity padTouchClick = false; sendInput(); findGamepad(); + if (gamepad == null) { + // The headset's own controllers play on, if the player picks them up. + nativeSetGamepad(false); + } } } diff --git a/shadps4-arm64-main/src/core/vr/vr_host_link.cpp b/shadps4-arm64-main/src/core/vr/vr_host_link.cpp index 2f47246..1901c25 100644 --- a/shadps4-arm64-main/src/core/vr/vr_host_link.cpp +++ b/shadps4-arm64-main/src/core/vr/vr_host_link.cpp @@ -223,13 +223,16 @@ void HostLink::ReadLoop() { } const Vec3 velocity{pad.linear_velocity[0], pad.linear_velocity[1], pad.linear_velocity[2]}; - if ((pad.flags & Protocol::PadPose::OrientationValid) != 0 && + const Quat orientation = Normalize( + {pad.orientation[0], pad.orientation[1], pad.orientation[2], pad.orientation[3]}); + const bool held = (pad.flags & Protocol::PadPose::HeldInHands) != 0; + if (!held && (pad.flags & Protocol::PadPose::OrientationValid) != 0 && (pad.flags & Protocol::PadPose::PositionValid) != 0) { - // A device fixed to the controller tells everything there is to know. + // A device fixed to the controller, or one that is the controller, tells + // everything there is to know. DeviceState state; state.pose.position = position; - state.pose.orientation = Normalize({pad.orientation[0], pad.orientation[1], - pad.orientation[2], pad.orientation[3]}); + state.pose.orientation = orientation; state.linear_velocity = velocity; state.angular_velocity = {pad.angular_velocity[0], pad.angular_velocity[1], pad.angular_velocity[2]}; @@ -237,6 +240,14 @@ void HostLink::ReadLoop() { runtime.UpdatePad(state); break; } + // Nothing tells everything (any more): a controller that was a tracked device a + // moment ago is placed by what else is known of it from here on. Without this it + // would stay where that device was last seen for as long as no motion sensor of + // its own speaks up, which for a controller that has none is for good. + runtime.ReleasePad(); + if (held && (pad.flags & Protocol::PadPose::OrientationValid) != 0) { + runtime.UpdatePadHeldOrientation(orientation); + } if ((pad.flags & Protocol::PadPose::PositionValid) != 0) { runtime.UpdatePadPosition(position, velocity); } else { diff --git a/shadps4-arm64-main/src/core/vr/vr_protocol.h b/shadps4-arm64-main/src/core/vr/vr_protocol.h index 060e619..8714d84 100644 --- a/shadps4-arm64-main/src/core/vr/vr_protocol.h +++ b/shadps4-arm64-main/src/core/vr/vr_protocol.h @@ -112,6 +112,10 @@ struct PadPose { /// The player asks for the view to be reset: where the head is now is where they sit, and /// the way it faces is straight ahead. (The message needs nothing else to be valid.) static constexpr std::uint32_t RecenterSeat = 1u << 5; + /// `orientation` is not the controller's own but what the hands holding it give away of + /// how it is turned and tilted: it stands in for motion sensors the controller does not + /// have, and `position` is taken as it is without `OrientationValid`. + static constexpr std::uint32_t HeldInHands = 1u << 6; Header header{.type = MessageType::PadPose}; std::uint32_t flags{}; diff --git a/shadps4-arm64-main/src/core/vr/vr_runtime.cpp b/shadps4-arm64-main/src/core/vr/vr_runtime.cpp index 637c2b9..089d2c9 100644 --- a/shadps4-arm64-main/src/core/vr/vr_runtime.cpp +++ b/shadps4-arm64-main/src/core/vr/vr_runtime.cpp @@ -405,6 +405,15 @@ void Runtime::UpdatePadOrientation(const Quat& orientation, const Vec3& angular_ pad_position_tracked = false; } +void Runtime::UpdatePadHeldOrientation(const Quat& host_orientation) { + std::scoped_lock lock{mutex}; + pad.pose.orientation = Normalize(Multiply(Conjugate(seat_yaw), host_orientation)); + pad.angular_velocity = {}; + ++pad.sequence; + pad.tracked = true; + pad_position_tracked = false; +} + void Runtime::UpdatePadAcceleration(const Vec3& acceleration) { std::scoped_lock lock{mutex}; pad_acceleration = acceleration; diff --git a/shadps4-arm64-main/src/core/vr/vr_runtime.h b/shadps4-arm64-main/src/core/vr/vr_runtime.h index 2c19bcc..ed9361e 100644 --- a/shadps4-arm64-main/src/core/vr/vr_runtime.h +++ b/shadps4-arm64-main/src/core/vr/vr_runtime.h @@ -186,6 +186,9 @@ public: /// For hosts that only know how the controller is turned (from its motion sensors) and not /// where it is. The runtime then places it with Config::pad_offset. void UpdatePadOrientation(const Quat& orientation, const Vec3& angular_velocity); + /// The same for a host that works out how the controller is turned from what it sees of + /// it (the hands holding a controller without motion sensors), in its own space. + void UpdatePadHeldOrientation(const Quat& host_orientation); /// The controller's own motion sensors, for hosts that have nothing better. Readings are in /// the controller's frame (+X right, +Y out of the face buttons, +Z towards the player), in /// rad/s and m/s² with gravity included, as SDL reports them. The runtime works out how the diff --git a/tools/quest-drystart.sh b/tools/quest-drystart.sh index 5d94bb0..26b9d09 100644 --- a/tools/quest-drystart.sh +++ b/tools/quest-drystart.sh @@ -16,7 +16,9 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) ADB="$LOCALAPPDATA/Android/Sdk/platform-tools/adb.exe" export ANDROID_SERIAL=${ANDROID_SERIAL:-$(cat "$(dirname "${BASH_SOURCE[0]}")/quest-serial.local" 2>/dev/null)} export MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*' -package=com.astrobotquest.vrhost +# (ASTRO_PACKAGE: the build made by quest-host/build.sh --test-package, if that is the +# one to run.) +package=${ASTRO_PACKAGE:-com.astrobotquest.vrhost} seconds=${1:-60} out="$root/build/quest/drystart" files=/sdcard/Android/data/$package/files @@ -27,7 +29,7 @@ if [ -n "${2:-}" ]; then "$ADB" shell "chmod 644 /data/local/tmp/astro/vrhost-test.txt" settings="-e settings /data/local/tmp/astro/vrhost-test.txt" fi -"$ADB" shell "am instrument -w -e drystart $seconds $settings $package/.SandboxShell" > /dev/null +"$ADB" shell "am instrument -w -e drystart $seconds $settings $package/com.astrobotquest.vrhost.SandboxShell" > /dev/null mkdir -p "$out" for name in host.log core.log; do # (Not "adb pull": with path conversion off, adb.exe is given a path it cannot write to.) diff --git a/tools/quest-probe.sh b/tools/quest-probe.sh index e6dd162..b477234 100644 --- a/tools/quest-probe.sh +++ b/tools/quest-probe.sh @@ -13,7 +13,9 @@ export MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*' ADB="$LOCALAPPDATA/Android/Sdk/platform-tools/adb.exe" # The headset also shows up over the network: name the USB connection. export ANDROID_SERIAL=${ANDROID_SERIAL:-$(cat "$(dirname "${BASH_SOURCE[0]}")/quest-serial.local" 2>/dev/null)} -package=com.astrobotquest.vrhost +# (ASTRO_PACKAGE: the build made by quest-host/build.sh --test-package, if that is the +# one to run.) +package=${ASTRO_PACKAGE:-com.astrobotquest.vrhost} stage=/data/local/tmp/astro driver=${ASTRO_DRIVER:-runtime/drivers/turnip} @@ -41,5 +43,5 @@ env HOME=\$PWD/home TMPDIR=\$PWD/tmp GLIBC_TUNABLES=glibc.pthread.rseq=0 \ timeout 120 \$ASTRO_LIB_DIR/libastro_ld.so --library-path \$PWD/runtime/host:\$PWD/$driver \ --preload \$PWD/runtime/host/libkgsl_compat.so ./$name $* 2>&1 " -"$ADB" shell "am instrument -w -e cmd '$command' $package/.SandboxShell" \ +"$ADB" shell "am instrument -w -e cmd '$command' $package/com.astrobotquest.vrhost.SandboxShell" \ | sed -e 's/^INSTRUMENTATION_RESULT: stream=//' -e '/^INSTRUMENTATION_CODE/d' diff --git a/tools/quest-sandbox-test.sh b/tools/quest-sandbox-test.sh index b28b6d1..72cb011 100644 --- a/tools/quest-sandbox-test.sh +++ b/tools/quest-sandbox-test.sh @@ -39,7 +39,9 @@ ADB="$LOCALAPPDATA/Android/Sdk/platform-tools/adb.exe" # The headset also shows up over the network once wireless debugging has been used: name the # USB connection so that adb does not have to guess. export ANDROID_SERIAL=${ANDROID_SERIAL:-$(cat "$(dirname "${BASH_SOURCE[0]}")/quest-serial.local" 2>/dev/null)} -package=com.astrobotquest.vrhost +# (ASTRO_PACKAGE: the build made by quest-host/build.sh --test-package, if that is the +# one to run.) +package=${ASTRO_PACKAGE:-com.astrobotquest.vrhost} stage=/data/local/tmp/astro game=${ASTRO_GAME:-$stage/games/CUSA12392/eboot.bin} driver=${ASTRO_DRIVER:-runtime/drivers/turnip} @@ -147,7 +149,7 @@ tail -c 2000 core.log echo "$(( $(date +%s) - start )) $(cat /sys/class/kgsl/kgsl-3d0/gpu_busy_percentage | tr -d " %") $(( $(cat /sys/class/kgsl/kgsl-3d0/gpuclk) / 1000000 )) $(( $(cat /sys/class/kgsl/kgsl-3d0/temp) / 1000 )) $(cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq | while read f; do printf "%s " $(( f / 1000 )); done)" sleep 5; done' > build/quest/run/gpu.txt 2>&1 & sampler=$! -"$ADB" shell "am instrument -w -e cmd '$command' $package/.SandboxShell" > build/quest/run/instrument.txt 2>&1 +"$ADB" shell "am instrument -w -e cmd '$command' $package/com.astrobotquest.vrhost.SandboxShell" > build/quest/run/instrument.txt 2>&1 kill $sampler 2>/dev/null rm -rf build/quest/run/screenshots build/quest/run/log build/quest/run/rt_dump diff --git a/tools/quest-selftest.sh b/tools/quest-selftest.sh index cc9a4f1..648d638 100644 --- a/tools/quest-selftest.sh +++ b/tools/quest-selftest.sh @@ -10,6 +10,10 @@ # ASTRO_FRESH=1 runs with the runtime as the installed app carries it (unpacked from the APK's # assets, like on the app's first start) instead of with the build in build/arm64. # +# HOST_... among the variables are the test's own (README-QUEST-VR.md, "Testing on the headset +# without wearing it"): a display and a compositor stand-in, where the head looks, what is +# said of the controller (HOST_PAD), how often a picture is kept (HOST_SHOTS). +# # ASTRO_MIC=real makes the game's microphone the headset's own instead of the test signal the # app otherwise feeds it (a second of noise every five); the app needs the permission for it # (adb shell pm grant com.astrobotquest.vrhost android.permission.RECORD_AUDIO). @@ -20,7 +24,9 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) ADB="$LOCALAPPDATA/Android/Sdk/platform-tools/adb.exe" # The headset also shows up over the network: name the USB connection. export ANDROID_SERIAL=${ANDROID_SERIAL:-$(cat "$(dirname "${BASH_SOURCE[0]}")/quest-serial.local" 2>/dev/null)} -package=com.astrobotquest.vrhost +# (ASTRO_PACKAGE: the build made by quest-host/build.sh --test-package, if that is the +# one to run.) +package=${ASTRO_PACKAGE:-com.astrobotquest.vrhost} stage=/data/local/tmp/astro seconds=${1:-60} shift $(( $# < 1 ? $# : 1 )) @@ -46,10 +52,10 @@ if [ -n "${ASTRO_FRESH:-}" ]; then else # The runtime the app unpacks on its first start, with the current build of the core in it. # Without its stamp, so that the app unpacks its own again when it is started for real. - "$ADB" shell "am instrument -w -e cmd 'mkdir -p runtime/host; rm -f runtime/.stamp; [ -d runtime/drivers ] || cp -r $stage/runtime/. runtime/; cp $stage/shadps4-arm64-fex runtime/host/shadps4-arm64-fex; cp $stage/libkgsl_compat.so runtime/host/libkgsl_compat.so' $package/.SandboxShell" > /dev/null + "$ADB" shell "am instrument -w -e cmd 'mkdir -p runtime/host; rm -f runtime/.stamp; [ -d runtime/drivers ] || cp -r $stage/runtime/. runtime/; cp $stage/shadps4-arm64-fex runtime/host/shadps4-arm64-fex; cp $stage/libkgsl_compat.so runtime/host/libkgsl_compat.so' $package/com.astrobotquest.vrhost.SandboxShell" > /dev/null fi -"$ADB" shell "am instrument -w -e selftest $seconds -e env '$extra_env' -e mic ${ASTRO_MIC:-test} $install $package/.SandboxShell" \ +"$ADB" shell "am instrument -w -e selftest $seconds -e env '$extra_env' -e mic ${ASTRO_MIC:-test} $install $package/com.astrobotquest.vrhost.SandboxShell" \ | sed -e 's/^INSTRUMENTATION_RESULT: stream=//' -e '/^INSTRUMENTATION_CODE/d' rm -rf build/quest/selftest/* diff --git a/tools/quest-xrprobe.sh b/tools/quest-xrprobe.sh index f8fc049..893042c 100644 --- a/tools/quest-xrprobe.sh +++ b/tools/quest-xrprobe.sh @@ -19,10 +19,12 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) ADB="$LOCALAPPDATA/Android/Sdk/platform-tools/adb.exe" export ANDROID_SERIAL=${ANDROID_SERIAL:-$(cat "$(dirname "${BASH_SOURCE[0]}")/quest-serial.local" 2>/dev/null)} export MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*' -package=com.astrobotquest.vrhost +# (ASTRO_PACKAGE: the build made by quest-host/build.sh --test-package, if that is the +# one to run.) +package=${ASTRO_PACKAGE:-com.astrobotquest.vrhost} level=${1:-3} -"$ADB" shell "am instrument -w -e xrprobe $level $package/.SandboxShell" \ +"$ADB" shell "am instrument -w -e xrprobe $level $package/com.astrobotquest.vrhost.SandboxShell" \ | sed -e 's/^INSTRUMENTATION_RESULT: stream=//' -e '/^INSTRUMENTATION_CODE/d' mkdir -p "$root/build/quest" "$ADB" exec-out "run-as $package cat files/xrprobe.log" > "$root/build/quest/xrprobe.log" 2>/dev/null diff --git a/tools/tests/held_pad_test.cpp b/tools/tests/held_pad_test.cpp new file mode 100644 index 0000000..22ae26d --- /dev/null +++ b/tools/tests/held_pad_test.cpp @@ -0,0 +1,204 @@ +// Checks what the Quest app makes of two hands holding a gamepad (quest-host/cpp/held_pad.h): +// where the gamepad is, which way it is turned, how far it is rolled and how far its front +// is tilted up, each with the right sign. +// +// clang-cl /std:c++latest /EHsc -fuse-ld=lld /I quest-host/cpp tools/tests/held_pad_test.cpp +// /Fe:build/tests/held_pad_test.exe +#include +#include +#include + +#include "held_pad.h" + +using namespace HeldPad; + +static int failures = 0; + +static void Check(bool ok, const std::string& what) { + std::printf("%s %s\n", ok ? "ok " : "FAIL", what.c_str()); + failures += ok ? 0 : 1; +} + +static bool Near(float a, float b, float within = 0.002f) { + return std::abs(a - b) <= within; +} + +static bool Near(const Vec& a, const Vec& b, float within = 0.002f) { + return Near(a.x, b.x, within) && Near(a.y, b.y, within) && Near(a.z, b.z, within); +} + +static constexpr float Pi = 3.14159265f; + +static float Radians(float degrees) { + return degrees * Pi / 180.0f; +} + +/// Two hands around a gamepad whose middle is at `middle`, turned `yaw` to the left and with +/// its right side `roll` above level, the hands pointing `pitch` above level. +static Hands Hold(const Vec& middle, float yaw, float roll, float pitch, float apart = 0.2f) { + const Vec right{std::cos(yaw) * std::cos(roll), std::sin(roll), -std::sin(yaw) * std::cos(roll)}; + const Vec level_front{-std::sin(yaw), 0.0f, -std::cos(yaw)}; + const Vec level_up = Cross(right, level_front); + const Vec pointing{level_front.x * std::cos(pitch) + level_up.x * std::sin(pitch), + level_front.y * std::cos(pitch) + level_up.y * std::sin(pitch), + level_front.z * std::cos(pitch) + level_up.z * std::sin(pitch)}; + Hands hands; + hands.left_palm = Sub(middle, Scale(right, apart * 0.5f)); + hands.right_palm = Sub(middle, Scale(right, -apart * 0.5f)); + hands.pointing_known = true; + // Each hand points inwards as well, by a third of a right angle. + const auto inwards = [&](float side) { + const Vec v{pointing.x * std::cos(0.5f) + right.x * side * std::sin(0.5f), + pointing.y * std::cos(0.5f) + right.y * side * std::sin(0.5f), + pointing.z * std::cos(0.5f) + right.z * side * std::sin(0.5f)}; + return v; + }; + hands.left_pointing = inwards(1.0f); + hands.right_pointing = inwards(-1.0f); + return hands; +} + +int main() { + const Vec x{1.0f, 0.0f, 0.0f}, y{0.0f, 1.0f, 0.0f}, z{0.0f, 0.0f, 1.0f}; + + // The rotation from three directions, and back. + { + const Quat q = FromAxes(x, y, z); + Check(Near(q.x, 0.0f) && Near(q.y, 0.0f) && Near(q.z, 0.0f) && Near(std::abs(q.w), 1.0f), + "right, up and back where they always are: no rotation"); + bool all = true; + for (int i = 0; i < 200; ++i) { + // Any attitude: turned, then tilted, then rolled. + const float a = 0.37f * static_cast(i), b = 0.11f * static_cast(i) - 1.4f, + c = 0.23f * static_cast(i); + const Vec right{std::cos(a) * std::cos(c), std::sin(c), -std::sin(a) * std::cos(c)}; + const Vec level_front{-std::sin(a), 0.0f, -std::cos(a)}; + const Vec level_up = Cross(right, level_front); + const Vec front{level_front.x * std::cos(b) + level_up.x * std::sin(b), + level_front.y * std::cos(b) + level_up.y * std::sin(b), + level_front.z * std::cos(b) + level_up.z * std::sin(b)}; + const Vec up = Cross(right, front); + const Quat turned = FromAxes(right, up, Scale(front, -1.0f)); + all = all && Near(Rotate(turned, x), right, 0.001f) && Near(Rotate(turned, y), up, 0.001f) && + Near(Rotate(turned, z), Scale(front, -1.0f), 0.001f); + } + Check(all, "any three directions make the rotation that takes the axes to them"); + } + + // Held level, straight ahead. + { + const auto pose = Locate(Hold({0.0f, -0.3f, -0.4f}, 0.0f, 0.0f, 0.0f), 0.0f, 0.5f); + Check(pose.has_value() && pose->turned_known, "two hands a gamepad's width apart hold one"); + Check(Near(pose->position, {0.0f, -0.285f, -0.435f}), + "its middle is a little ahead of and above the middle between the palms"); + Check(Near(pose->yaw, 0.0f) && Near(pose->pitch, 0.0f) && pose->pitch_measured, + "level hands pointing ahead: neither turned nor tilted"); + Check(Near(Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}), {0.0f, 0.0f, -1.0f}) && + Near(Rotate(pose->orientation, y), y), + "its front points straight ahead, its face up"); + } + + // Not a gamepad. + Check(!Locate(Hold({}, 0.0f, 0.0f, 0.0f, 0.4f), 0.0f, 0.0f).has_value(), + "hands further apart than a gamepad is wide hold none"); + Check(!Locate(Hold({}, 0.0f, 0.0f, 0.0f, 0.03f), 0.0f, 0.0f).has_value(), + "nor do hands that touch"); + + // Turned to the left. + { + const auto pose = Locate(Hold({0.1f, -0.3f, -0.4f}, Radians(30.0f), 0.0f, 0.0f), 0.0f, 0.0f); + Check(pose && Near(pose->yaw, Radians(30.0f)), "turned to the left: a heading to the left"); + const Vec front = Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}); + Check(Near(front, {-0.5f, 0.0f, -std::cos(Radians(30.0f))}), "its front points to the left"); + Check(Near(pose->position.x, 0.1f - 0.5f * 0.035f) && + Near(pose->position.z, -0.4f - std::cos(Radians(30.0f)) * 0.035f), + "and its middle is ahead of the palms the way it points"); + } + { + const auto pose = Locate(Hold({}, Radians(-45.0f), 0.0f, 0.0f), 0.0f, 0.0f); + Check(pose && Near(pose->yaw, Radians(-45.0f)) && + Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}).x > 0.7f, + "turned to the right: a heading to the right"); + } + + // Rolled: the right hand higher. + { + const auto pose = Locate(Hold({}, 0.0f, Radians(20.0f), 0.0f), 0.0f, 0.0f); + const Vec right = Rotate(pose->orientation, x); + const Vec front = Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}); + Check(pose && Near(right.y, std::sin(Radians(20.0f))), "the right hand higher: its right side is up"); + Check(Near(front, {0.0f, 0.0f, -1.0f}) && Near(pose->pitch, 0.0f), + "and its front still points level and ahead"); + } + + // Tilted: the hands point up. + { + const auto pose = Locate(Hold({}, 0.0f, 0.0f, Radians(30.0f)), 0.0f, 0.0f); + const Vec front = Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}); + Check(pose && pose->pitch_measured && Near(pose->pitch, Radians(30.0f)), + "hands pointing up: tilted up by as much, however far they point inwards"); + Check(Near(front, {0.0f, 0.5f, -std::cos(Radians(30.0f))}), "its front points up"); + Check(Rotate(pose->orientation, y).z > 0.49f, "and its face leans towards the player"); + } + { + const auto pose = Locate(Hold({}, 0.0f, 0.0f, Radians(-40.0f)), 0.0f, 0.0f); + Check(pose && Near(pose->pitch, Radians(-40.0f)) && + Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}).y < -0.6f, + "hands pointing down: tilted down"); + } + { + const auto pose = Locate(Hold({}, 0.0f, 0.0f, Radians(30.0f)), Radians(-10.0f), 0.0f); + Check(pose && Near(pose->pitch, Radians(30.0f)) && + Near(Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}).y, std::sin(Radians(20.0f))), + "a gamepad taken to point lower than the hands does, by so much"); + } + + // All at once. + { + const float yaw = Radians(25.0f), roll = Radians(-15.0f), pitch = Radians(35.0f); + const auto pose = Locate(Hold({0.05f, -0.2f, -0.5f}, yaw, roll, pitch), 0.0f, 0.0f); + const Vec right = Rotate(pose->orientation, x); + const Vec front = Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}); + const Vec up = Rotate(pose->orientation, y); + Check(pose && Near(pose->yaw, yaw) && Near(pose->pitch, pitch) && Near(right.y, std::sin(roll)), + "turned, rolled and tilted at once: each is told apart"); + Check(Near(Dot(right, front), 0.0f) && Near(Dot(right, up), 0.0f) && Near(Dot(front, up), 0.0f) && + Near(Length(right), 1.0f) && Near(Length(front), 1.0f), + "and what comes out is a rotation"); + } + + // What the headset cannot tell. + { + Hands hands = Hold({}, 0.0f, 0.0f, Radians(50.0f)); + hands.pointing_known = false; + const auto pose = Locate(hands, 0.0f, Radians(20.0f)); + Check(pose && pose->turned_known && !pose->pitch_measured && Near(pose->pitch, Radians(20.0f)) && + Near(Rotate(pose->orientation, {0.0f, 0.0f, -1.0f}).y, std::sin(Radians(20.0f))), + "where the hands point unknown: tilted the way it was taken to be"); + } + { + Hands hands; + hands.left_palm = {0.0f, -0.4f, -0.4f}; + hands.right_palm = {0.02f, -0.2f, -0.4f}; + const auto pose = Locate(hands, 0.0f, 0.0f); + Check(pose && !pose->turned_known && Near(pose->position, {0.01f, -0.285f, -0.4f}), + "one hand above the other: somewhere, but turned no telling how"); + } + + // Following, a little behind. + { + const Quat quarter = FromAxes({0.0f, 0.0f, -1.0f}, y, x); + const Quat half = Towards(Quat{}, quarter, 0.5f); + const Vec front = Rotate(half, {0.0f, 0.0f, -1.0f}); + Check(Near(front, {-std::sqrt(0.5f), 0.0f, -std::sqrt(0.5f)}, 0.01f), + "half the way to a quarter turn is an eighth of a turn"); + const Quat other{-quarter.x, -quarter.y, -quarter.z, -quarter.w}; + const Vec same = Rotate(Towards(Quat{}, other, 0.5f), {0.0f, 0.0f, -1.0f}); + Check(Near(same, front, 0.01f), "whichever of its two quaternions the attitude is given as"); + Check(Near(Rotate(Towards(Quat{}, quarter, 1.0f), x), {0.0f, 0.0f, -1.0f}, 0.01f), + "all the way is there"); + } + + std::printf("%d failed\n", failures); + return failures == 0 ? 0 : 1; +} diff --git a/tools/tests/pad_router_test.cpp b/tools/tests/pad_router_test.cpp new file mode 100644 index 0000000..627c82a --- /dev/null +++ b/tools/tests/pad_router_test.cpp @@ -0,0 +1,456 @@ +// Checks what the Quest app makes of the controllers a player has (quest-host/cpp/pad_router.h): +// which of a gamepad and the headset's own controllers the game is played with, what the +// headset's controllers' buttons and sticks become, the right stick as a finger on the +// touchpad, and what buttons mean about the view besides. +// +// clang-cl /std:c++latest /EHsc -fuse-ld=lld /I quest-host/cpp /I shadps4-arm64-main/src/input +// tools/tests/pad_router_test.cpp /Fe:build/tests/pad_router_test.exe +#include +#include +#include +#include + +#include "pad_router.h" + +static int failures = 0; + +static void Check(bool ok, const std::string& what) { + std::printf("%s %s\n", ok ? "ok " : "FAIL", what.c_str()); + failures += ok ? 0 : 1; +} + +/// A router with what it sends to the game and what it says kept for looking at. +struct Bench { + PadRouter router; + PadRouter::Output out; + int sent{}; + std::vector notices; + double time{}; + + Bench() { + router.SetSink([this](const PadRouter::Output& output) { + out = output; + ++sent; + }); + router.SetNotice([this](const char* text) { notices.emplace_back(text); }); + } + + /// Time passes, as it does once for every picture the headset shows. + void Pass(double seconds, const TouchState* touch = nullptr) { + const double until = time + seconds; + while (time < until - 1e-9) { + time += 1.0 / 90.0; + if (touch != nullptr) { + router.SetTouch(*touch, time); + } else { + router.Tick(time); + } + } + } + + bool Said(const std::string& part) const { + for (const std::string& notice : notices) { + if (notice.find(part) != std::string::npos) { + return true; + } + } + return false; + } + + PadRouter::Source Source() { + return router.Look().source; + } +}; + +static TouchState Held() { + TouchState touch; + touch.present = true; + return touch; +} + +int main() { + using Source = PadRouter::Source; + namespace Button = PadButton; + + { + Bench bench; + bench.router.SetPlaying(true, 0.0); + bench.Pass(0.1); + Check(bench.Source() == Source::None, "nothing to play with: no controller"); + const PadState& pad = bench.out.pad; + Check(bench.sent == 1 && pad.buttons == 0 && pad.left_x == 128 && pad.left_y == 128 && + pad.right_x == 128 && pad.right_y == 128 && pad.left_trigger == 0 && + pad.right_trigger == 0 && !pad.touch_down && !pad.has_motion && !bench.out.blowing, + "and the game is given a resting gamepad"); + } + + // --- a gamepad on its own --------------------------------------------------------------- + { + Bench bench; + bench.router.SetGamepadConnected(true, 0.0); + bench.router.SetPlaying(true, 0.0); + Check(bench.Source() == Source::Gamepad, "a gamepad alone is what the game is played with"); + + GamepadState pad; + pad.buttons = Button::Square | Button::L1; + pad.left_x = 1.0f; + pad.left_y = -1.0f; + pad.left_trigger = 0.5f; + bench.router.SetGamepad(pad, bench.time); + Check(bench.out.pad.buttons == (Button::Square | Button::L1), "its buttons are the game's"); + Check(bench.out.pad.left_x == 255 && bench.out.pad.left_y == 1 && bench.out.pad.right_x == 128 && + bench.out.pad.right_y == 128, + "its sticks are the game's (right and towards the player are the high numbers)"); + Check(bench.out.pad.left_trigger == 128 && bench.out.pad.right_trigger == 0, "and its triggers"); + Check(!bench.out.pad.has_motion, "a gamepad that says nothing of how it is held has no motion sensors"); + Check(bench.router.TakeRecenter() == 0, "square and L1 mean nothing about the view"); + + const float gyro[3]{0.1f, 0.2f, 0.3f}; + const float accel[3]{1.0f, 9.0f, 2.0f}; + bench.router.SetGamepadMotion(gyro, accel, bench.time); + Check(bench.out.pad.has_motion && bench.out.pad.gyro[1] == 0.2f && bench.out.pad.accel[2] == 2.0f, + "its motion sensors are passed on"); + Check(bench.router.Look().gamepad_motion && bench.router.Look().gamepad_accel[1] == 9.0f, + "and the host knows it has them"); + + // The first press of X. + pad = {}; + pad.buttons = Button::Cross; + bench.router.SetGamepad(pad, bench.time); + Check(bench.router.TakeRecenter() == PadRouter::RecenterSeat, "the first press of X takes the seat"); + Check(bench.Said("first press of X"), "and says so"); + bench.router.SetGamepad({}, bench.time); + bench.router.SetGamepad(pad, bench.time); + Check(bench.router.TakeRecenter() == 0, "the second does not"); + + // OPTIONS. + pad = {}; + pad.buttons = Button::Options; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.5); + Check(bench.out.pad.buttons == Button::Options, "the game sees OPTIONS"); + Check(bench.router.TakeRecenter() == 0, "OPTIONS held for half a second resets nothing"); + bench.Pass(0.6); + Check(bench.router.TakeRecenter() == (PadRouter::RecenterPad | PadRouter::RecenterSeat), + "held for a second it resets the view"); + bench.Pass(2.0); + Check(bench.router.TakeRecenter() == 0, "once"); + bench.router.SetGamepad({}, bench.time); + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.3); + bench.router.SetGamepad({}, bench.time); + bench.Pass(1.0); + Check(bench.router.TakeRecenter() == 0, "OPTIONS pressed briefly resets nothing"); + + // A gamepad that is gone. + pad = {}; + pad.buttons = Button::Triangle; + bench.router.SetGamepad(pad, bench.time); + bench.router.SetGamepadConnected(false, bench.time); + Check(bench.out.pad.buttons == 0 && !bench.out.pad.has_motion && bench.Source() == Source::None, + "nothing is held down on a gamepad that is gone"); + } + + // --- a gamepad's touchpad, or what stands in for it --------------------------------------- + { + Bench bench; + bench.router.SetGamepadConnected(true, 0.0); + bench.router.SetPlaying(true, 0.0); + GamepadState pad; + pad.touch_down = true; + pad.touch_x = 0.25f; + pad.touch_y = 1.0f; + bench.router.SetGamepad(pad, bench.time); + Check(bench.out.pad.touch_down && bench.out.pad.touch_x == 480 && bench.out.pad.touch_y == 1079, + "a finger on the gamepad's own touchpad is where it is"); + pad.right_y = 1.0f; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.2); + Check(bench.out.pad.touch_x == 480 && bench.out.pad.touch_y == 1079 && bench.out.pad.right_y == 255, + "the right stick moves no finger while a real one is on the pad"); + pad.touch_down = false; + bench.router.SetGamepad(pad, bench.time); + Check(!bench.out.pad.touch_down && bench.out.pad.touch_x == 480, + "the real finger lifts from where it was, and the stick's is not down the moment it does"); + pad.right_y = 0.0f; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.2); + + // The catapult at the end of a level: pull the stick towards you and let go. + pad.right_y = 1.0f; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.05); + Check(bench.out.pad.touch_down, "the right stick pulled puts a finger on the pad"); + bench.Pass(0.35); + Check(bench.out.pad.touch_down && bench.out.pad.touch_y == 1079 && bench.out.pad.touch_x == 960, + "at the pad's near edge"); + Check(bench.out.pad.right_y == 255, "(a gamepad's right stick stays a stick as well)"); + pad.right_y = 0.0f; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.2); + Check(!bench.out.pad.touch_down && bench.out.pad.touch_y == 1079, + "let go, the finger lifts there, not back in the middle: the catapult shoots"); + + // A button for the press. + pad = {}; + pad.buttons = Button::TouchPad; + bench.router.SetGamepad(pad, bench.time); + Check(bench.out.pad.touch_down && bench.out.pad.touch_x == 960 && bench.out.pad.touch_y == 540, + "a button that presses the touchpad touches it in the middle"); + bench.router.SetGamepad({}, bench.time); + Check(!bench.out.pad.touch_down, "and lets go of it"); + + bench.router.SetStickTouchpad(false); + pad = {}; + pad.right_x = -1.0f; + bench.router.SetGamepad(pad, bench.time); + bench.Pass(0.3); + Check(!bench.out.pad.touch_down && bench.out.pad.right_x == 1, + "switched off, the right stick is only a stick"); + } + + // --- the headset's own controllers on their own ------------------------------------------- + { + Bench bench; + bench.router.SetPlaying(true, 0.0); + TouchState touch = Held(); + bench.Pass(0.1, &touch); + Check(bench.Source() == Source::Touch, "the headset's controllers alone play the game once they are picked up"); + Check(bench.Said("headset's own controllers"), "and the log says so"); + + const auto buttons = [&](const TouchState& state) { + bench.Pass(0.05, &state); + return bench.out.pad.buttons; + }; + touch = Held(); + touch.a = true; + Check(buttons(touch) == Button::Cross, "A is cross"); + Check(bench.router.TakeRecenter() == PadRouter::RecenterSeat, "and its first press takes the seat"); + touch = Held(); + touch.b = true; + Check(buttons(touch) == Button::Square, "B is square"); + touch = Held(); + touch.x = true; + Check(buttons(touch) == Button::Circle && !bench.out.blowing, "X is circle"); + touch = Held(); + touch.y = true; + Check(buttons(touch) == Button::Triangle && !bench.out.blowing, "Y is triangle"); + touch.x = true; + Check(buttons(touch) == 0 && bench.out.blowing, "X and Y together are neither: they blow"); + touch = Held(); + Check(buttons(touch) == 0 && !bench.out.blowing, "for as long as they are held"); + touch.left_grip = 0.9f; + touch.right_grip = 0.4f; + Check(buttons(touch) == Button::L1, "a grip squeezed is L1, one half squeezed is nothing"); + touch = Held(); + touch.right_grip = 1.0f; + Check(buttons(touch) == Button::R1, "the other grip is R1"); + touch = Held(); + touch.left_trigger = 0.05f; + touch.right_trigger = 0.5f; + Check(buttons(touch) == Button::R2 && bench.out.pad.left_trigger == 0 && bench.out.pad.right_trigger == 128, + "a trigger pulled is R2 by as much as it is pulled; one not quite at rest is not pulled"); + touch = Held(); + touch.left_trigger = 1.0f; + Check(buttons(touch) == Button::L2 && bench.out.pad.left_trigger == 255, "the other trigger is L2"); + touch = Held(); + touch.menu = true; + Check(buttons(touch) == Button::Options, "the left controller's menu button is OPTIONS"); + bench.Pass(1.1, &touch); + Check(bench.router.TakeRecenter() == (PadRouter::RecenterPad | PadRouter::RecenterSeat), + "held for a second it resets the view"); + touch = Held(); + touch.left_stick_in = true; + Check(buttons(touch) == Button::L3 && !bench.out.pad.touch_down, "the left stick pressed in is L3"); + touch = Held(); + touch.right_stick_in = true; + Check(buttons(touch) == Button::TouchPad && bench.out.pad.touch_down && bench.out.pad.touch_x == 960 && + bench.out.pad.touch_y == 540, + "the right stick pressed in presses the touchpad, in its middle"); + touch.left_stick_in = true; + Check(buttons(touch) == 0 && !bench.out.pad.touch_down, "both sticks pressed in press nothing"); + Check(bench.router.TakeRecenter() == (PadRouter::RecenterPad | PadRouter::RecenterSeat), + "they reset the view"); + bench.Pass(0.5, &touch); + Check(bench.router.TakeRecenter() == 0, "once"); + + // Sticks: the runtime has away from the player as up, a gamepad the other way round. + touch = Held(); + touch.left_x = -1.0f; + touch.left_y = 1.0f; + bench.Pass(0.05, &touch); + Check(bench.out.pad.left_x == 1 && bench.out.pad.left_y == 1, + "the left stick pushed away and to the left is the gamepad's, pushed away and to the left"); + + // The right stick is the finger on the touchpad, and nothing else. + touch = Held(); + bench.Pass(0.2, &touch); + touch.right_y = -1.0f; + bench.Pass(0.4, &touch); + Check(bench.out.pad.touch_down && bench.out.pad.touch_y == 1079 && bench.out.pad.touch_x == 960, + "the right stick pulled towards the player drags a finger to the pad's near edge"); + Check(bench.out.pad.right_x == 128 && bench.out.pad.right_y == 128, + "while the game's right stick stays where it is"); + touch.right_y = 0.0f; + bench.Pass(0.2, &touch); + Check(!bench.out.pad.touch_down && bench.out.pad.touch_y == 1079, "let go, the finger lifts there"); + Check(!bench.out.pad.has_motion, "the headset's controllers have no motion sensors to pass on: they are tracked"); + + // The game loses the player's attention (the headset's menu): nothing stays held. + touch = Held(); + touch.a = true; + touch.left_x = 1.0f; + bench.Pass(0.05, &touch); + TouchState away = touch; + away.present = false; + bench.Pass(0.05, &away); + Check(bench.out.pad.buttons == 0 && bench.out.pad.left_x == 128 && bench.Source() == Source::Touch, + "what was held is let go when the controllers fall silent, and they stay the controller"); + } + + // --- both at hand --------------------------------------------------------------------------- + { + Bench bench; + bench.router.SetGamepadConnected(true, 0.0); + bench.router.SetPlaying(true, 0.0); + const float gyro[3]{}; + const float accel[3]{0.0f, 9.81f, 0.0f}; + bench.router.SetGamepadMotion(gyro, accel, 0.0); + TouchState touch = Held(); + bench.Pass(0.1, &touch); + Check(bench.Source() == Source::Gamepad, "with both at hand and neither used yet: the gamepad"); + + // What a hand can do without a controller in it never takes over from the gamepad. + touch.left_trigger = 1.0f; + touch.right_trigger = 1.0f; + touch.left_grip = 1.0f; + touch.right_grip = 1.0f; + touch.menu = true; + bench.Pass(0.2, &touch); + Check(bench.Source() == Source::Gamepad && bench.out.pad.buttons == 0, + "triggers, grips and the menu button of the headset's controllers do not take over"); + touch = Held(); + bench.Pass(0.1, &touch); + touch.a = true; + bench.Pass(0.05, &touch); + Check(bench.Source() == Source::Touch && bench.out.pad.buttons == Button::Cross, + "a button under a thumb does, and is the game's at once"); + Check(!bench.out.pad.has_motion, "the gamepad's motion sensors no longer count then"); + + touch = Held(); + bench.Pass(0.1, &touch); + bench.router.SetGamepadMotion(gyro, accel, bench.time); + Check(bench.Source() == Source::Touch && !bench.out.pad.has_motion, + "a gamepad that only lies there, sensors running, takes nothing back"); + GamepadState pad; + pad.left_x = 0.3f; + bench.router.SetGamepad(pad, bench.time); + Check(bench.Source() == Source::Touch, "nor does a stick that does not rest at its centre"); + pad.left_x = 0.9f; + bench.router.SetGamepad(pad, bench.time); + Check(bench.Source() == Source::Gamepad && bench.out.pad.left_x == 242 && bench.out.pad.has_motion, + "a stick pushed does"); + bench.Pass(0.1, &touch); + touch.right_x = 0.9f; + bench.Pass(0.05, &touch); + Check(bench.Source() == Source::Touch, "and a stick of the headset's controllers takes it back"); + pad = {}; + pad.buttons = Button::Circle; + bench.router.SetGamepad(pad, bench.time); + Check(bench.Source() == Source::Gamepad && bench.out.pad.buttons == Button::Circle, + "a button of the gamepad takes over again"); + pad = {}; + pad.right_trigger = 0.8f; + bench.router.SetGamepad({}, bench.time); + touch = Held(); + touch.b = true; + bench.Pass(0.05, &touch); + Check(bench.Source() == Source::Touch, "(back to the headset's)"); + bench.router.SetGamepad(pad, bench.time); + Check(bench.Source() == Source::Gamepad, "so does a trigger of the gamepad"); + + // The gamepad's battery gives out. + bench.router.SetGamepadConnected(false, bench.time); + touch = Held(); + bench.Pass(0.05, &touch); + Check(bench.Source() == Source::Touch, "the gamepad gone, the headset's controllers play on"); + // It comes back, and the headset's controllers are put down. + bench.router.SetGamepadConnected(true, bench.time); + Check(bench.Source() == Source::Touch, "a gamepad that comes back does not take over by being there"); + TouchState away; + bench.Pass(0.2, &away); + Check(bench.Source() == Source::Touch, "nor because the headset's controllers were put down"); + pad = {}; + pad.buttons = Button::Cross; + bench.router.SetGamepad(pad, bench.time); + Check(bench.Source() == Source::Gamepad, "its first button does"); + const int switches = static_cast(bench.notices.size()); + bench.Pass(1.0, &away); + Check(static_cast(bench.notices.size()) == switches, "and nothing changes while nothing is pressed"); + } + + // --- the host's own menu before the game --------------------------------------------------- + { + Bench bench; + TouchState touch = Held(); + bench.Pass(0.1, &touch); + Check(bench.router.TouchMenu() == 0, "the menu: nothing held, nothing said"); + touch.left_x = -0.9f; + bench.Pass(0.05, &touch); + Check(bench.router.TouchMenu() == PadRouter::MenuLeft, "the left stick to the left is left"); + touch = Held(); + touch.right_x = 0.9f; + bench.Pass(0.05, &touch); + Check(bench.router.TouchMenu() == PadRouter::MenuRight, "the right stick to the right is right"); + touch = Held(); + touch.right_trigger = 1.0f; + bench.Pass(0.05, &touch); + Check(bench.router.TouchMenu() == PadRouter::MenuConfirm, "a trigger confirms"); + Check(bench.out.pad.buttons == 0 && bench.sent <= 1, "none of which reaches the game"); + Check(bench.router.TakeRecenter() == 0, "or resets anything"); + + // The game starts with the trigger that confirmed still pulled, and A pressed besides. + touch.a = true; + bench.Pass(0.05, &touch); + Check(bench.router.TouchMenu() == PadRouter::MenuConfirm, "A confirms as well"); + bench.router.SetPlaying(true, bench.time); + bench.Pass(0.2, &touch); + Check(bench.out.pad.buttons == 0 && bench.out.pad.right_trigger == 0, + "what confirmed the menu is not pressed in the game"); + Check(bench.router.TakeRecenter() == 0, "and takes no seat"); + touch.right_trigger = 0.0f; + bench.Pass(0.05, &touch); + Check(bench.out.pad.buttons == 0, "A stays held back while it is held"); + touch.a = false; + bench.Pass(0.05, &touch); + touch.a = true; + touch.right_trigger = 1.0f; + bench.Pass(0.05, &touch); + Check(bench.out.pad.buttons == (Button::Cross | Button::R2) && bench.out.pad.right_trigger == 255, + "let go and pressed again, both are the game's"); + Check(bench.router.TakeRecenter() == PadRouter::RecenterSeat, "and that press of A takes the seat"); + + TouchState away; + bench.Pass(0.05, &away); + Check(bench.router.TouchMenu() == 0, "controllers that were put down say nothing to a menu"); + } + { + // The player chooses in the menu with the headset's controllers while a gamepad is on. + Bench bench; + bench.router.SetGamepadConnected(true, 0.0); + TouchState touch = Held(); + bench.Pass(0.1, &touch); + Check(bench.Source() == Source::Gamepad, "a gamepad that is on is the controller before the menu"); + bench.router.Use(Source::Touch, bench.time); + Check(bench.Source() == Source::Touch, "choosing in the menu with the headset's controllers makes them it"); + bench.router.Use(Source::Gamepad, bench.time); + Check(bench.Source() == Source::Gamepad, "and choosing with the gamepad, the gamepad"); + TouchState away; + bench.Pass(0.1, &away); + bench.router.Use(Source::Touch, bench.time); + Check(bench.Source() == Source::Gamepad, "controllers nobody holds are not chosen with"); + } + + std::printf("%d failed\n", failures); + return failures == 0 ? 0 : 1; +}