Merge one-eye: a new calibration of our tracker starts the eye weights over

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.dockerignore

@@ -16,3 +16,9 @@
 **/.env.*
 frametop-report-*.txt
 .frame-job.d/
+# the fine-tuned hand models: the image fetches its own pinned copy (hands/models.sh)
+hands/models/student/
+hands/models/.student-*
+# model versions downloaded from Input Settings (models/catalog.py): each install fetches its own
+hands/models/versions/
+gaze/tracker/models/

.github/workflows/release.yml

@@ -53,13 +53,19 @@ jobs:
         run: |
           podman run --rm --entrypoint sh "$FT_IMAGE" -c '
             set -e
-            for p in ft-screens ft-pointer ft-powerd ft-gaze ft-gazepanel ft-stream ft-eyegrab; do
+            for p in ft-screens ft-spotpanel ft-pointer ft-powerd ft-gaze ft-gazepanel ft-stream ft-eyegrab whisper-server; do
               test -x /opt/frametop/bin/$p
             done
             test -f /src/frametop/pointer/driver/build/driver_ft_pointer.so
+            test -x /src/frametop/hands/build/ft-camd
+            test -x /src/frametop/hands/build/ft-hands
+            test -x /src/frametop/hands/rec/build/ft-handpanel
+            /src/frametop/hands/models.sh check
             test -x /src/frametop/pack/build/distrobox/install
             /src/frametop/gaze/tracker/build/venv/bin/python -c "import numpy, cv2"
-            python3 -c "import PySide6"'
+            /src/frametop/voice/build/venv/bin/python -c "import numpy, sentencepiece, sherpa_onnx"
+            python3 -c "import PySide6, huggingface_hub"
+            command -v zstd >/dev/null'
 
       # The FrameDrop manifest names the zip's URL in this repo's release (a fork's, in a fork).
       - name: Frametop.zip
@@ -83,9 +89,17 @@ jobs:
         env:
           GH_TOKEN: ${{ github.token }}
         run: |
-          pre=()
-          [[ $VERSION == *-* ]] && pre=(--prerelease)
+          pre=() notes=pack/release-notes.md
+          if [[ $VERSION == *-* ]]; then
+            pre=(--prerelease)
+            # get.sh --release asks for a channel and offers stable first: an experimental
+            # release's install line names its channel.
+            notes=$RUNNER_TEMP/release-notes.md
+            sed 's/bash -s -- --release`/bash -s -- --release --experimental`/' pack/release-notes.md >"$notes"
+            grep -q -- '--release --experimental`' "$notes" ||
+              echo "::warning::pack/release-notes.md has no get.sh --release line to mark experimental"
+          fi
           gh release create "$GITHUB_REF_NAME" --draft --verify-tag "${pre[@]}" \
-            --title "Frametop $VERSION" --notes-file pack/release-notes.md --generate-notes \
+            --title "Frametop $VERSION" --notes-file "$notes" --generate-notes \
             framedrop/build/Frametop.zip framedrop/build/frametop.framedrop.json \
             framedrop/build/frametop-release.json framedrop/build/SHA256SUMS

.gitignore

@@ -11,3 +11,5 @@ frametop-report-*.txt
 *.pgm
 .venv/
 .frame-job.d/
+# eye net versions picked in Input Settings (models/catalog.py downloads them; CC BY-NC 4.0)
+gaze/tracker/models/

README.md

@@ -4,8 +4,10 @@ Frametop is a desktop for the Steam Frame that runs on the headset itself, with
 
 - **Screens that are real monitors.** Each KDE Plasma screen has its own resolution and shape: an ultrawide in front, portrait screens beside it. Move, resize, curve, and roll them, or pin one to your wrist or your head. They come back to your layout when the desktop starts.
 - **Windows that float on their own.** Take any app window off the screens into a panel of its own with Meta+Shift+F, its title bar button, or Launch as Standalone. It stays part of the desktop, so drag and drop and the clipboard still work between floating windows and the screens.
+- **Your other computers' monitors.** Stream a Windows PC's or a Mac's monitors, or a virtual monitor at any size, as screens of their own, with the same controls. See [Remote displays](#remote-displays).
 - **Profiles.** Save where your screens are, which ones show, and which apps are open where. Switch to a profile from Display Settings, a key, or its launcher entry, or start the desktop in one.
 - **One mouse for all of SteamVR.** A Bluetooth mouse drives a small dot anchored in the room. It works the screens, the dashboard, Steam, and overlays, and hands the laser back when you pick up a controller.
+- **Your hands (experimental).** Hand tracking from the headset's own cameras: poke a screen to click, pinch and grab screens to move, resize, and throw them, and see your hands over the screens.
 - **Look and click (experimental).** In gaze mode the pointer goes where you look. Meta+J and Meta+K, or the mouse buttons, click and fine-tune, and each correction teaches the tracker. Calibration runs in the headset.
 - **Made for long sessions.** The displays turn off when the headset isn't used, even on a stand that makes it seem worn. It stays awake on the charger, and you can reach the desktop remotely over VNC.
 
@@ -13,11 +15,13 @@ Two settings apps come with it: Frametop Display Settings for the screens, profi
 
 Frametop is an independent project, not made by or affiliated with Valve.
 
+Frametop lives at [Frametop/frametop](https://github.com/Frametop/frametop): code, releases, issues, and pull requests. It moved there from DeeJanuz/frametop on 2026-10-09; the old links and clones still work.
+
 Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions, ideas, and help with your setup.
 
 ## Install on the headset
 
-> **Frametop doesn't work on the SteamOS beta right now.** On the beta (SteamOS 0.4.3), gaze mode can't read the eye tracker, and the desktop has started without its taskbar ([#15](https://github.com/DeeJanuz/frametop/issues/15)). Use the stable SteamOS release until this note is gone.
+> **SteamOS 0.4:** SteamOS 0.4 moved the eye tracker's data that gaze mode reads. This version of Frametop reads both SteamOS 0.3's and 0.4's, and it's tested on 0.4.5. Run `scripts/doctor.sh` after the update: it says whether the eye tracker's layout is one Frametop knows. It also says whether the update deleted the Bluetooth fixes or our eye tracker's frame grabber, which happens when they were installed by Frametop 0.3.0-exp.3 or older. Reinstall what it names (`setup/bluetooth/install.sh install`, `gaze/tracker/install.sh`). From then on they're kept through updates.
 
 You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or the on-screen one), and about 3 GB of free space.
 
@@ -26,14 +30,14 @@ You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or th
 3. Run:
 
    ```
-   curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash
+   curl -fsSL https://frametop.github.io/frametop/get.sh | bash
    ```
 
-   It asks which version you want: stable (the `main` branch, tested releases) or experimental (the `experimental` branch, the newest features, less tested). Then it clones the repo into `~/frametop` and runs `install.sh`. To choose without the question, add `-s -- --stable` or `-s -- --experimental` after `bash`. By hand, the same is `git clone https://github.com/DeeJanuz/frametop.git ~/frametop`, then `cd ~/frametop` and `./install.sh` (add `--branch experimental` to the clone for experimental).
+   It asks which version you want: stable (the `main` branch, tested releases) or experimental (the `experimental` branch, the newest features, less tested). Then it clones the repo into `~/frametop` and runs `install.sh`. To choose without the question, add `-s -- --stable` or `-s -- --experimental` after `bash`. By hand, the same is `git clone https://github.com/Frametop/frametop.git ~/frametop`, then `cd ~/frametop` and `./install.sh` (add `--branch experimental` to the clone for experimental).
 
-   The third choice, experimental release, downloads Frametop already built (`Frametop.zip`, about 1.1 GB, from the [Frametop organization's releases](https://github.com/Frametop/frametop/releases)) and installs it without compiling anything. `-s -- --release --experimental` picks it without the question.
+   The third and fourth choices, stable release and experimental release, download Frametop already built (`Frametop.zip`, about 1.2 GB, from the [releases](https://github.com/Frametop/frametop/releases)) and install it without compiling anything. `-s -- --release` picks the stable release without the question, and `-s -- --release --experimental` the experimental one.
 
-   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you four things along the way: whether to install gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The eye tracker and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
+   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks along the way whether to install each optional part: voice dictation (yes by default; it downloads about 320 MB of speech models), hand tracking (experimental, yes by default), in a release the [Hand Recorder](https://frametop.github.io/frametop/hand-recorder) (yes by default; it records your hands for Frametop's open hand dataset), gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), in a release the Eye Recorder (yes by default, with our eye tracker; it records your eyes for Frametop's eye tracking dataset), and the Bluetooth fixes, then whether to restart SteamVR. The stable version asks only about gaze mode, the eye tracker, and the Bluetooth fixes. Hand tracking, the Hand Recorder, the eye tracker, and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
 
 After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings. SteamOS's own single-screen desktop is still there, as Native Desktop in the same list.
 
@@ -51,7 +55,7 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
 
 | Do this | To get this |
 | --- | --- |
-| Point near the bottom of a screen | Its controls fade in: the bar, the curve and roll buttons, and the resize tab on the corner |
+| Point near the bottom of a screen | Its controls fade in: the bar, the curve and roll buttons, and, near a corner, a soft glow round it: press there and drag to resize |
 | Drag the bar under a screen | Moves the screen; scroll while dragging to push it away or pull it closer. With the mouse, hold right while dragging to tilt it |
 | Drag the tab on a screen's bottom right corner | Resizes the screen |
 | Click the curve button (next to the bar) | Curves the screen around you, or flattens it |
@@ -60,6 +64,7 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
 | Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
 | Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
 | Meta+Alt+Tab, or Meta+Alt+Shift+Tab | Spins every screen and floating window around you together, like a lazy susan, so the next one on your right (or left) glides to straight ahead, with the pointer and typing going to it. Their arrangement stays the same: the room turns instead of you. Tap again to keep going; Meta+Shift+R puts the screens back. Pinned screens stay where they are |
+| Meta+Space, on any keyboard | Opens spotwatch, a search pill in front of you: type an app's or a remote display's name and press Enter. A running app's window spins to the front, an app that isn't running opens floating, and a remote display connects and spins to the front. Shift+Enter (or a right click) has more: a private window, a new window, docked on this screen, or one of its windows. Esc closes it. See [docs/spotwatch.md](docs/spotwatch.md) |
 | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
 | Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
 | Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
@@ -98,6 +103,22 @@ A profile is a named setup: where the screens are, with their sizes and pins, wh
 | Pick a profile under Arrangement and press Open profile | Switches to it: the screens move, open windows of its apps go to their places, and the apps that aren't open start. Nothing closes |
 | Pick a profile under Start in profile, or run its entry (Frametop: NAME) from SteamVR's Launch a program list | The desktop starts in that profile, or switches to it if it's running. A profile can also go on a key combination, mouse button, or controller button in Frametop Input Settings |
 
+### Remote displays
+
+Remote displays bring another computer's monitors into Frametop. Each one is a panel with a screen's controls: move it, resize it, curve it, pin it, and save it in a profile. The mouse, the keyboard and the lasers work on it, and its sound plays on the Frame. The computer streams with Sunshine's protocol, and the Frame decodes the streams in hardware.
+
+Each computer needs a host kit once:
+
+| Computer | What it streams | Set up with |
+| --- | --- | --- |
+| Windows PC | Its monitors, and virtual monitors at any size | Vibepollo with Frametop's build: [host/windows/README.md](host/windows/README.md) |
+| Mac with Apple silicon | Its monitors | Sunshine, one instance per monitor, and Frametop Streams in the menu bar to turn them on and off: [host/macos/README.md](host/macos/README.md) |
+| Linux PC (Arch, CachyOS) | Its monitors, and virtual monitors | Vibepollo with Frametop's build, not tested yet: [host/linux/README.md](host/linux/README.md) |
+
+Then, on the Frame, open Frametop Remote Displays (in the app menu, or Remote displays in Display Settings), choose Add computer, sign in with the computer's Web UI login, and pick its displays. Each display has a Connected switch, a Shown switch, and its stream's resolution, frame rate and bitrate. Meta+Space finds a remote display by name and brings it to the front. A PC with a Steam Link dongle can stream over it, which skips your router.
+
+A monitor you stream switches to the stream's resolution while it streams (Windows), so pick a size the monitor has, or one with the same shape. A computer that's asleep can't be reached. [docs/reference.md](docs/reference.md#frametop-remote-displays) has the details.
+
 ### Pause for VR games
 
 Frametop pauses while a VR game runs, so the game gets the headset's CPU and GPU, and comes back a few seconds after the game ends. Paused, the screens hide and the desktop nearly stops drawing, but its windows stay open. Gaze mode's eye tracking stops, and so do remote desktop and hand tracking if they run. The mouse works as a plain mouse in SteamVR.
@@ -124,6 +145,21 @@ In gaze mode the pointer goes where you look, and the mouse or the keyboard does
 
 By default the mouse only corrects: while the gaze has the pointer, moving the mouse does nothing until you hold a button. Each correction before a click teaches the tracker where it was off. A correction past the learning limit (55 degrees by default, about half of what you can see) starts a quick check instead. Calibrate, on the Gaze page, runs a full calibration in a panel in front of you: look at each dot and click. Check headset fit shows how well the eye tracker sees your eyes. [gaze/README.md](gaze/README.md) has the details.
 
+### Hand tracking (experimental)
+
+Hand tracking uses the headset's own cameras, with no controllers. The installer offers it (or run `hands/run.sh install` later; it needs `sudo` once, to let its camera broker read the cameras). Installed, it runs while the Frametop desktop does, and parks while Frametop is paused for a VR game. It uses a lot of the headset's CPU.
+
+| Do this | To get this |
+| --- | --- |
+| Poke a screen with your index finger | A click there |
+| Pinch a screen's bar, edge, or corner, and move | Moves or resizes the screen |
+| Hold a screen like a card, or in a fist | Carries it; turn your wrist to turn it. Flick it away, or let go fast to throw it |
+| Hold both hands on a screen and pull them apart | Stretches it |
+| Hold an open hand out to a far screen, then make a fist and pull it back ("grab that") | Brings it to arm's reach |
+| Turn a hand palm up toward your face, look at it, and pinch the ring that shows | Opens the Steam menu |
+
+The Hands page of Frametop Input Settings sets when it runs (with Frametop, always with SteamVR, or off) and switches each hand interaction on or off, live. `ft-handsctl status` shows how it's doing. [hands/README.md](hands/README.md) has the details.
+
 ### Leave the headset on a stand and reach it remotely
 
 To keep the Frame on and connected while you're not wearing it, for SSH, remote desktop, or anything else running on it, open the Power tab in Frametop Display Settings:
@@ -135,7 +171,7 @@ With the displays off, the headset keeps tracking and rendering, so it uses abou
 
 ## Known limitations
 
-This is an early release, tested on one Steam Frame (SteamOS 0.3.0 build 20260922, SteamVR 2.17.10).
+This is an early release, tested on one Steam Frame (SteamOS 0.4.5 build 20261007, SteamVR 2.18.2; before that SteamOS 0.3.0 build 20260922, SteamVR 2.17.10).
 
 - A SteamOS or SteamVR update can break parts of it until Frametop catches up. After an update, run `cd ~/frametop && scripts/doctor.sh` in a terminal. It checks what Frametop needs from SteamOS, and says what changed since the versions you last marked as working and what to try. Once everything works, `scripts/doctor.sh --mark-good` records the versions. If something stops working, please report it.
 - The first install downloads 1–2 GB for the build container and compiles everything on the headset, which takes several minutes.
@@ -159,14 +195,28 @@ In a terminal on the headset, run:
 cd ~/frametop && scripts/report.sh
 ```
 
-This writes `frametop-report-<date>.txt` with version numbers, service states, settings, and recent logs. Bluetooth addresses and the headset's serial number are masked. It takes up to half a minute, because it also checks gaze mode: it starts the gaze service for a moment to see whether the eye tracker sends. If gaze or its calibration doesn't work, run it while you wear the headset. `scripts/gaze-report.py` prints only the gaze part, with what looks wrong first. Then [open an issue](https://github.com/DeeJanuz/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.
+From a release, start in `~/.local/share/frametop/releases/current` instead of `~/frametop`.
+
+This writes `frametop-report-<date>.txt` with version numbers, settings, service states, CPU, memory and heat, and the state and recent logs of every part: the desktop (Frametop's keyboard, the Steam menu, the screens), gaze, hand tracking (its models, cameras, and how often it sees each hand), and voice (its models, the microphone, and whether whisper-server transcribes). It takes up to half a minute, because it starts the gaze service for a moment to see whether the eye tracker sends. Bluetooth addresses and the headset's serial number are masked, and what you said to Frametop Voice is left out.
+
+To report on one part only, name it: `--system`, `--desktop`, `--gaze`, `--hands` or `--voice` (several are fine, for example `scripts/report.sh --hands --voice`).
+
+If the problem is something you can make happen, like a window that won't drag, a pinch that doesn't register, or dictation that types nothing, add `--watch`. After the report it records for 60 seconds (`--watch 120` for longer) while you make it happen in the headset, and keeps the logs from that time:
+
+- desktop: when the Steam menu opens and closes, which laser drags what, where typing goes, and when Frametop's keyboard opens or why it doesn't
+- gaze: which eye tracker the gaze service uses, and whether its samples come in
+- hands: each second, how many frames hand tracking made, how often it saw each hand, and how close the fingertips came
+- voice: what Frametop Voice did (the number of words, not the words) and the microphone's level each second (no sound is kept)
+- system: every 5 seconds, the CPU's load, clocks and heat, and the busiest programs
+
+If gaze or its calibration doesn't work, run the report while you wear the headset. `scripts/gaze-report.py` prints only the gaze part, with what looks wrong first. Then [open an issue](https://github.com/Frametop/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.
 
 ## Update
 
 Run the same command again. It updates `~/frametop` to the latest of the version you have (or switches, if you pick the other one) and installs it:
 
 ```
-curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash
+curl -fsSL https://frametop.github.io/frametop/get.sh | bash
 ```
 
 Or by hand: `cd ~/frametop && git pull && ./install.sh`.
@@ -176,7 +226,7 @@ Or by hand: `cd ~/frametop && git pull && ./install.sh`.
 In a terminal on the headset, run:
 
 ```
-curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
+curl -fsSL https://frametop.github.io/frametop/uninstall.sh | bash
 ```
 
 It works in two steps, so it never takes away the keyboard, mouse, or desktop you're using while it runs:
@@ -208,10 +258,14 @@ A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland com
 | `input/` | The input relay (Bluetooth mice and keyboards, button maps). |
 | `pointer/` | The 3D mouse: SteamVR driver, helper service, and a probe tool. |
 | `power/` | ft-powerd: turns the displays off while the headset isn't used. |
+| `voice/` | Frametop Voice: dictation and wake words, recognised on the headset; the input relay types the text. See [voice/README.md](voice/README.md). |
 | `gaze/` | Gaze mode (experimental): the gaze service, its calibration panel, our own eye tracker, and the gaze probe. See [gaze/README.md](gaze/README.md). |
-| `hands/` | Hand tracking (experimental, deferred: the installer doesn't offer it). See [hands/README.md](hands/README.md). |
+| `hands/` | Hand tracking (experimental): the camera broker and the tracker. See [hands/README.md](hands/README.md). |
 | `display-settings/`, `input-settings/` | The two settings apps (Kirigami, Python). |
 | `remote/` | Frametop Remote Access, the app that turns remote desktop over VNC on and off. |
+| `remote-displays/` | Frametop Remote Displays, the app that adds other computers' monitors as screens. |
+| `stream/` | ft-stream, one per remote display: receives and decodes its stream for ft-screens. |
+| `host/` | The host kits for remote displays: Windows, macOS and Linux. |
 | `setup/` | The build container and the Bluetooth fixes. See [setup/README.md](setup/README.md). |
 | `scripts/` | Helpers the installers use. They run commands locally on the Frame, or over SSH from a PC. |
 
@@ -256,4 +310,9 @@ Programs built in the `dev` container link against its glibc, which is newer tha
 
 ## License
 
-MIT. See [LICENSE](LICENSE).
+MIT. See [LICENSE](LICENSE). The rendered hands' 3D models in `screens/assets/hands` (see its [README](screens/assets/hands/README.md); each folder has its licence) are from:
+
+- the W3C Immersive Web Working Group's WebXR Input Profiles, `generic-hand` (MIT, Copyright (c) 2019 Amazon), the default;
+- Babylon.js's hand meshes by Patrick Ryan, https://github.com/BabylonJS/Assets (CC BY 4.0, https://creativecommons.org/licenses/by/4.0/), unchanged;
+- the Mixed Reality Toolkit: MRTK3's OpenXR hand (BSD-3-Clause, Copyright (c) 2025, Mixed Reality Toolkit Contributors) and MRTK2's rigged hand (MIT, Copyright (c) Microsoft Corporation), converted to glTF;
+- Godot XR Tools' hands by DigitalN8m4r3 (Miodrag Sejic), CC0 1.0.

display-settings/ft_display_settings.py

@@ -10,8 +10,9 @@ the dev container:
     1920x1080 worth of pixels, rotation for portrait.)
   - Visibility (ft-screens): when the screens show (always, only with the SteamVR
     dashboard open, while you look at a controller, or only when toggled), the wrist
-    angle within which a pinned screen shows, and pinning each screen to a wrist or
-    your head.
+    angle within which a pinned screen shows, pinning each screen to a wrist or your
+    head, and whether the desktop's notifications go to the Steam session
+    (NOTIFY_FORWARD, session/ft-notifyfwd; its state from its log).
   - Layout: a preset (curved or flat, rows, distance, gap, height) or a named layout
     saved from where the screens are, with a preview; arrange now; save the current
     arrangement under a name; rename and delete; arrange automatically when the
@@ -62,6 +63,8 @@ FT_POWERD = "\0ft_powerd"
 # nothing was saved.
 STEAM_SLEEP_AC_DEFAULT = 3600
 SCALES = [0.75, 1.0, 1.25, 4 / 3, 1.5, 1.75, 2.0]
+NOTIFY_FORWARD = ("auto", "on", "off")  # session/ft-notifyfwd; frametop.conf's NOTIFY_FORWARD
+NOTIFYFWD_LOG = "/tmp/frametop-notifyfwd.log"  # its last line is its state (/tmp is the host's)
 ROTATIONS = [("normal", "Landscape"), ("left", "Portrait"), ("right", "Portrait (flipped)")]
 
 
@@ -121,6 +124,7 @@ class Backend(QObject):
         self._steam_busy = False
         self._steamDone.connect(self._steam_done, Qt.QueuedConnection)
         self._pins = []     # each running screen's pin: none | left | right | head
+        self._notify_state = ""
         self.poll = QTimer(interval=3000, timeout=self._check_running)
         self.poll.start()
         self._check_running()
@@ -133,8 +137,10 @@ class Backend(QObject):
                 return cast(conf.get(key, default))
             except ValueError:
                 return default
+        notify = conf.get("NOTIFY_FORWARD", "auto")
         return {"screens": num("SCREENS", 2, int), "width": num("WIDTH", 1920, int),
-                "height": num("HEIGHT", 1080, int), "physWidth": num("PHYS_WIDTH", 1.6)}
+                "height": num("HEIGHT", 1080, int), "physWidth": num("PHYS_WIDTH", 1.6),
+                "notifyForward": notify if notify in NOTIFY_FORWARD else "off"}
 
     def _check_running(self):
         # The session's own Wayland socket (host processes' environments aren't readable
@@ -142,12 +148,15 @@ class Backend(QObject):
         running = os.path.exists(f"/run/user/{os.getuid()}/frametop/wayland-0")
         count = self._screens_running() if running and ft_layout.backend() == "screens" else 0
         pins = self._read_pins(count)
-        if running != self._running or count != self._running_count or pins != self._pins:
+        notify_state = self._read_notify_state() if running else ""
+        if (running != self._running or count != self._running_count or pins != self._pins
+                or notify_state != self._notify_state):
             if running != self._running or count != self._running_count:
                 self._started = self._conf() if running else {}
             self._running = running
             self._running_count = count
             self._pins = pins
+            self._notify_state = notify_state
             self.changed.emit()
         self._check_powerd()
 
@@ -159,6 +168,22 @@ class Backend(QObject):
             pins.append(f[16] if len(f) > 16 else "none")
         return pins
 
+    @staticmethod
+    def _read_notify_state():
+        """ft-notifyfwd's: forwarding | waiting (no usable server on the user bus) | "" (not running)."""
+        try:
+            with open(NOTIFYFWD_LOG) as f:
+                lines = f.read().splitlines()
+        except OSError:
+            return ""
+        # the last state line: errors it logs in between (a failed Notify) don't change it
+        for line in reversed(lines):
+            if line == "ft-notifyfwd: forwarding":
+                return "forwarding"
+            if line.startswith("ft-notifyfwd: not forwarding"):
+                return "waiting"
+        return ""
+
     def _ask_screens(self, text):
         """Request/reply to ft-screens; None if it isn't running."""
         try:
@@ -383,6 +408,26 @@ class Backend(QObject):
     def toggleScreens(self):
         self._ask_screens("toggle")
 
+    # --- the desktop's notifications in the Steam session (session/ft-notifyfwd) ---
+    @Property(str, notify=changed)
+    def notifyForward(self):
+        return self._conf()["notifyForward"]
+
+    @Property(str, notify=changed)
+    def notifyForwardState(self):
+        return self._notify_state
+
+    @Property(bool, notify=changed)
+    def notifyRestartNeeded(self):
+        """The session writes ft-notifyfwd's autostart entry when it starts."""
+        return self._running and bool(self._started) and self._started["notifyForward"] != self._conf()["notifyForward"]
+
+    @Slot(str)
+    def setNotifyForward(self, mode):
+        if mode in NOTIFY_FORWARD:
+            write_conf_value("NOTIFY_FORWARD", mode)
+            self.changed.emit()
+
     # --- ft-screens: visibility and pinning ---
     @Property("QVariantMap", notify=changed)
     def visibility(self):

display-settings/main.qml

@@ -958,6 +958,60 @@ Kirigami.ApplicationWindow {
                           + "your head where you let go. Choosing a pin above keeps the screen where it is now, so place "
                           + "it first. Save as profile… (Layout) keeps pins."
                 }
+
+                Kirigami.FormLayout {
+                    Layout.fillWidth: true
+
+                    Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Notifications" }
+
+                    Repeater {
+                        model: [
+                            { value: "auto", text: "Show them in the Steam session", help: "The desktop's notifications go to the Steam session's notification server, such as one that shows them in VR, with their buttons. Plasma's own popups stay off meanwhile; its history keeps them. Not to SteamOS's own server, which can't show them on the Frame." },
+                            { value: "on", text: "Always send them to the Steam session", help: "Whatever serves notifications there, SteamOS's own server too, which can't show them on the Frame: they're lost then." },
+                            { value: "off", text: "Keep them in the desktop", help: "Plasma shows them on the desktop's screens." }
+                        ]
+                        delegate: ColumnLayout {
+                            required property var modelData
+                            spacing: 0
+                            // "on" can lose them (SteamOS's server): set in frametop.conf, shown only then
+                            visible: modelData.value !== "on" || backend.notifyForward === "on"
+                            Controls.RadioButton {
+                                text: modelData.text
+                                checked: backend.notifyForward === modelData.value
+                                onToggled: if (checked) backend.setNotifyForward(modelData.value)
+                            }
+                            Controls.Label {
+                                text: modelData.help
+                                opacity: 0.7
+                                font: Kirigami.Theme.smallFont
+                                leftPadding: Kirigami.Units.gridUnit * 1.6
+                                wrapMode: Text.Wrap
+                                Layout.maximumWidth: Kirigami.Units.gridUnit * 26
+                            }
+                        }
+                    }
+                    Controls.Label {
+                        Kirigami.FormData.label: "Now:"
+                        visible: backend.desktopRunning
+                        text: backend.notifyForwardState === "forwarding" ? "In the Steam session"
+                            : backend.notifyForwardState === "waiting" ? "In the desktop: the Steam session has no notification server that can show them"
+                            : "In the desktop"
+                        wrapMode: Text.Wrap
+                        Layout.maximumWidth: Kirigami.Units.gridUnit * 26
+                    }
+                    RowLayout {
+                        visible: backend.notifyRestartNeeded
+                        Controls.Label {
+                            text: "Applies when the desktop starts again."
+                            wrapMode: Text.Wrap
+                        }
+                        Controls.Button {
+                            text: "Restart desktop"
+                            icon.name: "system-reboot"
+                            onClicked: restartDialog.open()
+                        }
+                    }
+                }
             }
         }
     }

docs/design.md

@@ -20,10 +20,12 @@ SteamOS already shows a single-screen Plasma desktop in VR by running KWin neste
 
 It also leaves the panels to SteamVR's dashboard, which places them and doesn't expose their position to other programs.
 
-ft-screens replaces it. It's a small wlroots compositor that hosts the nested KWin. KWin's nested Wayland backend opens one window per output and resizes an output when the host configures its window, so ft-screens can give each screen any size, live. KWin needs `wl_compositor` v4 or later, `wl_shm`, `wl_seat`, `xdg_wm_base`, and `zwp_linux_dmabuf_v1` v4 with feedback from its host; the rest is optional.
+ft-screens replaces it. It's a small wlroots compositor that hosts the nested KWin. KWin's nested Wayland backend opens one window per output and resizes an output when the host configures its window, so ft-screens can give each screen any size, live. KWin needs `wl_compositor` v4 or later, `wl_shm`, `wl_seat`, `xdg_wm_base`, and `zwp_linux_dmabuf_v1` v4 with feedback from its host. KWin 6.6, which SteamOS build 20261009.6183636 ships, also refuses to start unless the host offers `wp_single_pixel_buffer_manager_v1`, `wp_viewporter`, `zwp_pointer_constraints_v1`, and `wp_presentation`. It draws each output's pixels on a synchronized subsurface and attaches a 1×1 single-pixel buffer to the toplevel, stretched to the output with viewporter. ft-screens presents the largest DMA-BUF in that tree, the toplevel's own included (KWin 6.2 draws there), and sends presentation feedback for the toplevel: KWin stops rendering after two frames without it. `xdg_toplevel_icon_manager_v1` and `zwp_keyboard_shortcuts_inhibit_manager_v1` stay optional. `scripts/update-check.py` fails when a newer KWin requires a host protocol this ft-screens binary does not name.
 
 Frames go to SteamVR the same way gamescope sends them: OpenVR's `IVRIPCResourceManagerClient` imports each DMA-BUF (`ImportDmabuf`), and the overlay shows it as a `TextureType_SharedTextureHandle` texture. There's no copy and no size limit. The Frame's SteamVR supports this interface, but the OpenVR header bundled with SteamVR's samples predates it, so the build fetches a pinned header from Valve's openvr repository.
 
+The small textures ft-screens draws itself (the controls, the hand glows, "grab that"'s halo, the carousel) go the same way, through `screens/overlaytex.cpp`: one linear DMA-BUF per distinct image, imported once and shared. Not `SetOverlayRaw`: the Frame's SteamVR never gives back the memory block each call takes, so a process's 201st call fails ("Refusing to create memory block because 201 blocks are already outstanding" in the journal) and the overlay keeps its old texture at its new size. That was the glow bug of 2026-10-15, after about 40 minutes of hand use. Other programs call it only a few times, or as a fallback.
+
 Before settling on this, we tried KWin's screencast virtual outputs. KWin 6.2.5 crashed (`std::out_of_range` in `textureForOutput`) streaming one while nested in gamescope, and its headless backend doesn't implement virtual outputs at all.
 
 A few wlroots details worth knowing: `wlr_shm_create` wants DRM format codes, not `wl_shm` ones, and KWin asks for server-side decorations before its first commit, when setting the mode would assert, so ft-screens answers on the first commit.
@@ -74,7 +76,7 @@ In a game, Frametop's panels work like SteamVR's own floating windows: point a c
 
 - Disabling a nested output keeps its host window. `Output::applyChanges` only flips `enabled`, KWin stops rendering it (no more commits), and Plasma drops its desktop view. So ft-screens keeps the same toplevel, and its screen numbers stay put. Enabling the output again resumes on the same toplevel.
 - Each output's host window is titled `KDE Wayland Compositor WL-<n>`, with `- Output disabled` appended while it's disabled (`WaylandOutput::updateWindowTitle`, on every `enabledChanged`). ft-screens reads the title to tell screens (`WL-0` to `WL-<SCREENS-1>`) from spares, and to see a spare turn on and off.
-- A spare resized while it's disabled comes up at the new size on its first frame, so floating a window needn't blink. Outputs with gaps between them are accepted, so ft-floatd places spares apart from the screens and from each other, within Xwayland's 32767-pixel limit.
+- A spare resized while it's disabled comes up at the new size on its first frame, so floating a window needn't blink. Outputs with gaps between them are accepted, so ft-floatd places spares apart from the screens and from each other: a grid below them, 4000 logical pixels a cell. Xwayland's coordinates are the layout's times its largest output scale, and they wrap past 32767, so an X11 window on a spare past that took no clicks (Brave on the 4th spare at x 27000 and scale 1.25 sat at x -31506; KWin still focused it). The grid ends at 20000 for 16 spares, within the limit up to scale 1.6.
 - KWin keeps a Wayland popup inside its parent's output (`XdgPopupWindow::updateRelativePlacement` uses the output's placement area), and X11 apps place their menus within the monitor. That's why a floating window's output has a margin around the window: menus and dropdowns open past the window's edges, into the margin.
 - KWin makes a nested output the size it's configured to times its scale, rounded (at 1.5 it lays out 1067 × 667 on a 1600 × 1000 buffer), and gives the buffer a whole buffer scale (1.2 becomes 2). A buffer whose size isn't a multiple of that is a protocol error that disconnects KWin, so ft-floatd sizes spares in multiples of it. After a scale change, ft-floatd asks for the output's size again in the new scale's terms, or the next configure would make it the old size times the scale.
 - **Virtual outputs don't work.** `createVirtualOutput` makes an output window but never adds it to the backend's `m_outputs`, so `findOutput()` returns null when the pointer enters it, and the next line dereferences it (`Q_ASSERT` is compiled out). A click on such a panel would crash KWin. This rules out virtual outputs (`stream_virtual_output`) for floating windows without a patched KWin.
@@ -117,7 +119,13 @@ Driver poses are in SteamVR's raw tracking space, and client programs work in th
 
 Frametop's SteamVR clients (ft-pointer, ft-screens, ft-gaze) connect as a background app first and switch to an overlay app only once that works. `VR_Init` as an overlay app starts vrserver itself when none is running, and one started that way from the dev container never finds the headset. At a boot where the gamescope session timed out, systemd dropped `steamvr.service`'s start job, the pointer service (ordered only `After=` it) started anyway, and its vrserver made every SteamVR launch fail with `HmdNotFound`. SteamOS's health check then kept resetting the Steam client and tried to fall back to the previous OS slot. The units also say `Requisite=steamvr.service`, so they don't start at all when SteamVR's start fails.
 
-The driver starts disconnected, because holding the right-hand role while SteamVR starts leaves the Steam UI stuck on its loading icon. It connects when the mouse is used and claims the right hand. SteamVR keeps a hand role reserved for a disconnected device that still asks for it, so the driver switches its role hint between right hand (connected) and opt-out (not connected).
+The driver starts disconnected, because holding the right-hand role while SteamVR starts leaves the Steam UI stuck on its loading icon. It connects when the mouse is used. With a hand role (`POINTER_ROLE=right`, the default before the treadmill role) it claims that hand: SteamVR keeps a hand role reserved for a disconnected device that still asks for it, so the driver switches its role hint between the hand (connected) and opt-out (not connected). The treadmill role (below), now the default, needs no switching.
+
+#### Treadmill role
+
+With a hand role, a Frame controller held in that hand takes the role back (its touch sensors count as use), and the virtual controller is left with no role and no clicks. So mixing a controller in one hand with the 3D mouse or a bare hand's ray in the other fails whenever that controller is merely held. The treadmill role (4) is no hand: the real controllers keep both hand roles, and the virtual controller never fights them. It held the dashboard laser in the gaze-first tests (2026-09-30, branch `gaze-first`), with two conditions. SteamVR gives a device the `/user/treadmill` path only when the device is added, so the driver must hint the role from `Activate`, not on connect: the `role` in its `default.vrsettings` (or `driver_ft_pointer/role` in SteamVR's settings), and switching it needs a SteamVR restart. And the compositor binding needs `/user/treadmill` entries next to the hand ones (`ft_pointer_vrcompositor.json`). A treadmill reports no controller role (`GetControllerRoleForTrackedDeviceIndex` is invalid), so the helper's "no hand role" release skips it. The laser claim (`/input/a`, bound to `switchlaserhand`) works the same; SteamVR took the laser back to our device 11-15 ms after a controller took it.
+
+A treadmill can't move SteamVR's own floating windows by their grab bars, though: SteamVR's dashboard (a web page in vrwebhelper) starts a window's move in `UndockedOverlay.startFloatingWindowMove`, which reads the pressing device from the mouse event's CapsLock and NumLock bits (head, left hand, right hand) and, for anything else, logs "mouse event carried no input path" and does nothing. Clicks and resizes don't ask, so only moves broke, from 2026-10-09, when treadmill became the default (found 2026-10-13, SteamVR 2.18.2; with role 2 the same laser moved them). `pointer/dashpatch` puts that back. `treadmill-move.js` wraps the grab bar's handler: a press with no device runs it as the right hand, with that one pose query answered by `/user/treadmill`, and the move then follows `/user/treadmill`. `ft-dashpatch.py run` (`frametop-dashpatch.service`, started with SteamVR) puts it into the dashboard over the Chrome DevTools Protocol, again whenever the page is new, and checks every 30 s that it's still in. The dashboard's debugger is off unless `VRWebHelper/DebuggerEnabled` is on; the installer turns it on (`ft-dashpatch.py on`, from SteamVR's next start) on port 8089, bound to 127.0.0.1. Not the usual 8087: SteamOS's `steamvr-web-debug-portforward.service` forwards 0.0.0.0:8088 to 127.0.0.1:8087, which would open the dashboard to the network. Like Steam's own debugger on 8080, any local program can use it. The uninstaller turns it off again if Frametop turned it on. The patch finds the handler by its name and React by its module, so a SteamVR update that renames them leaves windows unmovable for the treadmill again, without breaking anything else; `ft-dashpatch.py status` shows whether it's in and how many moves it made.
 
 ### The cursor
 
@@ -136,7 +144,7 @@ A few overlays need special handling:
 
 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
 
-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: the session now starts an AT-SPI registry, but apps still need to expose useful accessibility trees (and may need restarting), and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a rightLine truncated
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: the session now starts an AT-SPI registry, but apps still need to expose useful accessibility trees (and may need restarting), and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click, Enter, an index pinch or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still uLine truncated
 
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 
@@ -160,7 +168,7 @@ Keyboards aren't grabbed by default, because a grabbed keyboard's keys went into
 
 The relay never waits on the pointer helper. Its socket to the helper used to block, so when the helper stalled (a layout placement or `grabprobe` holds it for seconds, and the gaze service fills its socket 90 times a second meanwhile), the whole relay stopped with it: keyboards, the volume keys, and pausing. Now what the helper doesn't take waits in order and goes out on the next loops. Mouse moves add up into one while they wait, and a scroll notch is dropped, since scrolling seconds late is no use; presses and releases are kept, so no button stays down. Mouse motion goes to the helper at most every 4 ms, rather than once per report, which from a 1000 Hz mouse was 1000 datagrams a second to a helper that runs every 8 ms; a button sends the motion before it first, so the click lands where the pointer was.
 
-An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
+An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. Until 2026-10-10, `SHARE_KEYS=1` sent a grabbed keyboard's keys to `@frametop_keys` as datagrams for frame-voice. It was off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first got every key typed into the desktop, passwords included. Now frame-voice is part of Frametop (Frametop Voice), and the relay does its keyboard side itself: its push-to-talk key is a key combination, and its text comes to the relay as `type TEXT`, taken only from the relay's own user (`SO_PASSCRED`), and goes where typing goes, like a keyboard's keys: to ft-screens for the desktop, or the virtual keyboard for Steam, never both. Its own uinput keyboard went with it, so SteamVR no longer has to have opened a second keyboard for dictation to reach games.
 
 A mouse button that pointer mode passes through as a key (a side button for Back) is a pointer button to ft-screens, not a key, so it goes to the screen the pointer is on rather than where typing goes. As a key, its release was lost whenever typing moved while it was held (the dashboard closing, a click on another panel, a pause), and wlroots counts presses per button: from then on that button, the lasers' left click included, did nothing in the desktop, and KWin kept it held. So ft-screens tracks the relay's buttons itself, lets a release through whenever it took the press, and releases them when the pointer leaves the screens, its screen hides, or Frametop pauses.
 
@@ -168,7 +176,7 @@ Volume keys must never reach gamescope. With the openvr backend, gamescope sends
 
 Frametop's keyboard opens by itself for a text field on the desktop. The apps run inside the nested KWin, so only KWin knows when a text field has focus, and the way it tells anyone is its input method protocol (`zwp_input_method_v1`): KWin starts one input method program and activates it whenever the focused app turns on text input. `input/ft-textinput` is that program, speaking the Wayland wire protocol directly so it needs nothing but Python on the host. It only reports focus. The gamescope session puts `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` in the systemd user environment; with those, Qt and GTK apps use X input methods and never turn on Wayland text input, so the session script drops them.
 
-The keyboard itself is ft-screens' own panel (`screens/keyboard.cpp`). We tried SteamVR's first (`ShowKeyboardForOverlay`), and on the Frame it doesn't fit a desktop. It's Steam's own panel (`valve.steam.gamepadui.keyboard`), which SteamVR mounts in the dashboard's scene, so with the dashboard closed it opened but wasn't drawn. Placing it in the room ourselves (`SetKeyboardTransformAbsolute`) made it show, but SteamVR moves it to whichever overlay the laser goes to and mounts it again, and while it's open the controllers switch to SteamVR's own laser. Our panel is an overlay like the screens' controls: any laser or the 3D mouse clicks it, nothing moves it, and its keys go out as key presses on ft-screens' seat rather than as text handed back to the input method. So nothing typed leaves ft-screens (a socket to the input method could be claimed by any local process, like `@frametop_keys`), apps without text input (X11, Electron) take the keys too, and they mean what the desktop's keyboard layout says. It's drawn on the CPU and uploaded with `SetOverlayRaw` when a key's look changes; the labels come from stb_truetype, so the container needs no text rendering stack.
+The keyboard itself is ft-screens' own panel (`screens/keyboard.cpp`). We tried SteamVR's first (`ShowKeyboardForOverlay`), and on the Frame it doesn't fit a desktop. It's Steam's own panel (`valve.steam.gamepadui.keyboard`), which SteamVR mounts in the dashboard's scene, so with the dashboard closed it opened but wasn't drawn. Placing it in the room ourselves (`SetKeyboardTransformAbsolute`) made it show, but SteamVR moves it to whichever overlay the laser goes to and mounts it again, and while it's open the controllers switch to SteamVR's own laser. Our panel is an overlay like the screens' controls: any laser or the 3D mouse clicks it, nothing moves it, and its keys go out as key presses on ft-screens' seat rather than as text handed back to the input method. So nothing typed leaves ft-screens (a socket to the input method could be claimed by any local process, as the old `@frametop_keys` could), apps without text input (X11, Electron) take the keys too, and they mean what the desktop's keyboard layout says. It's drawn on the CPU into one of three shared buffers when a key's look changes; the labels come from stb_truetype, so the container needs no text rendering stack.
 
 The Frame controllers can be mapped like mouse buttons, but they aren't input devices on the host: they reach SteamVR over the headset's own radio, and no evdev or hidraw node exists for them. So only a SteamVR client can read them. Overlay apps normally get controller input only while they have input focus, which a background helper never has. SteamVR's experimental global action set priority (`steamvr/globalActionSetPriority`, "Enable global input from overlays") lets an overlay's action set receive input anyway, and takes the inputs it binds from the scene app. Binding every button would take them all from games, so the helper's action manifest puts each button in an action set of its own, and it activates only the sets of mapped buttons. The mapping itself stays in the relay, which does the action, so mice and controllers share one list of actions.
 
@@ -188,6 +196,16 @@ The fallback registry does not exit merely when its bus disconnects in the isola
 
 Run the isolated checks on the host with `/usr/bin/python3 session/test/test_accessibility.py`. They use private D-Bus buses, Xvfb and a GTK3 app, never the production display or input. Native activation uses a small `org.a11y.Bus` test provider pointing to a real private dbus-daemon with the installed registry service; the SteamOS fallback uses the installed bus launcher and broker. The tests check real app-tree discovery, existing owners, concurrent starts, session stop/restart, and teardown. They require test-only PyGObject (Gio and GTK3), Xvfb, and at-spi2-core; the runtime helper uses Python's standard library and the existing host `gdbus`. Actual Plasma autostart and VR desktop restart still require an approved hardware test.
 
+### Notifications outside the desktop
+
+The private bus keeps the desktop's notifications in it: apps notify Plasma, and Plasma's popups appear on the desktop's screens, whose panels may be hidden, or behind a VR game. Putting the desktop on the user bus instead would collide with the Steam session: Steam owns `org.freedesktop.ScreenSaver` there, the Steam session's portals and accessibility bus are there, Plasma would start its parts as units of the user's systemd and push its environment (`DISPLAY`, `WAYLAND_DISPLAY`, its runtime directory) into the bus's and systemd's activation environment, which programs started later in the Steam session would inherit.
+
+`session/ft-notifyfwd` forwards instead. Taking `org.freedesktop.Notifications` from Plasma isn't reliable: plasmashell, the bus's "D-Bus master", registers it with `ReplaceExistingService` and `DontAllowReplacement` (plasma-workspace's `libnotificationmanager/server_p.cpp`), so another owner only keeps it by starting first, and loses it to a Plasma restart in between. Plasma 6.2.5 has a watcher API for this: `org.kde.NotificationManager.RegisterWatcher` on Plasma's notification object, after which Plasma calls `org.kde.NotificationWatcher.Notify` (its own id first, then the spec's arguments with the body as sent and the app's hints, image data included) and `CloseNotification` on the watcher's `/NotificationWatcher`, and `InvokeAction(id, key)`, which makes Plasma emit `ActionInvoked` to the app. The app only accepts that signal from its notification server's owner, which is why a click has to go back through Plasma. The helper copies each notification to the user bus's server keeping the hints' D-Bus types, maps the two servers' ids (a replacement reuses Plasma's), and maps that server's `ActionInvoked` to `InvokeAction`. Its `NotificationClosed` with reason 2 (dismissed) closes the notification in Plasma, so the app sees it closed (as revoked, reason 3); an expired one stays in Plasma's history. While it forwards, it holds an `Inhibit` on Plasma, which turns its popups off as Do not disturb does, and ends with the helper's connection; watchers get every notification regardless. It follows both servers' owners: it registers and inhibits again when Plasma restarts, and in `auto` mode checks `GetServerInformation` whenever the user bus's server changes. SteamOS's `steam_notif_daemon` reports itself as `simple_notif_daemon`; it passes notifications to `~/.steam/root/ubuntu12_32/steam` with a `steam://open_xdg_notification` URL, an x86 program that can't run on the Frame, and the Frame's Steam client ignores that URL anyway, so forwarding there would lose them. The helper is event-driven (GLib's main loop, PyGObject), so it costs nothing between notifications, and exits when the desktop's bus closes.
+
+Notifications sent through the notification portal (`org.freedesktop.portal.Notification.AddNotification`, as Flatpak apps do) reach the watchers too: xdg-desktop-portal-kde passes them to Plasma's `Notify`. Plasma sends no activation token with `InvokeAction`, so KWin's focus stealing prevention may keep a window that an action raises from coming to the front. KDE's inline reply needs the `inline-reply` capability from the other server; without it, it's an action like any other.
+
+Run the isolated checks on the host with `/usr/bin/python3 session/test/test_notifyfwd.py`. Two private buses stand in for the desktop's and the user bus, with a fake Plasma (the watcher API, Inhibit) on one and a fake notification server on the other. They check forwarding with the hints' types, actions coming back, dismissed and expired notifications, replacements and closes mapping ids, `auto` skipping `simple_notif_daemon` until another server replaces it and giving Plasma its popups back when none is left, registering again after a Plasma restart, exiting with the desktop's bus, and doing nothing outside the Frametop desktop.
+
 ### Other session behavior
 
 Steam, not systemd, suspends the Frame: after `system_idle_suspend_ac_sec` (an hour by default) without input on AC power, it logs `Switching to power state: k_ESystemPowerState_Sleep` and suspends, even while charging. It's a Steam setting (Settings → Power → When Plugged In and Idle → Sleep after), which the Stay awake while plugged in switch in Frametop Display Settings sets to Never. SteamVR's standby, which turns the displays off when the headset comes off, is separate; see below.
@@ -200,12 +218,14 @@ A podman container's monitor process (conmon) stays in the cgroup of whatever st
 
 KWin renders with OpenGL through zink on Turnip, Vulkan on the same GPU vrcompositor needs to hit its frame time, and on the Frame that costs CPU too. The nested session started with KWin's defaults: blur and background contrast on (no `[Plugins]` group in its kwinrc) and animations at full length. Blur re-renders what's behind every translucent panel and menu each time it changes, and every animated frame is one more frame for KWin and ft-screens to draw and send. They're off by default in the Frametop desktop. The session script writes them before KWin starts, only where the desktop's own config has no value, once: System Settings deletes a setting put back to KDE's default rather than writing it, so without the marker in `frametoprc` a user who turned blur back on would lose it at the next start. The effect ids (`blur`, `contrast`) are the ones built into KWin 6.2.5 on SteamOS; KWin reads `<id>Enabled` from `[Plugins]`.
 
-The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
+The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. Steam's entry (`steam -silent`, from steamdeck-kde-presets) reached the running Steam client as a command line it ran (`ExecCommandLine` in its console log), since the desktop starts from Steam; SteamOS 0.4 added `-vrdisable -deckard` to it, for Desktop Mode, where Plasma starts Steam itself. The session hides all three for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
 
 Plasma 6.2.5 keeps each panel on a screen number (`lastScreen` in `plasma-org.kde.plasma.desktop-appletsrc`), and the numbers rank the enabled outputs by priority, so 0 is the primary screen. A panel whose number is past the screen count gets no view, and Plasma never moves it: the remap it runs at every start only moves a panel whose number has no desktop, and this desktop keeps a desktop for every output it has seen, spares included. So the taskbar was lost when the number of screens went down, and once it was found saved on a spare output, number 8 of a desktop with three screens ([#18](https://github.com/DeeJanuz/frametop/issues/18)). Before Plasma starts, the session runs `session/fix-panels.py`, which moves any panel numbered past the screen count, with its system tray's containment, to screen 0, keeping its widgets and settings. A panel stays put when screen 0 already has one on that edge, and comes back by itself if the screens do. Before each repair the file is backed up to `<file>.ft-bak.last`. `<file>.ft-bak` keeps it as it was before the first repair and is never overwritten. The repair writes over a moved panel's old screen number, so the backups are the only record of it, and `.ft-bak.last` also keeps everything changed since the first repair. Plasma's scripting can't do this while it runs (`panel.screen` is read-only in 6.2.5), so a lost taskbar comes back at the desktop's next start. `scripts/doctor.sh` and `scripts/report.sh` list the panels and their screens.
 
 Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
 
+KWin 6.2.5 can crash while Plasma logs out (a SIGSEGV in its libei backend's teardown, with a Deskflow client connected), and `kwin_wayland_wrapper` then starts a fresh KWin with no Plasma left to end it: a black desktop that never goes away. The session script watches for this. It finds Plasma's session manager (`ksmserver`) once it has started, waits for it to exit with `waitpid`, and if the session's KWin wrapper is still running 9 seconds later, ends it, which ends the session. Looking for `ksmserver` with `pgrep` every 3 seconds instead cost about 1.3% of a core (38 ms a look). `scripts/update-check.py` has a retest hint for KWin updates: if logout with Deskflow no longer crashes, the watcher can go.
+
 Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
 
 ## Displays off on a stand
@@ -231,7 +251,7 @@ Hiding the screens during a game kept them out of view, but Frametop kept using
 
 On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-steamvr-rel` package), next to KWin, gamescope, and the kernel, so every SteamOS update can bring a new SteamVR too. Frametop survives updates: it lives in the home folder and the `dev` container, the Bluetooth fixes are in `/etc`, which SteamOS keeps across updates, and nothing goes into `/usr`. What an update can break is what Frametop uses from the image. The public OpenVR API is versioned and stays put. The rest is less certain: `IVRIPCResourceManagerClient`, which is newer than the header SteamVR ships; the text `vrcmd --overlays` prints; the eye tracker's shared memory layout; XRService's camera buffers; KWin's nested backend; and behavior Frametop works around, such as the SteamVR Settings page that `ComputeOverlayIntersection` can't find or the scale KWin's nested backend doesn't undo.
 
-`scripts/update-check.py`, which `scripts/doctor.sh` runs, checks what it can directly: that SteamVR still serves every OpenVR interface version the installed programs were built against (read from the binaries), that `vrcmd`'s format still parses, that the eye tracker's shared memory still has a layout ft-gaze knows (stable's, or the 0.4.x beta's, with every field from the timestamp on 5 bytes later), by the same test ft-gaze uses to pick one, and the host files, services, sockets, and driver registration. Behavior can't be checked without someone in the headset, so it records the versions of the packages that matter once things work (`--mark-good`), and after an update names what changed and what to try by hand.
+`scripts/update-check.py`, which `scripts/doctor.sh` runs, checks what it can directly: that SteamVR still serves every OpenVR interface version the installed programs were built against (read from the binaries), that `vrcmd`'s format still parses, that the eye tracker's shared memory still has a layout ft-gaze knows (SteamOS 0.3's, or 0.4's, with every field from the timestamp on 5 bytes later), by the same test ft-gaze uses to pick one, and the host files, services, sockets, and driver registration. Behavior can't be checked without someone in the headset, so it records the versions of the packages that matter once things work (`--mark-good`), and after an update names what changed and what to try by hand.
 
 ## Approaches we dropped
 

docs/floating-windows.md

@@ -28,7 +28,7 @@ The numbers are cited in the code, so they stay as they are. Struck-out text was
 
 | # | Question | Decision |
 |---|---|---|
-| 1 | How many windows can float at once | 8 spare outputs by default, configurable with `FLOAT_SLOTS` (at most 16); a change needs a desktop restart. A Display Settings control isn't built |
+| 1 | How many windows can float at once | 16 spare outputs by default (also the most), configurable with `FLOAT_SLOTS`; a change needs a desktop restart. A Display Settings control isn't built |
 | 2 | Menus and dropdowns | Each floating output has a margin around the window. The panel shows only the window, and each open popup gets a small overlay of its own, cut from the same buffer |
 | 3 | Tearing off | Not built. Drag the title bar past a screen's edge and let go in the air, with a small dead zone past the edge |
 | 4 | Docking by dragging | Not built. Push the window flush against a screen (within about 10 cm), with the landing spot highlighted, and let go |
@@ -73,7 +73,7 @@ Alternatives considered:
 
 ### Where the extra outputs come from: spare outputs
 
-KWin's nested backend opens its outputs at start (`--output-count`). The session starts KWin with the screen count plus `FLOAT_SLOTS` outputs (default 8, at most 16). ft-floatd turns off the spares nothing floats on with `kscreen-doctor` once it starts. Floating a window enables a spare, and docking the window disables it again. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. How KWin's nested backend treats disabled outputs, and why its virtual outputs can't be used instead, is in [design.md](design.md#floating-windows).
+KWin's nested backend opens its outputs at start (`--output-count`). The session starts KWin with the screen count plus `FLOAT_SLOTS` outputs (default and most 16). ft-floatd turns off the spares nothing floats on with `kscreen-doctor` once it starts. Floating a window enables a spare, and docking the window disables it again. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. How KWin's nested backend treats disabled outputs, and why its virtual outputs can't be used instead, is in [design.md](design.md#floating-windows).
 
 ft-screens creates a `screen` for each toplevel in the order they appear, and indexes its settings by that order. Spares come after the screens, so they get indices `SCREENS` and up. Their panels are hidden while their output is disabled.
 

docs/hands-typing.md

@@ -0,0 +1,87 @@
+# Hands while typing on a real keyboard
+
+Branch `hands-typing` (2026-10-10). The problem, from the user's headset session that day (hand
+tracking with student-v10a): typing on the real keyboard made hands click (pinches, middle pinches,
+grips, pokes), and the hand ray took the pointer from the mouse or the gaze during or right after
+typing.
+
+## What the logs showed (2026-10-10)
+
+Sources: `journalctl --user -u frametop-pointer` (the helper), `-u frametop-hands`, `-u
+frametop-input-relay`, and the four flags the user said "flag that" for
+(`frame-hands/captures/flags/flag-20261010-145753`, `-145841`, `-145857`, `-150115`: 16 s of hands
+in the head frame plus head poses). Keys aren't logged anywhere, so how long after a key a gesture
+came is known only as "within 1 s" (the old guard's "ignored (no ray up; typing)") or not.
+
+- The old guard (`POINTER_PINCH_TYPING`, 1 s) caught 23 pinches today (09:12-09:42 and
+  14:59-15:02), always "no ray up" ones. It never looked at ray pinches, middle pinches, grips,
+  ft-screens' pokes and grabs, or the ray showing.
+- 14:57:10-14:57:30, typing into a text field on the TK-KB006 keyboard (connected 14:56:27, field
+  focused 14:56:34): 11 pinches became "no ray up: a press where the pointer is, on nothing" and 2
+  more were "ignored (a press is already held)". All began at 1.7-2.0 cm (deliberate pinches read
+  0.4-1.0 cm), so typing thumbs. None was within 1 s of a key: typing pauses are longer than that.
+- 14:57:37 the right hand's ray came up ("hand ray: none -> right (raised)") with the hand on the
+  keyboard, and at 14:57:38 a typing pinch became a ray press on `frametop.remote.103.bar`, held
+  1.89 s. Flag 145753 shows that moment: both palms 0.29-0.33 m under the eyes, 0.26-0.34 m ahead,
+  palms down (0.90-1.00 of the normal pointing down), 0.17-0.22 m apart. The knuckles sat 0.29-0.31 m
+  under the eyes, right at the ray's "raised" limit (0.30 m to show).
+- 15:00:37 and 15:02:24: two more presses at the pointer between "ignored (typing)" pinches.
+- 15:01:15 (flag 150115): both hands typing at 0.40-0.42 m under the eyes, 0.18-0.21 m ahead, palms
+  down 0.84-0.97, 0.15-0.16 m apart; the right hand rose, gripped, dragged a panel, then came back
+  to 0.28 m under the eyes, palm down 0.84, and a ray pinch clicked `frametop.screen.1`.
+- Flags 145841 and 145857 are deliberate ray use: the pointing hand 0.15-0.30 m under the eyes and
+  0.33-0.46 m ahead, palm anywhere from sideways to down; the other hand resting low (0.41-0.52 m
+  under the eyes, -0.1 to 0.2 m ahead), 0.35-0.6 m away.
+- "pinch right: left to ft-screens (the hand is near its panel)" bursts (14:58:26-14:58:42,
+  09:14:51-09:15:02) are hands near a low panel; with typing nearby those are ft-screens' to judge.
+
+The hand recorder's desk take (PR #3, `frame-hands/dumps/pr3-04-desk-work`, head poses known):
+
+| phase | palms under the eyes | ahead | palm down | apart | height difference |
+| --- | --- | --- | --- | --- | --- |
+| typing | 0.36-0.39 m | 0.22-0.29 m | 0.85-0.99 | 0.17 m (p90 0.40) | 0.00-0.01 m |
+| mouse (one hand on the keys) | 0.38-0.41 m | 0.21-0.29 m | 0.82-0.99 | 0.41 m | 0.00-0.02 m |
+| switching | 0.37-0.40 m | 0.21-0.29 m | 0.82-1.00 | 0.17-0.41 m | 0.00-0.02 m |
+
+Deliberate gestures in the same recording (`pr3-03-gestures`): the hand 0.0-0.10 m under the eyes,
+0.26-0.37 m ahead. The touch take (`pr3-06-touch`, low dots) had the touching hand 0.24-0.30 m
+under the eyes, palm down 0.7-0.9, but 0.10-0.17 m higher than the other hand.
+
+So typing hands sit anywhere from 0.29 to 0.42 m under the eyes depending on the desk and chair,
+which is also where deliberate gaze pinches with the elbow resting sit (0.35-0.45 m): no fixed
+height tells them apart. What does: both hands down together (level within a few cm, 0.15-0.25 m
+apart, palms down), and a hand that types rising well above where it typed when it's meant.
+
+## What the branch does
+
+`pointer/helper/typing.h` (tests: `pointer/tests/typing-test.cpp`) decides, per side, whether typing
+holds a hand back; the pointer helper feeds it the relay's keys and each hands frame, and ft-screens
+hears the result (`typing <L> <R>` on @ft_screens).
+
+- Keys: for `POINTER_PINCH_TYPING` (2 s, was 1) after a key, every hand is held back, except one
+  that has left the keyboard or is busy out of the keyboard zone (its ray up, a gesture held:
+  pointing with one hand and pressing Enter with the other).
+- On the keyboard: a hand in the keyboard zone (palm at least `POINTER_TYPING_BELOW`, 0.2 m, under
+  the eyes, 0.05-0.45 m ahead, palm down) while typing is parked, and stays held back through any
+  pause until it rises `POINTER_TYPING_RAISE` (0.08 m) above where it typed, or turns its palm up,
+  for 0.1 s. Freed, it works at once, keys or not. A hand on the mouse beside the keyboard is parked
+  too once a key comes (its pinches are mouse clicks). Unseen 3 s, it's forgotten.
+- The posture: both hands in the zone, level within 8 cm and at most 32 cm apart, for
+  `POINTER_TYPING_POSTURE` (0.3 s), is typing before any key, within `POINTER_TYPING_RECENT` (600 s)
+  of a key. Without that limit, both hands resting on a desk would hold gaze pinches back.
+- Held back: no pinch, middle pinch or grip (ray, gaze or plain) in the helper; no ray (an owner
+  drops the pointer to the mouse or the gaze, "hand ray: right -> none (typing)"); no poke, near grab
+  or fingertip cursor in ft-screens. A gesture already held carries on.
+
+Replaying the four flags and the PR #3 takes through typing.h (`/tmp` driver, not in the repo; the
+flags have no keys, so a key a minute before the window was assumed):
+
+| recording | result |
+| --- | --- |
+| flag 145753 (typing, the ray click at 14:57:38) | both hands parked by the posture 0.3 s into the window, held back 98% of it; the 14:57:38 pinch came 0.27 s into the window, before the replay's posture (live, the hands had been on the keys since 14:57:10) |
+| flag 150115 (typing, then the right hand up for a grip, a drag and a click) | posture at once; the right hand freed by its raise (0.18 m), its grip and two pinches allowed; the left held back 98% |
+| flags 145841, 145857 (deliberate ray use, the other hand low) | nothing held back, all 11 pinches allowed |
+| desk take, keys during the typing prompt | typing, mouse and switching: both hands held back 100% |
+| desk take, no keys | the posture alone: typing 84-100%, mouse phase: the mouse hand free (0.41 m from the other hand) |
+| gestures take (pinches, grabs, pointing) | nothing held back |
+| touch take (low dots) | the posture took both hands while they waited low; the touching hand freed by its first raise, then held back 9% |

docs/hazards.md

@@ -25,6 +25,6 @@ ft-screens drops keys while no screen has focus or the SteamVR dashboard is open
 
 ## Typing and grabbed keyboards
 
-- **Programs that watch every keyboard lose grabbed ones.** While typing goes to the desktop, the relay grabs pass-through keyboards, so a hotkey tool reading them directly stops seeing their keys. `SHARE_KEYS=1` in `~/.config/frametop.conf` sends their keys to the abstract socket `@frametop_keys` instead. It's off by default: abstract sockets have no permissions, and any local process that binds the name first receives every key typed into the desktop, passwords included.
+- **Programs that watch every keyboard lose grabbed ones.** While typing goes to the desktop, the relay grabs pass-through keyboards, so a hotkey tool reading them directly stops seeing their keys. Bind the tool's key as a Frametop key combination that runs a command (`command:CMD`) instead: the relay sees every keyboard. Frametop Voice's push-to-talk key works this way. (Until 2026-10-10, `SHARE_KEYS=1` sent every key typed into the desktop to the abstract socket `@frametop_keys`, which any local process could bind first.)
 - **Typing starts out going to Steam.** After the desktop starts, keys go to Steam until you click a screen.
 - **Controller clicks don't move typing.** Overlay apps don't see controller clicks on other panels, so after one typing stays where it was. A mouse click, or a click on a screen, moves it.

docs/reference.md

@@ -30,7 +30,9 @@ kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key contrastEna
 kwriteconfig6 --file ~/.config/frametop/kdeglobals --group KDE --key AnimationDurationFactor 1
 ```
 
-Two of the system's autostart programs don't start in this desktop: Discover's update notifier (`org.kde.discover.notifier`), which starts Discover to check for updates, and IBus (`ibus`), which can't reach the desktop's apps because KWin's input method is `input/ft-textinput`. The session script puts copies with `Hidden=true` in `~/.config/frametop/autostart` once (marked in `frametoprc`), and skips a name you already have a file for. Delete a copy to start that program again.
+The desktop's notifications can show outside it, in the Steam session. Plasma there runs on the desktop's own D-Bus, so its popups only appear on the desktop's screens. `session/ft-notifyfwd` (Plasma's autostart; log: `/tmp/frametop-notifyfwd.log`) hands each one to the notification server on the user bus, the one the Steam session's apps use, for example a VR notification server that draws them as panels. Its buttons work: a click there reaches the app as if made in Plasma, and dismissing it there closes it in Plasma. While it forwards, Plasma's own popups are off (do not disturb, held by "frametop"), and Plasma's notification history still has every one. `NOTIFY_FORWARD` in `~/.config/frametop.conf` (or Notifications on the Visibility & pins tab of Frametop Display Settings) sets it: `auto` (the default) forwards only when the user bus's server isn't SteamOS's own `steam_notif_daemon`, which hands notifications to a Steam client that can't show them on the Frame, so with stock SteamOS nothing changes; `on` forwards to any server, SteamOS's too, where they're lost (it's in the file only; the settings app shows it while it's set); `off` keeps them in the desktop. Restart the desktop after a change. Notifications Flatpak apps send through the notification portal go along too. KDE's inline reply shows as a plain Reply button there; see [design.md](design.md#notifications-outside-the-desktop).
+
+Three of the system's autostart programs don't start in this desktop: Discover's update notifier (`org.kde.discover.notifier`), which starts Discover to check for updates, IBus (`ibus`), which can't reach the desktop's apps because KWin's input method is `input/ft-textinput`, and Steam (`steam`), which is already running. The session script puts copies with `Hidden=true` in `~/.config/frametop/autostart` once (marked in `frametoprc`; Steam's was added later and is hidden once on desktops that already had the other two), and skips a name you already have a file for. Delete a copy to start that program again.
 
 Settings are in two files, and Frametop Display Settings edits both. The screens (resolution, width in metres, scale, curve, which one has the taskbar) and their layout are in `~/.config/frametop-layout.json`. The backend, remote desktop, and pointer settings are in `~/.config/frametop.conf`; `session/frametop.conf.example` lists every key.
 
@@ -47,7 +49,7 @@ Every screen is an overlay named `frametop.screen.N` with five controls:
 - `.bar` moves the screen. Drag it with any laser or with the 3D mouse, whose right-drag tilts. Scrolling while you drag pushes the screen away or pulls it closer, along the line from your head.
 - `.curve` bends the screen into a cylinder around you, using your current distance as the radius, or makes it flat again.
 - `.roll` rolls the screen when you drag it sideways, like a knob. It snaps level within 2.5°, and scrolling on it turns 5° per notch.
-- `.resize`, the tab on the bottom right corner, sets the width. Screens go down to 15 cm wide.
+- `.resize`, the handle on a corner, sets the size from there, the opposite corner staying put (a screen keeps its aspect; a floating window gets that many pixels). It sits on the bottom right, and goes to whichever corner a laser, the gaze or a hand's pinch comes near, brightening as it comes; lit blue while resizing. Screens go down to 15 cm wide.
 - `.reset`, left of the bar, is the quick reset, like Meta+Shift+R (`ft-layout reset`): with a profile in use it opens that profile again, as Open profile does, and otherwise it puts every screen back in its layout around where you are now.
 
 The controls are sized from both the screen's width and its distance from you, follow the surface of a curved screen, and stay invisible until a laser or the 3D mouse's cursor lands on one or comes within about 1.5 times a button's size of it. While invisible they're still there, fully transparent, so SteamVR's laser can find them. They're translucent until a laser is on them, like SteamVR's own window controls.
@@ -68,7 +70,7 @@ In the last three modes the hotkey shows the screens anyway. A screen can also b
 - During VR games, the Always mode hides the screens unless the dashboard is open (the default), or leaves them up.
 - Controllers on the screens. Visible screens can keep SteamVR's laser mouse on, so controllers work them with the dashboard closed, but that also takes the controllers away from a game. By default this is off while a VR game runs, and the 3D mouse or the dashboard works the screens. Pointing a controller at a screen, a floating window, or the keyboard still turns its laser on, like SteamVR's own floating windows, and pointing away gives the game the controllers back. The other choices are always on, or only with the dashboard open, which also suits flatscreen games since they aren't scene apps.
 
-Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay: from pass-through keyboards, a keyboard's media keys (its Consumer Control node), and any key a pointer device passes through. A mouse button passed through as a key in pointer mode (a side button for Back) goes to the screen the pointer is on instead, wherever typing goes, and only while the pointer is on a screen that shows and Frametop isn't paused. Its release always goes through, and ft-screens releases it itself when the pointer leaves the screens, its screen hides, or Frametop pauses (`scripts/test-relay-buttons.sh` checks this offline). Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.
+Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay: from pass-through keyboards, a keyboard's media keys (its Consumer Control node), and any key a pointer device passes through. A mouse button passed through as a key in pointer mode (a side button for Back) goes to the screen the pointer is on instead, wherever typing goes, and only while the pointer is on a screen that shows and Frametop isn't paused. Its release always goes through, and ft-screens releases it itself when the pointer leaves the screens, its screen hides, or Frametop pauses (`scripts/test-relay-buttons.sh` checks this offline). Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; bind its key as a key combination with a command instead (Frametop Input Settings, Keyboard). Frametop Voice types its text through the relay the same way, so it lands where typing goes. Hidden screens don't take typing.
 
 Frametop's keyboard opens by itself when a text field on the desktop gets focus, and stays open until its Close key, a layout reset, or a mapped button closes it (or, with Keep it open off in Frametop Input Settings, until the text field loses focus). While the Steam menu (the dashboard) or Steam's own keyboard is up, it steps aside, and it comes back where it was when they're gone; one asked for meanwhile appears then. In the "only with the dashboard" visibility mode, the dashboard doesn't count. It doesn't open without a head pose (the headset in standby). It's a panel of keys (a US laptop layout, with Esc where Caps Lock would be, arrows, and a Close key) that ft-screens shows 0.7 m in front of you and below your eyes, facing you. It stays where it opened, and its grab bar (the pill along the top) moves it like a screen's. Type on it with a controller's laser or the 3D mouse. Shift, Ctrl and Alt latch for the next key, and a held key repeats. KWin starts `input/ft-textinput` as the desktop's input method, and KWin activates it whenever the focused app turns on text input for a field. It tells the relay (`textfield 1` or `0`), the relay decides by the Keyboard setting in Frametop Input Settings, and ft-screens opens the keyboard for the screen that has keyboard focus (`vrkeyboard show`, `hide`, or `toggle` from a mapped button). Its keys reach the focused screen as key presses, so it works in every app, but only apps that use Wayland text input (Qt, GTK, Firefox) open it by themselves; Chromium, Electron and X11 apps need the button. The session drops the `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` that the gamescope session sets, or Qt and GTK apps wouldn't use Wayland text input either.
 
@@ -83,13 +85,18 @@ get N    screens    head    state    key code value    scale N s    vrkeyboard s
 visibility always|dashboard|gesture|toggle    wrist degrees    gesture left|right degrees
 hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible    pause on|off|state
 conceal N|all    reveal N|all    concealed    cutouts on|off|state    cutouts predict on|off    cutouts lead ms
+cutouts matte circle|mesh|capsules    cutouts margin deg [ms [deg]]    cutouts shift off|spec    cutouts roomview auto|on|off
+hands L R allowed    handinput on|off|force|state    palmmenu on|off|state    grabthat on|off|state
 float N mpp x y w h title    unfloat N    pose N matrix    sub N k x y w h | sub N k off    minimized N 0|1    carry N
 rates focused in_view hidden    rates?    watch seconds    phase ms
+spin next|prev|DEGREES|to N    front N    spotwatch on|off
 ```
 
 Each screen draws at a frame rate for how much of it you see. KWin draws a screen only after ft-screens gives it a frame callback, and its apps wait for theirs, so the rate of callbacks is the screen's frame rate, for KWin and the apps on it alike. A screen is focused while you look at it (within 12 degrees of where your head points), while a laser or the mouse is on it or was in the last 1.5 seconds, while it's carried, and while you type on it; it gets every display frame. The rest of what you can see (within 60 degrees) gets 15 frames a second, and a hidden screen, one behind you, and everything while paused get one a second. A level goes up at once and comes down after a moment (1.5 s from focused, 0.5 s from in view). A video, or anything moving over a large part of a screen (6% or more of it, redrawn on 8 commits in a row, 10 or more a second), keeps every frame while in view. A floating window is a screen of its own here. Nothing that stands still costs anything at any rate: KWin sends a frame only when something on the screen changed. `rates F V H` sets the three rates in Hz (0: every display frame; default `0 15 1`, also `ft-screens --rates 0,15,1`), and `rates?` shows them, the display's rate, and for each screen its level, its milliseconds between frames, and whether it counts as a video. `watch S` gives every screen full rate for S seconds: remote desktop renews it while a VNC client is connected, since a viewer sees what KWin draws. The ticks (SteamVR events and the callbacks) come once per display frame, 1 ms after the vsync (`phase ms` changes that, for tuning), in step with the display rather than on a timer that drifted through the frame.
 
-`conceal` and `reveal` hide and show one screen on its own (`ft-layout hide` and `show` send them), and `concealed` lists those screens. `pause on` (from the input relay, when Frametop pauses for a VR game) hides every screen and floating window whatever else says, and slows the desktop down; `pause off` undoes it. `cutouts` turns the hand cutouts on and off (`ft-handsctl cutouts`). The last line is ft-floatd's, for floating windows: N is a floating window's panel, numbered on from the screens, one per spare output. `float` gives the window's rectangle in its output, metres per pixel, and the title bar's height, and shows the panel; `unfloat` hides it. `pose` places it (a 3x4 matrix, standing universe), `sub` shows popup or dialog k over it, `minimized` hides it while its window is minimized, and `carry` moves it with the laser that pressed the window's own title bar.
+`spin` is the lazy susan. By default it brings panels to a front slot at reading distance and leaves the rest where they are: `next` and `prev` bring the next one round on the right or left (the one in the slot goes home), and `to N` brings panel N. With `SCREEN_SPIN_BRING=0` in `~/.config/frametop.conf` it turns every panel in the room about your head instead: `next` and `prev` bring the next one on the right or left to straight ahead, and `to N` turns the short way to panel N. Either way, a number of degrees turns the room that far. Typing goes to the panel that ends up in front. `front N` gives panel N the typing where it is, as a spin to it would. `spotwatch on`, renewed every second, gives spotwatch's search the typing, and `spotwatch off` gives it back ([Spotwatch](#spotwatch)).
+
+`conceal` and `reveal` hide and show one screen on its own (`ft-layout hide` and `show` send them), and `concealed` lists those screens. `pause on` (from the input relay, when Frametop pauses for a VR game) hides every screen and floating window whatever else says, and slows the desktop down; `pause off` undoes it. `cutouts` turns the hand cutouts on and off (`ft-handsctl cutouts`), and sets their matte, its margin and the per-distance correction (hands/README.md, "Hand cutouts and near input"); they cut only with Room View on, detected or forced (`cutouts roomview`, hands/README.md "Rendered hands"). `hands` is the pointer helper's word on hand input (each side `hand`, `controller` or `off`, and whether it's allowed now); `handinput` turns near input on (following that word), off, or on without it, and `handinput state` reports what each hand is doing, its counts (presses, one-frame presses, pinch clicks, grabs by card and fist, poke attempts and those refused, crossings off the content, presses at the edge, flicks and throws, holds through tracking loss, second hands joining a held panel, pinches that waited to become fists, buttons pinched) and what held presses back; each press, release, refused poke and grab also goes to ft-screens' log as a `hands:` line. `palmmenu` turns the palm menu (a bare hand palm up, pinch: the Steam menu) on and off, and `palmmenu state` reports it. `grabthat` turns "grab that" (an open hand held out to a panel farther than arm's reach, then a fist pulled back: it comes to arm's reach) on and off, and `grabthat state` reports each hand's state, its counts and the panels that came. The last line is ft-floatd's, for floating windows: N is a floating window's panel, numbered on from the screens, one per spare output. `float` gives the window's rectangle in its output, metres per pixel, and the title bar's height, and shows the panel; `unfloat` hides it. `pose` places it (a 3x4 matrix, standing universe), `sub` shows popup or dialog k over it, `minimized` hides it while its window is minimized, and `carry` moves it with the laser that pressed the window's own title bar.
 
 ## Input relay
 
@@ -115,9 +122,10 @@ A mouse drives SteamVR the way a controller's laser does, but shows up as a smal
 
 - The input relay, in pointer mode (`POINTER=1`), sends mouse motion, clicks, and scrolling to the helper. A deliberate movement or a click wakes the pointer, and 30 seconds without mouse input releases it.
 - The helper, `pointer/helper/ft-pointer`, runs in the `dev` container as `frametop-pointer.service` and starts with SteamVR. It keeps the cursor, tests it against every visible overlay, draws the dot, and sends the driver an exact pose.
-- The driver, `pointer/driver/` (`ft_pointer`), is loaded by SteamVR. It's an invisible virtual right-hand controller whose laser follows the cursor.
+- The driver, `pointer/driver/` (`ft_pointer`), is loaded by SteamVR. It's an invisible virtual controller whose laser follows the cursor. It takes the treadmill role, no hand, so the real controllers keep theirs and one can be used beside it (`POINTER_ROLE` and the driver's `role` setting; see docs/design.md, "Treadmill role").
+- The dashboard patch, `pointer/dashpatch/` (`frametop-dashpatch.service`, starts with SteamVR), lets that treadmill move SteamVR's own windows by their grab bars, which SteamVR allows only for the head or a hand. It goes into SteamVR's dashboard page through the dashboard's debugger, which the installer turns on at 127.0.0.1:8089 (`VRWebHelper/DebuggerEnabled`; the uninstaller turns it off). `pointer/dashpatch/run.sh status` shows whether it's in.
 
-Whichever device you used last wins. Picking up a controller hands the laser back at once, and moving the mouse takes it again. When the headset comes off, the pointer lets go, so the displays can sleep, and it stays off until you're wearing the headset again.
+Whichever device you used last wins. Picking up a controller hands the laser back at once, and moving the mouse takes it again. With hand interaction (`POINTER_HANDS=1`), this goes per hand: a controller used recently wins its hand, and the other hand can stay bare. When the headset comes off, the pointer lets go, so the displays can sleep, and it stays off until you're wearing the headset again.
 
 To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pulls it while you drag. Hold the right button while dragging and move the mouse to tilt the panel around the grab point; the right press isn't sent as a click. The tilt stays for the rest of the drag, and releasing the left button drops the panel as it is. A mapped Toggle dashboard button wakes the pointer if needed and holds the virtual system button for 0.12 s, because SteamVR ignores a press and release in the same instant. Open Steam menu / close dashboard (`steam_menu`) needs no pointer: `steam/ft-steam menu` asks Steam's UI, over its debugging port (`steam/steamui.py`), to show its dashboard overlay and focus the Steam frame's menu, or to hide the dashboard if it's up.
 
@@ -125,20 +133,22 @@ To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pu
 pointer/driver/build.sh && pointer/driver/install.sh install   # then restart SteamVR
 pointer/helper/build.sh && pointer/helper/run.sh install
 pointer/helper/run.sh status | log | restart
-pointer/driver/install.sh probe     # devices, hand roles, who owns the dashboard pointer
+pointer/driver/install.sh probe     # devices, hand roles, activity, charging, who owns the dashboard pointer
+pointer/probe/build.sh              # probes: vrprobe, lasertest (who has the laser), ctlnoise (controller noise)
 ```
 
-The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, `POINTER_GAZE_DOT`, `POINTER_GAZE_SHOW`, `POINTER_GAZE_MOUSE`, `POINTER_GAZE_MOUSE_MOVE`, and the keyboard clicks' `POINTER_HEAD_DEADZONE` and `POINTER_KEY_TAP`, and the gaze service's `GAZE_TRACKER` (`auto`, the default: our own eye tracker when it's installed, else SteamVR's; or `own` or `steam`) and `GAZE_EYE` (the eye bias). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its two again when the file changes).
+The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, `POINTER_GAZE_CONTROL_HOLD`, `POINTER_GAZE_DOT`, `POINTER_GAZE_SHOW`, `POINTER_GAZE_MOUSE`, `POINTER_GAZE_MOUSE_MOVE`, and the keyboard clicks' `POINTER_HEAD_DEADZONE` and `POINTER_KEY_TAP`, and the gaze service's `GAZE_TRACKER` (`auto`, the default: our own eye tracker when it's installed, else SteamVR's; or `own` or `steam`), `GAZE_EYE` (the eye bias), `GAZE_FIT_WATCH` (the put-on check and the slip note, off by default: [gaze/README.md](../gaze/README.md), "Headset fit"), and `GAZE_FUSION` (our tracker's gaze mixed with SteamVR's, off by default: [gaze/README.md](../gaze/README.md), "Gaze pointer"), and our eye tracker's `EYES_SIZE_GUARD` (`both`, the default, `long`, or `off`: [gaze/README.md](../gaze/README.md), "Our own eye tracker"). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its four again when the file changes; ft-eyes reads `EYES_SIZE_GUARD` when it starts).
 
 ## Frametop Input Settings
 
-A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has nine pages:
+A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has ten pages:
 
 - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, whose key combinations work everywhere), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
 - Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, pause or resume Frametop ([Pausing for VR games](#pausing-for-vr-games)), Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
 - Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
 - Game optimization has the pause for VR games: its state with Pause now or Resume, whether VR games pause Frametop by themselves, the controller gesture (one or two buttons, pressed once or twice; one button always takes two presses), what happens to the desktop, and the sound. They're saved as `pause_auto`, `pause_gesture`, `pause_desktop`, and `pause_sound` in `~/.config/frametop-input.json`. See [Pausing for VR games](#pausing-for-vr-games).
-- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), Meta+Shift+F (float window in VR or put it back), and Meta+Alt+Tab and Meta+Alt+Shift+Tab (spin the panels: every screen and floating window turns about your head, so the next one on the right or left comes to the front; ft-screens' `spin next|prev|<degrees>`); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
+- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), Meta+Shift+F (float window in VR or put it back), and Meta+Alt+Tab and Meta+Alt+Shift+Tab (spin the panels: every screen and floating window turns about your head, so the next one on the right or left comes to the front; ft-screens' `spin next|prev|<degrees>`); remove them or add others there. With Frametop Voice installed, a Right Ctrl tap starts and stops dictation too. A tap is a press with no other modifier held, then a release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. Tapping a left modifier binds either side's; tapping a right one (Right Ctrl) binds that key alone. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
+- Voice has Frametop Voice's settings (see Voice dictation): on or off, the dictation keys (set on the Keyboard page), the controller buttons and their mode, typing as you talk, sounds, and the wake word and its phrases. They're saved in `~/.config/frametop-voice.json`, and the service applies them within 2 seconds.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
 - Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), what the mouse's left button and movement do, the gaze dot, the eye tracker and eye bias, the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
@@ -156,7 +166,7 @@ Frametop Display Settings has four tabs (three with the gamescope backend, which
 
 - Screens: add and remove screens, and set each one's resolution (presets from 1080p to 4K, ultrawide, super ultrawide, portrait, or custom), its width in VR (0.5 to 6 m), its scale, whether it's curved, and whether it has the taskbar. Resolution, width, and curve apply at once. Adding or removing a screen takes a desktop restart, which the app offers.
 - Layout (the Layout & profiles page): the Arrangement list starts with two presets: Curved around you, with the screens hinged edge to edge like monitors on a desk and each turned to face you, and Flat wall. Both take rows, distance, gap, and height, and Arrange now applies them. Save as profile… saves where the screens are now (positions, sizes, curves, and pins), which ones are hidden, and the open apps and where their windows are, under a name. Profiles are listed in Arrangement after the presets: pick one and Open profile switches to it, and the buttons next to the list rename or delete it. Below it, Apps lists the profile's windows and where they go, and Leave out drops one. A profile saved with fewer screens than you have now leaves the others where they were saved last, or where the preset would put them. Start in profile picks the profile the desktop starts in; with None, a switch turns auto-arrange at startup on or off. A preview shows the layout from above and from the front. See [profiles.md](profiles.md).
-- Visibility & pins: the visibility, game, and controller settings described above, the wrist angle, where each screen is pinned (in the room, a wrist, or your head), and buttons to pin all screens or unpin them.
+- Visibility & pins: the visibility, game, and controller settings described above, the wrist angle, where each screen is pinned (in the room, a wrist, or your head), and buttons to pin all screens or unpin them. Its Notifications section sets `NOTIFY_FORWARD` (whether the desktop's notifications show in the Steam session, see [The desktop](#the-desktop)), shows where they go now, and offers a desktop restart after a change.
 - Power: when the displays turn off while the headset isn't used, their state now, Turn displays off now (to try it), and Stay awake while plugged in. See [Displays off and sleep](#displays-off-and-sleep).
 
 `layout/ft-layout` does the arranging. It's a Python script that uses only the standard library and runs on the host:
@@ -181,21 +191,24 @@ The layout is stored relative to your head when it's applied. `/run/user/<uid>/f
 
 ## Frametop Remote Displays
 
-Other computers' monitors as Frametop screens (ft-screens only), streamed from Vibepollo with Moonlight's protocol. Frametop Remote Displays (`remote-displays/`, also opened by the Remote displays button on Display Settings' Screens page) finds Vibepollo computers on the network, signs in to one with its Web UI login (it keeps a narrow API token, not the password, and pins the host's certificate), shows whether it answers, and adds its displays: its monitors, or a virtual one at any size. A computer with a Steam Link dongle on the Frame's hotspot streams over it (Connection: auto, network or dongle only); the others use the network. Each display has a Connected switch (and the host one for all of its displays), Shown, its stream's resolution, frame rate and bitrate, and its width in VR. A disconnected display keeps its settings, and within the same desktop run, its place. In VR each one is a panel with a screen's controls, and profiles keep where they are. See [remote-displays.md](remote-displays.md).
+Other computers' monitors as Frametop screens (ft-screens only), streamed from Vibepollo or Sunshine with Moonlight's protocol. Frametop Remote Displays (`remote-displays/`, also opened by the Remote displays button on Display Settings' Screens page) finds Vibepollo and Sunshine computers on the network, signs in to one with its Web UI login (it keeps a narrow API token, not the password, and pins the host's certificate), shows whether it answers, and adds its displays: its monitors, or a virtual one at any size. A computer with a Steam Link dongle on the Frame's hotspot streams over it (Connection: auto, network or dongle only); the others use the network. Each display has a Connected switch (and the host one for all of its displays), Shown, its stream's resolution, frame rate and bitrate, and its width in VR. On Windows, a monitor switches to its stream's resolution while it streams (or its closest mode with the same shape), and back when the stream ends. That takes Frametop's build `frametop-2.0.0-2` or newer; `frametop-2.0.0-1` scaled the monitor's own resolution into the stream. A Windows host with Frametop's build can also get the Frame's microphone (Send my microphone on its card, off by default): apps on the PC record it as "Microphone (Steam Streaming Microphone)" while its displays stream. A disconnected display keeps its settings, and within the same desktop run, its place. In VR each one is a panel with a screen's controls, and profiles keep where they are. See [remote-displays.md](remote-displays.md).
 
 ```
 layout/ft-layout remote list                 # the remote displays and their streams' state
 layout/ft-layout remote connect|disconnect ID...
+layout/ft-layout remote host NAME mic=on|off # send it the Frame's microphone (streams pick it up within 3 s)
 remote-displays/install.sh                   # its menu entry
 ```
 
-On the PC, `host/windows/Setup Frametop host.cmd` sets it up for Frametop: Vibepollo 2.0.0 (installed if missing), Frametop's build of its `sunshine.exe`, the settings Frametop needs, the Web UI login you sign in with from the Frame, and a firewall check (`-Check` to see what it would change, `-Undo` to put things back).
+On the PC, `host/windows/Setup Frametop host.cmd` sets it up for Frametop: Vibepollo 2.0.0 (installed if missing), Frametop's build of its `sunshine.exe`, the settings Frametop needs, the Web UI login you sign in with from the Frame, and a firewall check (`-Check` to see what it would change, `-Undo` to put things back). [host/windows/README.md](../host/windows/README.md) has the steps, and the [`windows-host-1`](https://github.com/Frametop/frametop/releases/tag/windows-host-1) release has the kit as a zip. A Linux PC (Arch or CachyOS) installs Frametop's Vibepollo package with Vibepollo's installer, as [host/linux/README.md](../host/linux/README.md) says; it hasn't been tested yet.
+
+On a Mac, `host/macos/frametop-host-setup.sh` (or `Setup Frametop host.command`) installs Sunshine and runs one instance per real display, each on its own port (47989 + 100·N), started at login. You allow Sunshine Screen Recording and Accessibility once in System Settings. Without Accessibility the Frame's pointer moves on the Mac but clicks, scrolling and typing don't arrive; the setup checks both, and offers to ask again if Accessibility is missing. Remote Displays finds the Mac as one computer; signing in with the Web UI login pairs the displays you pick, and the password isn't kept, so adding more later signs in again. Macs have no virtual displays (`--check`, `--undo` as on the PC). Frametop Streams in the menu bar turns each display on or off with a click, or all of them, and `ft-sunshine off|on|status` does the same in Terminal. Off lasts through logins until on. [host/macos/README.md](../host/macos/README.md) has the steps for setting up a Mac, and the [`macos-host-1`](https://github.com/Frametop/frametop/releases/tag/macos-host-1) release has the kit as a zip.
 
 ## Floating windows
 
 A desktop window can float in VR as a panel of its own, away from the screens. Meta+Shift+F floats the window under the pointer (or the active one, over the wallpaper), or puts it back on its screen if it floats. So do Float in VR in every window's menu (Alt+F3; Back to Desktop on a floating one), the button left of Close in its title bar, and a mouse button, controller button, or key combination mapped to Float window in VR in Frametop Input Settings; Put all floating windows back is mappable too. Launch as Standalone, in an app's right-click menu in the Application Launcher or the taskbar, starts the app with its first window floating, where that app last floated or in front of you. [floating-windows.md](floating-windows.md) explains how it works.
 
-`float/ft-floatd` does this. It runs inside the desktop's Plasma session (log: `/tmp/frametop-floatd.log`), loads the KWin script `float/frametop-float.js`, and moves each floating window to a spare KWin output of its own, which ft-screens shows as a panel cropped to the window. The settings are in `~/.config/frametop.conf`: `FLOAT_SLOTS` is how many windows can float at once (8, at most 16; 0 turns floating off; restart the desktop after a change), and `FLOAT_MARGIN` the pixels around each window on its output, so menus have room past its edges (300). Each app's last floating place, size, and scale are kept in `~/.config/frametop-float.json` by desktop file name, relative to the primary screen, so they move with the screens. `FT_FLOAT_DEBUG=1` in ft-floatd's environment logs every event from the KWin script.
+`float/ft-floatd` does this. It runs inside the desktop's Plasma session (log: `/tmp/frametop-floatd.log`), loads the KWin script `float/frametop-float.js`, and moves each floating window to a spare KWin output of its own, which ft-screens shows as a panel cropped to the window. The settings are in `~/.config/frametop.conf`: `FLOAT_SLOTS` is how many windows can float at once (16, the most; 0 turns floating off; restart the desktop after a change), and `FLOAT_MARGIN` the pixels around each window on its output, so menus have room past its edges (300). Each app's last floating place, size, and scale are kept in `~/.config/frametop-float.json` by desktop file name, relative to the primary screen, so they move with the screens. `FT_FLOAT_DEBUG=1` in ft-floatd's environment logs every event from the KWin script.
 
 With floating on, the session gives the desktop's windows Frametop's own decoration (`decoration/`): Breeze's look plus the float button. Apps that draw their own title bar, like Chromium and Electron apps, don't have the button. `decoration/apply.sh` puts a changed copy into the running desktop, and `decoration/apply.sh --off` goes back to Breeze until the next desktop start.
 
@@ -207,6 +220,20 @@ float/ft-float launch org.kde.dolphin # start an app (its desktop file name) flo
 float/ft-float run COMMAND [ARG...]   # the same for a command
 float/ft-float close ID               # close a window
 float/ft-float list                   # the spare outputs and what floats on them
+float/ft-float launch firefox action=new-private-window   # one of the app's own actions, floating
+float/ft-float launch org.kde.dolphin screen=2            # its first window docked on screen 2
+float/ft-float activate ID            # make a window the active one, unminimized
+```
+
+## Spotwatch
+
+Meta+Space opens spotwatch, a search pill straight ahead of where your head points, for apps and remote displays (`{host} - {display}`). Tab, Shift+Tab, and the arrow keys move through what it finds, and Enter or a click opens it. A running app's most recent window spins to the front with the lazy susan. An app that isn't running starts floating. A remote display connects if needed and spins to the front. Shift+Enter or a right click opens an item's menu: an app's own actions (a private window), Open new window, Open docked on this screen, and its windows; a remote display's Go to and Disconnect. Esc, Meta+Space again, or a click elsewhere closes it. It's the `spotwatch` action in Frametop Input Settings, bindable to mouse and controller buttons too, and opened that way it brings Frametop's keyboard. [spotwatch.md](spotwatch.md) has the decisions and how it works.
+
+`spotwatch/ft-spotwatch` runs inside the desktop's Plasma session (log: `/tmp/frametop-spotwatch.log`) and runs the pill, `ft-spotpanel` (built by `screens/build.sh`), in the container. While the search is open, ft-screens sends it every key. Use counts are in `~/.config/frametop-spotwatch.json` and icons in `~/.cache/frametop/spotwatch`.
+
+```
+spotwatch/test/spotwatch-test.py   # the ranking and the search against a fake ft-screens and ft-floatd (offline)
+spotwatch/tools/render.py OUT.png   # the panel's picture, without a headset
 ```
 
 ## Displays off and sleep
@@ -237,7 +264,7 @@ Paused, Frametop leaves the headset's CPU and GPU to a VR game. The input relay
 - The gaze service stops (`frametop-gaze`: ft-gazed, ft-gaze, our own eye tracker, the gaze panel), so nothing reads SteamVR's eye tracking. Our frame grabber, the root service `ft-eyegrab`, goes idle by itself 3 seconds after our eye tracker stops asking it for frames.
 - Hand tracking stops if it runs (`frametop-camd`, `frametop-hands`).
 - The desktop, as the Game optimization page of Frametop Input Settings says (`pause_desktop`): hidden (the default) or closed. Hidden, ft-screens hides every screen and floating window whatever the visibility mode, the hotkey, or the dashboard says, and gives KWin a frame callback once a second instead of every display frame. KWin draws a screen only after its frame callback, and its apps wait for theirs, so the desktop hardly draws, but its windows stay open. Remote desktop stops if it runs (`session/remote-ctl.sh`). Closed, `desktops.sh stop` closes the desktop and its windows, and resuming starts it again (about 12 seconds), in its start profile if it has one.
-- The relay lets go of the 3D mouse, releasing any click still held on it (a mouse button, a mapped controller button, or a key combination), and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, and commands; a key combination that does nothing is typed as usual.
+- The relay lets go of the 3D mouse, releasing any click still held on it (a mouse button, a mapped controller button, or a key combination), and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, dictation, and commands; a key combination that does nothing is typed as usual.
 
 Resuming starts again only what pausing stopped, and plays a second sound. The pointer helper and ft-powerd keep running: they cost little, the helper is what says a game started, and stopping it would leave its virtual controller connected with its last pose.
 
@@ -258,6 +285,23 @@ input/test/pause-buttons-test.py   # a click held into a pause comes up, offline
 
 The state outlives a relay restart, in `/run/user/UID/frametop-pause.json`. A SteamVR restart while paused starts the gaze service with it, and the relay stops it again when the pointer helper comes back.
 
+## Voice dictation
+
+Frametop Voice (`voice/`, [voice/README.md](../voice/README.md)) types what you say where you type, recognised on the headset; nothing leaves it. `voice/ft-voiced` runs in Frametop's container as `frametop-voice.service` (installed by `voice/run.sh install`, which `install.sh` offers, yes by default), starting at login. It records with `pw-record` and transcribes live with Moonshine on the CPU; `frametop-voice-whisper.service` (whisper.cpp on the GPU) runs only while the wake word is on, for its commands. Its models, about 320 MB, are downloaded on install into `~/.local/share/frametop/voice/models` (`voice/models.sh`: fixed URLs, checked by sha256).
+
+The input relay does the keyboard side. Dictation's keys are key combinations with the actions `dictate` (a press starts or stops) and `dictate_hold` (records while held); with Frametop Voice installed, the default key combinations add a Right Ctrl tap (`"97": "dictate"`, Right Ctrl alone: a right-hand modifier's own code is that key's tap only). Like every tap, it doesn't count with another modifier held, or with a key, click or scroll in between. The relay sends `toggle|start|stop <source>` to `@frametop_voice`, and dictation works while Frametop is paused. ft-voiced sends its text back as `type TEXT` on `@frametop_relay`; the relay types it, one key every 4 ms on a US layout, to ft-screens while typing goes to the desktop, otherwise on its virtual keyboard for Steam and games, and holds a character back while a modifier is held. Both sockets take these only from their own user. Frame controller buttons (left d-pad down, held, by default) are read by ft-voiced itself from vrserver's web socket (`input/vrws.py`), so games still get them.
+
+Frametop Input Settings' Voice page turns it on and off and sets the controller buttons and their mode (hold or press to toggle), typing as you talk, sounds, and the wake word (off by default) and its phrases, in `~/.config/frametop-voice.json`; the keys are on its Keyboard page. Wake word commands that run a program, and advanced settings, are edited in that file by hand (voice/README.md).
+
+```
+voice/build.sh && voice/run.sh install
+voice/run.sh status          # the services, and ft-voiced's state as JSON
+voice/ft-voice toggle        # start or stop dictation; also start, stop, cancel, type TEXT, wake on|off, events
+voice/run.sh log
+voice/models.sh check        # every model's sha256
+voice/test/test_daemon.py    # typing through a stand-in relay, push-to-talk, offline
+```
+
 ## Gaze pointer (experimental)
 
 In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the keyboard does the last bit. It needs the gaze service, which `install.sh` offers (yes by default) and `gaze/run.sh install` installs on its own: it builds it and runs `gaze/ft-gazed` as `frametop-gaze.service`, which starts with SteamVR. The service idles while the gaze isn't used: its eye tracker reader and our own eye tracker run only while gaze mode is on and someone wears the headset, while a check or the calibration runs, or while the Gaze page of Frametop Input Settings is open, and stop 30 seconds after ([gaze/README.md](../gaze/README.md)). Turn gaze mode on with the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1`, or a button or key combination mapped to Gaze pointer on/off.
@@ -265,20 +309,20 @@ In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the ke
 - Meta+J left-clicks and Meta+K right-clicks where you look. A quick tap clicks where the dot was at the press. Hold instead, and the dot stays put in your view: turn your head until it's on what you meant, and let go to click there. Held still for `POINTER_GAZE_HOLD` (0.5 s), the press becomes a real one, and your head drags. Meta+K with Meta+J held presses where the dot is now, to drag from there, and a second Meta+K during that drag (a double Meta+K) pans and tilts what you're dragging while it's held.
 - The mouse's buttons work the same way, with the mouse steering instead of your head (`POINTER_GAZE_MOUSE=precision`, the default): the right button with the left held starts a drag, and a double right click pans and tilts what you're dragging. With `POINTER_GAZE_MOUSE_MOVE=held`, the default, the mouse only corrects: while the gaze has the pointer, moving it does nothing unless a button is held. `free` lets the mouse take the pointer any time. With the gaze stale for a second, in a game, or with the headset off, the mouse works as usual.
 - A correction before a click teaches the gaze service the tracker's error there. A correction bigger than `POINTER_GAZE_NUDGE_MAX` (55 degrees) isn't learned; it opens a quick check instead.
-- Calibration and checks run in a panel fixed to the headset (`gaze/panel/ft-gazepanel`, which the gaze service runs), from the Gaze page: Quick check is one dot, and also opens when you put the headset on. Calibrate is three rounds of dots, dark to bright; look at each dot and left click or press Meta+J to take it. Check headset fit shows, live, how well the tracker sees each eye. A right click or Meta+K closes the panel. Gaze mode on without a calibration opens Calibrate by itself, as soon as your eyes are seen. If gaze mode is on but can't follow your eyes yet (no calibration, the calibration can't open, the gaze service not running), the Gaze page says why under the Gaze pointer switch, and `gaze/ft-gazectl on` notes it.
+- Calibration and checks run in a panel fixed to the headset (`gaze/panel/ft-gazepanel`, which the gaze service runs), from the Gaze page: Quick check is one dot, and also opens when you put the headset on. Calibrate is three rounds of dots, dark to bright; look at each dot and left click, press Enter, pinch (index to thumb, with hand tracking) or press Meta+J to take it. Check headset fit shows, live, how well the tracker sees each eye. A right click or Meta+K closes the panel. Gaze mode on without a calibration opens Calibrate by itself, as soon as your eyes are seen. If gaze mode is on but can't follow your eyes yet (no calibration, the calibration can't open, the gaze service not running), the Gaze page says why under the Gaze pointer switch, and `gaze/ft-gazectl on` notes it.
 
 [gaze/README.md](../gaze/README.md) has the details, our own eye tracker, and the gaze probe, a development tool.
 
-## Hand tracking (experimental, deferred)
+## Hand tracking (experimental)
 
-Deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it. It still builds and runs, installed by hand, for working on it.
+It costs a lot of the headset's CPU. `install.sh` offers it (yes by default, `--no-hands` skips it, and it needs `sudo` for ft-camd's file capabilities), and a release brings `ft-camd` and `ft-hands` built in its image. Installed, it runs while the Frametop desktop does (`HANDS_MODE=frametop`).
 
-Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
+Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen (a round hole per hand, `HANDS_MATTE=circle`; `mesh` and `capsules` are the older shapes). ft-handview (`frametop-handview.service`) draws your hands as a 3D hand model posed by the tracking: over games and the void with Room View off, and over the hole with it on (`HANDS_VIEW_ROOMVIEW=1`, the default). `HANDS_VIEW_MODEL` picks the model: `asset` (the default), `babylon`, `mrtk3`, `godot`, `mrtk2`, `robot` (a robot hand built in code from the tracked skeleton: light shells, dark joints, cyan accents), or `capsules` for the older tubes; `HANDS_VIEW_STYLE=ghost|solid` the look. The models, in `screens/assets/hands` with their licences: the WebXR Input Profiles' generic hand (`asset`; MIT, Copyright (c) 2019 Amazon), Babylon.js's hand meshes by Patrick Ryan (`babylon`; CC BY 4.0, https://github.com/BabylonJS/Assets), MRTK3's OpenXR hand (`mrtk3`; BSD-3-Clause, Copyright (c) 2025, Mixed Reality Toolkit Contributors), Godot XR Tools' hands by DigitalN8m4r3 (`godot`; CC0 1.0) and MRTK2's rigged hand (`mrtk2`; MIT, Copyright (c) Microsoft Corporation). The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode your eyes aim and your hands click, Vision Pro style: a pinch clicks where you were looking as it began, and the hand ray stays off (`POINTER_GAZE_HAND_RAY=1` brings it back beside the gaze). Pinch, hold and move the hand a few centimetres to nudge the pointer slowly onto the target, then let go to click there; the correction teaches the gaze tracker, as the mouse's do. Pinch and hold still for half a second to press and drag with the hand. A middle pinch right-clicks the same way. Hold a pinch with one hand and pinch with the other to hold both buttons, left and right, each until its pinch opens. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`. With `POINTER_HANDS=1` and gaze mode off, a bare hand raised and pointing also shows a ray that drives the pointer like a controller's laser (it comes up as the thumb nears the index or middle tip; a pinch clicks, a middle pinch right-clicks or scrolls, a three-finger pinch scrolls, reaching out and in pushes and pulls a carried panel), a controller used recently takes over its own hand only, and during a paused VR game hand tracking is parked rather than stopped, waking when the dashboard opens. Details: [hands/README.md](../hands/README.md), "Pinches and grips in the pointer". When tracking goes wrong, `ft-handsctl flag [note]` (or a voice command through frame-voice, "flag that") saves the last 12 seconds of hand crops and what the tracker made of them for the offline tools: ft-hands keeps them in RAM (`HANDS_FLAG_BUFFER=1`, about 50 MB), the drawn hands pulse cyan when a flag is taken (`HANDS_FLAG_FLASH`, `HANDS_FLAG_TONE`), and flags go to `HANDS_FLAG_DIR`. See [hands/README.md](../hands/README.md), "Flags".
 
 - `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`. The Hand Recorder's installer (`hands/rec/install.sh`) sets them the same way. `hands/run.sh uncaps` takes them back while neither the services nor the Hand Recorder is installed. `hands/run.sh uninstall` and `hands/rec/install.sh uninstall` run it after removing their own part, and `uninstall.sh` always takes them back.
 - `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
-- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
-- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
+- `HANDS_MODE` says when they run: `off` (the default until install.sh installs hand tracking, which sets `frametop`) only after `ft-handsctl on`, until `ft-handsctl off` or SteamVR stops; `frametop` while the Frametop desktop runs; `always` whenever SteamVR runs, in games too. Frametop Input Settings' Hands page sets it and applies it at once (`hands/handsmode.py`), next to the hand cutouts' margin (`HANDS_MATTE_MARGIN`, live). The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
+- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, typing's `POINTER_PINCH_TYPING`, `POINTER_TYPING_RAISE`, `POINTER_TYPING_BELOW`, `POINTER_TYPING_POSTURE`, `POINTER_TYPING_RECENT`, the gaze pinch's `POINTER_GAZE_PINCH`, `POINTER_GAZE_HAND_RAY`, `POINTER_GAZE_PINCH_DEADZONE`, and `POINTER_GAZE_PINCH_GAIN`, the hand ray's `POINTER_HAND_RAY`, `POINTER_RAY_DEADZONE`, `POINTER_RAY_PUSH`, `POINTER_RAY_SCROLL`, `POINTER_RAY_RELEASE`, `POINTER_RAY_GRACE`, its smoothing and taps (`POINTER_RAY_SMOOTH_MINCUTOFF`, `POINTER_RAY_SMOOTH_BETA`, `POINTER_RAY_SMOOTH_HOLD`, `POINTER_RAY_STEADY`, `POINTER_RAY_TAP`, `POINTER_RAY_TAP_DEADZONE`), its body and hand models (`POINTER_RAY_TORSO_WRISTS`, `POINTER_RAY_HAND_WEIGHT`, `POINTER_RAY_HAND_PITCH`, `POINTER_RAY_HAND_YAW`, `POINTER_RAY_ANCHOR`), and ft-screens' near reach (`HANDS_NEAR_REACH`, 0.075 m), pokes (`HANDS_POKE_ANGLE`, `HANDS_POKE_TRAVEL`, `HANDS_POKE_REST_BELOW`, `HANDS_POKE_REST_NEAR`) and near grabs (the border band `HANDS_GRAB_BAND`, a pinch on the content clicking `HANDS_PINCH_CLICK`, the card pinch `HANDS_CARD_PINCH`, the fist grab `HANDS_FIST_GRAB` and its reach `HANDS_FIST_REACH`, a second hand joining a held panel `HANDS_JOIN_REACH`, two hands turning a panel on every axis `HANDS_STRETCH_TURN`, the round buttons' hit area for hands `HANDS_BUTTON_REACH` and pinches pressing them `HANDS_PINCH_BUTTONS`, `HANDS_GRAB_SMOOTH_MINCUTOFF`, `HANDS_GRAB_SMOOTH_BETA`, `HANDS_GRAB_DEADZONE`, the wrist turning a carried panel: `HANDS_GRAB_ROTATE`, `HANDS_GRAB_ROTATE_DEADZONE`, holding through tracking loss `HANDS_HOLD_LOST_MS`, ignoring tracking no hand makes `HANDS_TRACK_GATE`), throws (the flick `HANDS_FLICK`, `HANDS_FLICK_SPEED`, `HANDS_FLICK_OPENING`, a carry let go fast `HANDS_THROW`, `HANDS_THROW_SPEED`, the push `HANDS_PUSH`, `HANDS_PUSH_SPEED`, where they go, `HANDS_THROW_DISTANCE`, and at what width, `HANDS_SEND_AWAY_WIDTH`), its "grab that" gesture (`HANDS_GRAB_THAT`, `HANDS_GRAB_THAT_DISTANCE`, `HANDS_GRAB_THAT_WIDTH`, which panels: `HANDS_GRAB_THAT_FROM`, `HANDS_GRAB_THAT_MARGIN`, `HANDS_GRAB_THAT_MARGIN_DEG`, `HANDS_GRAB_THAT_FACING`), the soft white halo round a gesture's target panel (`HANDS_TARGET_GLOW`), and the soft glow round the corner, edge or button a hand is at (`HANDS_EDGE_GLOW`). The example config explains each.
 
 Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
 

docs/remote-displays.md

@@ -1,6 +1,6 @@
-# Remote displays (exploration)
+# Remote displays: design notes
 
-Status: exploration. Spike S1 is done; nothing in Frametop has changed yet. Branch `remote-displays`, written 2026-10-06.
+These are the notes from building remote displays, from the first exploration (2026-10-06) on: what the Frame's hardware gives us, the spikes, and the decisions. To set a computer up, see [Remote displays in the README](../README.md#remote-displays) and the host kits: [Windows](../host/windows/README.md), [macOS](../host/macos/README.md), [Linux](../host/linux/README.md). How the pieces work now is in [reference.md](reference.md#frametop-remote-displays).
 
 The idea: show desktops streamed from other machines as Frametop panels that behave like the native screens. They get placement, curve, pinning, layouts, lasers, gaze, and attention-based frame rates. The target is up to 5 remote displays from 2 hosts, with no more headset overhead than 5 native screens.
 
@@ -33,7 +33,7 @@ Five 1440p displays fit at full rate with room left. Five 4K displays only fit a
 
 ### The planned setup
 
-Two hosts that share the same two desk monitors: a Mac (14-inch MacBook Pro) with those two plus its built-in display, and the test PC (Windows, RTX 5090) with the two. Windows and Linux hosts run Vibepollo, which streams real displays and creates virtual ones on demand. The Mac runs Sunshine and is limited to one display for now (see "Host side"). At 60 fps:
+Two hosts that share the same two desk monitors: a Mac (14-inch MacBook Pro) with those two plus its built-in display, and the test PC (Windows, RTX 5090) with the two. Windows and Linux hosts run Vibepollo, which streams real displays and creates virtual ones on demand. The Mac runs Sunshine, which began with one display; it now runs one instance per display (see "Host side"). At 60 fps:
 
 | Display | Share of the decoder |
 |---|---|
@@ -175,7 +175,7 @@ The requirement: each host streams its real displays, and virtual displays it cr
 |---|---|---|
 | Windows | Vibepollo | 1 main session (real or virtual) + up to 4 virtual Remote Monitors |
 | Linux (Arch, CachyOS; beta) | Vibepollo | The same, with at most 4 virtual displays at once |
-| macOS | Sunshine | 1, the Mac's main display, for now |
+| macOS | Sunshine | Every real display, one Sunshine instance each (`host/macos`); no virtual displays |
 
 ### Windows and Linux: Vibepollo
 
@@ -210,19 +210,15 @@ The config folder is admin-only, so the change is a script for the user to run (
 
 **Risk.** Vibepollo is new, moves fast, and has one maintainer, and its README says about 99% of its code is AI-generated. Frametop depends only on its Moonlight-protocol behaviour and the Remote Monitor app ids, so on Windows Apollo (virtual displays through the SudoVDA driver) or plain Sunshine (real displays) stay as fallbacks.
 
-### macOS: Sunshine, one display for now
+### macOS: Sunshine, one instance per display
 
-Vibepollo has no macOS build. Sunshine (v2026.914, labelled experimental on macOS) is the only Moonlight-protocol host for the Mac, so the Mac streams one display: its main display, from one Sunshine instance. The reasons:
+Vibepollo has no macOS build. Sunshine (labelled experimental on macOS) is the only Moonlight-protocol host for the Mac. One Sunshine streams one display, so the Mac runs one instance per real display, each with its own config file, ports (`port` 47989 + 100·N) and `output_name` (the display's CGDirectDisplayID). `host/macos/frametop-host-setup.sh` sets them up (see "The Mac" below). Sunshine can't create virtual displays on macOS, and no PR is open for it.
 
-- Since v2026.906, absolute mouse input only reaches the main display (Sunshine #5733). Any other display would show but couldn't be clicked.
-- Nobody reports running several Sunshine instances on one Mac yet.
-- Sunshine can't create virtual displays on macOS.
+- Absolute mouse input reached only the main display from v2026.906 (Sunshine #5733). libvirtualhid #145 and Sunshine #5739 fixed it on 2026-10-09, first in the pre-release v2026.1010.141104, which the host setup pins until a release has it.
+- Capture is at backing pixel size, so the built-in display streams at 3024x1964. Closing the lid removes the built-in display, so a stream of it ends. Sunshine now keeps a display awake while it captures it (#5509 was the stall when one slept).
+- Sound comes from a Core Audio tap (macOS 14+), so nothing like BlackHole is needed. Every instance carries it; ft-stream's per-address lock plays one.
 
-The main display is the one with the menu bar (System Settings → Displays). Capture is at backing pixel size, so the built-in display streams at 3024x1964. Closing the lid removes the built-in display, so a stream of it ends. Capture stalls if the display sleeps mid-stream (#5509).
-
-One display can still reach the encoder's limit. Sunshine forces VideoToolbox's low-latency mode, which roughly halves throughput (#5814); an M4 Pro managed 0.36-0.46 Gpix/s in that mode. The super ultrawide at 60 fps is 0.44 Gpix/s, so full-motion video on it may need a lower fps or a smaller stream. The other two displays need 0.30 and 0.36 Gpix/s.
-
-**What would lift the limit.** The mouse fix is libvirtualhid PR #145 ("target configured mouse viewport", open), which Sunshine's draft PR #5739 pulls in. Once it ships, spike S4 tries one Sunshine instance per display (own config file, `port` about 100 apart, `output_name`) and BetterDisplay virtual displays switched on by `global_prep_cmd`. Open PR #5817 (ScreenCaptureKit + OBS's VideoToolbox encoder) may raise the encoder limit. Helping #145 along upstream is the one Mac contribution worth making.
+Full-motion video can still reach the encoder's limit. Sunshine forces VideoToolbox's low-latency mode, which roughly halves throughput (#5814); an M4 Pro managed 0.36-0.46 Gpix/s in that mode. The super ultrawide at 60 fps is 0.44 Gpix/s; the test Mac is an M3 Max, with two encode engines. Open PR #5817 (ScreenCaptureKit + OBS's VideoToolbox encoder) may raise the limit. Its capture (AVFoundation) also sends 30-55 fps of a still desktop, not `minimum_fps_target`'s 1 fps, at a low bitrate (1-6 Mbit/s).
 
 ### What stock hosts cost us
 
@@ -232,8 +228,8 @@ One display can still reach the encoder's limit. Sunshine forces VideoToolbox's
 | One stream per display | Five ft-stream processes, connections, and client pairings on the Frame | Measure the CPU (S3) |
 | One real display per Vibepollo install | A second real display needs a second host instance | Use virtual displays; S3 tries a second instance on Windows |
 | Cursor drawn into the video | Lags the laser by a round trip; each mouse move over a still page becomes a video frame | Accept |
-| Mac: one display | Sunshine's mouse reaches only the main display, and there are no virtual displays | Upstream fix in progress, then S4 |
-| Mac encoder | Full-motion video on the super ultrawide at 60 fps is at the limit | Lower fps or size for that stream |
+| Mac: an instance per display | Each display pairs on its own, with the Web UI login (no tokens); no virtual displays | Remote Displays pairs all the picked displays from one sign-in |
+| Mac encoder | Full-motion video on the super ultrawide at 60 fps is at the limit; a still desktop still streams 30-55 fps | Lower fps or size for that stream; ScreenCaptureKit (#5817) upstream |
 
 What Frametop maintains: ft-stream (pairing, decoder, keyframe sampling, input mapping, Remote Monitor launch and resume; the protocol itself is moonlight-common-c's), the remote screen type in ft-screens, the layout and settings changes, and host setup notes or scripts.
 
@@ -439,22 +435,55 @@ The user decided remote displays are Frametop displays, with the same controls a
 **The PC side: Frametop host setup (2026-10-07, ~17:40):**
 - `host/windows/Setup Frametop host.cmd` (it runs `frametop-host-setup.ps1` and asks for admin) does what the test PC got by hand, nothing else of its settings:
   1. Vibepollo 2.0.0 with its own installer when it isn't there (downloaded from Nonary/Vibepollo's release, SHA-256 checked first; you click through it).
-  2. Frametop's build of `sunshine.exe` over the original (kept as `sunshine.exe.2.0.0-original`), SHA-256 checked: from `-FrametopBuild PATH|URL`, the script's `$BuildUrl`, or `sunshine-frametop.exe` next to it. Only over Vibepollo 2.0.0's own exe; another version stops it with a message. Without the build, virtual displays still work, but not the PC's own monitors.
+  2. Frametop's build of `sunshine.exe` over the original (kept as `sunshine.exe.2.0.0-original`), SHA-256 checked: from `-FrametopBuild PATH|URL`, the script's `$BuildUrl` (the fork's release), or `sunshine-frametop.exe` next to it. Only over Vibepollo 2.0.0's own exe; another version stops it with a message. Without the build, virtual displays still work, but not the PC's own monitors.
   3. `remote_monitor_mute_audio = disabled`, `remote_monitor_disconnect_on_client_disconnect = enabled`, `remote_monitor_disconnect_on_stream_end = disabled`. The rest of `sunshine.conf` stays as it is.
   4. The Web UI login: keep the one there is, or set one (`sunshine.exe --creds`, the password typed into a hidden prompt and passed to it quoted).
   5. Checks: an inbound firewall rule for `sunshine.exe` on every network type (added if missing; Windows puts a Steam Link dongle's network in Public), whether a Steam Link dongle (an adapter "For Valve") is connected, and that the Web UI answers. It ends with what to pick and sign in as on the Frame.
   `-Check` only says what it would change, `-SkipLogin` leaves the login, `-Undo` puts back the original exe and the oldest backup of the settings. Backups and a log go to `%ProgramData%\Frametop`.
 - Tested on the test PC (admin over SSH, `FRAMETOP_NO_PAUSE=1`): `-Check`; a run with nothing to change (no restart); `-Undo` (the original exe back) and a run with `-FrametopBuild D:\vp-build\src\build\sunshine.exe` (backed up, swapped, Vibepollo restarted, the streams came back on their own). Not tested: a PC without Vibepollo (the download and its installer), setting the login (needs the user at the PC), adding the firewall rule.
-- Still to decide: where other PCs get Frametop's build. `$BuildUrl` is empty; publishing it needs the fork's source published with it (GPL-3.0).
+- Where other PCs get Frametop's build: the fork's GitHub releases, with the source as the release's tag (GPL-3.0). See below (2026-10-09).
 - The Frame side: `install.sh` now builds ft-stream (`stream/build.sh`, step 7) with Remote Displays' menu entry, and `uninstall.sh` offers to delete `~/.local/share/frametop-stream` (the clients' keys, the hosts' tokens and pins) with the other settings.
 - Linux hosts (Vibepollo's Arch package): a host setup like this one, later.
 - After Vibepollo restarted, a stream checked the dongle in 300 ms, too soon, and went over the network; ft-stream now gives it a second.
 
+**Frametop's build released (2026-10-09):**
+- Release [`frametop-2.0.0-2`](https://github.com/Frametop/frametop-vibepollo/releases/tag/frametop-2.0.0-2) of Frametop/frametop-vibepollo (2026-10-14; `frametop-2.0.0-1` was the first) has `sunshine.exe` and the Arch package (`pacman -U`), each with its `.sha256`. The fork's CI (`frametop-build.yml`, Depot's runners) builds them from the tag and publishes them; a `frametop-*` tag is what makes a release. The tag is the source.
+- The host setup's `$BuildUrl` points at that `sunshine.exe`, so a PC needs only `host/windows`. It replaces the original, the hand-built `2f032252`, and the first CI build (`a84b6cfc`).
+- New in it: a Frametop display stream that asks for SDR turns the display's HDR off while it captures it, and back on when the stream ends (with `dd_hdr_option` automatic, Vibepollo's default). The test PC's HDR monitor looked washed out in SDR: Vibepollo's conversion clips at 80 nits while Windows draws SDR content at its SDR white level (240 nits there), and scaling for that left the colours heavily oversaturated. Displays it turned off are listed in `config\frametop_display_hdr.json` until they're back on, so the next start turns them back on after a crash or a forced stop. On the test PC's dev build: HDR off 0.4 s after the stream started, 8-bit capture, back on at the end, off again on a reconnect.
+- Tested on the test PC: with the first CI build installed and only `host/windows` in a folder, `-Check` downloaded the release and matched its SHA-256, and the run backed up, swapped and restarted Vibepollo; the Web UI answered.
+
+**The monitor follows the stream's resolution (frametop-vibepollo, 2026-10-10):**
+- Before, a Frametop display stream captured the monitor at its own resolution and Vibepollo fit that picture into the stream's size, so a stream smaller than the monitor, or of another shape, never showed its desktop pixel for pixel.
+- Now the stream switches the monitor to the stream's width and height, at the monitor's own refresh rate, before capture starts, and puts it back when the stream ends, the way `dd_resolution_option` automatic (Vibepollo's default) matches a main session's display to the client. With it set to manual or disabled, nothing changes. A monitor without that mode gets its closest mode with the same shape, and the stream scales that; one with no mode of that shape keeps its own and is scaled as before.
+- The change is temporary: Windows' saved display settings stay as they were, so a reboot undoes it too. Windows may move the other monitors to keep them touching one that changed size, so every monitor's place is noted before the first change and put back after the last, in one change that either fully applies or doesn't.
+- Two streams of the same monitor share the first one's mode. HDR (above) goes off after the mode changes and back on before it's put back.
+- Monitors it changed are listed in `config\frametop_display_mode.json` (each one's own mode and the one it set, and the places) until they're back, so the next start puts them back after a crash or a forced stop. One someone has changed since keeps theirs.
+- In Frametop nothing changes: ft-stream already asks for the size set in Remote Displays, and changing it starts the stream over, which switches the monitor again. The note under a monitor's stream settings says what happens.
+
+**The Mac: one Sunshine instance per display (2026-10-10, branch `mac-host`):**
+- `host/macos/frametop-host-setup.sh` (or `Setup Frametop host.command`): installs the pinned Sunshine pre-release (SHA-256 checked, notarized by LizardByte), lists the displays (`displays.js`, JavaScript for Automation, no compiler), and writes an instance per display in `~/Library/Application Support/Frametop/sunshine/instance-N`, the main display's first, each started at login by the LaunchAgent `frametop.sunshine.N`. Its settings: `port`, `sunshine_name` "COMPUTER: DISPLAY (WxH)", its own state and apps files, `minimum_fps_target = 1`, and the menu bar icon on the first only. The rest of each `sunshine.conf` (Web UI changes) is kept on a rerun. `--check`, `--undo` (removes the LaunchAgents; settings, pairings and Sunshine stay), `--set-login`, `--skip-login`.
+- Turning them off and on (2026-10-12): Frametop Streams, a menu bar icon, lists the displays with a check by each one that's on. A click turns one on or off, Turn All On and Turn All Off do every one, and the icon dims while they're all off. It's `Frametop Streams.applescript`, a stay-open AppleScriptObjC app (`NSStatusItem`, so no compiler) that the setup compiles into `~/Applications`, marks `LSUIElement` (no Dock icon), signs again ad hoc after that change, and starts at login with the LaunchAgent `frametop.streams`. Quit from its menu, it stays quit until the next login. It runs `ft-sunshine off|on|status [--tsv] [N...]`, in the background for a change, with the output in `~/Library/Logs/Frametop/streams.log`. `ft-sunshine` is installed next to `ft-sunshine-instance` and linked into `~/.local/bin` when there is one. Off disables each instance's LaunchAgent (`launchctl disable`) and stops it, so it stays off through logins. On enables it and starts it, or starts one quit from Sunshine's own menu bar icon, which launchd leaves stopped after a clean exit. Rerunning the setup turns them all on. The Frame's streams of the Mac keep trying while it's off and reconnect after on. Tested with throwaway LaunchAgents serving fake instances, through the menu (UI scripting) and `ft-sunshine`: on, on again (the running one kept its pid), off, on after off, one display, a stopped one, unknown numbers, a display that isn't connected, and the app started by its LaunchAgent and quit from its menu. On the M3 Max: the setup from the kit zip, unpacked with a download's quarantine (the installed scripts keep it, and run anyway: launchd and the app run them through bash and sh), which turned the three on; then Turn All Off, Turn All On and Turn All Off again from the menu, each done within 3 s.
+- The kit as a release (2026-10-12): [`macos-host-1`](https://github.com/Frametop/frametop/releases/tag/macos-host-1) of Frametop/frametop has `frametop-macos-host-1.zip`, `host/macos` of 188f107 in a `frametop-macos-host` folder (`zip -qr -X`), and its `.sha256`. Its notes are `host/macos/README.md` without the title, plus the files. It isn't marked Latest and has no `Frametop.zip`, so get.sh skips it: stable takes `releases/latest`, experimental the newest release with a `Frametop.zip`. Its tag doesn't start with `v`, so `release.yml` doesn't build it. A new kit is a new release, `macos-host-N`, with the zip renamed to match.
+- An instance is tied to its monitor by vendor, model and serial, not to a display id: `ft-sunshine-instance` looks the id up at every start and passes `output_name=ID`. While the monitor is unplugged it doesn't start Sunshine, which would fall back to the main display, and launchd tries again.
+- One certificate for all the instances, so Remote Displays pins one key per Mac; one Web UI login file for all, kept from an earlier Sunshine install if there was one. An earlier Homebrew Sunshine's LaunchAgent (the same ports) is turned off, and back on with `--undo`.
+- Screen Recording and Accessibility are granted by hand, once for all the instances: macOS lists them as Sunshine (the LaunchAgent runs a bash script that execs it; `AssociatedBundleIdentifiers` makes Login Items say Sunshine too). Sunshine started from a terminal gets that terminal's grants instead.
+- Without Accessibility (Post Event access) the pointer still moves, since libvirtualhid also warps it (`CGWarpMouseCursorPosition`, no permission needed), but clicks, scrolling and keys go only through `CGEventPost`, which macOS drops without a word: every Frame input method moved the pointer and clicked nothing, while the Mac's own trackpad worked. Sunshine notes at each start whether it had each permission (`defaults read dev.lizardbyte.app.Sunshine inputPostEventPermissionState`, 2 = granted, 1 = asked and not granted) and asks only once, so a grant macOS lost track of, or one given to an older Homebrew Sunshine (a separate app to macOS), stays broken with nothing asking again. The setup reports both permissions at the end and with `--check`. After the first instance's restart, if Accessibility is still missing, it offers to clear Sunshine's records (`tccutil reset Accessibility` and `PostEvent` for its bundle id, and that note) and restart the instance so macOS asks again, which fixed the test Mac (2026-10-10).
+- Frametop side: a display has a `port` when its instance isn't on 47989, and ft-stream takes `HOST:PORT`, so ft-screens passes it through as it is. The token, pin and audio lock stay per address. Given HOST alone, ft-stream takes its display's port from the layout, by its client: an ft-layout from before ports (an older release's Reset Screen Layout and profile launchers) restarted the Mac's streams at HOST, the instance on 47989, where they froze unpaired and their pointer went to the main display. `host_answers` checks the displays' ports. Remote Displays tries `/api/token` at sign-in; a bare 400 "Bad Request" (upstream Sunshine's answer to a path it doesn't have, before any login check) or a 404 means plain Sunshine, and the login is checked on `/api/clients/list`. It's kept in memory only, passed on stdin to `ft-layout remote add --port P --login` and on to `ft-stream pair --login` for each display picked, then forgotten. Upstream Sunshine (v2026.1010) lists a waiting pairing request with an id (`GET /api/pin`); ft-stream approves its own (libgamestream's device name "roth", from its own address) with `POST /api/pin {pairing_id, pin, name}`. The login is checked first: a wrong password's request, left waiting, blocked the next try. There are no permissions to grant. Add computer merges the instances' mDNS entries ("Mac (2)") into one computer, and Add displays lists the instances from their serverinfo on ports 47989 + 100·N.
+- Tested on the M3 Max (macOS 26.6.2: built-in 3024x1964, MPG491CX OLED 5120x1440, LC34G55T 3440x1440): the setup with `--check`, a first run and a rerun; the three instances answer from the Frame. With throwaway test instances (their own login, removed after): pairing with a wrong password then the right one; `ft-layout remote add --port --login` into a scratch layout; streams through a stand-in for ft-screens, the three at once (no losses; 35, 52 and 36 fps of still desktops, 3-9% of a Frame core each, about 6% CPU for Sunshine on the Mac for the OLED); the host's sound; a pointer move to two corners of the LC34G55T, not the main display, landing within a pixel of where it should. Not tested: the Remote Displays app itself, the live desktop, drags across the Mac's displays (separate processes; the release goes through the stream the press went down on), full-motion video's encode rate, replugging a monitor.
+
 **Tested in the live desktop (frame-testbench, the 3D mouse through `@ft_pointer_helper`, 2026-10-07):**
 - An Explorer window carried by its title bar from the Remote Monitor to the OLED stayed there, and carried back, stayed there too. The log showed one change of screen each way.
 - Disconnect, connect: the panel came back where it was, at its width. Connect in a new run with the headset off: placed from screen 1's anchor.
 - Windows rescales a window that moves between displays with different scaling, so the point you held moves on the window. A second drag from the same spot can land on the address bar instead of the title bar. That's Windows, not the routing.
 
+**The Frame's microphone on the host (2026-10-18):**
+- Send my microphone on a host's card (`"mic": true` in its layout entry, `ft-layout remote host NAME mic=on|off`, off by default) sends the Frame's microphone to it. The stream that plays the host's sound (the audio lock's holder) records PipeWire's default source, which on the Frame runs through echo cancelling, so a host gets it once however many displays stream. It reads the setting every 3 s, so switching needs no restart. Its 10 s report adds `mic N packets/s`.
+- Moonlight has no microphone. `stream/moonlight-mic.patch` adds `LiSendMicrophoneOpusData` to moonlight-common-c: packet type 0x3003 with `[sequence (2 bytes, big-endian), channel 1, flags 0]` then the Opus packet, the layout of Vibepollo's draft PR #169. It refuses (-1) when the control stream isn't encrypted, so the microphone never goes in the clear. Mono, 48 kHz, 20 ms packets, 64 kbit/s, Opus VOIP with in-band FEC.
+- ENet's throttle drops unreliable packets whenever round trips wobble. Over Wi-Fi beside three video streams it dropped 27% of them. The microphone now goes reliable, on ENet channel 0x07 of its own (moonlight-common-c uses 0x00-0x06, and 0x10 up for controllers), so a resent packet holds up only the microphone. Loss went to 0.
+- The host side is in our fork (`frametop/mic`, `src/frametop_mic*.cpp`). It takes the packets only on an encrypted control stream from a client allowed keyboard input, one client at a time (another waits and tries every 2 s). It decodes them on the control thread, hides up to 5 lost packets in a row (the last from FEC) and starts over after longer gaps, and plays them on a thread of its own. On Windows that's WASAPI into "Speakers (Steam Streaming Microphone)", which apps record as "Microphone (Steam Streaming Microphone)". The driver comes with Steam; the build installs `SteamStreamingMicrophone.inf` once if it's missing and puts the default devices back after. That capture endpoint adds +30 dB at Windows' default level (its range goes to +30), 32 times too loud, so the build plays that much quieter and follows the level when it changes.
+- On Linux the fork makes a "Frametop Microphone" source (`module-pipe-source`), but only when Sunshine runs as the desktop user. Vibepollo's packaged Linux service runs as its own user and reaches the desktop's sound only through its root session broker (`pactl`, `parec`), which has no operation for this. There it logs that once and does nothing. A broker operation would fix it, once a Linux host can test it.
+- Tested on the test PC with a 440 Hz test tone from the Frame instead of the microphone: every packet arrived encrypted, and a recorder on "Microphone (Steam Streaming Microphone)" heard the level sent (RMS 0.345 sent, 0.345 recorded), with no gaps. The Linux source was tested here on PipeWire. Not tested yet: a real voice in a real app, by ear, in the headset.
+
 **Still to do:**
 - A headset test of all of it in the live desktop:
   - the row above the screens;
@@ -475,6 +504,6 @@ The user decided remote displays are Frametop displays, with the same controls a
 ## Open questions
 
 - Which of the test PC's two desk monitors is the real one (the main session)?
-- The Mac's chip (Max chips have two video encode engines) and BetterDisplay Pro matter only once the Mac goes past one display.
+- Mac virtual displays: BetterDisplay switched on by `global_prep_cmd`, or a small helper on the private `CGVirtualDisplay` API. Not needed for v1.
 - Audio: none, the focused display's host, or a fixed one?
 - Remote displays outside the desktop: should they also show over games, where the decoder is shared with Steam Link VR?

docs/spotwatch.md

@@ -0,0 +1,76 @@
+# Spotwatch
+
+Spotwatch is Frametop's search for apps and remote displays, like Spotlight on a Mac. Press Meta+Space and a glass search pill opens straight ahead of where your head points. Type part of an app's name, or a remote display's `{host} - {display}`, and pick from what it finds:
+
+- **An app that's already running:** its most recent window comes to you. The lazy susan spins the panel it's on, a screen or its own floating window, to straight ahead, and typing goes to it.
+- **An app that isn't running:** it starts as a floating window, where that app last floated or in front of you.
+- **A remote display:** spotwatch connects it if needed, then spins it to the front.
+
+Use Tab, Shift+Tab, or the arrow keys to move through the results, and Enter or a click to open one. Shift+Enter or a right click opens the item's menu:
+
+- **For an app:** its own launch options, such as a browser's New Private Window, plus Open new window (even when one is open), Open docked on this screen, and each of its open windows.
+- **For a remote display:** Go to (connecting it first if needed) and Disconnect.
+
+Esc goes back from a menu and closes the search. Meta+Space again, or a click anywhere else, closes it too.
+
+Spotwatch works while the Frametop desktop is running and Frametop isn't paused for a VR game. It's the `spotwatch` action in Frametop Input Settings, so a mouse button, a controller button, or another key combination can open it too. When it opens from a mouse or controller, Frametop's keyboard opens with it.
+
+## Not built
+
+- Searching open windows by title, Frametop's own actions ("Reset layout"), and profiles (decision 4).
+- Moving the caret within the query. Typing and Backspace work at the end only.
+- A setting for its distance and size (`--distance` and `--width` on ft-spotpanel; 0.75 m and 0.42 m, `spotwatch/reserve.h`).
+
+## Decisions
+
+These were settled with the user on 2026-10-10. The numbers may be cited in the code, so they stay as they are.
+
+| # | Question | Decision |
+|---|---|---|
+| 1 | The look | A rounded glass pill with a results card under it. The glass is faux, because SteamVR's overlays have alpha but no backdrop blur: a translucent dark tint, a soft sheen from the top, a bright rim fading down the edges, and a shadow outside. Real blur would mean reading the panels back every frame on the Frame's GPU |
+| 2 | Where it opens | Fixed in the room, centred where the head points (not the eye gaze), at 0.75 m, upright, tilted no more than 60° up or down. The card hangs below the pill, so the pill stays put as results change. That spot is reserved (2026-10-17): the lazy susan's front slot stays at least 0.15 m behind it, and the search draws over any panel nearer than it |
+| 3 | An app with several windows | One row per app. Enter goes to its most recently active window, and Shift+Enter or a right click lists the others with the menu items |
+| 4 | What it searches | Apps and remote displays only. Hidden remote displays are left out, and disconnected ones show with Connect |
+| 5 | Disconnected remote displays | Enter connects the display (`ft-layout remote connect`), waits for its panel, then spins to it |
+| 6 | When it works | Only while Frametop is running and active: the session starts it, and the input relay doesn't run the action while Frametop is paused. A pause while it's open closes it |
+| 7 | Without a keyboard | `spotwatch` is an action like the others, so it can be bound to a mouse button or a controller button. Opened that way, Frametop's keyboard opens too and closes with it |
+| 8 | The menu | The app's desktop actions, Open new window (the app's own new-window action when it has one, else a plain launch), Open docked on this screen (the screen nearest straight ahead), and its windows |
+| 9 | An app that isn't running | It floats, where ft-floatd puts any launched app (where it last floated, or in front of you) |
+| 10 | A window on a pinned panel | Pinned panels don't spin. Typing goes to it where it is (`front N`), and a short note says where it is ("on your left wrist") |
+| 11 | Shortcut | Meta+Space by default. Plasma's KRunner has Alt+Space, and nothing in Frametop used Meta+Space |
+| 12 | Branch | `spotwatch`, off `experimental`, merged into `experimental` |
+
+## How it works
+
+Spotwatch has three parts, and needs changes in ft-screens, ft-floatd and the input relay:
+
+- **`spotwatch/ft-spotwatch`** (`ft_spotwatch.py`) runs inside the desktop's Plasma session, started from its autostart like ft-floatd, so it's up exactly while the desktop is. It keeps the app index (Gio's, so it sees what the desktop's menus see), the remote displays from the layout, the usage counts, and the search state. It decides what each key and click does. Its log is `/tmp/frametop-spotwatch.log`.
+- **`spotwatch/ft-spotpanel.cpp`** is the pill: a SteamVR overlay drawn on the CPU, like ft-gazepanel, built by `screens/build.sh` and run by ft-spotwatch in the container. It holds no state beyond the picture. ft-spotwatch sends it the whole state each time, and it sends back hovers, clicks, right clicks, scrolls, and clicks beside the card. Its clear area catches those clicks.
+- **`spotwatch/spotsearch.py`** ranks the results with plain Python, so its test runs anywhere. Name matches come first, from the strongest kind of match to the weakest: the name's start, a word's start, initials, anywhere in the name, then letters in order. Generic names, keywords, and program names count for less and never match fuzzily. Every word of the query must match. Use counts and use in the last day raise an item, and so does an app that's running.
+
+The search has to take the typing from wherever it went, Steam included. While it's open, ft-screens sends every key to `@ft_spotwatch` instead of KWin or a remote display's stream, from keyboards and Frametop's keyboard alike, and tells the input relay that typing goes to the desktop, so the relay grabs the keyboards. ft-spotwatch holds this with `spotwatch on`, renewed every second. ft-screens lets it lapse after 3 seconds, so a crashed ft-spotwatch can't keep the keys. A click on any other panel, or a pause, ends it, and ft-screens says `dismiss` so the pill closes.
+
+ft-spotwatch turns key codes into text on the desktop's keyboard layout (Plasma's `kxkbrc`) through libxkbcommon, and falls back to a US layout without it. The input relay sends no key repeats, so ft-spotwatch repeats a held Backspace or arrow key itself.
+
+Bringing a window to the front takes three commands:
+
+1. ft-floatd's `activate ID` makes the window KWin's active window, unminimized.
+2. ft-screens' `spin to N` brings panel N to the front slot at reading distance (or, with `SCREEN_SPIN_BRING=0`, turns the ring of panels the short way so it ends straight ahead). Screens are 1 to S, floating windows come after them, and remote displays start at 101.
+3. When the spin settles, ft-screens gives that panel the typing, as any spin does. When the panel can't spin (pinned, or a floating window that's minimized), `front N` gives it the typing where it is.
+
+ft-floatd's `spotlist` lists every app window with its desktop file name, title, panel, and when it was last active. The KWin script keeps that timestamp.
+
+Launching uses ft-floatd's `launch APP.desktop`, which now takes `action=ACTION` (one of the desktop file's actions) and `screen=N` (open docked on screen N; the KWin script's `to-output`). Without ft-floatd (`FLOAT_SLOTS=0`), spotwatch launches through Gio itself, and nothing counts as running.
+
+Apps' icons come from the desktop's icon theme (`kdeglobals`, then breeze and hicolor). GdkPixbuf rasterizes them to 64x64 RGBA files in `~/.cache/frametop/spotwatch` for the panel, one at a time in idle moments after the desktop starts. Use counts are kept in `~/.config/frametop-spotwatch.json`.
+
+```
+spotwatch/test/spotwatch-test.py    # the ranking, then ft-spotwatch against a fake ft-screens and ft-floatd (offline)
+spotwatch/tools/render.py OUT.png    # what the panel draws, without a headset (ft-spotpanel --render)
+```
+
+## Risks
+
+- **Stray key releases.** A key pressed before the search opened is released to wherever it went, so nothing stays held. A key pressed in the search and released after it closes (the Enter that opened something) reaches the window that now has the typing as a release without a press. Apps ignore those.
+- **Laser input.** While the pill shows, it makes SteamVR's lasers active (`MakeOverlaysInteractiveIfVisible`), as Frametop's keyboard does. The search is short-lived, but during that time a game loses its controllers.
+- **Matching windows to apps.** Matching uses KWin's desktop file name, then the window class. An app whose windows report neither, such as some X11 apps under Flatpak, doesn't show as running, and Enter starts it again.

float/frametop-float.js

@@ -17,6 +17,7 @@ const settled = {};    // id -> {output, frame, fullScreen, maximized}: where a
 const held = {};       // id -> {w, h, until, asked}: a size asked for, for a second (see hold)
 const moved = {};      // id -> true, or "back" once put back: it moved while KWin changed the outputs
 let layout = "", layoutOutputs = {}, layoutSince = 0;  // the outputs at the last screensChanged
+const activeAt = {};   // id -> when it was last KWin's active window (ms), for spotwatch's most recent
 
 function send(ev) {
     callDBus(SERVICE, PATH, IFACE, "Event", JSON.stringify(ev));
@@ -52,7 +53,7 @@ function info(w) {
         frame: rect(w.frameGeometry), client: rect(w.clientGeometry), popup: w.popupWindow,
         transient: w.transient, parent: w.transientFor ? String(w.transientFor.internalId) : "",
         normal: w.normalWindow, dialog: w.dialog, fullScreen: w.fullScreen, minimized: w.minimized,
-        onAllDesktops: w.onAllDesktops, maximized: isMaximized(w)
+        onAllDesktops: w.onAllDesktops, maximized: isMaximized(w), activeAt: activeAt[String(w.internalId)] || 0
     };
 }
 
@@ -268,8 +269,10 @@ workspace.windowRemoved.connect(w => {
     delete settled[id];
     delete held[id];
     delete moved[id];
+    delete activeAt[id];
 });
 workspace.windowActivated.connect(w => {
+    if (w) activeAt[String(w.internalId)] = Date.now();
     if (w && isSpare(w.output)) send({ev: "activated", id: String(w.internalId)});
 });
 takeLayout();
@@ -376,8 +379,15 @@ function run(c) {
             if (w) w.closeWindow();
             break;
         case "activate":
+            if (w && w.minimized) w.minimized = false;
             if (w) workspace.activeWindow = w;
             break;
+        case "to-output": {  // a launched window that opens docked, onto that screen (spotwatch)
+            const o = outputByName(c.output);
+            if (w && o && (!w.output || w.output.name !== o.name)) workspace.sendClientToScreen(w, o);
+            if (w) workspace.activeWindow = w;
+            break;
+        }
         case "activate-output": {  // the top window on that output (a spin brought it to the front)
             const order = workspace.stackingOrder;
             for (let i = order.length - 1; i >= 0; --i) {

float/ft-float

@@ -8,7 +8,10 @@
   ft-float dock all            put every floating window back
   ft-float launch APP          start an app (its desktop file name, e.g. org.kde.dolphin) and
                                float its first window where that app last floated
+  ft-float launch APP action=ACTION   ...one of its desktop file's actions (a private window)
+  ft-float launch APP screen=N   ...its first window docked on screen N instead
   ft-float run COMMAND [ARG...]  the same for a command
+  ft-float activate ID         make a window KWin's active one, unminimized
   ft-float close ID            close a window
   ft-float list                the spare outputs and what floats on them
 FT_FLOAT_SOCKET names ft-floatd's socket (default frametop_float).

float/ft_floatd.py

@@ -12,8 +12,12 @@ which window floats on which spare output and panel, and connects three parts:
 Commands and ft-screens' events arrive as datagrams on @frametop_float (ft-float is the
 command-line side). Replies go to the sender:
   float [ID|active|pointer]   dock [ID|active|all]   close ID   list   quit   (ft-float)
-  launch APP.desktop   run ["argv", ...]   (start an app and float its first window)
+  launch APP.desktop [action=ACTION] [screen=N]   run ["argv", ...]
+            (start an app, or one of its desktop file's actions, and float its first window;
+            with screen=N its first window opens docked on screen N instead)
   windows   (-> JSON: the open apps' windows and where they are, for a profile; ft-layout save)
+  spotlist  (-> JSON: every app window, its title, panel, and when it was last active; spotwatch)
+  activate ID   (make that window KWin's active one, unminimized; spotwatch)
   profile NAME   (open a profile's apps: move the matching windows, launch the missing ones)
   ("float pointer" is the float key: the window under the pointer, else the active one,
   floated or docked; the input relay sends it for float_toggle, and "dock all" for dock_all)
@@ -24,15 +28,15 @@ command-line side). Replies go to the sender:
 Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its
 frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows
 only the window. Spares are placed apart from the screens and from each other in KWin's
-layout, all within Xwayland's 32767-pixel limit.
+layout, within Xwayland's 32767-pixel limit at scales up to 1.6.
 
 Usage: ft-floatd [--screens N] [--slots N] [--margin PX] [--control NAME] [--socket NAME]
 Defaults: FT_SCREEN_COUNT (from the session) or the layout's count, FLOAT_SLOTS and
 FLOAT_MARGIN from ~/.config/frametop.conf (8 and 300), @ft_screens, @frametop_float.
 
 Launching floating: "launch" starts a desktop file's app (Gio), "run" a command. The next
-normal window from that process (or a child), or with that desktop file name, within
-LAUNCH_SECONDS floats: where that app last floated, or in front of you. Single-instance and
+normal window from that process (or a child), or of that desktop file (desktop_name: by its app
+id, else its class, so Flatpak and X11 apps match too), within LAUNCH_SECONDS floats: where that app last floated, or in front of you. Single-instance and
 D-Bus-activated apps open their window from a process that was already running, hence the
 name. Each app's last floating place (its pose relative to the primary screen, so it moves
 with the screens' layout), size in pixels, and scale are kept in PLACES_PATH, by desktop
@@ -73,7 +77,11 @@ SERVICE = IFACE = "org.frametop.Float"
 PATH = "/Float"
 SCRIPT = "frametop-float"
 POLL_SECONDS = 20        # NextCommand answers empty after this (KWin's D-Bus timeout is 25 s)
-SPARE_X, SPARE_CELL = 12000, 5000  # spares in KWin's layout: a grid from here, 4 across
+# Spares in KWin's layout: a grid 4 across, below the screens. X11 windows get Xwayland
+# coordinates of the layout's times its largest scale, which wrap past 32767: a window there
+# takes no clicks. 16 spares end at 20000, fine up to scale 1.6 (12000 across had the 4th
+# column past it at 1.25).
+SPARE_Y, SPARE_CELL = 4000, 4000
 PULL_OUT = 0.3           # a floated window starts this far in front of its screen (metres), clear of it
 DEFAULT_MPP = 1.6 / 1920  # metres per pixel when ft-screens can't say (no SteamVR)
 PLACES_PATH = os.path.expanduser("~/.config/frametop-float.json")  # each app's last floating place
@@ -223,6 +231,14 @@ def recordable(ev):
     return app not in SKIP_APPS and not any(n.startswith(("ft-", "frametop", "ksplash")) for n in (app, cls))
 
 
+def listable(ev):
+    """An app's top-level window, the kind spotwatch finds: Frametop's own settings apps too,
+    which profiles leave out (an open Input Settings launched a second copy, 2026-10-13)."""
+    if not ev.get("normal") or ev.get("popup") or ev.get("transient"):
+        return False
+    return ev.get("app", "") not in SKIP_APPS and not any(n.startswith("ksplash") for n in (ev.get("app", ""), ev.get("cls", "")))
+
+
 def cmdline(pid):
     try:
         with open(f"/proc/{int(pid)}/cmdline", "rb") as f:
@@ -231,22 +247,56 @@ def cmdline(pid):
         return []
 
 
+_names = {}  # (app, cls) -> (desktop file name, when): desktop_name's answers for a minute
+
+
 def desktop_name(app, cls):
-    """The desktop file name for a window's app id and class. A Flatpak app's window can give an
-    id with no desktop file: RustDesk's (an X11 window) says com.carriez.flutter_hbb, and its
-    desktop file is com.rustdesk.RustDesk. That file's StartupWMClass names the window's class
-    then. The app id itself when nothing matches."""
+    now = time.monotonic()
+    hit = _names.get((app, cls))
+    if hit and now - hit[1] < 60:
+        return hit[0]
+    if len(_names) > 512:
+        _names.clear()
+    name = _desktop_name(app, cls)
+    _names[(app, cls)] = (name, now)
+    return name
+
+
+def _desktop_name(app, cls):
+    """The desktop file name (without .desktop) for a window's app id and class, or "" when no
+    desktop file is its. Most windows give their desktop file name as the app id. The others:
+    - A Flatpak app's window can give an id with no desktop file: RustDesk's (an X11 window)
+      says com.carriez.flutter_hbb, and its desktop file is com.rustdesk.RustDesk.
+    - An X11 window can give no app id at all, only its class: Brave's Flatpak says
+      "Brave-browser" (2026-10-13), and its desktop file is com.brave.Browser.
+    Then, by the class (or the id), in this order: a desktop file's StartupWMClass; a desktop
+    file whose name, or its last dotted part, is that name; the program a desktop file runs
+    (Flatpak's --command=, else the Exec's own), by its name."""
     try:
-        if not app or Gio.DesktopAppInfo.new(app + ".desktop"):
+        if app and Gio.DesktopAppInfo.new(app + ".desktop"):
             return app
     except TypeError:  # PyGObject raises for the NULL a missing desktop file returns
         pass
     names = {n.lower() for n in (app, cls) if n}
-    for info in Gio.AppInfo.get_all():
-        wm = info.get_startup_wm_class() if isinstance(info, Gio.DesktopAppInfo) else None
+    if not names:
+        return ""
+    infos = [i for i in Gio.AppInfo.get_all() if isinstance(i, Gio.DesktopAppInfo) and i.get_id()]
+    for info in infos:
+        wm = info.get_startup_wm_class()
         if wm and wm.lower() in names:
             return info.get_id().removesuffix(".desktop")
-    return app
+    for info in infos:
+        did = info.get_id().removesuffix(".desktop").lower()
+        if did in names or did.rsplit(".", 1)[-1] in names:
+            return info.get_id().removesuffix(".desktop")
+    for info in infos:
+        line = info.get_commandline() or ""
+        m = re.search(r"--command=(\S+)", line)
+        prog = os.path.basename(m.group(1) if m else (info.get_executable() or ""))
+        if prog and prog.lower() not in ("flatpak", "env", "sh", "bash", "python3", "snap") and \
+                any(n == prog.lower() or n.startswith(prog.lower() + "-") for n in names):
+            return info.get_id().removesuffix(".desktop")
+    return ""
 
 
 class Launch:
@@ -259,6 +309,7 @@ class Launch:
         self.pids = [pid] if pid else []  # the processes we started (none when D-Bus started it)
         self.entries = entries    # a profile's entries still waiting for a window, or None
         self.relaunched = False
+        self.dock = None          # its first window opens docked on this output (launch screen=N)
         self.until = time.monotonic() + LAUNCH_SECONDS
 
 
@@ -287,7 +338,7 @@ class Slot:
         self.k = k
         self.output = f"WL-{screens + k}"
         self.index = screens + k + 1  # ft-screens' number (1-based)
-        self.pos = (SPARE_X + (k % 4) * SPARE_CELL, (k // 4) * SPARE_CELL)
+        self.pos = ((k % 4) * SPARE_CELL, SPARE_Y + (k // 4) * SPARE_CELL)
         self.size = None              # its output's size in pixels, as last set
         self.want = None              # the size in pixels asked for (size is at least that)
         self.kscale = 1.0             # its output's scale in KWin, as last set
@@ -308,7 +359,7 @@ class Daemon:
         self.sub_numbers = {}    # popup/dialog id -> (window id, number)
         self.next_sub = 1
         self.launches = []       # Launch: apps we started, waiting for their window
-        self.captures = {}       # token -> sender: "windows" requests waiting for the script's report
+        self.captures = {}       # token -> (sender, what): "windows" and "spotlist" requests waiting for the script's report
         self.next_token = 1
         self.sock = None         # our socket (set by main), for deferred replies
         self.places = load_places()  # app -> {"rel": pose by the primary screen, "pixels", "scale"}
@@ -394,6 +445,11 @@ class Daemon:
             return
         if kind == "added" and self.launches and not f:
             launch = self.launched(ev)
+            if launch and launch.dock:
+                log(f"{wid[:9]} ({ev.get('cls')}) opens docked on {launch.dock}")
+                self.command(cmd="to-output", id=wid, output=launch.dock)
+                self.seen(ev)
+                return
             if launch and launch.entries is None:
                 self.float_launched(ev, launch)
                 return
@@ -469,7 +525,7 @@ class Daemon:
                 return
             # Floating when ft-floatd (re)started: take it over where it is.
             f = Float(wid, slot, None)
-            f.app = ev.get("app", "")
+            f.app = desktop_name(ev.get("app", ""), ev.get("cls", "")) or ev.get("app", "")
             slot.window = f
             self.floats[wid] = f
             f.scale = slot.kscale = output_scales().get(slot.output, 1.0)
@@ -557,7 +613,7 @@ class Daemon:
                         "to float another.")
             return None
         f = Float(wid, slot, {"output": ev["output"], "frame": ev["frame"], "onAllDesktops": ev.get("onAllDesktops")})
-        f.app = ev.get("app", "")
+        f.app = desktop_name(ev.get("app", ""), ev.get("cls", "")) or ev.get("app", "")
         slot.window = f
         self.floats[wid] = f
         scales = output_scales()
@@ -760,9 +816,11 @@ class Daemon:
         c[1] -= 0.1 * d  # a little below eye level, like a screen
         return c, ft_layout.turn_yaw((1.0, 0.0, 0.0), heading), (0.0, 1.0, 0.0), ft_layout.turn_yaw((0.0, 0.0, 1.0), heading)
 
-    def launch(self, app=None, argv=None):
-        """Start an app (a desktop file name) or a command; its first window will float.
-        Returns (reply, pid or None)."""
+    def launch(self, app=None, argv=None, action=None, dock=None):
+        """Start an app (a desktop file name) or a command; its first window will float, or
+        with dock (an output name) open docked there. action: one of the desktop file's
+        actions instead (no pid: its window is known by the app's name). Returns (reply, pid or
+        None)."""
         if app:
             app = app.removesuffix(".desktop")
             try:
@@ -771,14 +829,22 @@ class Daemon:
                 info = None
             if info is None:
                 return f"error no app {app}", None
+            if action and action not in info.list_actions():
+                return f"error {app} has no action {action}", None
             pids = []
             try:
-                info.launch_uris_as_manager([], Gio.AppLaunchContext(), GLib.SpawnFlags.SEARCH_PATH,
-                                            None, None, lambda _info, pid, *_: pids.append(pid), None)
+                if action:
+                    info.launch_action(action, Gio.AppLaunchContext())
+                else:
+                    info.launch_uris_as_manager([], Gio.AppLaunchContext(), GLib.SpawnFlags.SEARCH_PATH,
+                                                None, None, lambda _info, pid, *_: pids.append(pid), None)
             except GLib.Error as e:
                 return f"error {app}: {e.message}", None
-            self.launches.append(Launch(app, pids[0] if pids else None))
-            log(f"launched {app} (pid {pids[0] if pids else 'by D-Bus'})")
+            la = Launch(app, pids[0] if pids else None)
+            la.dock = dock
+            self.launches.append(la)
+            log(f"launched {app}{' (' + action + ')' if action else ''}"
+                f"{' docked on ' + dock if dock else ''} (pid {pids[0] if pids else 'by D-Bus'})")
             return "ok", (pids[0] if pids else None)
         else:
             try:
@@ -798,16 +864,26 @@ class Daemon:
         if not ev.get("normal") or ev.get("popup") or ev.get("transient"):
             return None
         chain = pid_chain(int(ev.get("pid") or 0))
-        app = ev.get("app", "")
-        # (An X11 window in a Flatpak gives its pid in the sandbox: only its app id can match.)
-        apps = {app, desktop_name(app, ev.get("cls", ""))} if app and any(la.app for la in self.launches) else {app}
+        # A window in a sandbox (Flatpak, Snap) gives its pid in there, and a single-instance app
+        # opens it from a process that was already running: then only its desktop file can match.
+        name = desktop_name(ev.get("app", ""), ev.get("cls", "")) if any(la.app for la in self.launches) else ""
         for la in self.launches:
-            if any(p in chain for p in la.pids) or (la.app and app and la.app in apps):
-                if not la.entries or len(la.entries) <= 1:
-                    self.launches.remove(la)
-                return la
+            if any(p in chain for p in la.pids) or (la.app and name == la.app):
+                return self.took(la)
+        # Last: a window no desktop file claims, while exactly one app launch waits and none of
+        # its processes made it (an app whose window names itself after nothing we can find).
+        waiting = [la for la in self.launches if la.app]
+        if not name and len(waiting) == 1 and not any(p in chain for la in self.launches for p in la.pids):
+            log(f"{ev.get('id', '')[:9]} ({ev.get('cls', '') or '?'}): no desktop file claims it; "
+                f"taken as {waiting[0].app}'s, the one launch waiting")
+            return self.took(waiting[0])
         return None
 
+    def took(self, la):
+        if not la.entries or len(la.entries) <= 1:
+            self.launches.remove(la)
+        return la
+
     def concealed(self):
         """The screens (1-based) hidden on their own (ft-layout hide N)."""
         reply = self.screens.ask("concealed", quiet=True)
@@ -825,7 +901,7 @@ class Daemon:
     def float_launched(self, ev, launch):
         """Float a launched app's window (or one that opened on a hidden screen) where that app
         last floated (its size too), or in front of you, at the primary screen's density."""
-        app = ev.get("app") or (launch.app if launch else "")
+        app = (launch.app if launch and launch.app else "") or desktop_name(ev.get("app", ""), ev.get("cls", "")) or ev.get("app", "")
         known = self.places.get(app) if app else None
         ref = self.reference()
         place = to_world(ref, known["rel"]) if known and ref and len(known.get("rel", [])) == 12 else self.in_front()
@@ -838,25 +914,42 @@ class Daemon:
 
     # ------------------------------------------------------------ profiles (docs/profiles.md)
 
-    def capture(self, sender):
-        """"windows": have the script report every window as it is now, and answer when the
-        report is in (or after 3 seconds with what we know)."""
+    def capture(self, sender, what="windows"):
+        """"windows" and "spotlist": have the script report every window as it is now, and
+        answer when the report is in (or after 3 seconds with what we know)."""
         if not sender:
-            return "error windows needs a reply address"
+            return f"error {what} needs a reply address"
         token = self.next_token
         self.next_token += 1
-        self.captures[token] = sender
+        self.captures[token] = (sender, what)
         self.command(cmd="report-all", token=token)
         GLib.timeout_add(3000, lambda: self.captured(token) and False)
         return None  # answered by captured
 
     def captured(self, token):
-        sender = self.captures.pop(token, None)
+        sender, what = self.captures.pop(token, (None, None))
         if sender and self.sock:
+            entries = self.spot_entries() if what == "spotlist" else self.window_entries()
             try:
-                self.sock.sendto(("ok " + json.dumps(self.window_entries())).encode(), sender)
+                self.sock.sendto(("ok " + json.dumps(entries)).encode(), sender)
             except OSError as e:
-                log(f"windows: {e}")
+                log(f"{what}: {e}")
+
+    def spot_entries(self):
+        """Every app window for spotwatch's search: its desktop file name, title, the panel
+        it's on (a screen or a floating window's; None elsewhere), and when it was last active."""
+        out = []
+        for wid, ev in self.windows.items():
+            if not listable(ev):
+                continue
+            f = self.floats.get(wid)
+            m = re.match(r"WL-(\d+)$", ev.get("output", ""))
+            panel = f.slot.index if f else (int(m.group(1)) + 1 if m and int(m.group(1)) < self.screens_n else None)
+            out.append({"id": wid, "app": desktop_name(ev.get("app", ""), ev.get("cls", "")),
+                        "class": ev.get("cls", ""), "pid": ev.get("pid") or 0, "title": ev.get("caption", ""),
+                        "panel": panel, "floating": bool(f), "minimized": bool(ev.get("minimized")),
+                        "activeAt": ev.get("activeAt") or 0})
+        return out
 
     def window_entries(self):
         """Every app window and where it is, as a profile keeps it."""
@@ -866,7 +959,8 @@ class Daemon:
         for wid, ev in self.windows.items():
             if not recordable(ev):
                 continue
-            entry = {"app": desktop_name(ev["app"], ev.get("cls", ""))} if ev.get("app") else {"cmd": cmdline(ev.get("pid")), "class": ev.get("cls", "")}
+            app = desktop_name(ev.get("app", ""), ev.get("cls", ""))
+            entry = {"app": app} if app else {"cmd": cmdline(ev.get("pid")), "class": ev.get("cls", "")}
             if not entry.get("app") and not entry.get("cmd"):
                 continue
             f = self.floats.get(wid)
@@ -1036,8 +1130,15 @@ class Daemon:
         if not words:
             return "error empty"
         cmd, rest = words[0], words[1:]
-        if cmd == "launch" and len(rest) == 1:
-            return self.launch(app=rest[0])[0]
+        if cmd == "launch" and rest:
+            opts = dict(w.split("=", 1) for w in rest[1:] if "=" in w)
+            if len(opts) != len(rest) - 1 or set(opts) - {"action", "screen"}:
+                return "error launch APP.desktop [action=ACTION] [screen=N]"
+            screen = opts.get("screen")
+            if screen is not None and not (screen.isdigit() and 1 <= int(screen) <= self.screens_n):
+                return f"error no screen {screen}"
+            return self.launch(app=rest[0], action=opts.get("action"),
+                               dock=f"WL-{int(screen) - 1}" if screen else None)[0]
         if cmd == "run" and rest:
             try:
                 argv = json.loads(text.split(None, 1)[1])
@@ -1046,8 +1147,13 @@ class Daemon:
             if not isinstance(argv, list) or not argv or not all(isinstance(a, str) for a in argv):
                 return "error run takes a JSON list"
             return self.launch(argv=argv)[0]
-        if cmd == "windows":
-            return self.capture(sender)
+        if cmd in ("windows", "spotlist"):
+            return self.capture(sender, cmd)
+        if cmd == "activate" and len(rest) == 1:
+            if rest[0] not in self.windows:
+                return f"error no window {rest[0]}"
+            self.command(cmd="activate", id=rest[0])
+            return "ok"
         if cmd == "profile" and rest:
             return self.open_profile(text.split(None, 1)[1])
         if cmd == "list":
@@ -1160,7 +1266,7 @@ def main():
     conf = ft_layout.read_conf()
     p = argparse.ArgumentParser(description="Floating windows for the Frametop desktop")
     p.add_argument("--screens", type=int, default=int(os.environ.get("FT_SCREEN_COUNT") or 0))
-    p.add_argument("--slots", type=int, default=int(conf.get("FLOAT_SLOTS") or 8))
+    p.add_argument("--slots", type=int, default=int(conf.get("FLOAT_SLOTS") or 16))
     p.add_argument("--margin", type=int, default=int(conf.get("FLOAT_MARGIN") or 300))
     p.add_argument("--control", default="ft_screens")
     p.add_argument("--socket", default="frametop_float")

framedrop/README.md

@@ -1,6 +1,6 @@
 # Install with FrameDrop (proof of concept)
 
-[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and two optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.
+[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and four optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.
 
 Nothing here is published yet: no release, no button.
 
@@ -26,7 +26,7 @@ It uses Valve's SteamOS Devkit path: pair once with the headset's devkit service
 
 1. In the container, it starts itself again on the host with `flatpak-spawn --host`.
 2. It clears Steam's preload and library paths, and opens `installer/progress.py` (GTK 4 and libadwaita).
-3. The window asks which optional parts to install: our own eye tracker (on by default) and the Bluetooth fixes. Both need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves both out.
+3. The window asks which optional parts to install: voice dictation (on by default), hand tracking (on by default), the Hand Recorder (on by default; a release's zip only, since a clone builds it with `hands/rec/install.sh`), our own eye tracker (on by default), and the Bluetooth fixes. All but voice dictation need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves those out.
 4. It runs the zip's `install-release.sh --yes` (a test zip: `get.sh --yes`) in a transient user service, `frametop-framedrop-install`. The service is used because Steam ends the title's whole process tree when it's quit, and starts it with an OOM score of 900. Opened from a VR desktop (the zip unpacked by hand), it still uses the user's real bus and runtime folder for the service.
 5. It follows the service's log, shows the steps as a progress bar, and reports the result. Closing it leaves the install running. Playing the title again reattaches.
 
@@ -62,7 +62,7 @@ framedrop/build.sh --image REF --version V [--commit SHA] [--channel C] [ZIP_URL
 framedrop/build.sh [ZIP_URL]                                                        # a test zip
 ```
 
-This writes `framedrop/build/Frametop.zip` (reproducible), `frametop.framedrop.json` (FrameDrop's manifest with the zip's sha256), and `SHA256SUMS`. A release's zip has the image REF (`podman save`) with its `frametop-release.json` (`pack/release-info.py`) and `install-release.sh`; CI builds it on a tag (`.github/workflows/release.yml`). A test zip has `get.sh` instead. By default, `ZIP_URL` is the release's asset (`releases/download/vV/Frametop.zip`; for a test zip, a `framedrop-installer` release's). Each release carries its manifest, so the button's link can point at the newest stable one: `https://framedropvr.com/install?manifest=https://github.com/DeeJanuz/frametop/releases/latest/download/frametop.framedrop.json` (the exact URL format is FrameDrop's to confirm).
+This writes `framedrop/build/Frametop.zip` (reproducible), `frametop.framedrop.json` (FrameDrop's manifest with the zip's sha256), and `SHA256SUMS`. A release's zip has the image REF (`podman save`) with its `frametop-release.json` (`pack/release-info.py`) and `install-release.sh`; CI builds it on a tag (`.github/workflows/release.yml`). A test zip has `get.sh` instead. By default, `ZIP_URL` is the release's asset (`releases/download/vV/Frametop.zip`; for a test zip, a `framedrop-installer` release's). Each release carries its manifest, so the button's link can point at the newest stable one: `https://framedropvr.com/install?manifest=https://github.com/Frametop/frametop/releases/latest/download/frametop.framedrop.json` (the exact URL format is FrameDrop's to confirm).
 
 ## Open questions, for a test with FrameDrop on a Windows PC
 

framedrop/build.sh

@@ -40,14 +40,14 @@ files=("$here/installer/frametop-install.sh" "$here/installer/progress.py" "$her
 if [ -n "$image" ]; then
   [ -n "$version" ] || { echo "--image needs --version" >&2; exit 2; }
   commit=${commit:-$(git -C "$repo" rev-parse HEAD)}
-  url=${1:-https://github.com/DeeJanuz/frametop/releases/download/v$version/Frametop.zip}
+  url=${1:-https://github.com/Frametop/frametop/releases/download/v$version/Frametop.zip}
   echo "saving $image"
   podman save -q --format oci-archive -o "$out/frametop-image.tar" "$image"
   python3 "$repo/pack/release-info.py" --image-file "$out/frametop-image.tar" --version "$version" \
     --commit "$commit" ${channel:+--channel "$channel"} >"$out/frametop-release.json"
   files+=("$repo/pack/install-release.sh" "$out/frametop-release.json" "$out/frametop-image.tar")
 else
-  url=${1:-https://github.com/DeeJanuz/frametop/releases/download/framedrop-installer/Frametop.zip}
+  url=${1:-https://github.com/Frametop/frametop/releases/download/framedrop-installer/Frametop.zip}
   files+=("$repo/get.sh")
 fi
 

framedrop/installer/progress.py

@@ -13,7 +13,7 @@ The install runs in a user service of its own (UNIT): Steam ends the title's who
 tree when it's quit, and starts it with a high OOM score. Closing the window doesn't stop the
 install; playing the title again reattaches to it.
 
-Our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
+Hand tracking, the Hand Recorder, our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
 headset, checks it with sudo, and keeps it in this process's memory until the install ends.
 sudo in the install gets it through askpass (SUDO_ASKPASS), from a socket in a folder only you
 can open, and the window answers only programs in the install's service. It's never written to
@@ -46,7 +46,7 @@ RELEASE = (HERE / "install-release.sh").exists() and (HERE / "frametop-image.tar
 SOCK_DIR = Path(f"/run/user/{os.getuid()}/frametop-install")
 SOCK = SOCK_DIR / "askpass"
 ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
-# get.sh and install.sh mark each step with "== N/10 what it is"
+# get.sh and install.sh mark each step with "== N/M what it is"
 STEP = re.compile(r"^== (\d+)/(\d+) (.*)$")
 DONE_TEXT = ("Frametop is installed. Restart SteamVR once, or reboot the headset, so it loads "
              "Frametop's driver. Then open Launch a program, then Desktop.")
@@ -101,7 +101,7 @@ def in_unit(pid):
         return False
 
 
-def install_command(eye_tracker, bluetooth):
+def install_command(eye_tracker, bluetooth, voice=True, hands=True, hand_recorder=True):
     if RELEASE:
         args = [str(HERE / "install-release.sh"), "--yes"]
         if DRY_RUN:
@@ -110,6 +110,12 @@ def install_command(eye_tracker, bluetooth):
         args = [str(HERE / "get.sh"), "--yes"]
         if DRY_RUN:
             args += ["--clone-only", "--dir", str(STATE / "dry-run")]
+    if not voice:
+        args.append("--no-voice")
+    if not hands:
+        args.append("--no-hands")
+    if RELEASE and not hand_recorder:   # a clone (get.sh) doesn't offer it: hands/rec/install.sh builds it
+        args.append("--no-hand-recorder")
     if not eye_tracker:
         args.append("--no-eye-tracker")
     if bluetooth:
@@ -302,8 +308,16 @@ class Window(Adw.ApplicationWindow):
     def show_choices(self):
         what = f"Frametop {release_version()}" if RELEASE else "Frametop from GitHub"
         self.detail.set_label(f"This installs {what}: the multi-screen desktop, the 3D mouse, gaze "
-                              "mode, and Frametop's settings apps. Two optional parts need your "
-                              "SteamOS password (sudo):")
+                              "mode, and Frametop's settings apps. Optional: voice dictation, and "
+                              "parts that need your SteamOS password (sudo):")
+        self.voice = Gtk.CheckButton(label="Voice dictation: a Right Ctrl tap or holding the left d-pad down, and "
+                                           "what you say is typed (downloads about 320 MB)", active=True)
+        self.hands = Gtk.CheckButton(label="Hand tracking (experimental): poke, pinch and grab the screens with "
+                                           "your hands. It uses a lot of the headset's CPU", active=True)
+        # A release brings it built; a clone builds it with hands/rec/install.sh, so isn't offered it
+        self.recorder = Gtk.CheckButton(label="Hand Recorder: record your hands for Frametop's open hand "
+                                              "dataset, which trains hand tracking (nothing is uploaded "
+                                              "until you press Upload)", active=True, visible=RELEASE)
         self.eye = Gtk.CheckButton(label="Our own eye tracker for gaze mode (more accurate than SteamVR's)",
                                    active=True)
         self.bt = Gtk.CheckButton(label="Bluetooth fixes (LE mice and keyboards, like the Swiftpoint Z3, "
@@ -319,27 +333,28 @@ class Window(Adw.ApplicationWindow):
         self.go.connect("clicked", self.install)
         self.pw.connect("activate", self.install)
         pw_row, keypad = keypad_row(self.pw)
-        self.choices = [self.eye, self.bt, pw_row, keypad, pw_note, self.error, self.go]
+        self.choices = [self.voice, self.hands, self.recorder, self.eye, self.bt, pw_row, keypad, pw_note,
+                        self.error, self.go]
         if not password_set():
-            for c in (self.eye, self.bt):
+            for c in (self.hands, self.recorder, self.eye, self.bt):
                 c.set_active(False)
                 c.set_sensitive(False)
             pw_row.set_visible(False)
             keypad.set_visible(False)
-            pw_note.set_label("Your user has no password, so sudo can't run and these two can't be "
+            pw_note.set_label("Your user has no password, so sudo can't run and these can't be "
                               "installed from here. To set one, run passwd in Konsole; then play "
                               "Frametop again, or install them later from a terminal "
-                              "(gaze/tracker/install.sh, setup/bluetooth/install.sh).")
-        for c in (self.eye, self.bt):
-            c.connect("toggled", lambda *_: pw_row.set_sensitive(self.eye.get_active() or self.bt.get_active()))
+                              "(hands/run.sh install, hands/rec/install.sh, gaze/tracker/install.sh, "
+                              "setup/bluetooth/install.sh).")
+        for c in (self.hands, self.recorder, self.eye, self.bt):
+            c.connect("toggled", lambda *_: pw_row.set_sensitive(self.needs_password()))
         for w in self.choices:
             self.box.append(w)
 
     def install(self, *_):
         if self.checking:
             return
-        needs = self.eye.get_active() or self.bt.get_active()
-        if not needs:
+        if not self.needs_password():
             return self.begin(None)
         text = self.pw.get_text()
         if not text:
@@ -366,12 +381,17 @@ class Window(Adw.ApplicationWindow):
 
         threading.Thread(target=check, daemon=True).start()
 
+    def needs_password(self):
+        return (self.hands.get_active() or (RELEASE and self.recorder.get_active()) or self.eye.get_active()
+                or self.bt.get_active())
+
     def say(self, text):
         self.error.set_label(text)
         self.error.set_visible(True)
 
     def begin(self, pw):
-        args = install_command(self.eye.get_active(), self.bt.get_active())
+        args = install_command(self.eye.get_active(), self.bt.get_active(), self.voice.get_active(),
+                               self.hands.get_active(), self.recorder.get_active())
         if pw is not None:
             try:
                 self.askpass = Askpass(self.ask_again)

framedrop/probe/probe.sh

@@ -61,8 +61,8 @@ done
 section "what an installer needs"
 check "home writable" sh -c 'f=$HOME/.cache/frametop-framedrop/.w && : >"$f" && rm "$f" && echo yes'
 check "~/frametop visible" sh -c 'ls -d "$HOME/frametop" && git -C "$HOME/frametop" log -1 --format=%h'
-have git && check "git ls-remote github" git ls-remote --heads https://github.com/DeeJanuz/frametop.git experimental
-have curl && check "curl get.sh" sh -c 'curl -fsSL https://deejanuz.github.io/frametop/get.sh | head -1'
+have git && check "git ls-remote github" git ls-remote --heads https://github.com/Frametop/frametop.git experimental
+have curl && check "curl get.sh" sh -c 'curl -fsSL https://frametop.github.io/frametop/get.sh | head -1'
 have podman && check "podman ps" podman ps --format '{{.Names}}'
 have distrobox && check "distrobox list" distrobox list
 have systemctl && check "systemctl --user" systemctl --user is-system-running

gaze/README.md

@@ -9,8 +9,11 @@ The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (t
 
 Day to day, install the gaze service, then turn gaze mode on and calibrate on the Gaze page of Frametop Input Settings (Calibrate). The installer offers the gaze service (`gaze/run.sh install`) and then our own tracker (`gaze/tracker/install.sh`), which gaze mode uses once it's installed; the probe is installed by hand.
 
+The gaze service's install also turns on SteamVR's `steamvr.gazeDepEyeToHeadEnable` (`gaze/eye-rate.py on`), which counts from SteamVR's next start. The Frame's SteamVR driver runs the eye cameras at 15 to 90 fps (`eyeTrackingRateMin` and `eyeTrackingRateMax` in its `frame_hmd_additional.vrsettings`) and picks the rate as SteamVR starts: with its defaults, 15 (`eyetracking.txt`: "Set framerate 15"). That's enough for SteamVR's own tracker, but our tracker then sees 8 or 9 frames in a calibration dot's 0.6 s, where it needs 15, so it can never calibrate (a 2026-10-10 report: "the right eye was seen in only 8 frames" on every dot, while SteamVR's fit screen showed both eyes tracked). With that setting on, SteamVR asks for 90 at every start (vrserver.txt: "cv: Request eye tracking framerate: 90"). Valve's driver leaves it off and SteamVR's settings have no switch for it; what else it changes isn't documented, and the headset Frametop is developed on runs with it on. Uninstalling turns it off again only if Frametop turned it on. `scripts/doctor.sh` and `scripts/gaze-report.py` report the cameras' rate.
+
 ```
 gaze/run.sh install           # the gaze service: builds ft-gaze and the panel, starts with SteamVR
+gaze/eye-rate.py status       # SteamVR's eye cameras: the setting that keeps them at 90 fps, and their rate now
 gaze/ft-gazectl on            # the pointer follows your gaze (off: the mouse alone)
 gaze/tracker/install.sh       # our own eye tracker's frame grabber (asks for sudo)
 gaze/build.sh                 # build ft-gaze and the panel by hand
@@ -26,6 +29,8 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
 - `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. Two settings on the Gaze page of Frametop Input Settings (`GAZE_TRACKER` and `GAZE_EYE` in `~/.config/frametop.conf`, read again when the file changes) pick whose eye tracking it uses and how it weights the eyes:
   - **Eye tracker:** our own (Own tracker: see "Our own eye tracker" below) or SteamVR's. The default, `GAZE_TRACKER=auto`, is ours when it's installed (its frame grabber, and ft-eyes' Python in the gaze service's checkout), else SteamVR's, and it switches when ours is installed or removed; picking one on the Gaze page sets it for good. The gaze service runs ours while it's the one in use. It keeps its own calibration: with Own tracker chosen, Calibrate on the Gaze page calibrates it. The gaze pointer's settings (hand back, nudges, hold to drag, the dot) are the pointer helper's, so they're the same with either.
   - **Eye bias:** Auto, Left, or Right. The gaze combines both eyes, each calibrated on its own, because their errors partly cancel: on 306 clicks with our tracker, the eyes' sideways errors were correlated -0.37, and both together were 0.65 degrees off (median) against 0.96 for the left eye alone and 1.11 for the right. So Left or Right leans instead of choosing: that eye counts twice as much as the other (0.03 degrees worse there toward the better eye, 0.13 toward the worse). Auto weights each eye by the inverse square of how far off it was at your last 20 nudges, once each eye has 5, and evenly before that. Each eye's miss is measured before that nudge teaches anything, so each is a fresh test. The calibration's own fit isn't used for this: on SteamVR's test of 2026-09-29, the calibration dots said the left eye was the better one, and new spots said the right. Either eye carries the gaze alone while the other is closed or lost.
+  - **One eye:** SteamOS 0.4's Track Dominant Eye Only (SteamVR's settings, General, with Show advanced settings on; `steamvr.eyeTrackingDominantEyeOnly` with `steamvr.dominantEye`) makes SteamVR's tracker ignore the other eye. With SteamVR's tracker, Frametop then goes by that eye alone: the calibration and the checks wait only for it, the gaze is that eye's own reading once it's calibrated (SteamVR's combined gaze before, which follows that eye then), and the fit check shows the other eye as not tracked. The service reads the setting again when SteamVR's settings file changes. Not yet tried in the headset: what SteamVR's shared memory says about the ignored eye wasn't measured.
+  - **Fusion** (`GAZE_FUSION=on` in `~/.config/frametop.conf`, off by default, not on the Gaze page yet): with our tracker, the gaze takes in SteamVR's too. SteamVR's is its two eyes averaged, corrected by an affine fit (offset, gain, and tilt) on the same dots as our calibration: Calibrate with Own tracker saves it in `~/.local/state/frametop/gaze/fusion.json`, and `gaze/bench/compare.py --fit-fusion` can make one from bench runs. While ours sees both eyes, the gaze is half ours and half SteamVR's. While ours has one eye or none, SteamVR's carries it alone, and without SteamVR's eyes (closed, lost, or its one-eye setting) it's ours. The two trackers' errors partly cancel. On the gaze bench's two head-locked runs of 2026-10-17 (25 presses), each fitted on the other presses: ours alone was 2.01 degrees off (median; 6.47 at the 90th percentile), SteamVR's eyes averaged 2.67 (5.72), SteamVR's corrected 1.78 (4.26), and the fusion 1.51 (3.10). Fitted on the other run only, the fusion was 1.41 (4.59). The fit holds only while SteamVR's tracker reads as it did at the calibration, and SteamVR starts its eye model over each time the headset goes on, so it stays off until benches across wears say it holds.
 
   With SteamVR, each eye is its own reading (set 2), corrected by its calibration from the probe (the Left eye and Right eye sources) plus what the pointer has taught that eye since. On that test, the two eyes each calibrated and averaged were 1.70 degrees off (median; mean 1.62) against 1.72 (mean 1.84) for SteamVR's combined gaze with its calibration. A calibration from before the probe had the eyes as sources, or `--source`, uses the older path. That path runs on SteamVR's combined gaze (mmap set 1), corrected as a whole. When the tracker loses one eye (its variance for that eye jumps from about 0.001 to 0.02), the gaze comes from the other eye instead: that eye's own reading (set 2) plus what it usually reads against the combined gaze, learned while both eyes are seen, in 10 degree cells of where it looks. Set 1 keeps going on one eye too, but it holds the lost eye's yaw where it was, so the gaze moves half as far sideways as your eyes do. On a recording, one eye alone came out a median 0.8 degrees from both eyes' gaze over a steady look, a little more jittery.
 
@@ -34,7 +39,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
 - **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
 - **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
 - **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
-- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow: the middle, and 12 degrees left and right and 9 up and down, in a see-through panel 40 degrees wide (the quick check's 16 degree square left four of them off the panel, unseen). The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
+- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click, Enter, an index pinch (with hand tracking) or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow: the middle, and 12 degrees left and right and 9 up and down, in a see-through panel 40 degrees wide (the quick check's 16 degree square left four of them off the panel, unseen). The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click, press Enter, pinch (index to thumb, with hand tracking) or press Meta+J, and the gaze held still up to then is taken. A button mapped to gaze precision or gaze drag counts as the left click there (before 2026-10-09 the panel ignored it). Enter works on a keyboard while typing goes to the desktop (the relay takes it while the panel is up, so it isn't typed; while typing goes to Steam, Steam sees it too). A pinch counts from its start, whether or not hand interaction is on, but not in a VR game, with the headset off, while typing, or at a controller. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). The dot's ring shows full while ft-eyes checks it; the service doesn't wait for that answer (before 2026-10-09 it did, up to 3 s a dot, so the pointer could come back over the panel, and a slow answer closed the calibration as if the headset came off). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
 - **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
 
@@ -55,8 +60,8 @@ The tracker stops when the headset is off your head. SteamVR also calibrates gaz
 
 `gaze/tracker/` is an eye tracker of our own, because SteamVR's is about 1.5 degrees off after the best correction the gaze service can learn, and what's left is mostly look-to-look noise that no correction on top of its output can remove. Ours processes the eye cameras itself: 0.59 degrees (median) in its best live session against 0.83 for SteamVR's with the probe's correction, and after the headset was taken off and put back without recalibrating, 0.58 once your first clicks had taught it where the headset sat (`tracker/findings.md` has the measurements).
 
-- `ft-eyegrab` (C, root, the system service `frametop-eyegrab.service`) copies the eye-camera frames (512x400, 90 fps per eye) out of the DMA-BUFs SteamVR's `eyetracking` process holds into `/dev/shm/frametop-eyes-cams`, owned by you. It maps them read-only, and it only copies while someone touches `/dev/shm/frametop-eyes-want` (ft-eyes and the recorder do, every second). Otherwise it holds none of the tracker's buffers. Its unit keeps only the capabilities that needs (`CAP_SYS_PTRACE`, `CAP_DAC_READ_SEARCH`, `CAP_CHOWN`). `gaze/tracker/install.sh` builds it and installs it to `/etc/frametop` with sudo, which it asks for (`uninstall`, `status`, and `log` too).
-- `ft-eyes` (Python with numpy and OpenCV, in the dev container: `gaze/tracker/build.sh` puts the pinned `requirements.txt` in `gaze/tracker/build/venv`) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (`eyes_pupil.py`), and maps them to a gaze with a quadratic fit per eye (`eyes_model.py`). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. It publishes the gaze in `/dev/shm/frametop-eyes-gaze` (ft-gaze's source `own`) and takes calibration dots and clicks on `@ft_eyes`. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in `~/.local/state/frametop/gaze/eyes/`.
+- `ft-eyegrab` (C, root, the system service `frametop-eyegrab.service`) copies the eye-camera frames (512x400, 90 fps per eye, or 15 without `gaze/eye-rate.py on`) out of the DMA-BUFs SteamVR's `eyetracking` process holds into `/dev/shm/frametop-eyes-cams`, owned by you. It maps them read-only, and it only copies while someone touches `/dev/shm/frametop-eyes-want` (ft-eyes and the recorder do, every second). Otherwise it holds none of the tracker's buffers. Its unit keeps only the capabilities that needs (`CAP_SYS_PTRACE`, `CAP_DAC_READ_SEARCH`, `CAP_CHOWN`). `gaze/tracker/install.sh` builds it and installs it to `/etc/frametop` with sudo, which it asks for (`uninstall`, `status`, and `log` too). It also adds it to SteamOS's update keep list (`/etc/atomic-update.conf.d/frametop-eyegrab.conf`): an update deletes `/etc` files that list doesn't name, and without the grabber gaze mode falls back to SteamVR's tracker and its calibration. `scripts/doctor.sh` reports a grabber an update deleted.
+- `ft-eyes` (Python with numpy and OpenCV, in the dev container: `gaze/tracker/build.sh` puts the pinned `requirements.txt` in `gaze/tracker/build/venv`) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (`eyes_pupil.py`), and maps them to a gaze with a quadratic fit per eye (`eyes_model.py`). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. A find much smaller than that eye's usual pupil (the median over the last 10 s) counts as not found, since it's usually a dark spot beside the pupil (`EYES_SIZE_GUARD`): with `both`, the default, under 0.75 of the other eye's ratio, so light shrinking both pupils gets through; with `long`, under 0.6 whatever the other eye does; `off` takes every find. On the bench runs of 2026-10-17, good finds were 0.84 of usual size or more 95% of the time, and the right camera's misfinds a median of 0.77. It publishes the gaze in `/dev/shm/frametop-eyes-gaze` (ft-gaze's source `own`) and takes calibration dots and clicks on `@ft_eyes`. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in `~/.local/state/frametop/gaze/eyes/`. Instead of the pupil finder it can run an eye net (`eyes_net.py`): `EYES_MODEL` is `classic` (the default), `latest`, or a version from `gaze/tracker/models.json`, picked with its notes and benchmarks on Input Settings' Gaze page, which downloads it into `gaze/tracker/models/NAME` (`models/README.md`). Each net keeps its own calibration in `eyes/net-NAME/`.
 - `lab/` has the tools for improving it on recordings. `ft-eyes-record NAME` (or `ft-eyes-session`, with SteamVR's gaze alongside) records the cameras. `ft-eyes-score` fits and scores on recordings against the probe's practice clicks. `ft-eyes-e2e` runs the whole live path on two recordings (calibrate on one, click through the other). `ft-eyes-replay` plays a recording into a scratch share. Heavy ones are meant for a PC: if you have `frame-job` (a personal tool, not in this repo), `gaze/tracker/.frame-job` sends them there. `lab/py` runs them with that Python (in the dev container on the Frame; on a PC, the same venv from `requirements.txt`, which frame-job's setup makes).
 
 Ground rules, for anyone changing it:
@@ -68,7 +73,12 @@ Ground rules, for anyone changing it:
 
 ## Headset fit
 
-Check headset fit on the Gaze page opens it in the headset panel: a card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and the hints, live while you adjust the headset. A left click or Meta+J runs the guided check (dots, then looks down, up, left and right), and a right click or Meta+K closes it. The probe's Headset fit mode (`ft-gazeprobe --mode fit`, for development) has the same check with maps: it shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
+Check headset fit on the Gaze page opens it in the headset panel: a card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and the hints, live while you adjust the headset. A left click, Enter, a pinch or Meta+J runs the guided check (dots, then looks down, up, left and right), and a right click or Meta+K closes it. The probe's Headset fit mode (`ft-gazeprobe --mode fit`, for development) has the same check with maps: it shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
+
+**The fit watch** (`GAZE_FIT_WATCH=1` in `~/.config/frametop.conf`; off by default until it's been tried in the headset). The fit check keeps a good fit as yours: when its coach (`gaze/fitcoach.py`, from ft-eyes' pupils and glints, with the shipped reference) last read it as good while you looked straight ahead, closing the check saves that reading to `~/.local/state/frametop/gaze/fit-baseline.json`, with the IPD setting. The eye recorder's fitment step saves it too. Then, with the fit watch on:
+- **Put-on check:** when the headset goes on, the quick check first holds its dot for up to 4 s while the coach reads the fit against your saved one. If it sits the same, the dot is taken as usual. If it moved (5 degrees of gaze or more), the panel widens and says which way to move the headset, and a changed IPD over that, live, until it's back ("Good: it's back where it was"); then the dot is taken. A click takes the dot as the headset sits, and after 30 s it's taken anyway. It needs our tracker installed (ft-eyes runs for it with either tracker in use), and without a saved fit it's the plain quick check.
+- **Slip note:** with our tracker in use and gaze mode on, ft-eyes' glint slip (its `jump_px`, how far each pupil moved in its image since the last click) is turned into degrees through the reference. Past 5 degrees for 2 s, a small strip low in your view says "Your headset slipped" and which way to move it, for 8 s or until the slip is gone, at most once every 2 minutes. It isn't a check: the pointer stays, and a click or the quick check still teaches the tracker the new seating.
+`gaze/test/fit-watch-test.py` tests both offline.
 
 Losing an eye is usually about where you look, not the tracker. On this Frame the left eye was lost 57 to 64 % of the time looking 30 to 50 degrees down (at the keyboard) and the right eye never; at screen height both were seen over 98 % of the time. Looking down, the lids come down over the eyes. That's harmless, since the gaze service ignores looks down past the screens: they show on the maps, but not in the counts or as a problem.
 

gaze/bench/README.md

@@ -0,0 +1,101 @@
+# Gaze Bench
+
+A test bench for gaze tracking. It shows dots in the places gaze tracking finds hard (far out
+toward the edges and corners, long holds, light changes, a moving head, after the headset moved).
+You look at each one and press the headset's button. Every press is recorded with everything the
+trackers saw, so gaze techniques can be compared afterwards on the same looks (`compare.py`),
+including ones that didn't run live.
+
+The dots show on a panel in the headset, 64 degrees across, that moves with your head: a dot is
+always at the same direction from your head, wherever you face. So every run asks the same of the
+eyes, and you never have to turn your head to find a target. Your eyes do all the looking.
+
+## Use
+
+1. Put the headset on and open **Frametop Gaze Bench** from the app menu (`gaze/bench/install.sh`
+   adds it), or run `gaze/bench/ft-gazebench` from Frametop's desktop. A window opens; its
+   status lines say whether the headset panel, the headset button and the trackers are ready.
+   Gaze mode can be on or off: the bench keeps the trackers running itself.
+2. Pick a scenario. The panel shows what to do; press the button on the headset's right side
+   to begin.
+3. For each dot: look at it and press the button. The dot turns green and the next one shows.
+   Where you looked is the dot, so only press while you're on it.
+   Two presses take the last one back. Holding the button ends the scenario.
+   As in the gaze calibration, a left click, Enter or an index pinch (index to thumb, with hand
+   tracking) marks too, and a right click or Meta+K ends: while the panel shows, the bench claims
+   the pointer helper's calibration panel (`calpanel 1 ft_gazebench.PID`), so the helper sends
+   those here and hides the pointer. Those marks are recorded `"by": "helper"`, at the time the
+   helper's answer came in (a pinch's a frame or two after it began).
+   The window's keys do the same: Space marks, Backspace takes the last one back, S skips a
+   dot, Esc ends.
+4. At the end, the panel and the window say how far our tracker and SteamVR's were off. A press
+   (or a click, Enter or a pinch) closes the panel. Then run `compare.py` on the runs.
+
+| Scenario | What it tests |
+|---|---|
+| Warm-up | nine dots around the middle |
+| Edges | sixteen dots along the panel's edges and in its corners, 24 to 37 degrees out |
+| Hold at the edges | edge dots you look at for 3 s before pressing (the tracker loses an eye out there) |
+| Out from the middle | twenty-four dots on eight spokes, at a third, two thirds and all the way out |
+| Head moving | nine dots; keep looking while you turn or nod your head, press while it moves |
+| Light changes | dark, bright and grey backgrounds: pupil size |
+| Small steps | twenty dots, each 2 to 4 degrees from the last |
+| After a nudge | lift the headset and set it back, then nine dots: recovery |
+| Long session | sixty dots anywhere at your own pace, a few seconds apart, about six minutes: drift and stale corrections |
+
+Scenarios live in `scenarios.json` (the generators are described at its top).
+
+## Compare
+
+```
+scripts/in-box /opt/frametop/venv/bin/python gaze/bench/compare.py ~/.local/share/frametop/gaze/bench
+```
+
+Give it runs, or the folder they're in. For every press it prints how far each technique was from
+where you looked: overall, by scenario, by where the dot was (edge, corner, inside), by how far
+from straight ahead, and by the time since the last nudge. `--per-click` adds a line per press;
+`--json OUT` writes the records. The techniques are described at the top of `compare.py`: what
+ran live (our tracker, SteamVR's combined gaze and its two eyes averaged, a 70/30 mix, SteamVR's
+corrected by the presses), SteamVR's eyes corrected by the run's `fusion.json` (copied into
+`start/` when the run begins) and the fusion with ours, the same fitted on the run's other presses
+(`-loo`) or on the other runs only (`-xrun`), and replays of our classic tracker on its recorded
+finds with changed rules (no glint jump, a jump only when both eyes see it, no lessons past the
+calibrated range, learning from every press, bridging a lost eye, the size guard's two rules or
+none). `fusion-loo:REPLAY` fuses a replay with SteamVR's.
+`--fit-fusion OUT` fits SteamVR's correction on all the given runs' presses and saves it as a
+`fusion.json` the gaze service reads (`~/.local/state/frametop/gaze/fusion.json`). The replay of the live rules should land close to what our tracker published (the
+"replay vs live" line); the variants are only as trustworthy as that.
+
+Runs from before 2026-10-17 showed their targets on a Frametop screen, with clicks. `compare.py`
+still reads them: their truth comes from ft-gaze's screen hits, and they have a "pointer"
+technique (where the gaze pointer was).
+
+A long session's run (gaze.jsonl and own.jsonl) takes a while to replay: run it through
+`frame-job` when the headset is in use.
+
+## What a run records
+
+`~/.local/share/frametop/gaze/bench/<time>-<scenario>/` (`benchrec.py` describes every file):
+the dots and the presses, the panel's log of what the dot did and when it reached the eyes,
+the headset's pose 250 times a second, ft-gaze's samples of every source except SteamVR's input
+action (the one that costs vrserver anything), our tracker's per-frame pupil and glint finds,
+and our tracker's and the gaze service's calibration and state at the start. With **Record the
+eye cameras** on (the default), both eye cameras too, as the Eye Recorder records them: about
+1.2 GB a minute.
+
+Eye images identify a person. They stay on the headset: the bench uploads nothing, and the runs
+are yours to delete (the folder above).
+
+## Files
+
+| File | |
+|---|---|
+| `gazebench.py`, `bench.qml` | the app and its window (PySide6, in Frametop's container); it runs its own `ft-gazepanel` |
+| `benchrec.py` | the recording |
+| `scenarios.json` | the scenarios |
+| `compare.py` | the comparison (numpy, OpenCV) |
+| `ft-gazebench`, `ft-gazebench.desktop`, `install.sh` | the launcher and its menu entry |
+| `tests/` | `test_bench.py` (runs with stand-ins for ft-gaze, the panel, the button and the services), `test_compare.py` (a made-up run with known errors) |
+
+Tests: `scripts/in-box python3 gaze/bench/tests/test_bench.py`, and
+`scripts/in-box /opt/frametop/venv/bin/python gaze/bench/tests/test_compare.py`.

gaze/bench/bench.qml

@@ -0,0 +1,188 @@
+// Frametop Gaze Bench's window (gazebench.py), one page at a time: "menu" picks a scenario,
+// "run" follows it (its targets show on the headset's panel, not here), "done" shows how far the
+// trackers were off. The keys work as the headset button does.
+import QtQuick
+import QtQuick.Controls
+import QtQuick.Layouts
+import QtQuick.Window
+
+Window {
+    id: win
+    title: "Frametop Gaze Bench"
+    width: 960
+    height: 720
+    visible: true
+    color: "#262626"
+
+    readonly property color ink: "#eeeeee"
+    readonly property color dim: "#9a9a9a"
+
+    Shortcut { sequence: "Esc"; onActivated: bench.stop() }
+    Shortcut { sequence: "Backspace"; onActivated: bench.undo() }
+    Shortcut { sequence: "S"; enabled: bench.page === "run"; onActivated: bench.skip() }
+    Shortcut { sequences: ["Space", "Return", "Enter"]; enabled: bench.page !== "menu"; onActivated: bench.markKey() }
+
+    // --- menu
+
+    Item {
+        anchors.fill: parent
+        visible: bench.page === "menu"
+
+        ColumnLayout {
+            anchors.centerIn: parent
+            width: Math.min(parent.width - 60, 900)
+            spacing: 12
+
+            Label {
+                text: "Gaze Bench"
+                font.pixelSize: 30
+                color: win.ink
+            }
+            Label {
+                Layout.fillWidth: true
+                wrapMode: Text.Wrap
+                color: win.dim
+                font.pixelSize: 15
+                text: "Each scenario shows dots one at a time on a panel in front of you in the headset. "
+                    + "It moves with your head, so your eyes do all the looking. Look at the dot "
+                    + "and press the button on the headset's right side. Every look is recorded, so different "
+                    + "gaze techniques can be compared on the same looks afterwards."
+            }
+            Label {
+                Layout.fillWidth: true
+                visible: bench.notice !== ""
+                wrapMode: Text.Wrap
+                color: "#ffb347"
+                font.pixelSize: 15
+                text: bench.notice
+            }
+            GridLayout {
+                Layout.fillWidth: true
+                Layout.topMargin: 6
+                columns: 2
+                columnSpacing: 12
+                rowSpacing: 8
+                Repeater {
+                    model: bench.scenarios
+                    delegate: Button {
+                        required property var modelData
+                        Layout.fillWidth: true
+                        Layout.preferredWidth: 1
+                        implicitHeight: 64
+                        focusPolicy: Qt.NoFocus
+                        onClicked: bench.start(modelData.id)
+                        contentItem: ColumnLayout {
+                            spacing: 2
+                            Label {
+                                text: modelData.name + "  (" + modelData.count + ")"
+                                font.pixelSize: 16
+                                font.bold: true
+                            }
+                            Label {
+                                Layout.fillWidth: true
+                                text: modelData.about
+                                wrapMode: Text.Wrap
+                                maximumLineCount: 2
+                                elide: Text.ElideRight
+                                font.pixelSize: 12
+                                opacity: 0.8
+                            }
+                        }
+                    }
+                }
+            }
+            CheckBox {
+                text: "Record the eye cameras too (about 1.2 GB a minute; they stay on this headset)"
+                checked: bench.images
+                focusPolicy: Qt.NoFocus
+                onToggled: bench.images = checked
+            }
+            Label {
+                Layout.fillWidth: true
+                wrapMode: Text.Wrap
+                color: win.dim
+                font.pixelSize: 13
+                text: bench.status
+            }
+            RowLayout {
+                Layout.topMargin: 4
+                Label {
+                    Layout.fillWidth: true
+                    wrapMode: Text.Wrap
+                    color: win.dim
+                    font.pixelSize: 13
+                    text: "The headset button: press = marked, two presses = back one, hold = end. A left click, "
+                        + "Enter or an index pinch marks too; a right click ends. "
+                        + "Here: Space marks, Backspace goes back one, S skips, Esc ends."
+                }
+                Button { text: "Quit"; focusPolicy: Qt.NoFocus; onClicked: Qt.quit() }
+            }
+        }
+    }
+
+    // --- a run (the dots are on the headset's panel)
+
+    ColumnLayout {
+        anchors.centerIn: parent
+        width: Math.min(parent.width - 60, 760)
+        visible: bench.page === "run"
+        spacing: 14
+        Label {
+            Layout.fillWidth: true
+            wrapMode: Text.Wrap
+            text: bench.progress
+            color: win.ink
+            font.pixelSize: 22
+        }
+        Label {
+            Layout.fillWidth: true
+            wrapMode: Text.Wrap
+            text: bench.message
+            color: win.dim
+            font.pixelSize: 16
+        }
+        Label {
+            Layout.fillWidth: true
+            text: bench.last
+            color: win.ink
+            font.pixelSize: 16
+        }
+        Label {
+            Layout.fillWidth: true
+            wrapMode: Text.Wrap
+            color: win.dim
+            font.pixelSize: 13
+            text: "The headset button: press = marked, two presses = back one, hold = end. A left click, "
+                + "Enter or an index pinch marks too; a right click ends. "
+                + "Here: Space marks, Backspace goes back one, S skips, Esc ends."
+        }
+        RowLayout {
+            spacing: 10
+            Button { text: "Back one"; focusPolicy: Qt.NoFocus; onClicked: bench.undo() }
+            Button { text: "Skip"; focusPolicy: Qt.NoFocus; onClicked: bench.skip() }
+            Button { text: "End"; focusPolicy: Qt.NoFocus; onClicked: bench.stop() }
+        }
+    }
+
+    // --- done
+
+    ColumnLayout {
+        anchors.centerIn: parent
+        width: Math.min(parent.width - 60, 860)
+        visible: bench.page === "done"
+        spacing: 16
+        Label {
+            Layout.fillWidth: true
+            wrapMode: Text.Wrap
+            text: bench.summary
+            color: win.ink
+            font.pixelSize: 18
+            lineHeight: 1.3
+        }
+        RowLayout {
+            spacing: 12
+            Button { text: "Again"; focusPolicy: Qt.NoFocus; onClicked: bench.again() }
+            Button { text: "Scenarios"; focusPolicy: Qt.NoFocus; onClicked: bench.stop() }
+        }
+    }
+}

gaze/bench/benchrec.py

@@ -0,0 +1,348 @@
+"""Gaze bench recording (README.md): everything a run needs to replay gaze techniques later.
+
+A run is one scenario played once. Its folder, under BASE:
+  run.json          the scenario, the panel, the settings, and the gaze service's and our
+                    tracker's status at the start and the end
+  trials.jsonl      a line per target: where it was, when it showed, and when you marked it
+                    (gazebench.py's own record: "click" is the mark, by the button or a key)
+  targets.jsonl     the panel's record of what the dot did, with when each change reached the
+  poses.jsonl       eyes; the headset's pose 250 times a second; SteamVR's IPD setting
+  ipd.jsonl         (ft-gazepanel's log, gaze/panel/ft-gazepanel.cpp)
+  gaze.jsonl        ft-gaze's samples, every source but SteamVR's input action (the one that
+                    calls into vrserver): SteamVR's eyes (mmap1, mmap2, left, right), our
+                    tracker's published gaze (own), the head ray, and where each lands on
+                    Frametop's screens, in pixels, with pixels per degree there (ft-gaze.cpp)
+  own.jsonl         our tracker's finds, a line per eye-camera frame (ft-eyes "finds start"):
+                    pupil, ellipse, glints, and each eye's gaze as it published it
+  lessons.jsonl     the gaze service's lesson log (pointer-lessons.jsonl) over the run: each
+                    nudge the pointer sent, its raw gaze and the true direction
+  eyes-clicks.jsonl our tracker's clicks over the run (clicks.jsonl): what each nudge taught it
+  start/            copies of our tracker's calibration, its dots and its state (the shift
+                    clicks taught it), and the gaze service's calibration, lessons and fusion
+                    (gazed-fusion.json), as they were when the run started, so a replay starts
+                    where the live tracker did
+  images/           both eye cameras, zstd chunks (gaze/rec/eyerec.py), unless images are off
+
+Eye images identify a person: they stay on this headset (and the maintainer's own training
+machines). Nothing here uploads anything.
+"""
+import json
+import os
+import shutil
+import socket
+import subprocess
+import sys
+import threading
+import time
+from collections import deque
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+GAZE = HERE.parent
+
+# ft-gaze: this tree's build (gaze/build.sh), else the installed release's (a worktree with no build)
+FT_GAZES = [Path(p) for p in (os.environ.get("FT_GAZE"), GAZE / "build" / "ft-gaze",
+                              Path.home() / ".local/share/frametop/releases/current/gaze/build/ft-gaze") if p]
+BASE = Path(os.environ.get("FT_BENCH_BASE", Path.home() / ".local/share/frametop/gaze/bench"))
+STATE = Path.home() / ".local/state/frametop/gaze"
+EYES_STATE = STATE / "eyes"
+LESSON_LOG = STATE / "pointer-lessons.jsonl"
+CONF = Path.home() / ".config/frametop.conf"
+CONF_KEYS = ("GAZE_", "POINTER_GAZE", "EYES_MODEL")
+SOURCES = "mmap1,mmap2,left,right,own"
+EYES_SOCK = os.environ.get("FT_BENCH_EYES_SOCKET", "ft_eyes")     # (the tests' stand-ins)
+GAZED_SOCK = os.environ.get("FT_BENCH_GAZED_SOCKET", "ft_gazed")
+RECENT = 3.0        # s of parsed samples kept for the app's quick scores
+RESTART = 3.0       # s before ft-gaze is started again after it stopped (SteamVR restarted)
+KEEPALIVE = 2.0     # s between checks that our tracker is still logging finds
+LOG_SLACK = 3.0     # s either side of the run when copying the lesson logs
+
+
+def raw_now():
+    return time.clock_gettime(time.CLOCK_MONOTONIC_RAW)
+
+
+class Sock:
+    """A datagram client for one of the gaze services' abstract sockets (@ft_eyes, @ft_gazed)."""
+
+    def __init__(self, name):
+        self.name = "\0" + name
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC)
+        self.sock.bind("")
+
+    def ask(self, text, timeout=0.5):
+        """Send, and the reply (text) or None."""
+        self.sock.setblocking(False)
+        while True:  # drop late replies to earlier questions
+            try:
+                self.sock.recv(65536)
+            except OSError:
+                break
+        try:
+            self.sock.sendto(text.encode(), self.name)
+            self.sock.settimeout(timeout)
+            return self.sock.recv(1 << 20).decode(errors="replace")
+        except OSError:
+            return None
+
+    def json(self, text, timeout=0.5):
+        r = self.ask(text, timeout)
+        try:
+            return json.loads(r) if r else None
+        except ValueError:
+            return None
+
+    def close(self):
+        self.sock.close()
+
+
+class GazeStream:
+    """ft-gaze, for the app's life (it takes a moment to connect to SteamVR and the screens), and
+    again RESTART seconds after it stops: it quits when SteamVR does. Its lines go to the current
+    run's gaze.jsonl while one records, and the last RECENT seconds are kept parsed for the app's
+    quick scores and status."""
+
+    def __init__(self, prog=None, log=print):
+        self.prog = Path(prog) if prog else next((p for p in FT_GAZES if p.exists()), FT_GAZES[0])
+        self.log = log
+        self.proc = None
+        self.out = None
+        self.lock = threading.Lock()
+        self.recent = deque()
+        self.lines = 0
+        self.error = None
+        self.restarts = 0
+        self.stopping = threading.Event()
+        self.reader = None
+
+    def start(self):
+        if not self.prog.exists():
+            self.error = f"{self.prog} isn't built (gaze/build.sh)"
+            return self
+        self.reader = threading.Thread(target=self._run, name="ft-gaze", daemon=True)
+        self.reader.start()
+        return self
+
+    def _run(self):
+        while True:
+            with self.lock:
+                if self.stopping.is_set():
+                    return
+                self.proc = subprocess.Popen([str(self.prog), "--watch-stdin", "--sources", SOURCES],
+                                             stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
+            before = self.lines
+            self._read()
+            code = self.proc.wait()
+            self.proc.stdout.close()
+            try:
+                self.proc.stdin.close()
+            except OSError:
+                pass
+            if self.stopping.is_set():
+                return
+            self.error = f"ft-gaze stopped (exit {code}); starting it again"
+            if self.lines > before or not self.restarts:   # not every few seconds while SteamVR is down
+                self.log(self.error)
+            self.restarts += 1
+            self.stopping.wait(RESTART)
+
+    def _read(self):
+        for line in self.proc.stdout:
+            with self.lock:
+                if self.out:
+                    self.out.write(line)
+            self.lines += 1
+            try:
+                s = json.loads(line)
+            except ValueError:
+                continue
+            self.error = None
+            with self.lock:
+                self.recent.append(s)
+                while self.recent and self.recent[0].get("t", 0) < s.get("t", 0) - RECENT:
+                    self.recent.popleft()
+
+    def to(self, path):
+        """Write the samples to path from now on (None: stop writing)."""
+        with self.lock:
+            if self.out:
+                self.out.close()
+            self.out = open(path, "ab") if path else None
+
+    def window(self, t0, t1):
+        with self.lock:
+            return [s for s in self.recent if t0 <= s.get("t", 0) <= t1]
+
+    def latest(self):
+        with self.lock:
+            return self.recent[-1] if self.recent else None
+
+    def stop(self):
+        self.to(None)
+        with self.lock:
+            self.stopping.set()
+            proc = self.proc
+        if proc:
+            try:
+                proc.stdin.close()
+            except OSError:
+                pass
+            try:
+                proc.wait(3)
+            except subprocess.TimeoutExpired:
+                proc.kill()
+        if self.reader:
+            self.reader.join(2)
+
+
+def conf_lines():
+    try:
+        return [l for l in CONF.read_text().splitlines() if l.startswith(CONF_KEYS)]
+    except OSError:
+        return []
+
+
+def eyes_state_dir(status):
+    """Our tracker's state folder for the model it runs (ft-eyes: net-NAME/ for a net)."""
+    model = (status or {}).get("model")
+    if model:
+        try:
+            name = json.loads((Path(model) / "model.json").read_text())["name"]
+        except (OSError, ValueError, KeyError):
+            name = Path(model).name
+        return EYES_STATE / ("net-" + str(name))
+    return EYES_STATE
+
+
+def last_time(path, before):
+    """The latest "time" (wall clock) in a JSON lines log before `before`, or None: the last nudge
+    before a run, for compare.py's time since the last nudge."""
+    last = None
+    try:
+        with open(path) as f:
+            for line in f:
+                try:
+                    tm = json.loads(line).get("time")
+                except ValueError:
+                    continue
+                if tm is not None and tm < before and "refused" not in line:
+                    last = tm if last is None else max(last, tm)
+    except OSError:
+        pass
+    return last
+
+
+def copy_rows(src, dst, t0, t1):
+    """The JSON lines of src whose "time" (wall clock) is within [t0, t1], appended to dst."""
+    n = 0
+    try:
+        with open(src) as f, open(dst, "a") as out:
+            for line in f:
+                try:
+                    tm = json.loads(line).get("time", 0)
+                except ValueError:
+                    continue
+                if t0 <= tm <= t1:
+                    out.write(line if line.endswith("\n") else line + "\n")
+                    n += 1
+    except OSError:
+        pass
+    return n
+
+
+class Run:
+    """One scenario played once: its folder, and what records into it (see the top)."""
+
+    def __init__(self, scenario, gaze, images=True, base=BASE, log=print, info=None):
+        self.scenario, self.gaze, self.images, self.log = scenario, gaze, images, log
+        stamp = time.strftime("%Y%m%d-%H%M%S")
+        self.dir = Path(base) / f"{stamp}-{scenario['id']}"
+        self.info = dict(info or {})
+        self.eyes = Sock(EYES_SOCK)
+        self.gazed = Sock(GAZED_SOCK)
+        self.writer = None
+        self.finds_on = False
+        self.checked = 0.0
+        self.trials = None
+
+    def status(self):
+        return {"eyes": self.eyes.json("status"), "gazed": self.gazed.json("status"), "t_raw": raw_now(),
+                "time": time.time()}
+
+    def start(self):
+        self.dir.mkdir(parents=True)
+        (self.dir / "start").mkdir()
+        st = self.status()
+        estate = eyes_state_dir(st["eyes"])
+        for name in ("calibration.json", "calibration-dots.jsonl", "state.json"):
+            if (estate / name).exists():
+                shutil.copy2(estate / name, self.dir / "start" / ("eyes-" + name))
+        for name in ("calibration.json", "pointer-lessons.json", "fusion.json"):
+            if (STATE / name).exists():
+                shutil.copy2(STATE / name, self.dir / "start" / ("gazed-" + name))
+        self.info.update(scenario=self.scenario, start=st, conf=conf_lines(), eyes_state=str(estate),
+                         images=self.images, sources=SOURCES, last_lesson=last_time(LESSON_LOG, st["time"]))
+        self.write_info()
+        self.trials = open(self.dir / "trials.jsonl", "a", buffering=1)
+        self.lines0 = self.gaze.lines
+        self.gaze.to(self.dir / "gaze.jsonl")
+        self.arm_finds()
+        if self.images:
+            try:
+                if str(GAZE / "rec") not in sys.path:
+                    sys.path.insert(0, str(GAZE / "rec"))
+                import eyerec
+                self.writer = eyerec.Writer(str(self.dir / "images"), log=self.log).start()
+                self.writer.record(True)
+            except Exception as e:  # no zstd, no share: record the rest
+                self.log(f"images: {e}")
+                self.writer = None
+                self.info["images_error"] = str(e)
+        return self
+
+    def arm_finds(self):
+        r = self.eyes.ask("finds start " + str(self.dir / "own.jsonl"))
+        self.finds_on = bool(r and r.startswith("ok"))
+        if not self.finds_on:
+            self.log(f"our tracker isn't logging finds ({r or 'no reply: is it running?'})")
+        return self.finds_on
+
+    def keepalive(self):
+        """Call now and then: our tracker may have restarted (or started), and dropped the log."""
+        if time.monotonic() - self.checked < KEEPALIVE:
+            return
+        self.checked = time.monotonic()
+        st = self.eyes.json("status")
+        if st is None:
+            self.finds_on = False
+        elif not self.finds_on:
+            self.arm_finds()
+
+    def trial(self, rec):
+        if self.trials:
+            self.trials.write(json.dumps(rec) + "\n")
+
+    def write_info(self):
+        tmp = self.dir / "run.json.tmp"
+        tmp.write_text(json.dumps(self.info, indent=1))
+        tmp.replace(self.dir / "run.json")
+
+    def stop(self, summary=None):
+        rows = self.eyes.ask("finds stop")
+        self.gaze.to(None)
+        if self.trials:
+            self.trials.close()
+            self.trials = None
+        if self.writer:
+            self.info["images_stats"] = self.writer.stop()
+        time.sleep(0.5)  # a nudge's lesson reaches the logs just after its click
+        st = self.status()
+        t0, t1 = self.info["start"]["time"] - LOG_SLACK, st["time"] + LOG_SLACK
+        n_lessons = copy_rows(LESSON_LOG, self.dir / "lessons.jsonl", t0, t1)
+        n_clicks = copy_rows(Path(self.info["eyes_state"]) / "clicks.jsonl", self.dir / "eyes-clicks.jsonl", t0, t1)
+        self.info.update(end=st, finds_rows=rows, lessons=n_lessons, eyes_clicks=n_clicks,
+                         gaze_lines=self.gaze.lines - self.lines0, summary=summary)
+        self.write_info()
+        self.eyes.close()
+        self.gazed.close()
+        return self.info

gaze/bench/compare.py

@@ -0,0 +1,670 @@
+#!/usr/bin/env python3
+"""Compare gaze techniques on gaze bench runs (README.md): how far each was from where you looked,
+at every click.
+
+  compare.py RUN [RUN...] [--per-click] [--json OUT] [--only a,b]
+
+RUN is a run's folder (benchrec.py), or a folder of them (all runs in it). A click is a mark:
+you looked at the target and pressed. Where you looked is the target, a head-relative direction
+on runs with the headset's panel (run.json "display": "panel", from 2026-10-17). The runs before
+showed targets on a Frametop screen: there it's the target's middle, or the click if it landed off
+the target, turned into a direction through ft-gaze's screen hits just before the click (the hit
+nearest that point, and its pixels per degree there: ft-gaze.cpp's HIT "j"). Each technique's
+gaze is its median over the same LOOK window (run.json), and its error the angle between the two.
+
+Techniques: what ran live, from gaze.jsonl:
+  own-live        our tracker's published gaze (ft-eyes, with its calibration and the shifts the
+                  nudges taught it)
+  steamvr         SteamVR's combined gaze (ft-gaze's mmap1, its own calibration)
+  steamvr-eyes    the mean of SteamVR's two eyes (left, right: set 2, no calibration of ours)
+  pointer         where the pointer was: a nudge's raw gaze (lessons.jsonl), else the click
+                  (screen runs only: on the panel there's no pointer)
+  fuse            70 % our tracker, 30 % SteamVR's eyes' mean (both live)
+  steamvr+clicks  SteamVR's, less the median error of the last SHIFT_KEEP clicks before
+With the fusion the run started with (start/gazed-fusion.json: gazecal.Fusion, fitted at our
+tracker's last calibration, or by --fit-fusion), what ft-gazed's GAZE_FUSION=on sends:
+  steamvr-affine  SteamVR's eyes' mean with its affine correction
+  steamvr-affine+clicks   ...plus the median of what the run's clicks before said was still off
+  fusion          per sample: ours (its eyes, weighted as ft-gazed does) mixed half and half with
+                  steamvr-affine where both our eyes are found, steamvr-affine alone where we
+                  have one, ours alone where SteamVR has no gaze; the look's median
+The same, fitted here on the other clicks (all runs given), for runs without one:
+  steamvr-affine-loo, fusion-loo     on every other click
+  ours-else-steam-loo                fusion-loo, but ours alone where both our eyes are found
+  steamvr-affine-xrun(+clicks), fusion-xrun(+clicks)   on the other runs' clicks only: a
+                  calibration from another sitting
+  fusion-loo:REPLAY   fusion-loo with our eyes from that replay (FUSE_REPLAYS, below), not
+                  live: a guard's worth with SteamVR covering the eyes it refuses
+Replays of our tracker on its finds (own.jsonl), from the calibration and the shift it had when
+the run started (start/), with ft-eyes' own rules (eyes_model.py), and nudges teaching it when
+they did live (lessons.jsonl):
+  own-replay      the live rules: check this against own-live (the "replay vs live" line)
+  no-jump         no glint jump: only clicks move the shift
+  both-jump       a glint jump counts only while both eyes see one
+  edge-guard      nudges past the calibration's dots (by EDGE_MARGIN) teach nothing
+  every-click     every click teaches it, not only the nudges
+  bridge          one eye lost: the other, plus how far it usually reads from the two together
+  guard-both      eyes_pupil.SizeGuard "both" over the finds (refused ones count as lost)
+  guard-long      SizeGuard "long"
+  no-guard        the finds the live SizeGuard refused (own.jsonl "guard": true) back in
+Each click is scored before anything it teaches. The breakdowns: by scenario, by where the target
+was (edge, corner or inside), by how far from straight ahead you looked, and by the time since
+the last nudge (run.json's last_lesson for the first).
+
+Needs numpy and (for the glint pairs) OpenCV: the tracker's Python, e.g.
+  scripts/in-box /opt/frametop/venv/bin/python gaze/bench/compare.py ~/.local/share/frametop/gaze/bench
+
+--fit-fusion OUT fits a gazecal.Fusion on all the clicks given (SteamVR's eyes' mean at each, and
+the target) and writes it to OUT, e.g. ~/.local/state/frametop/gaze/fusion.json for ft-gazed.
+"""
+import argparse
+import bisect
+import json
+import math
+import statistics
+import sys
+import time
+from collections import deque
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve().parent
+sys.path.insert(0, str(HERE.parent / "tracker"))
+sys.path.insert(0, str(HERE.parent))
+import eyes_model  # noqa: E402
+from gazecal import EyeWeights, Fusion  # noqa: E402
+
+LOOK = (0.35, 0.05)       # s before a click: as gazebench.py, for runs that don't say
+CLICK_BEFORE = 0.3        # ft-eyes: a click's frames are the 300 ms before it
+CLICK_MIN = 5             # ...at least this many
+FRESH = 0.03              # ft-eyes: an eye counts toward the output this long after its frame
+GAP = 3.0                 # ft-eyes: frames stopping this long is the headset off (a re-seat)
+HISTORY = 12.0            # ft-eyes: seconds of pupil positions kept per eye
+LESSON_MATCH = 1.5        # s between a click and its nudge's lesson
+EDGE_MARGIN = 2.0         # degrees past the calibration's outermost dots for edge-guard
+BRIDGE_TAU = 2.0          # s: bridge's average of each eye against the two together
+FUSE = 0.7                # fuse: our tracker's share
+SHIFT_KEEP = eyes_model.SHIFT_KEEP
+LIVE = ("own-live", "steamvr", "steamvr-eyes", "pointer", "fuse", "steamvr+clicks",
+        "steamvr-affine", "steamvr-affine+clicks", "fusion")
+CROSS = ("steamvr-affine-loo", "fusion-loo", "ours-else-steam-loo",
+         "steamvr-affine-xrun", "steamvr-affine-xrun+clicks", "fusion-xrun", "fusion-xrun+clicks")
+FUSE_REPLAYS = ("own-replay", "guard-both", "guard-long")  # fusion-loo:NAME, with that replay's eyes
+REPLAYS = {  # name: Replay options
+    "own-replay": {},
+    "no-jump": {"jump": None},
+    "both-jump": {"jump": "both"},
+    "edge-guard": {"guard": True},
+    "every-click": {"teach": "every"},
+    "bridge": {"bridge": True},
+    "guard-both": {"size": "both"},
+    "guard-long": {"size": "long"},
+    "no-guard": {"size": "none"},
+}
+ECC_BINS = ((0, 10, "<10°"), (10, 20, "10-20°"), (20, 30, "20-30°"), (30, 999, "30°+"))
+SINCE_BINS = ((0, 30, "<30 s"), (30, 120, "30-120 s"), (120, math.inf, "120 s+"))
+
+
+def jsonl(path):
+    try:
+        with open(path) as f:
+            for line in f:
+                try:
+                    yield json.loads(line)
+                except ValueError:
+                    continue
+    except FileNotFoundError:
+        return
+
+
+def unit(hy, hp):
+    """ft-gaze's head-relative yaw (+left), pitch (+up), degrees, as a unit vector (-Z ahead)."""
+    y, p = math.radians(hy), math.radians(hp)
+    return np.array([-math.sin(y) * math.cos(p), math.sin(p), -math.cos(y) * math.cos(p)])
+
+
+def angle(a, b):
+    if a is None or b is None:
+        return None
+    return math.degrees(math.acos(max(-1.0, min(1.0, float(unit(*a) @ unit(*b))))))
+
+
+def median2(rows):
+    rows = [r for r in rows if r is not None]
+    if not rows:
+        return None
+    return (statistics.median(r[0] for r in rows), statistics.median(r[1] for r in rows))
+
+
+def rays(s):
+    """A sample's rays with screen hits: (yaw, pitch, hit), head-relative."""
+    head = s.get("head") or {}
+    if head.get("hit"):
+        yield 0.0, 0.0, head["hit"]
+    for name, src in (s.get("src") or {}).items():
+        if src.get("hit") and "hy" in src:
+            yield src["hy"], src["hp"], src["hit"]
+        for e, h in zip(src.get("eyes") or (), src.get("ehit") or ()):
+            if e and h:
+                yield e[0], e[1], h
+
+
+class Run:
+    def __init__(self, d):
+        self.dir = Path(d)
+        self.info = json.loads((self.dir / "run.json").read_text())
+        st = self.info["start"]
+        self.wall = st["time"] - st["t_raw"]          # wall clock = raw + this
+        self.look = tuple(self.info.get("look") or LOOK)
+        self.scenario = self.info["scenario"]["id"]
+        self.samples = sorted(jsonl(self.dir / "gaze.jsonl"), key=lambda s: s.get("t", 0))
+        self.ts = [s.get("t", 0) for s in self.samples]
+        self.notes = []
+        self.replay_eyes = {}   # replay name: (times, [(left, right) eyes' gazes]) (replays())
+        self.lessons = sorted((r for r in jsonl(self.dir / "lessons.jsonl") if "refused" not in r),
+                              key=lambda r: r["time"])
+        self.panel = self.info.get("display") == "panel"
+        if self.panel:   # the targets' directions are in trials.jsonl
+            self.screen, self.sx, self.sy = None, 1.0, 1.0
+            self.trials = self.load_trials()
+            return
+        votes = {}
+        for s in self.samples:
+            hit = (s.get("head") or {}).get("hit")
+            if hit:
+                votes[hit["s"]] = votes.get(hit["s"], 0) + 1
+        self.screen = max(votes, key=votes.get) if votes else None
+        # ft-screens' pixels per window pixel: its size over the window's (gazebench.Bench.scale)
+        win, size = self.info.get("screen") or {}, (self.info.get("screens") or {}).get(str(self.screen))
+        if size and win.get("w") and win.get("h"):
+            self.sx, self.sy = size[0] / win["w"], size[1] / win["h"]
+        else:
+            self.sx = self.sy = win.get("dpr") or 1.0
+            self.notes.append("no screen sizes in run.json (a run before 2026-10-17): window pixels at Qt's "
+                              "pixel ratio, wrong where it isn't the desktop's scale")
+        self.trials = self.load_trials()
+
+    def load_trials(self):
+        """The trials that stand: the last click on each target, unless taken back or skipped. On a
+        screen run, each with the nudge's lesson that came with its click, if there was one."""
+        by_i = {}
+        for r in jsonl(self.dir / "trials.jsonl"):
+            if r.get("kind") == "trial":
+                by_i[r["i"]] = r
+            elif r.get("kind") == "undo":
+                by_i.pop(r["i"], None)
+        out = []
+        used = set()
+        for r in sorted(by_i.values(), key=lambda r: r["click"]["t"]):
+            if r.get("skipped") or "truth" not in r:
+                continue
+            r = dict(r)
+            if self.panel:
+                out.append(r)
+                continue
+            wall = r["click"]["t"] + self.wall
+            best = None
+            for k, ls in enumerate(self.lessons):
+                if k not in used and abs(ls["time"] - wall) <= LESSON_MATCH and (
+                        best is None or abs(ls["time"] - wall) < abs(self.lessons[best]["time"] - wall)):
+                    best = k
+            if best is not None:
+                used.add(best)
+                r["lesson"] = self.lessons[best]
+            out.append(r)
+        return out
+
+    def window(self, t0, t1):
+        return self.samples[bisect.bisect_left(self.ts, t0):bisect.bisect_right(self.ts, t1)]
+
+    def look_window(self, r):
+        """The click's look: before the press for a nudge (the lesson's look_t), else before the click."""
+        ls = r.get("lesson")
+        if ls and ls.get("look_t") is not None:
+            return ls["look_t"] - (self.look[0] - self.look[1]), ls["look_t"]
+        t = r["click"]["t"]
+        return t - self.look[0], t - self.look[1]
+
+    def direction(self, samples, px, py):
+        """Head-relative (yaw, pitch) of a point on the bench's screen (ft-screens' pixels): per
+        sample, from the hit nearest it, through that hit's pixels per degree; the median."""
+        out = []
+        for s in samples:
+            best = None
+            for hy, hp, h in rays(s):
+                if h.get("s") != self.screen or not h.get("j"):
+                    continue
+                d = math.hypot(h["x"] - px, h["y"] - py)
+                if best is None or d < best[0]:
+                    best = (d, hy, hp, h)
+            if best is None:
+                continue
+            _, hy, hp, h = best
+            j = h["j"]  # [dx/dhy, dy/dhy, dx/dhp, dy/dhp]
+            det = j[0] * j[3] - j[2] * j[1]
+            if abs(det) < 1e-9:
+                continue
+            ex, ey = px - h["x"], py - h["y"]
+            out.append((hy + (j[3] * ex - j[2] * ey) / det, hp + (-j[1] * ex + j[0] * ey) / det))
+        return median2(out)
+
+    def truth(self, r):
+        if "yaw" in r["truth"]:   # the panel's target
+            return r["truth"]["yaw"], r["truth"]["pitch"]
+        t0, t1 = self.look_window(r)
+        return self.direction(self.window(t0, t1), r["truth"]["x"] * self.sx, r["truth"]["y"] * self.sy)
+
+    def source(self, name, t0, t1):
+        rows = []
+        for s in self.window(t0, t1):
+            src = (s.get("src") or {}).get(name) or {}
+            if "hy" in src:
+                rows.append((src["hy"], src["hp"]))
+        return median2(rows)
+
+    def frames(self, size=None):
+        """Our tracker's finds, in time order: (t, cam, x, y, glint pair midpoint or None, gaze).
+        size: None, the live rules (the finds its SizeGuard refused are out); "none", those back
+        in; or a SizeGuard rule, run here over all of them instead."""
+        import eyes_pupil  # OpenCV
+        rows = sorted((r for r in jsonl(self.dir / "own.jsonl") if not r.get("lost") and "x" in r),
+                      key=lambda r: (r["t"], r["cam"]))
+        guard = eyes_pupil.SizeGuard(size) if size not in (None, "none") else None
+        out = []
+        for r in rows:
+            if size is None and r.get("guard"):
+                continue
+            if guard and not guard.keep(r["cam"], r["t"], r["a"]):
+                continue
+            pair = eyes_pupil.glint_pair({"x": r["x"], "y": r["y"], "glints": r.get("glints") or []})
+            out.append((r["t"], r["cam"], r["x"], r["y"], eyes_model.pair_mid(pair) if pair else None, r.get("gaze")))
+        return out
+
+    def fusion_inputs(self, t0, t1, replay=None):
+        """Per sample over [t0, t1]: (our eyes [left, right], SteamVR's left, right), for fusions.
+        Our eyes as published, or as a replay had them then (its newest output, if FRESH)."""
+        out = []
+        if replay:
+            rt, re = self.replay_eyes[replay]
+        for s in self.window(t0, t1):
+            src = s.get("src") or {}
+            if replay:
+                k = bisect.bisect_right(rt, s.get("t", 0)) - 1
+                eyes = list(re[k]) if k >= 0 and s.get("t", 0) - rt[k] < FRESH else [None, None]
+            else:
+                eyes = [tuple(e) if e else None for e in ((src.get("own") or {}).get("eyes") or [None, None])]
+            lr = [(src[n]["hy"], src[n]["hp"]) if "hy" in (src.get(n) or {}) else None for n in ("left", "right")]
+            out.append((eyes, lr[0], lr[1]))
+        return out
+
+
+class Replay:
+    """ft-eyes' classic tracker (Eye.feed, Tracker.command "click", the output loop) over a run's
+    finds, with variations."""
+
+    def __init__(self, run, frames, cal, shifts, status, clicks, jump="each", guard=False, bridge=False, box=None):
+        self.run, self.frames, self.cal = run, frames, cal
+        self.clicks = sorted(clicks, key=lambda c: c[0])  # (apply at, look t, (yaw, pitch))
+        self.jump, self.guard, self.bridge, self.box = jump, guard, bridge, box
+        self.shift = {}
+        for e in (0, 1):
+            sh = eyes_model.Shift.from_json(shifts[e]) if e in shifts else eyes_model.Shift()
+            st = ((status or {}).get("eyes") or {}).get(("right", "left")[e]) or {}
+            sh.reseated = bool(st.get("reseat", e not in shifts))
+            if st.get("jump") and jump is not None:
+                sh.jump = np.asarray(st.get("jump_px", (0, 0)), float)
+            self.shift[e] = sh
+        self.slip = {e: eyes_model.SlipTracker(cal, e, window=eyes_model.JUMP_WINDOW) for e in (0, 1)}
+        self.history = {0: deque(), 1: deque()}
+        self.gaze = {0: None, 1: None}
+        self.last_t = {0: None, 1: None}
+        self.off = {0: None, 1: None}
+        self.taught = self.skipped = 0
+        self.out_e = []   # with each output: the eyes' gazes in it, (left, right)
+
+    def value(self, e):
+        sh = self.shift[e]
+        if self.jump == "both" and not all(np.any(self.shift[o].jump) for o in (0, 1)):
+            return sh.base
+        return sh.value
+
+    def click(self, t, truth):
+        if self.guard and self.box and not (self.box[0] - EDGE_MARGIN <= truth[0] <= self.box[1] + EDGE_MARGIN
+                                            and self.box[2] - EDGE_MARGIN <= truth[1] <= self.box[3] + EDGE_MARGIN):
+            self.skipped += 1
+            return
+        both = all(np.any(self.shift[o].jump) for o in (0, 1))
+        for e in (0, 1):
+            rows = [h for h in self.history[e] if t - CLICK_BEFORE <= h[0] <= t]
+            if len(rows) < CLICK_MIN or not self.cal.has("where", e):
+                continue
+            pupil = np.median(np.array([h[1:3] for h in rows]), axis=0)
+            d = self.cal.click_shift(e, pupil, truth)
+            sh = self.shift[e]
+            # A jump in force (both-jump: in both eyes) means the headset moved: the older clicks are stale
+            if (both if self.jump == "both" else np.any(sh.jump)) or sh.reseated:
+                sh.meas, sh.reseated = [], False
+            sh.meas = (sh.meas + [np.asarray(d, float)])[-SHIFT_KEEP:]
+            sh.base = np.median(sh.meas, axis=0)
+            sh.jump, sh.held = np.zeros(2), None
+            if self.slip[e].value is not None:
+                sh.ref = np.asarray(self.slip[e].value, float)
+        self.taught += 1
+
+    def __call__(self):
+        """The output, (times, yaw, pitch) arrays, one per frame that changed it."""
+        out_t, out_g = [], []
+        ci = 0
+        last_out = None
+        for t, e, x, y, mid, _ in self.frames:
+            while ci < len(self.clicks) and self.clicks[ci][0] <= t:
+                self.click(*self.clicks[ci][1:])
+                ci += 1
+            if self.last_t[e] is not None and t - self.last_t[e] > GAP:
+                self.shift[e].reseat()
+            self.last_t[e] = t
+            h = self.history[e]
+            h.append((t, x, y))
+            while h[0][0] < t - HISTORY:
+                h.popleft()
+            if not self.cal.has("pupil", e):
+                continue
+            if mid is not None:
+                self.slip[e].add(t, (x, y), mid)
+            if self.jump is not None:
+                self.shift[e].glint(self.slip[e].get(t), t)
+            g = self.cal.gaze(e, x, y, self.value(e))
+            self.gaze[e] = (t, float(g[0]), float(g[1]))
+            latest = max(v[0] for v in self.gaze.values() if v)
+            use = {k: v[1:] for k, v in self.gaze.items() if v and latest - v[0] < FRESH}
+            out = self.cal.combine(use)
+            if self.bridge:
+                if len(use) == 2:
+                    a = 1.0 if last_out is None else 1 - math.exp(-(t - last_out) / BRIDGE_TAU)
+                    for k, v in use.items():
+                        d = np.asarray(out) - np.asarray(v)
+                        self.off[k] = d if self.off[k] is None else self.off[k] + a * (d - self.off[k])
+                elif len(use) == 1:
+                    (k, v), = use.items()
+                    if self.off[k] is not None:
+                        out = np.asarray(v) + self.off[k]
+            last_out = t
+            out_t.append(latest)
+            out_g.append((float(out[0]), float(out[1])))
+            self.out_e.append(tuple(None if k not in use else (float(use[k][0]), float(use[k][1])) for k in (1, 0)))
+        return out_t, out_g
+
+
+def calibration_box(run, cal):
+    """The calibration's dots' extent (yaw min, max, pitch min, max), from calibration-dots.jsonl."""
+    made = cal.info.get("made")
+    pts = [r["truth"] for r in jsonl(run.dir / "start" / "eyes-calibration-dots.jsonl") if r.get("made") == made]
+    if len(pts) < 4:
+        return (-20.0, 20.0, -20.0, 20.0)
+    return (min(p[0] for p in pts), max(p[0] for p in pts), min(p[1] for p in pts), max(p[1] for p in pts))
+
+
+def replays(run, truths, only):
+    """{name: function(t0, t1) -> (yaw, pitch) or None} for the replays, or {} with a note."""
+    want = [n for n in REPLAYS if not only or n in only]
+    if not want:
+        return {}
+    status = (run.info["start"] or {}).get("eyes")
+    if status is None:
+        run.notes.append("our tracker wasn't running at the start: no replays")
+        return {}
+    if status.get("feature", "pupil") != "pupil":
+        run.notes.append(f"our tracker ran {status.get('feature')}: replays are of the classic tracker only")
+        return {}
+    calp = run.dir / "start" / "eyes-calibration.json"
+    if not calp.exists():
+        run.notes.append("no calibration in start/: no replays")
+        return {}
+    cal = eyes_model.Calibration.load(calp)
+    shifts = {}
+    try:
+        st = json.loads((run.dir / "start" / "eyes-state.json").read_text())
+        if st.get("calibration") == cal.info.get("made"):
+            shifts = {int(k): v for k, v in st.get("shift", {}).items()}
+    except (OSError, ValueError):
+        pass
+    frames = {}
+    if not run.frames():
+        run.notes.append("no finds in own.jsonl: no replays")
+        return {}
+    nudges = [(ls["time"] - run.wall, ls["look_t"], tuple(ls["true"])) for ls in run.lessons
+              if ls.get("kind") == "own" and ls.get("look_t") is not None]
+    every = [(r["click"]["t"], r["click"]["t"] - run.look[1], truths[id(r)]) for r in run.trials
+             if truths.get(id(r)) is not None]
+    box = calibration_box(run, cal)
+    out = {}
+    for name in want:
+        opt = dict(REPLAYS[name])
+        teach = every if opt.pop("teach", None) == "every" else nudges
+        size = opt.pop("size", None)
+        if size not in frames:
+            frames[size] = run.frames(size)
+        replay = Replay(run, frames[size], cal, shifts, status, teach, box=box, **opt)
+        t, g = replay()
+        run.replay_eyes[name] = (t, replay.out_e)
+
+        def look(t0, t1, t=t, g=g):
+            return median2(g[bisect.bisect_left(t, t0):bisect.bisect_right(t, t1)])
+        out[name] = look
+    return out
+
+
+def fuse_look(inputs, weights, fusion, share=None, clicks=False):
+    """Fusion.combine per sample (fusion_inputs), ours combined as ft-gazed does; the look's median."""
+    out = []
+    for eyes, left, right in inputs:
+        own = weights.combine(eyes)
+        out.append(fusion.combine(own, sum(e is not None for e in eyes), fusion.steam_gaze(left, right, clicks), share))
+    return median2(out)
+
+
+def own_weights(run):
+    """ft-gazed's weights for our tracker's eyes when the run started (its lessons' misses)."""
+    try:
+        misses = json.loads((run.dir / "start" / "gazed-pointer-lessons.json").read_text())["misses"]["own"]
+    except (OSError, ValueError, KeyError, TypeError):
+        misses = None
+    return EyeWeights(((run.info["start"] or {}).get("gazed") or {}).get("eye_bias", "auto"), misses)
+
+
+def score(run, only=None):
+    """A record per click: where it was, and each technique's error (degrees, or None)."""
+    truths = {id(r): run.truth(r) for r in run.trials}
+    rep = replays(run, truths, only)
+    last_lesson = run.info.get("last_lesson")
+    weights = own_weights(run)
+    fusion = Fusion.load(run.dir / "start" / "gazed-fusion.json")
+    if fusion and not fusion.ready():
+        fusion = None
+    if fusion:
+        fusion.wear = run.dir.name
+    recs, steam_offsets = [], []
+    for r in run.trials:
+        truth = truths[id(r)]
+        t0, t1 = run.look_window(r)
+        own, steam = run.source("own", t0, t1), run.source("mmap1", t0, t1)
+        left, right = run.source("left", t0, t1), run.source("right", t0, t1)
+        mean = Fusion.mean(left, right)
+        inputs = run.fusion_inputs(t0, t1)
+        g = {"own-live": own, "steamvr": steam, "steamvr-eyes": mean}
+        ls = r.get("lesson")
+        if ls:
+            g["pointer"] = tuple(ls["raw"])
+        elif not run.panel:
+            c = r["click"]
+            g["pointer"] = run.direction(run.window(t0, t1), c["x"] * run.sx, c["y"] * run.sy)
+        g["fuse"] = (tuple(FUSE * a + (1 - FUSE) * b for a, b in zip(own, mean)) if own and mean else None)
+        if fusion:
+            g["steamvr-affine"] = fusion.steam_gaze(left, right, clicks=False)
+            g["steamvr-affine+clicks"] = fusion.steam_gaze(left, right)
+            g["fusion"] = fuse_look(inputs, weights, fusion)
+            if truth:
+                fusion.click(left, right, *truth)
+        if steam and steam_offsets:
+            keep = steam_offsets[-SHIFT_KEEP:]
+            g["steamvr+clicks"] = (steam[0] + statistics.median(o[0] for o in keep),
+                                   steam[1] + statistics.median(o[1] for o in keep))
+        else:
+            g["steamvr+clicks"] = steam
+        if steam and truth:
+            steam_offsets.append((truth[0] - steam[0], truth[1] - steam[1]))
+        for name, look in rep.items():
+            g[name] = look(t0, t1)
+        wall = r["click"]["t"] + run.wall
+        before = [ls_["time"] for ls_ in run.lessons if ls_["time"] < wall - LESSON_MATCH]
+        last = before[-1] if before else last_lesson
+        u, v = r["u"], r["v"]
+        edges = sum(1 for w in (u, v) if w in (0.0, 1.0))
+        rec = {"run": run.dir.name, "scenario": run.scenario, "i": r["i"], "u": u, "v": v,
+               "where": ("corner" if edges == 2 else "edge" if edges else "inside"),
+               "truth": truth, "ecc": math.hypot(*truth) if truth else None,
+               "since": None if last is None else wall - last, "nudged": bool(ls),
+               "err": {k: angle(val, truth) for k, val in g.items() if not only or k in only},
+               "_fz": {"left": left, "right": right, "mean": mean, "inputs": inputs, "weights": weights,
+                       "replays": {n: run.fusion_inputs(t0, t1, n) for n in FUSE_REPLAYS if n in run.replay_eyes}}}
+        if ls:
+            c = r["click"]
+            cd = run.direction(run.window(t0, t1), c["x"] * run.sx, c["y"] * run.sy)
+            rec["check"] = angle(cd, tuple(ls["true"]))
+        if g.get("own-replay") and own:
+            rec["replay_vs_live"] = angle(g["own-replay"], own)
+        recs.append(rec)
+    return recs
+
+
+def cross(recs, only=None):
+    """CROSS: SteamVR's affine and the fusion fitted on the other clicks (every other one: -loo;
+    the other runs' only: -xrun), each record's errors with them."""
+    want = [n for n in CROSS if not only or n in only]
+    have = [r for r in recs if r["truth"] and r["_fz"]["mean"]]
+    if not want or not have:
+        return
+    pts = {id(r): (*r["_fz"]["mean"], *r["truth"]) for r in have}
+    xrun = {}
+    for name in dict.fromkeys(r["run"] for r in recs):
+        xrun[name] = Fusion()
+        xrun[name].fit([pts[id(r)] for r in have if r["run"] != name])
+        xrun[name].wear = name
+    for r in recs:
+        fz, g = r["_fz"], {}
+        loo = Fusion()
+        loo.fit([pts[id(q)] for q in have if q is not r])
+        if loo.ready():
+            g["steamvr-affine-loo"] = loo.steam_gaze(fz["left"], fz["right"])
+            g["fusion-loo"] = fuse_look(fz["inputs"], fz["weights"], loo)
+            g["ours-else-steam-loo"] = fuse_look(fz["inputs"], fz["weights"], loo, share=1.0)
+            for name, inputs in fz["replays"].items():
+                g[f"fusion-loo:{name}"] = fuse_look(inputs, fz["weights"], loo)
+        x = xrun[r["run"]]
+        if x.ready():
+            g["steamvr-affine-xrun"] = x.steam_gaze(fz["left"], fz["right"], clicks=False)
+            g["steamvr-affine-xrun+clicks"] = x.steam_gaze(fz["left"], fz["right"])
+            g["fusion-xrun"] = fuse_look(fz["inputs"], fz["weights"], x)
+            g["fusion-xrun+clicks"] = fuse_look(fz["inputs"], fz["weights"], x, clicks=True)
+            if r["truth"]:
+                x.click(fz["left"], fz["right"], *r["truth"])
+        for k, v in g.items():
+            if not only or k in only:
+                r["err"][k] = angle(v, r["truth"])
+
+
+def fit_fusion(recs, out, runs):
+    """A Fusion fitted on all the clicks, written to out."""
+    pts = [(*r["_fz"]["mean"], *r["truth"]) for r in recs if r["truth"] and r["_fz"]["mean"]]
+    f = Fusion()
+    f.fit(pts, {"how": "bench", "runs": runs, "made": time.strftime("%Y-%m-%d %H:%M:%S")})
+    if not f.ready():
+        sys.exit(f"--fit-fusion: only {len(pts)} clicks with SteamVR's two eyes (need {Fusion.MIN})")
+    f.save(out)
+    print(f"\nfusion fitted on {len(pts)} clicks from {len(runs)} runs (SteamVR's affine leaves "
+          f"{statistics.median(f.misses()):.2f}° at the median on them): {out}")
+
+
+def bin_of(value, bins):
+    if value is None:
+        return "none" if bins is SINCE_BINS else "?"
+    return next((name for lo, hi, name in bins if lo <= value < hi), "?")
+
+
+def table(title, recs, key, names):
+    groups = {}
+    for r in recs:
+        groups.setdefault(key(r), []).append(r)
+    cols = list(groups)
+    if not cols:
+        return
+    w = max(len(n) for n in names) + 2
+    print(f"\n{title}: median degrees off (clicks)")
+    print(" " * w + "".join(f"{c:>13}" for c in cols))
+    for n in names:
+        cells = []
+        for c in cols:
+            errs = sorted(e for e in (r["err"].get(n) for r in groups[c]) if e is not None)
+            cells.append(f"{statistics.median(errs):6.2f} ({len(errs):3d})" if errs else f"{'-':>12}")
+        print(f"{n:<{w}}" + "".join(f"{x:>13}" for x in cells))
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("runs", nargs="+", help="run folders, or folders of them")
+    ap.add_argument("--per-click", action="store_true", help="a line per click")
+    ap.add_argument("--json", help="write the per-click records here")
+    ap.add_argument("--only", help="comma-separated techniques")
+    ap.add_argument("--fit-fusion", metavar="OUT", help="fit a gazecal.Fusion on all the clicks, write it here")
+    a = ap.parse_args()
+    only = set(a.only.split(",")) if a.only else None
+    dirs = []
+    for p in map(Path, a.runs):
+        dirs += [p] if (p / "run.json").exists() else sorted(q.parent for q in p.glob("*/run.json"))
+    recs = []
+    for d in dirs:
+        run = Run(d)
+        got = score(run, only)
+        recs += got
+        print(f"{d.name}: {len(got)} clicks, {sum(r['nudged'] for r in got)} nudged"
+              + "".join(f"; {n}" for n in run.notes))
+    if not recs:
+        sys.exit("no clicks")
+    cross(recs, only)
+    names = [n for n in LIVE + CROSS + tuple(f"fusion-loo:{n}" for n in FUSE_REPLAYS) + tuple(REPLAYS)
+             if any(r["err"].get(n) is not None for r in recs)]
+    print()
+    w = max(len(n) for n in names) + 2
+    print(f"{'':<{w}}{'median':>8}{'p90':>8}{'mean':>8}{'n':>6}")
+    for n in names:
+        errs = sorted(e for e in (r["err"].get(n) for r in recs) if e is not None)
+        p90 = errs[min(len(errs) - 1, int(0.9 * len(errs)))]
+        print(f"{n:<{w}}{statistics.median(errs):8.2f}{p90:8.2f}{statistics.fmean(errs):8.2f}{len(errs):6d}")
+    apart = sorted(r["replay_vs_live"] for r in recs if r.get("replay_vs_live") is not None)
+    if apart:
+        p90 = apart[min(len(apart) - 1, int(0.9 * len(apart)))]
+        print(f"\nreplay vs live: our tracker's replay is {statistics.median(apart):.2f}° from what it published, "
+              f"at the median ({p90:.2f}° p90, {len(apart)} clicks); the replays are only as good as this")
+    checks = [r["check"] for r in recs if r.get("check") is not None]
+    if checks:
+        print(f"screen to direction vs the pointer's own: {statistics.median(checks):.2f}° apart at the median "
+              f"({len(checks)} nudges)")
+    table("By scenario", recs, lambda r: r["scenario"], names)
+    table("By where the target was", recs, lambda r: r["where"], names)
+    table("By how far from straight ahead", recs, lambda r: bin_of(r["ecc"], ECC_BINS), names)
+    table("By time since the last nudge", recs, lambda r: bin_of(r["since"], SINCE_BINS), names)
+    if a.per_click:
+        print()
+        for r in recs:
+            print(f"{r['run']} #{r['i']:<3} {r['where']:<6} ecc {r['ecc'] or 0:5.1f}°"
+                  + (" nudged" if r["nudged"] else "       ")
+                  + "".join(f"  {n} {r['err'][n]:.2f}" for n in names if r["err"].get(n) is not None))
+    if a.fit_fusion:
+        fit_fusion(recs, a.fit_fusion, [d.name for d in dirs])
+    if a.json:
+        Path(a.json).write_text(json.dumps([{k: v for k, v in r.items() if k != "_fz"} for r in recs], indent=1))
+
+
+if __name__ == "__main__":
+    main()

gaze/bench/ft-gazebench

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Launch the Frametop Gaze Bench from a Plasma session on the Frame host (its app menu entry,
+# gaze/bench/install.sh), as the eye recorder's gaze/rec/ft-eyerec: the app runs in Frametop's
+# container (PySide6), on the session's Wayland socket and bus; podman gets the real
+# XDG_RUNTIME_DIR and user bus.
+# Options go to gazebench.py: --base DIR (where runs go), --no-images, --no-button.
+here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
+wl=${WAYLAND_DISPLAY:-wayland-0}
+case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
+session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
+export XDG_RUNTIME_DIR=/run/user/$(id -u)
+export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
+cd "$HOME" || exit 1
+exec "$here/../../scripts/in-box" env PATH="/opt/frametop/venv/bin:/opt/frametop/bin:$PATH:/usr/sbin:/sbin" \
+  WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+  XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
+  QT_QPA_PLATFORM="wayland;xcb" ${FT_GAZE:+FT_GAZE="$FT_GAZE"} ${FT_BENCH_BASE:+FT_BENCH_BASE="$FT_BENCH_BASE"} \
+  python3 "$here/gazebench.py" "$@"

gaze/bench/ft-gazebench.desktop

@@ -0,0 +1,9 @@
+[Desktop Entry]
+Type=Application
+Name=Frametop Gaze Bench
+GenericName=Test gaze tracking on hard targets
+Comment=Look at dots in hard places on a panel in the headset and press its button, to compare gaze techniques
+Exec=@REPO@/gaze/bench/ft-gazebench
+Icon=view-visible
+Categories=Utility;
+Keywords=eyes;eye tracking;gaze;test;bench;frametop;

gaze/bench/gazebench.py

@@ -0,0 +1,851 @@
+#!/usr/bin/env python3
+"""Frametop Gaze Bench: targets in hard places for gaze tracking, and a recording of every look
+at them, so gaze techniques can be compared on the same looks afterwards (README.md).
+
+The targets show on a panel fixed to the headset, our own ft-gazepanel in the eye recorder's
+mode (gaze/rec/session.py): wherever the head turns, a target stays at the same direction from
+it, so every run asks the same of the eyes. Pick a scenario (scenarios.json) in the window; the
+panel shows one dot at a time. Look at the dot and press the button on the headset's right side:
+where you looked is the dot. Two presses take the last one back, a hold ends the scenario. The
+window's keys do the same: Space marks, Backspace takes back, S skips, Esc ends. As in Frametop's
+gaze calibration, a left click, Enter, an index pinch (hand tracking) or Meta+J marks too, and a
+right click or Meta+K ends: the bench claims the pointer helper's calibration panel while its
+panel shows (pointer/helper/calpanel.h), so the helper answers those here and hides the pointer.
+
+Each scenario played is a run, recorded by benchrec.py into BASE/<time>-<scenario>/, with the
+panel's own log of what the dot did and when it reached the eyes, and the headset's pose. After
+each mark the window works out how far our tracker and SteamVR were off (their median over the
+LOOK before the press) for the summary; compare.py replays every technique on the recording.
+
+Runs in Frametop's container (PySide6), launched by ft-gazebench. --base DIR, --no-images.
+"""
+import argparse
+import json
+import math
+import os
+import random
+import signal
+import socket
+import statistics
+import subprocess
+import sys
+import threading
+import time
+from pathlib import Path
+
+from PySide6.QtCore import Property, QObject, QSocketNotifier, QTimer, QUrl, Signal, Slot
+from PySide6.QtGui import QGuiApplication
+from PySide6.QtQml import QQmlApplicationEngine
+from PySide6.QtQuickControls2 import QQuickStyle
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.dirname(HERE)
+REPO = os.path.dirname(GAZE)
+sys.path.insert(0, HERE)
+sys.path.insert(0, REPO)
+import benchrec  # noqa: E402
+from recorder.button import ButtonReader, find_button, read_input_devices  # noqa: E402
+from recorder.common import in_container  # noqa: E402
+
+PANEL_DEG, PANEL_ASPECT = 64.0, 0.75   # ft-gazepanel's rec panel (kFullDeg, 4:3), as gaze/rec/session.py
+MARGIN = 1.5            # degrees from the panel's edge to a target at 0 or 1, measured in the panel's plane
+                        # from straight ahead: about 1.1 out at the edges, where the dot's ring (0.75
+                        # out from its middle) still fits
+PANEL_SOCK = os.environ.get("FT_BENCH_PANEL_SOCKET", "ft_benchpanel")   # (the tests' stand-in)
+PANEL_OVERLAY = "frametop.benchpanel"
+HELPER_SOCK = os.environ.get("FT_BENCH_HELPER_SOCKET", "ft_pointer_helper")   # (the tests' stand-in)
+CLAIM_EVERY = 1.0       # s between claims of the helper's calibration panel (it lets one go after 3 s)
+PANELS = [Path(p) for p in (os.environ.get("FT_GAZEPANEL"), os.path.join(GAZE, "build", "ft-gazepanel"),
+                            Path.home() / ".local/share/frametop/releases/current/gaze/build/ft-gazepanel") if p]
+IN_BOX = os.path.join(REPO, "scripts", "in-box")
+PANEL_WAIT = 20.0       # s for a new panel to answer (SteamVR's overlays can take a while)
+LOOK = (0.4, 0.1)       # s before a press: the gaze samples a mark takes. It ends a little before the
+                        # press (a click's was 0.05 s): the push on the headset's side comes before it
+DONE_S = 0.25           # s a dot marked from the window's keys shows done before the next (the
+                        # button's shows done from its down, a double-press time before)
+WAKE = 60               # s: the gaze service's wake lease while a run records (Input Settings' Gaze
+                        # page does the same), so the trackers run whether gaze mode is on or not
+# SteamVR's is the mean of its two eyes (set 2's left and right): on the 2026-10-17 runs it was
+# nearer than its combined gaze (mmap1), and it's what ft-gazed's fusion corrects.
+QUICK_SOURCES = (("own", "Our tracker"), ("steam-eyes", "SteamVR"))
+
+
+def quick_gaze(src, key):
+    """A sample's (yaw, pitch) for a QUICK_SOURCES key, or None."""
+    if key == "steam-eyes":
+        left, right = src.get("left") or {}, src.get("right") or {}
+        if "hy" in left and "hy" in right:
+            return (left["hy"] + right["hy"]) / 2, (left["hp"] + right["hp"]) / 2
+        return None
+    g = src.get(key) or {}
+    return (g["hy"], g["hp"]) if "hy" in g and g.get("ok", 1) != 0 else None
+KEYS_START = "Press the button, click, Enter or pinch to start. Hold the button or right click to end."
+KEYS_END = "Press the button, click, Enter or pinch to close this."
+
+
+def boxed(argv):
+    """A program built in the dev container: run it there (scripts/in-box from the host)."""
+    return list(argv) if in_container() or not os.path.exists(IN_BOX) else [IN_BOX] + list(argv)
+
+
+def margins(margin=MARGIN):
+    """MARGIN as fractions of the panel across and down."""
+    half = math.tan(math.radians(PANEL_DEG / 2))
+    m = math.tan(math.radians(margin)) / (2 * half)
+    return m, m / PANEL_ASPECT
+
+
+def panel_direction(u, v, margin=MARGIN):
+    """The head-relative direction (yaw +left, pitch +up, degrees) of a target u across and v down
+    the panel, 0 and 1 being margin degrees in from its edges: ft-gazepanel's ToPixel backwards."""
+    half = math.tan(math.radians(PANEL_DEG / 2))
+    mx, my = margins(margin)
+    x, y = mx + u * (1 - 2 * mx), my + v * (1 - 2 * my)
+    yaw = math.atan((0.5 - x) * 2 * half)
+    pitch = math.atan((0.5 - y) * 2 * half * PANEL_ASPECT * math.cos(yaw))
+    return math.degrees(yaw), math.degrees(pitch)
+
+
+def panel_place(yaw, pitch, margin=MARGIN):
+    """panel_direction backwards: u, v for a direction."""
+    half = math.tan(math.radians(PANEL_DEG / 2))
+    mx, my = margins(margin)
+    y_r = math.radians(yaw)
+    x = 0.5 - math.tan(y_r) / (2 * half)
+    y = 0.5 - math.tan(math.radians(pitch)) / math.cos(y_r) / (2 * half) / PANEL_ASPECT
+    return (x - mx) / (1 - 2 * mx), (y - my) / (1 - 2 * my)
+
+
+def unit(yaw, pitch):
+    """ft-gaze's head-relative yaw (+left), pitch (+up), degrees, as a unit vector (-Z ahead)."""
+    y, p = math.radians(yaw), math.radians(pitch)
+    return (-math.sin(y) * math.cos(p), math.sin(p), -math.cos(y) * math.cos(p))
+
+
+def angle(a, b):
+    """Degrees between two (yaw, pitch) directions."""
+    d = sum(x * y for x, y in zip(unit(*a), unit(*b)))
+    return math.degrees(math.acos(max(-1.0, min(1.0, d))))
+
+
+def make_trials(sc, rng):
+    """The scenario's targets, in order: dicts with u, v (fractions of the panel across and down,
+    panel_direction), and maybe bg (the panel's brightness, 0 to 1) and dwell (s)."""
+    spec = sc["targets"]
+    pts = []
+    if "grid" in spec:
+        cols, rows = spec["grid"]
+        lo, hi = spec.get("span", (0.2, 0.8))
+        at = lambda k, n: lo + (hi - lo) * k / (n - 1) if n > 1 else 0.5  # noqa: E731
+        pts = [{"u": at(i, cols), "v": at(j, rows)} for j in range(rows) for i in range(cols)]
+    elif "edges" in spec:
+        n = spec["edges"]
+        seen = []
+        for k in range(n):
+            t = k / (n - 1)
+            for p in ((t, 0.0), (t, 1.0), (0.0, t), (1.0, t)):
+                if p not in seen:
+                    seen.append(p)
+        pts = [{"u": u, "v": v} for u, v in seen]
+    elif "rays" in spec:
+        dirs = spec.get("dirs", 8)
+        for f in spec["rays"]:
+            for k in range(dirs):
+                a = 2 * math.pi * k / dirs
+                c, s = math.cos(a), math.sin(a)
+                m = max(abs(c), abs(s))  # so 1 reaches the edge, or the corner on a diagonal
+                pts.append({"u": 0.5 + 0.5 * f * c / m, "v": 0.5 - 0.5 * f * s / m})
+    elif "random" in spec:
+        for _ in range(spec["random"]):
+            u, v = rng.random(), rng.random()
+            if rng.random() < spec.get("edge_share", 0.0):
+                if rng.random() < 0.5:
+                    u = float(round(u))
+                else:
+                    v = float(round(v))
+            pts.append({"u": u, "v": v})
+    elif "steps" in spec:
+        # A walk of small steps from the middle: each target lo to hi degrees from the last, within
+        # the box (degrees either side of straight ahead)
+        lo, hi = spec.get("size", (2.0, 4.0))
+        by, bp = spec.get("box", (12.0, 9.0))
+        at = (0.0, 0.0)
+        for _ in range(spec["steps"]):
+            for _ in range(100):
+                a, d = rng.uniform(0, 2 * math.pi), rng.uniform(lo, hi)
+                nxt = (at[0] + d * math.cos(a), at[1] + d * math.sin(a))
+                if abs(nxt[0]) <= by and abs(nxt[1]) <= bp:
+                    break
+            at = nxt
+            u, v = panel_place(*at)
+            pts.append({"u": u, "v": v})
+    if sc.get("shuffle"):
+        rng.shuffle(pts)
+    for i, p in enumerate(pts):
+        if sc.get("backgrounds"):
+            p["bg"] = sc["backgrounds"][i % len(sc["backgrounds"])]
+        if sc.get("dwell"):
+            p["dwell"] = sc["dwell"]
+    return pts
+
+
+def git_rev():
+    try:
+        return subprocess.run(["git", "-C", HERE, "describe", "--always", "--dirty"], capture_output=True,
+                              text=True, timeout=5).stdout.strip() or None
+    except (OSError, subprocess.TimeoutExpired):
+        return None
+
+
+class Panel:
+    """Our own ft-gazepanel in its rec mode (as gaze/rec/session.py runs one): 64 degrees across,
+    fixed to the headset, a solid background and one dot. Commands go to its control socket
+    without waiting; it answers each one, in order, and the answers are read later (errors
+    logged), or waited for by ask()."""
+
+    def __init__(self, prog=None, log=print, no_vr=False):
+        self.prog = Path(prog) if prog else next((p for p in PANELS if p.exists()), PANELS[0])
+        self.log, self.no_vr = log, no_vr
+        self.proc = None
+        self.ready = False
+        self.error = None
+        self.sent = {}
+        self.pending = 0     # commands sent and not answered yet
+        self.lock = threading.Lock()
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC)
+        self.sock.bind("")
+        self.sock.setblocking(False)
+
+    def start(self):
+        """Start it and wait for its answer: call from a thread (SteamVR can take a while)."""
+        self.ready, self.error = False, None
+        if not self.prog.exists():
+            self.error = f"{self.prog} isn't there: run gaze/build.sh in the dev container"
+            return False
+        argv = [str(self.prog), "--watch-stdin", "--socket", PANEL_SOCK, "--overlay", PANEL_OVERLAY]
+        try:
+            self.proc = subprocess.Popen(boxed(argv + (["--no-vr"] if self.no_vr else [])), stdin=subprocess.PIPE,
+                                         stdout=subprocess.DEVNULL, start_new_session=True)
+        except OSError as e:
+            self.error = f"can't start {self.prog.name}: {e}"
+            return False
+        self.log(f"started {self.prog} (pid {self.proc.pid})")
+        end = time.monotonic() + PANEL_WAIT
+        while time.monotonic() < end:
+            if self.ask("ping", 0.3):
+                self.ready = True
+                return True
+            if self.proc.poll() is not None:
+                self.error = f"{self.prog.name} exited ({self.proc.returncode}): is SteamVR running?"
+                return False
+        self.error = f"{self.prog.name} doesn't answer"
+        self.stop()
+        return False
+
+    def alive(self):
+        return self.ready and self.proc is not None and self.proc.poll() is None
+
+    def _got(self, r):
+        self.pending = max(0, self.pending - 1)
+        if r.startswith("error"):
+            self.log(f"panel: {r}")
+
+    def _drain(self):
+        while True:
+            try:
+                r = self.sock.recv(65536).decode(errors="replace")
+            except OSError:
+                return
+            self._got(r)
+
+    def cmd(self, text):
+        with self.lock:
+            self._drain()
+            try:
+                self.sock.sendto(text.encode(), "\0" + PANEL_SOCK)
+                self.pending += 1
+            except OSError:
+                pass
+
+    def ask(self, text, timeout=0.5):
+        """Send, and its answer (or None): the one after those of the commands still unanswered."""
+        with self.lock:
+            self._drain()
+            try:
+                self.sock.sendto(text.encode(), "\0" + PANEL_SOCK)
+            except OSError:
+                return None
+            self.pending += 1
+            end = time.monotonic() + timeout
+            while (left := end - time.monotonic()) > 0:
+                try:
+                    self.sock.settimeout(left)
+                    r = self.sock.recv(65536).decode(errors="replace")
+                except OSError:
+                    break
+                finally:
+                    self.sock.setblocking(False)
+                last = self.pending == 1
+                self._got(r)
+                if last:
+                    return r
+            self.pending = 0   # a lost answer: count afresh
+            return None
+
+    def set(self, key, text):
+        """Send only if it changes that part of the panel."""
+        if self.sent.get(key) != text:
+            self.sent[key] = text
+            self.cmd(text)
+
+    def show(self, bg):
+        self.cmd("show rec")   # an empty panel
+        self.sent = {}
+        self.set("bg", f"bg {bg:.3f}")
+
+    def lines(self, title="", text="", keys="", step=""):
+        for key, value in (("title", title), ("text", text), ("keys", keys), ("step", step)):
+            self.set(key, f"{key} {value}" if value else key)
+
+    def dot(self, yaw, pitch, state, progress=None):
+        self.set("dot", f"dot {yaw:.4f} {pitch:.4f} {state}" + ("" if progress is None else f" {progress:.2f}"))
+
+    def close(self):
+        self.sock.close()
+
+    def stop(self):
+        p, self.proc, self.ready = self.proc, None, False
+        if not p:
+            return
+        try:
+            p.stdin.close()   # --watch-stdin: it quits
+            p.wait(5)
+        except (OSError, subprocess.TimeoutExpired):
+            try:
+                os.killpg(p.pid, signal.SIGTERM)
+                p.wait(3)
+            except (OSError, subprocess.TimeoutExpired):
+                p.kill()
+        self.log(f"panel stopped (exit {p.returncode})")
+
+
+def text_line(text):
+    """One line for the panel's socket: its own line breaks are "|"."""
+    return " ".join(text.replace("\n", "|").split())
+
+
+class Bench(QObject):
+    changed = Signal()
+    statusChanged = Signal()
+    # The headset button's events, from its reader's thread: "down", "press", "double", "hold",
+    # and the raw time of the press they're about (its down)
+    button = Signal(str, float)
+
+    def __init__(self, scenarios, base, images, gaze_prog=None, panel_prog=None, panel_no_vr=False, button=None):
+        super().__init__()
+        self.all = scenarios
+        self.base = base
+        self._images = images
+        self._page = "menu"
+        self._status = "Looking for the gaze service..."
+        self._progress = self._message = self._summary = self._last = self._notice = ""
+        self.bg = float(scenarios.get("background", 0.15))
+        self.gaze = benchrec.GazeStream(prog=gaze_prog, log=self.log).start()
+        self.gazed = benchrec.Sock(benchrec.GAZED_SOCK)
+        self.panel = Panel(panel_prog, self.log, panel_no_vr)
+        self.panel_starting = None
+        self.run = self.sc = None
+        self.stopping = None       # the last run's stop (it copies the logs), for shutdown to wait for
+        self.trials, self.i, self.results = [], 0, []
+        # A run's phase: intro (its message, until a press), target (a dot to look at), marked (its
+        # done shows), gap (no dot, a scenario's gap), end (the summary on the panel, until a press)
+        self.phase = None
+        self.target = None         # the dot showing: (yaw, pitch)
+        self.shown_t = self.ready_t = None
+        self.flashed = False       # the button's down showed this dot done before its press came
+        self.next_at = None        # monotonic: when the marked or gap phase ends
+        self.rev = git_rev()
+        self.button.connect(self.on_button)
+        self.down_t = None         # the reader's thread: its last down's raw time
+        self.reader = None
+        path = find_button(read_input_devices()) if button is None else button
+        if path:
+            self.reader = ButtonReader(path, self._gesture, self.log, on_down=self._down).start()
+        self.button_path = path or None
+        # The pointer helper's calibration panel (see the top): claimed as @ft_gazebench.PID while
+        # a run or its summary shows; its answers ("calaccept", "calquit") come back there.
+        self.claim_name = f"ft_gazebench.{os.getpid()}"
+        self.claim_at = None       # monotonic: the last claim; None while not claiming
+        self.helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK | socket.SOCK_CLOEXEC)
+        self.helper_note = None
+        try:
+            self.helper.bind("\0" + self.claim_name)
+            self.helper_note = QSocketNotifier(self.helper.fileno(), QSocketNotifier.Type.Read, self)
+            self.helper_note.activated.connect(self.from_helper)
+        except OSError as e:
+            self.log(f"pointer helper socket: {e} (clicks, Enter and pinches won't mark)")
+        self.timer = QTimer(self)
+        self.timer.timeout.connect(self.tick)
+        self.timer.start(50)
+        self.status_at = 0.0
+        self.wake_at = 0.0
+        self.status_busy = False
+        self.status_thread = None
+        self.start_panel()
+
+    def log(self, line):
+        print("ft-gazebench: " + line, file=sys.stderr, flush=True)
+
+    def start_panel(self):
+        if self.panel_starting and self.panel_starting.is_alive():
+            return
+        self.panel_starting = threading.Thread(target=self._start_panel, daemon=True)
+        self.panel_starting.start()
+
+    def _start_panel(self):
+        if not self.panel.start():
+            self.log(f"panel: {self.panel.error}")
+        self.refresh_status()
+
+    # --- properties for bench.qml
+
+    def _get(name):  # noqa: N805
+        return lambda self: getattr(self, name)
+
+    page = Property(str, _get("_page"), notify=changed)
+    progress = Property(str, _get("_progress"), notify=changed)
+    message = Property(str, _get("_message"), notify=changed)
+    summary = Property(str, _get("_summary"), notify=changed)
+    last = Property(str, _get("_last"), notify=changed)
+    notice = Property(str, _get("_notice"), notify=changed)
+    status = Property(str, _get("_status"), notify=statusChanged)
+
+    def _set_images(self, v):
+        self._images = bool(v)
+        self.changed.emit()
+
+    images = Property(bool, _get("_images"), _set_images, notify=changed)
+
+    @Property("QVariantList", constant=True)
+    def scenarios(self):
+        out = []
+        for sc in self.all["scenarios"]:
+            n = len(make_trials(sc, random.Random(0)))
+            out.append({"id": sc["id"], "name": sc["name"], "about": sc["about"], "count": n})
+        return out
+
+    # --- status (in a thread: the sockets wait for replies)
+
+    def refresh_status(self):
+        if self.status_busy:
+            return
+        self.status_busy = True
+        self.status_thread = threading.Thread(target=self._status_thread, daemon=True)
+        self.status_thread.start()
+
+    def _status_thread(self):
+        try:
+            self._status = self.status_text()
+        finally:
+            self.status_busy = False
+        self.statusChanged.emit()
+
+    def status_text(self):
+        lines = []
+        if self.panel.alive():
+            lines.append("Headset panel: ready")
+        elif self.panel_starting and self.panel_starting.is_alive():
+            lines.append("Headset panel: starting...")
+        else:
+            lines.append(f"Headset panel: not running ({self.panel.error or 'it stopped'}). Starting a scenario tries again.")
+        lines.append(f"Headset button: {self.button_path}" if self.button_path else
+                     "Headset button: not found. Click, press Enter or pinch, or use this window's keys (Space marks).")
+        g = self.gazed.json("status")
+        if g is None:
+            lines.append("The gaze service isn't answering (frametop-gaze.service).")
+        else:
+            tracker = "our tracker" if g.get("tracker") == "own" else "SteamVR's tracker"
+            checks = g.get("checks") or {}
+            lines.append(f"Gaze service: {tracker}" + ("; gaze mode is on (the bench doesn't need it)"
+                                                        if checks.get("gaze_mode") else ""))
+            if checks.get("headset_worn") is False:
+                lines.append("Nobody is wearing the headset.")
+            if g.get("tracker") == "own":
+                made = g.get("calibration_made")
+                lines.append(f"Our tracker: calibrated {made}" if made else "Our tracker: not calibrated yet")
+        s = self.gaze.latest()
+        if self.gaze.error:
+            lines.append(f"ft-gaze: {self.gaze.error}")
+        elif s is None or benchrec.raw_now() - s.get("t", 0) > 2:
+            lines.append("ft-gaze: no samples yet (is SteamVR running?)")
+        else:
+            src = s.get("src", {})
+            ok = [n for k, n in QUICK_SOURCES if quick_gaze(src, k)]
+            lines.append("Gaze now from: " + (", ".join(ok) or "nothing (eyes closed, or the headset off?)"))
+        lines.append("Eye images: " + ("recorded (they stay on this headset)" if self._images else "off"))
+        lines.append(f"Runs go to {self.base}")
+        return "\n".join(lines)
+
+    # --- the headset button (its thread: to the Qt thread through the signal)
+
+    def _down(self, t):
+        self.down_t = benchrec.raw_now() - (time.monotonic() - t)
+        self.button.emit("down", self.down_t)
+
+    def _gesture(self, gesture):
+        self.button.emit(gesture, self.down_t or benchrec.raw_now())
+
+    @Slot(str, float)
+    def on_button(self, what, t):
+        if not self.run and self.phase != "end":
+            return
+        if what == "down":
+            # Done at once, so the press feels taken; it counts when "press" comes (a double is undo)
+            if self.phase == "target" and self.ready_t is not None and t >= self.ready_t:
+                self.panel.dot(*self.target, "done")
+                self.flashed = True
+        elif what == "press":
+            if self.phase == "end":
+                self.close_panel()
+            elif self.phase == "intro":
+                self.begin()
+            else:
+                self.mark(t, "button")
+        elif what == "double":
+            self.undo()
+        elif what == "hold" and self.run:
+            self.log("headset button: hold (end)")
+            self.finish(early=True)
+
+    # --- the pointer helper's calibration panel: clicks, Enter and pinches
+
+    def claim(self):
+        """Claim the helper's panel while a run or its summary shows on ours, and let it go after."""
+        want = (self.run is not None or self.phase == "end") and self.panel.alive()
+        now = time.monotonic()
+        if want and (self.claim_at is None or now - self.claim_at >= CLAIM_EVERY):
+            self.claim_at = now
+            self.to_helper(f"calpanel 1 {self.claim_name}")
+        elif not want and self.claim_at is not None:
+            self.claim_at = None
+            self.to_helper(f"calpanel 0 {self.claim_name}")
+
+    def to_helper(self, text):
+        try:
+            self.helper.sendto(text.encode(), "\0" + HELPER_SOCK)
+        except OSError:
+            pass  # the helper isn't running (SteamVR isn't)
+
+    @Slot()
+    def from_helper(self):
+        while True:
+            try:
+                data = self.helper.recv(256)
+            except OSError:
+                return
+            t = benchrec.raw_now()
+            what = data.decode(errors="replace").strip()
+            if self.claim_at is None:
+                continue  # a late answer: the panel is gone
+            if what == "calaccept":     # a press: as the button's, at its arrival
+                if self.phase == "end":
+                    self.close_panel()
+                elif self.phase == "intro":
+                    self.begin()
+                elif self.run:
+                    self.mark(t, "helper")
+            elif what == "calquit":     # as holding the button
+                if self.run:
+                    self.log("pointer helper: quit (end)")
+                    self.finish(early=True)
+                else:
+                    self.close_panel()
+
+    # --- a run
+
+    @Slot(str)
+    def start(self, sid):
+        if self.run:
+            return
+        if not self.panel.alive():
+            self._notice = ("The headset panel isn't running" + (f": {self.panel.error}" if self.panel.error else "")
+                            + ". Trying again; start the scenario once it's ready.")
+            self.start_panel()
+            self.refresh_status()
+            self.changed.emit()
+            return
+        self.sc = next(s for s in self.all["scenarios"] if s["id"] == sid)
+        seed = self.sc.get("seed", random.randrange(1 << 30))
+        self.trials = make_trials(self.sc, random.Random(seed))
+        self.i, self.results = 0, []
+        info = {"display": "panel", "panel": {"deg": PANEL_DEG, "aspect": PANEL_ASPECT, "margin": MARGIN,
+                                              "prog": str(self.panel.prog)},
+                "seed": seed, "look": LOOK, "app": {"rev": self.rev}, "trials_planned": len(self.trials),
+                "button": self.button_path}
+        self.run = benchrec.Run(self.sc, self.gaze, images=self._images, base=self.base, log=self.log, info=info).start()
+        r = self.panel.ask(f"log start {self.run.dir}")
+        if not (r or "").startswith("ok"):
+            self.log(f"panel log: {r or 'no answer'}")
+            self.run.info["panel"]["log_error"] = r or "no answer"
+            self.run.write_info()
+        self._page, self._summary, self._last, self._notice = "run", "", "", ""
+        prompt = self.sc.get("prompt", self.all.get("prompt", "Look at the dot and press the button (or click, Enter or pinch)."))
+        intro = [self.sc["message"]] if self.sc.get("message") else []
+        self._message = "\n".join(intro + [prompt])
+        self.phase, self.target, self.next_at = "intro", None, None
+        self.panel.show(self.bg)
+        self.panel.dot(0, 0, "off")
+        self.panel.lines(self.sc["name"], text_line(self._message), KEYS_START, f"{len(self.trials)} targets")
+        self._progress = f"{self.sc['name']}: {len(self.trials)} targets. Press the headset button (or Space) to start."
+        self.run.trial({"kind": "message", "text": self._message, "t": benchrec.raw_now()})
+        self.wake_at = 0.0
+        self.claim()
+        self.changed.emit()
+
+    def begin(self):
+        """The intro read: the first target."""
+        self.run.trial({"kind": "message-done", "t": benchrec.raw_now()})
+        self.panel.lines()
+        self.show(self.i)
+
+    @Slot()
+    def again(self):
+        if self._page == "done" and self.sc:
+            self.start(self.sc["id"])
+
+    def show(self, i):
+        self.i = i
+        if i >= len(self.trials):
+            return self.finish()
+        p = self.trials[i]
+        self.target = panel_direction(p["u"], p["v"])
+        bg = float(p.get("bg", self.bg))
+        self.panel.set("bg", f"bg {bg:.3f}")
+        self.shown_t = benchrec.raw_now()
+        dwell = p.get("dwell")
+        self.ready_t = None if dwell else self.shown_t
+        self.panel.dot(*self.target, "capture" if dwell else "look", 0.0 if dwell else None)
+        self.phase, self.flashed, self.next_at = "target", False, None
+        self._progress = f"{self.sc['name']}: {i + 1} of {len(self.trials)}"
+        self.changed.emit()
+
+    def quick(self, t, target):
+        """How far our tracker and SteamVR were from the target: their median direction over the
+        LOOK before t, {source: {"yaw", "pitch", "n", "deg"}}."""
+        samples = self.gaze.window(t - LOOK[0], t - LOOK[1])
+        out = {}
+        for key, _ in QUICK_SOURCES:
+            rows = [g for g in (quick_gaze(s.get("src") or {}, key) for s in samples) if g]
+            if len(rows) < 3:
+                continue
+            g = (statistics.median(r[0] for r in rows), statistics.median(r[1] for r in rows))
+            out[key] = {"yaw": round(g[0], 2), "pitch": round(g[1], 2), "n": len(rows), "deg": round(angle(g, target), 2)}
+        return out
+
+    @Slot()
+    def markKey(self):
+        """Space in the window: a mark now."""
+        if self.phase == "intro":
+            self.begin()
+        elif self.phase == "end":
+            self.close_panel()
+        elif self.run:
+            self.mark(benchrec.raw_now(), "key")
+
+    def mark(self, t, by):
+        """You were looking at the dot at t (a press's down)."""
+        if self.phase != "target" or t < self.shown_t:
+            self.run.trial({"kind": "stray", "t": t, "by": by, "phase": self.phase})
+            return
+        if self.ready_t is None or t < self.ready_t:
+            self.run.trial({"kind": "early", "i": self.i, "t": t, "by": by})
+            return
+        p = self.trials[self.i]
+        yaw, pitch = self.target
+        rec = {"kind": "trial", "i": self.i, "u": p["u"], "v": p["v"], "yaw": round(yaw, 4), "pitch": round(pitch, 4),
+               "bg": float(p.get("bg", self.bg)), "shown_t": self.shown_t, "ready_t": self.ready_t,
+               "click": {"t": t, "by": by}, "truth": {"yaw": round(yaw, 4), "pitch": round(pitch, 4), "from": "target"}}
+        rec["quick"] = self.quick(t, self.target)
+        self.results.append(rec)
+        self.run.trial(rec)
+        self._last = "Last: " + ", ".join(f"{label} {rec['quick'][k]['deg']:.1f}° off" if k in rec["quick"] else
+                                          f"{label} no gaze" for k, label in QUICK_SOURCES)
+        self.panel.dot(yaw, pitch, "done")
+        self.after(0.0 if self.flashed else DONE_S)
+
+    def after(self, wait):
+        """The next target, after wait (and the scenario's gap)."""
+        self.i += 1
+        self.phase, self.next_at = "marked", time.monotonic() + wait
+        self.changed.emit()
+
+    @Slot()
+    def skip(self):
+        if self.phase == "target":
+            p = self.trials[self.i]
+            self.run.trial({"kind": "trial", "i": self.i, "u": p["u"], "v": p["v"], "yaw": round(self.target[0], 4),
+                            "pitch": round(self.target[1], 4), "shown_t": self.shown_t,
+                            "click": {"t": benchrec.raw_now(), "by": "key"}, "skipped": True})
+            self.panel.dot(*self.target, "off")
+            self.after(0.0)
+
+    @Slot()
+    def undo(self):
+        """Back to the last target (self.i is the one showing, or the next one while marked or in
+        a gap)."""
+        if not self.run or self.phase in ("intro", "end"):
+            return
+        back = self.i - 1
+        if back < 0:
+            if self.phase == "target" and self.flashed:   # the first dot's down: show it again
+                self.show(self.i)
+            return
+        if self.results and self.results[-1]["i"] == back:
+            self.results.pop()
+        self.run.trial({"kind": "undo", "i": back, "t": benchrec.raw_now()})
+        self._last = "Took the last one back"
+        self.show(back)
+
+    @Slot()
+    def stop(self):
+        if self.run:
+            self.finish(early=True)
+        elif self._page == "done":
+            self.close_panel()
+            self._page = "menu"
+            self.changed.emit()
+        else:
+            QGuiApplication.quit()
+
+    def finish(self, early=False, why=None):
+        summary = self.summarize(early, why)
+        self.phase, self.next_at = "end", None
+        self._page = "done"
+        self._summary = summary["text"]
+        self.changed.emit()
+        run, self.run = self.run, None
+        run.info["panel"]["log_rows"] = self.panel.ask("log stop") if self.panel.alive() else None
+        if self.panel.alive():
+            self.panel.dot(0, 0, "off")
+            self.panel.set("bg", f"bg {self.bg:.3f}")
+            self.panel.lines("Ended early" if early else "Done", text_line(summary["panel"]), KEYS_END)
+        self.stopping = threading.Thread(target=lambda: run.stop(summary), daemon=True)
+        self.stopping.start()
+
+    def close_panel(self):
+        if self.phase == "end":
+            self.phase = None
+            self.panel.cmd("hide")
+            self.claim()
+
+    def summarize(self, early, why=None):
+        marks = [r for r in self.results if not r.get("skipped")]
+        lines = [f"{self.sc['name']}: {len(marks)} marked" + (" (ended early)" if early else "")]
+        if why:
+            lines.append(why)
+        out = {"marks": len(marks), "early": early}
+        for key, label in QUICK_SOURCES:
+            errs = sorted(r["quick"][key]["deg"] for r in marks if key in r.get("quick", {}))
+            if errs:
+                med, worst = statistics.median(errs), errs[-1]
+                p90 = errs[min(len(errs) - 1, int(0.9 * len(errs)))]
+                out[key] = {"median": med, "p90": p90, "worst": worst, "n": len(errs)}
+                lines.append(f"{label}: {med:.1f}° off at the median, {p90:.1f}° at the 90th percentile, worst {worst:.1f}°")
+            else:
+                lines.append(f"{label}: no gaze at the marks")
+        out["panel"] = "\n".join(lines)
+        if self.run:
+            lines.append(f"Saved in {self.run.dir}")
+        out["text"] = "\n".join(lines)
+        return out
+
+    # --- timers
+
+    def tick(self):
+        now = time.monotonic()
+        if self._page != "run" and now - self.status_at > 2:
+            self.status_at = now
+            self.refresh_status()
+        self.claim()
+        if not self.run:
+            return
+        if now - self.wake_at > WAKE / 2:
+            self.wake_at = now
+            threading.Thread(target=lambda: self.gazed.ask(f"wake {WAKE}"), daemon=True).start()
+        self.run.keepalive()
+        if not self.panel.alive():
+            self.log("the panel stopped")
+            return self.finish(early=True, why="The headset panel stopped.")
+        if self.phase == "target" and self.ready_t is None:
+            dwell = self.trials[self.i]["dwell"]
+            done = min(1.0, (benchrec.raw_now() - self.shown_t) / dwell)
+            self.panel.dot(*self.target, "capture", round(done, 2))
+            if done >= 1.0:
+                self.ready_t = benchrec.raw_now()
+        if self.next_at is not None and now >= self.next_at:
+            gap = self.sc.get("gap")
+            if self.phase == "marked" and gap and self.i < len(self.trials):
+                self.panel.dot(*self.target, "off")
+                self.phase, self.next_at = "gap", now + random.uniform(*gap)
+            else:
+                self.show(self.i)
+
+    def shutdown(self):
+        self.timer.stop()
+        if self.run:
+            self.finish(early=True)
+        if self.stopping:
+            self.stopping.join(10)
+        if self.reader:
+            self.reader.stop()
+        if self.panel.alive():
+            self.panel.cmd("hide")
+        if self.claim_at is not None:
+            self.to_helper(f"calpanel 0 {self.claim_name}")
+            self.claim_at = None
+        if self.helper_note:
+            self.helper_note.setEnabled(False)
+        self.helper.close()
+        for t in (self.panel_starting, self.status_thread):
+            if t:
+                t.join(PANEL_WAIT + 1)
+        self.panel.stop()
+        self.panel.close()
+        self.gaze.stop()
+        self.gazed.close()
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Frametop Gaze Bench")
+    ap.add_argument("--base", default=str(benchrec.BASE), help="where runs are recorded")
+    ap.add_argument("--no-images", action="store_true", help="don't record the eye cameras")
+    ap.add_argument("--scenarios", default=os.path.join(HERE, "scenarios.json"))
+    ap.add_argument("--ft-gaze", help="the ft-gaze to run (default: this tree's build, else the installed release's)")
+    ap.add_argument("--ft-gazepanel", help="the ft-gazepanel to run (default: this tree's build, else the release's)")
+    ap.add_argument("--panel-no-vr", action="store_true", help="the panel without SteamVR (its --no-vr)")
+    ap.add_argument("--no-button", action="store_true", help="don't read the headset button")
+    ap.add_argument("--style", default="Fusion", help="the Qt Quick Controls style")
+    a, qt_args = ap.parse_known_args()
+    app = QGuiApplication([sys.argv[0]] + qt_args)
+    app.setApplicationName("ft-gazebench")
+    app.setApplicationDisplayName("Frametop Gaze Bench")
+    app.setDesktopFileName("ft-gazebench")
+    QQuickStyle.setStyle(a.style)
+    with open(a.scenarios) as f:
+        scenarios = json.load(f)
+    bench = Bench(scenarios, a.base, not a.no_images, gaze_prog=a.ft_gaze, panel_prog=a.ft_gazepanel,
+                  panel_no_vr=a.panel_no_vr, button=False if a.no_button else None)
+    app.aboutToQuit.connect(bench.shutdown)
+    # A stop (the launcher's container doesn't pass one on, but kill does) ends a run properly
+    for sig in (signal.SIGTERM, signal.SIGINT):
+        signal.signal(sig, lambda *_: QGuiApplication.quit())
+    engine = QQmlApplicationEngine()
+    engine.rootContext().setContextProperty("bench", bench)
+    engine.load(QUrl.fromLocalFile(os.path.join(HERE, "bench.qml")))
+    if not engine.rootObjects():
+        sys.exit(1)
+    sys.exit(app.exec())
+
+
+if __name__ == "__main__":
+    main()

gaze/bench/install.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Add "Frametop Gaze Bench" to the app menu on a Frame with Frametop, for this tree (README.md).
+# It needs ft-gaze and ft-gazepanel (gaze/build.sh; a release brings them built, and a tree
+# without a build uses the installed release's) and, to record our tracker, ft-eyes
+# (gaze/tracker/install.sh).
+# Usage: gaze/bench/install.sh            install or update
+#        gaze/bench/install.sh uninstall  remove the menu entry. Runs stay where they are.
+set -euo pipefail
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
+. "$root/scripts/_env.sh"
+entry='~/.local/share/applications/frametop-gazebench.desktop'
+
+case ${1:-install} in
+  install)
+    fill_template "$root/gaze/bench/ft-gazebench.desktop" |
+      on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
+    echo "Installed. On Frametop's desktop, open \"Frametop Gaze Bench\" from the app menu."
+    echo "Runs go to ~/.local/share/frametop/gaze/bench."
+    ;;
+  uninstall)
+    on_frame "rm -f $entry"
+    echo "Removed the menu entry. Runs are still in ~/.local/share/frametop/gaze/bench."
+    ;;
+  *) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
+esac

gaze/bench/scenarios.json

@@ -0,0 +1,75 @@
+{
+  "version": 2,
+  "about": "The gaze bench's scenarios (README.md). Targets show on the headset's panel (gazebench.py), 64 degrees across (4:3), fixed to the head. They're fractions of the panel across (u) and down (v); 0 and 1 are 1 degree in from its edges: 31 degrees left or right, 24 up or down, the corners 37 out. Generators: grid [cols, rows] over span; edges N per edge, corners included; rays: fractions of the way from the middle to the edge along dirs directions; random N with edge_share of them on an edge; steps N, each size [lo, hi] degrees from the last, within box [yaw, pitch] degrees of the middle. background and backgrounds are the panel's brightness, 0 to 1.",
+  "background": 0.15,
+  "prompt": "Look at the dot and press the button (or click, Enter or pinch).",
+  "scenarios": [
+    {
+      "id": "warmup",
+      "name": "Warm-up",
+      "about": "Nine dots around the middle.",
+      "targets": {"grid": [3, 3], "span": [0.2, 0.8]},
+      "shuffle": true
+    },
+    {
+      "id": "edges",
+      "name": "Edges",
+      "about": "Sixteen dots along the panel's edges and in its corners, 24 to 37 degrees out.",
+      "targets": {"edges": 5},
+      "shuffle": true
+    },
+    {
+      "id": "hold",
+      "name": "Hold at the edges",
+      "about": "Look at an edge dot until its ring fills (3 seconds), then press: the tracker loses an eye out there.",
+      "prompt": "Keep looking at the dot. Press when its ring is full.",
+      "targets": {"edges": 3},
+      "dwell": 3.0,
+      "shuffle": true
+    },
+    {
+      "id": "spokes",
+      "name": "Out from the middle",
+      "about": "Twenty-four dots on eight spokes, a third, two thirds and all the way out.",
+      "targets": {"rays": [0.35, 0.7, 1.0], "dirs": 8},
+      "shuffle": true
+    },
+    {
+      "id": "head-moving",
+      "name": "Head moving",
+      "about": "Keep looking at the dot while you turn or nod your head slowly, and press while it moves.",
+      "prompt": "Keep looking at the dot while you turn or nod your head slowly. Press while your head is still moving.",
+      "targets": {"grid": [3, 3], "span": [0.2, 0.8]},
+      "shuffle": true
+    },
+    {
+      "id": "light",
+      "name": "Light changes",
+      "about": "The panel switches between dark, bright and grey: your pupils change size.",
+      "targets": {"grid": [4, 3], "span": [0.1, 0.9]},
+      "backgrounds": [0.0, 0.95, 0.42],
+      "shuffle": true
+    },
+    {
+      "id": "steps",
+      "name": "Small steps",
+      "about": "Twenty dots, each 2 to 4 degrees from the last, near the middle.",
+      "targets": {"steps": 20, "size": [2.0, 4.0], "box": [12.0, 9.0]}
+    },
+    {
+      "id": "reseat",
+      "name": "After a nudge",
+      "about": "Lift the headset a little and set it back, then nine dots: how fast the tracker recovers.",
+      "message": "First lift the headset a little off your face and set it back down.",
+      "targets": {"grid": [3, 3], "span": [0.15, 0.85]},
+      "shuffle": true
+    },
+    {
+      "id": "long",
+      "name": "Long session",
+      "about": "Sixty dots anywhere at your own pace, a few seconds apart, about six minutes: drift, slips and stale corrections.",
+      "targets": {"random": 60, "edge_share": 0.3},
+      "gap": [1.0, 5.0]
+    }
+  ]
+}

gaze/bench/tests/fake_ft_gaze.py

@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+"""A stand-in for ft-gaze in the bench's tests: 90 lines a second, the head facing the middle of
+a 1000 x 800 screen (number 2) at 20 px per degree, our tracker 0.5 degrees right of straight
+ahead and SteamVR 1 degree up. Quits when stdin closes, as ft-gaze --watch-stdin, or after
+$FAKE_GAZE_QUIT_AFTER seconds."""
+import json
+import os
+import sys
+import threading
+import time
+
+PPD = 20.0
+
+
+def hit(hy, hp):
+    return {"s": 2, "x": 500 - PPD * hy, "y": 400 - PPD * hp, "j": [-PPD, 0.0, 0.0, -PPD], "dpp": 1 / PPD}
+
+
+def watch():
+    while sys.stdin.buffer.read(4096):
+        pass
+    os._exit(0)
+
+
+threading.Thread(target=watch, daemon=True).start()
+quit_after = float(os.environ.get("FAKE_GAZE_QUIT_AFTER", "inf"))   # s: then exit, as when SteamVR quits
+start = time.monotonic()
+n = 0
+while time.monotonic() - start < quit_after:
+    t = time.clock_gettime(time.CLOCK_MONOTONIC_RAW)
+    own = {"hy": -0.5, "hp": 0.0, "hit": hit(-0.5, 0.0), "eyes": [[-0.5, 0.0], [-0.5, 0.0]],
+           "ehit": [hit(-0.5, 0.0), hit(-0.5, 0.0)]}
+    s = {"t": t, "age": 5, "n": n, "head": {"yaw": 0, "pitch": 0, "hit": hit(0, 0)},
+         "src": {"action": {"ok": 0}, "mmap1": {"hy": 0.0, "hp": 1.0, "hit": hit(0.0, 1.0)}, "mmap2": {"ok": 0},
+                 "left": {"hy": 0.0, "hp": 1.0, "hit": hit(0.0, 1.0)}, "right": {"hy": 0.0, "hp": 1.0, "hit": hit(0.0, 1.0)},
+                 "own": own}, "eye": None}
+    try:
+        sys.stdout.write(json.dumps(s) + "\n")
+        sys.stdout.flush()
+    except BrokenPipeError:
+        break
+    n += 1
+    time.sleep(1 / 90)

gaze/bench/tests/fake_ft_gazepanel.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+"""A stand-in for ft-gazepanel in the bench's tests: its control socket (--socket NAME) answers
+"ok" to everything, "log start DIR" writes the three logs' files there, "log stop" answers
+"ok 3 100 1". Each command goes on a line of $FT_FAKE_PANEL_LOG. Quits when stdin closes, as
+ft-gazepanel --watch-stdin."""
+import os
+import socket
+import sys
+import threading
+from pathlib import Path
+
+
+def watch():
+    while sys.stdin.buffer.read(4096):
+        pass
+    os._exit(0)
+
+
+name = sys.argv[sys.argv.index("--socket") + 1]
+log = os.environ.get("FT_FAKE_PANEL_LOG")
+threading.Thread(target=watch, daemon=True).start()
+sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+sock.bind("\0" + name)
+while True:
+    data, addr = sock.recvfrom(65536)
+    text = data.decode()
+    if log:
+        with open(log, "a") as f:
+            f.write(text + "\n")
+    reply = "ok"
+    if text.startswith("log start "):
+        for f in ("targets.jsonl", "poses.jsonl", "ipd.jsonl"):
+            (Path(text[10:]) / f).write_text("{}\n")
+    elif text == "log stop":
+        reply = "ok 3 100 1"
+    if addr:
+        sock.sendto(reply.encode(), addr)

gaze/bench/tests/test_bench.py

@@ -0,0 +1,442 @@
+#!/usr/bin/env python3
+"""The gaze bench without a headset: the scenarios' targets, bench.qml against gazebench.Bench,
+and whole runs recorded with stand-ins for ft-gaze (fake_ft_gaze.py), the panel
+(fake_ft_gazepanel.py), the headset button (a FIFO), our tracker, the gaze service and the pointer
+helper (their sockets), into a temporary folder. Needs PySide6 (the dev container); the parts that need it are
+skipped without it.
+
+  scripts/in-box python3 gaze/bench/tests/test_bench.py
+"""
+import json
+import os
+import random
+import re
+import socket
+import sys
+import tempfile
+import threading
+import time
+import unittest
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+BENCH = HERE.parent
+sys.path.insert(0, str(BENCH))
+sys.path.insert(0, str(BENCH.parent.parent))
+os.environ["FT_BENCH_EYES_SOCKET"] = f"ft_eyes_benchtest_{os.getpid()}"
+os.environ["FT_BENCH_GAZED_SOCKET"] = f"ft_gazed_benchtest_{os.getpid()}"
+os.environ["FT_BENCH_PANEL_SOCKET"] = f"ft_benchpanel_test_{os.getpid()}"
+os.environ["FT_BENCH_HELPER_SOCKET"] = f"ft_pointer_helper_benchtest_{os.getpid()}"   # (not the live helper)
+os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")
+import benchrec  # noqa: E402
+from recorder.button import EV_KEY, INPUT_EVENT, KEY_SELECT  # noqa: E402
+
+try:
+    from PySide6.QtCore import QUrl
+    from PySide6.QtGui import QGuiApplication
+    from PySide6.QtQml import QQmlApplicationEngine
+    from PySide6.QtQuickControls2 import QQuickStyle
+    import gazebench
+    APP = QGuiApplication.instance() or QGuiApplication([sys.argv[0]])
+    QQuickStyle.setStyle("Fusion")  # the app's
+except ImportError as e:
+    gazebench, APP, NO_QT = None, None, str(e)
+
+SCENARIOS = json.loads((BENCH / "scenarios.json").read_text())
+FAKE_GAZE = HERE / "fake_ft_gaze.py"
+FAKE_PANEL = HERE / "fake_ft_gazepanel.py"
+OWN, STEAMVR = (-0.5, 0.0), (0.0, 1.0)   # fake_ft_gaze.py's gaze, head-relative degrees
+
+
+class FakeService(threading.Thread):
+    """A datagram server on an abstract socket that answers with reply(text)."""
+
+    def __init__(self, name, reply):
+        super().__init__(daemon=True)
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        self.sock.bind("\0" + name)
+        self.reply, self.heard, self.done = reply, [], False
+        self.start()
+
+    def stop(self):
+        self.done = True
+        self.join(1)
+        self.sock.close()
+
+    def run(self):
+        self.sock.settimeout(0.05)
+        while not self.done:
+            try:
+                data, addr = self.sock.recvfrom(65536)
+            except TimeoutError:
+                continue
+            except OSError:
+                return
+            text = data.decode()
+            self.heard.append(text)
+            r = self.reply(text)
+            if r is not None and addr:
+                self.sock.sendto(r.encode(), addr)
+
+
+class GazeStreamTest(unittest.TestCase):
+    def test_restarts(self):
+        """ft-gaze quits with SteamVR: it's started again, and the samples go on."""
+        saved = benchrec.RESTART
+        benchrec.RESTART = 0.2
+        os.environ["FAKE_GAZE_QUIT_AFTER"] = "0.3"
+        logs = []
+        g = benchrec.GazeStream(prog=FAKE_GAZE, log=logs.append).start()
+        try:
+            end = time.monotonic() + 5
+            while g.restarts < 2 and time.monotonic() < end:
+                time.sleep(0.05)
+            self.assertGreaterEqual(g.restarts, 2, logs)
+            lines = g.lines
+            end = time.monotonic() + 3
+            while g.lines <= lines and time.monotonic() < end:
+                time.sleep(0.05)
+            self.assertGreater(g.lines, lines)
+            self.assertIn("starting it again", logs[0])
+        finally:
+            del os.environ["FAKE_GAZE_QUIT_AFTER"]
+            benchrec.RESTART = saved
+            g.stop()
+        self.assertFalse(g.reader.is_alive())
+
+
+class ScenarioTest(unittest.TestCase):
+    def setUp(self):
+        if gazebench is None:
+            self.skipTest(f"no PySide6: {NO_QT}")
+
+    def test_targets(self):
+        for sc in SCENARIOS["scenarios"]:
+            ts = gazebench.make_trials(sc, random.Random(1))
+            self.assertTrue(ts, sc["id"])
+            for p in ts:
+                self.assertTrue(-1e-9 <= p["u"] <= 1 + 1e-9 and -1e-9 <= p["v"] <= 1 + 1e-9, (sc["id"], p))
+            if sc["id"] == "edges":
+                self.assertEqual(len(ts), 16)
+                self.assertTrue(all(p["u"] in (0, 1) or p["v"] in (0, 1) for p in ts))
+            if sc["id"] == "hold":
+                self.assertTrue(all(p["dwell"] == 3.0 for p in ts))
+            if sc["id"] == "light":
+                self.assertEqual({p["bg"] for p in ts}, set(sc["backgrounds"]))
+            if sc["id"] == "steps":
+                dirs = [(0.0, 0.0)] + [gazebench.panel_direction(p["u"], p["v"]) for p in ts]
+                steps = [gazebench.angle(a, b) for a, b in zip(dirs, dirs[1:])]
+                self.assertTrue(all(1.9 < d < 4.1 for d in steps), steps)
+                self.assertTrue(all(abs(y) <= 12.01 and abs(p) <= 9.01 for y, p in dirs), dirs)
+        rays = gazebench.make_trials(next(s for s in SCENARIOS["scenarios"] if s["id"] == "spokes"), random.Random(1))
+        outer = [p for p in rays if p["u"] in (0, 1) or p["v"] in (0, 1)]
+        self.assertEqual(len(outer), 8)  # the 1.0 ring reaches the edges and corners
+
+    def test_panel_direction(self):
+        d = gazebench.panel_direction
+        self.assertEqual(d(0.5, 0.5), (0.0, 0.0))
+        self.assertGreater(d(0, 0.5)[0], 30)    # u 0: left (+yaw)
+        self.assertGreater(d(0.5, 0)[1], 20)    # v 0: up (+pitch)
+        for u, v in ((0, 0), (1, 1), (0.2, 0.7), (0.9, 0.1)):
+            back = gazebench.panel_place(*d(u, v))
+            self.assertAlmostEqual(back[0], u, places=9)
+            self.assertAlmostEqual(back[1], v, places=9)
+        # Every scenario's dot shows whole: its ring (0.75 degrees out) is on the panel
+        for sc in SCENARIOS["scenarios"]:
+            for p in gazebench.make_trials(sc, random.Random(2)):
+                yaw, pitch = d(p["u"], p["v"])
+                for dy, dp in ((0.75, 0), (-0.75, 0), (0, 0.75), (0, -0.75)):
+                    x, y = gazebench.panel_place(yaw + dy, pitch + dp, margin=0)
+                    self.assertTrue(0 <= x <= 1 and 0 <= y <= 1, (sc["id"], p, dy, dp))
+
+
+class QmlTest(unittest.TestCase):
+    """bench.qml's bench.name(...) calls are slots and its bench.name reads are properties or slots."""
+
+    def setUp(self):
+        if gazebench is None:
+            self.skipTest(f"no PySide6: {NO_QT}")
+        meta = gazebench.Bench.staticMetaObject
+        self.slots = {bytes(meta.method(i).name()).decode() for i in range(meta.methodCount())}
+        self.props = {meta.property(i).name() for i in range(meta.propertyCount())}
+        self.qml = (BENCH / "bench.qml").read_text()
+
+    def test_names(self):
+        called = set(re.findall(r"\bbench\.(\w+)\s*\(", self.qml))
+        self.assertEqual(sorted(called - self.slots), [])
+        read = set(re.findall(r"\bbench\.(\w+)\b(?!\s*\()", self.qml)) - {"qml"}
+        self.assertEqual(sorted(read - self.props - self.slots), [])
+
+
+class RunTest(unittest.TestCase):
+    """A warm-up played through with the button and the keys, recorded and summed up; a hold
+    scenario ended by holding the button; bench.qml loaded on it."""
+
+    def setUp(self):
+        if gazebench is None:
+            self.skipTest(f"no PySide6: {NO_QT}")
+        self.tmp = tempfile.TemporaryDirectory()
+        tmp = Path(self.tmp.name)
+        self.saved = {k: getattr(benchrec, k) for k in ("STATE", "EYES_STATE", "LESSON_LOG", "CONF")}
+        benchrec.STATE = tmp / "state"
+        benchrec.EYES_STATE = benchrec.STATE / "eyes"
+        benchrec.LESSON_LOG = benchrec.STATE / "pointer-lessons.jsonl"
+        benchrec.CONF = tmp / "frametop.conf"
+        benchrec.EYES_STATE.mkdir(parents=True)
+        (benchrec.EYES_STATE / "calibration.json").write_text('{"version": 1, "fits": []}')
+        benchrec.CONF.write_text("GAZE_TRACKER=own\nOTHER=1\n")
+        benchrec.LESSON_LOG.write_text(json.dumps({"time": time.time() - 100, "kind": "own"}) + "\n")
+        self.finds = []
+        self.panel_log = tmp / "panel.log"
+        os.environ["FT_FAKE_PANEL_LOG"] = str(self.panel_log)
+        self.fifo = tmp / "button"
+        os.mkfifo(self.fifo)
+        self.button = os.open(self.fifo, os.O_RDWR)   # (Linux: doesn't wait for the reader)
+
+        def eyes(text):
+            w = text.split()
+            if w[0] == "status":
+                return json.dumps({"calibration": {"made": "2026-10-17 10:00:00"}, "feature": "pupil", "model": None,
+                                   "eyes": {"right": {"shift": [0, 0], "clicks": 0, "jump": False, "jump_px": [0, 0],
+                                                      "reseat": False}}})
+            if w[:2] == ["finds", "start"]:
+                self.finds.append(w[2])
+                Path(w[2]).write_text(json.dumps({"t": 1.0, "cam": 0, "lost": True}) + "\n")
+                return "ok"
+            if w[:2] == ["finds", "stop"]:
+                return "ok 1"
+            return "fail"
+
+        def gazed(text):
+            if text == "status":
+                return json.dumps({"tracker": "own", "calibration_made": "2026-10-17 10:00:00",
+                                   "checks": {"gaze_mode": False, "headset_worn": True}})
+            return "ok"
+
+        self.eyes = FakeService(benchrec.EYES_SOCK, eyes)
+        self.gazed = FakeService(benchrec.GAZED_SOCK, gazed)
+        self.helper = FakeService(os.environ["FT_BENCH_HELPER_SOCKET"], lambda text: None)
+
+    def tearDown(self):
+        for k, v in self.saved.items():
+            setattr(benchrec, k, v)
+        self.eyes.stop()
+        self.gazed.stop()
+        self.helper.stop()
+        os.close(self.button)
+        self.tmp.cleanup()
+
+    def wait(self, cond, timeout=5.0):
+        end = time.monotonic() + timeout
+        while time.monotonic() < end:
+            APP.processEvents()
+            if cond():
+                return True
+            time.sleep(0.02)
+        return False
+
+    def key(self, down):
+        t = time.time()
+        os.write(self.button, INPUT_EVENT.pack(int(t), int(t % 1 * 1e6), EV_KEY, KEY_SELECT, 1 if down else 0))
+
+    def press(self, held=0.08):
+        self.key(True)
+        self.wait(lambda: False, held)
+        self.key(False)
+
+    def panel_said(self):
+        return self.panel_log.read_text().splitlines() if self.panel_log.exists() else []
+
+    def bench(self, base):
+        bench = gazebench.Bench(SCENARIOS, str(base), images=False, gaze_prog=FAKE_GAZE, panel_prog=FAKE_PANEL,
+                                button=str(self.fifo))
+        self.assertTrue(self.wait(lambda: bench.gaze.latest() is not None), "no samples from the fake ft-gaze")
+        self.assertTrue(self.wait(bench.panel.alive), bench.panel.error)
+        return bench
+
+    def test_run(self):
+        base = Path(self.tmp.name) / "runs"
+        bench = self.bench(base)
+        try:
+            self.assertTrue(self.wait(lambda: "Headset panel: ready" in bench.status), bench.status)
+            self.assertIn(str(self.fifo), bench.status)
+            bench.start("warmup")
+            self.assertEqual((bench.page, bench.phase), ("run", "intro"))
+            run = next(base.iterdir())
+            self.assertTrue(self.wait(lambda: "title Warm-up" in self.panel_said()), self.panel_said())
+            said = self.panel_said()
+            self.assertIn("show rec", said)
+            self.assertIn(f"log start {run}", said)
+            self.press()                                # the intro: a press starts it
+            self.assertTrue(self.wait(lambda: bench.phase == "target"))
+            n = len(bench.trials)
+            pressed_at = None
+            for k in range(n):
+                self.assertTrue(self.wait(lambda: bench.phase == "target"), (k, bench.phase))
+                self.wait(lambda: False, 0.45)          # samples in the look window
+                if k == 0:                               # the button marks it, at its down
+                    pressed_at = benchrec.raw_now()
+                    self.press()
+                    self.assertTrue(self.wait(lambda: len(bench.results) == 1))
+                    continue
+                if k == 2:                               # two presses: back to 1, marked again
+                    self.press()
+                    self.wait(lambda: False, 0.1)
+                    self.press()
+                    self.assertTrue(self.wait(lambda: bench.i == 1 and bench.phase == "target"), (bench.i, bench.phase))
+                    self.wait(lambda: False, 0.45)
+                    bench.markKey()
+                    self.assertTrue(self.wait(lambda: bench.phase == "target"))
+                    self.wait(lambda: False, 0.45)
+                if k == 4:
+                    bench.skip()
+                    continue
+                bench.markKey()
+            self.assertTrue(self.wait(lambda: bench.page == "done"))
+            self.assertEqual(bench.phase, "end")
+            self.assertIn("Our tracker", bench.summary)
+            self.assertIn("log stop", self.panel_said())
+            self.assertTrue(self.wait(lambda: "title Done" in self.panel_said()), self.panel_said()[-5:])
+            self.press()                                 # closes the panel
+            self.assertTrue(self.wait(lambda: self.panel_said()[-1] == "hide"))
+            self.assertTrue(self.wait(lambda: "end" in json.loads((run / "run.json").read_text())))
+        finally:
+            bench.shutdown()
+        info = json.loads((run / "run.json").read_text())
+        self.assertEqual(info["scenario"]["id"], "warmup")
+        self.assertEqual(info["display"], "panel")
+        self.assertEqual(info["panel"]["log_rows"], "ok 3 100 1")
+        self.assertEqual(info["button"], str(self.fifo))
+        self.assertEqual(info["conf"], ["GAZE_TRACKER=own"])
+        self.assertIsNotNone(info["last_lesson"])
+        self.assertGreater(info["gaze_lines"], 50)
+        self.assertEqual(info["finds_rows"], "ok 1")
+        self.assertTrue((run / "start" / "eyes-calibration.json").exists())
+        self.assertTrue((run / "targets.jsonl").exists())
+        self.assertEqual(self.finds, [str(run / "own.jsonl")])
+        rows = [json.loads(line) for line in (run / "trials.jsonl").read_text().splitlines()]
+        kinds = [r["kind"] for r in rows]
+        self.assertEqual(kinds[:2], ["message", "message-done"])
+        self.assertIn("undo", kinds)
+        trials = [r for r in rows if r["kind"] == "trial"]
+        self.assertEqual(sum(1 for r in trials if r.get("skipped")), 1)
+        by_button = [r for r in trials if r["click"]["by"] == "button"]
+        self.assertEqual(len(by_button), 1)
+        # The mark's time is the press's down, not when the press was sure (a double-press time later)
+        self.assertLess(abs(by_button[0]["click"]["t"] - pressed_at), 0.2)
+        for r in trials:
+            if r.get("skipped"):
+                continue
+            target = gazebench.panel_direction(r["u"], r["v"])
+            self.assertAlmostEqual(r["truth"]["yaw"], target[0], places=3)
+            self.assertAlmostEqual(r["truth"]["pitch"], target[1], places=3)
+            q = r["quick"]
+            self.assertAlmostEqual(q["own"]["deg"], gazebench.angle(OWN, target), places=1)
+            self.assertAlmostEqual(q["steam-eyes"]["deg"], gazebench.angle(STEAMVR, target), places=1)
+        # The button's down showed the dot done before the press counted
+        self.assertTrue(any(c.startswith("dot ") and c.endswith(" done") for c in self.panel_said()))
+        self.assertGreater(len((run / "gaze.jsonl").read_text().splitlines()), 50)
+
+    def answer(self, bench, text):
+        """The pointer helper's answer to the bench's panel claim."""
+        c = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        c.sendto(text.encode(), "\0" + bench.claim_name)
+        c.close()
+
+    def test_helper(self):
+        """The pointer helper's calibration panel: claimed while a run or its summary shows, renewed;
+        its calaccept starts, marks (at its arrival) and closes, calquit ends; then it's let go."""
+        base = Path(self.tmp.name) / "runs"
+        bench = self.bench(base)
+        claim, release = f"calpanel 1 {bench.claim_name}", f"calpanel 0 {bench.claim_name}"
+        try:
+            self.wait(lambda: False, 0.2)
+            self.assertEqual(self.helper.heard, [], "no claim before a run")
+            bench.start("warmup")
+            self.assertTrue(self.wait(lambda: claim in self.helper.heard))
+            self.assertTrue(self.wait(lambda: self.helper.heard.count(claim) >= 2, 2.5), self.helper.heard)
+            self.answer(bench, "calaccept")             # the intro: starts it
+            self.assertTrue(self.wait(lambda: bench.phase == "target"))
+            self.wait(lambda: False, 0.45)
+            sent = benchrec.raw_now()
+            self.answer(bench, "calaccept")
+            self.assertTrue(self.wait(lambda: len(bench.results) == 1))
+            self.assertEqual(bench.results[0]["click"]["by"], "helper")
+            self.assertLess(abs(bench.results[0]["click"]["t"] - sent), 0.1)
+            self.answer(bench, "calquit")                # ends it
+            self.assertTrue(self.wait(lambda: bench.page == "done"))
+            self.assertIn("ended early", bench.summary)
+            self.assertNotIn(release, self.helper.heard, "the summary still shows")
+            self.answer(bench, "calaccept")             # closes the summary
+            self.assertTrue(self.wait(lambda: self.panel_said()[-1] == "hide"))
+            self.assertTrue(self.wait(lambda: self.helper.heard[-1] == release), self.helper.heard[-3:])
+            n = len(self.helper.heard)
+            self.answer(bench, "calaccept")             # late: nothing
+            self.wait(lambda: False, 1.3)
+            self.assertEqual(len(self.helper.heard), n, "no claim once the panel is gone")
+            self.assertIsNone(bench.run)
+            run = next(base.iterdir())
+            self.assertTrue(self.wait(lambda: "end" in json.loads((run / "run.json").read_text())))
+        finally:
+            bench.shutdown()
+
+    def test_hold(self):
+        """Hold at the edges: a press before the ring fills doesn't count; holding the button ends it."""
+        base = Path(self.tmp.name) / "runs"
+        bench = self.bench(base)
+        try:
+            bench.start("hold")
+            bench.markKey()
+            self.assertEqual(bench.phase, "target")
+            bench.markKey()                              # too soon
+            self.assertTrue(self.wait(lambda: any(re.fullmatch(r"dot \S+ \S+ capture 0\.\d+", c) for c in self.panel_said())))
+            self.key(True)
+            self.assertTrue(self.wait(lambda: bench.page == "done", 3))
+            self.key(False)
+            self.assertIn("ended early", bench.summary)
+            run = next(base.iterdir())
+            self.assertTrue(self.wait(lambda: "end" in json.loads((run / "run.json").read_text())))
+        finally:
+            bench.shutdown()
+        kinds = [json.loads(line)["kind"] for line in (run / "trials.jsonl").read_text().splitlines()]
+        self.assertIn("early", kinds)
+        self.assertNotIn("trial", kinds)
+
+    def test_no_panel(self):
+        bench = gazebench.Bench(SCENARIOS, str(Path(self.tmp.name) / "runs"), images=False, gaze_prog=FAKE_GAZE,
+                                panel_prog="/nonexistent/ft-gazepanel", button=False)
+        try:
+            self.wait(lambda: bench.panel.error is not None, 2)
+            bench.start("warmup")
+            self.assertEqual(bench.page, "menu")
+            self.assertIn("isn't running", bench.notice)
+            self.assertTrue(self.wait(lambda: "Headset panel: not running" in bench.status), bench.status)
+            self.assertIn("Headset button: not found", bench.status)
+        finally:
+            bench.shutdown()
+
+    def test_qml_loads(self):
+        bench = self.bench(Path(self.tmp.name) / "runs")
+        warnings = []
+        try:
+            engine = QQmlApplicationEngine()
+            engine.warnings.connect(lambda ws: warnings.extend(w.toString() for w in ws))
+            engine.rootContext().setContextProperty("bench", bench)
+            engine.load(QUrl.fromLocalFile(str(BENCH / "bench.qml")))
+            self.assertTrue(engine.rootObjects())
+            self.wait(lambda: False, 0.3)
+            bench.start("steps")
+            self.wait(lambda: False, 0.2)
+            bench.markKey()
+            self.wait(lambda: False, 0.2)
+            bench.stop()
+            self.wait(lambda: bench.page == "done", 1)
+            self.wait(lambda: False, 0.5)
+            self.assertEqual(warnings, [])
+            del engine
+        finally:
+            bench.shutdown()
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/bench/tests/test_compare.py

@@ -0,0 +1,291 @@
+#!/usr/bin/env python3
+"""compare.py on a made-up run with known errors: SteamVR 1 degree high (its eyes 1 degree either
+side of that), our live tracker 0.5 degrees left (both its eyes), and our tracker's finds made
+from a calibration, with the headset slipping 8 px (2 degrees) a third of the way in, so the
+replays' jump rules differ in a known way. Needs
+numpy and OpenCV (the tracker's Python):
+
+  scripts/in-box /opt/frametop/venv/bin/python gaze/bench/tests/test_compare.py
+"""
+import json
+import math
+import shutil
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+sys.path.insert(0, str(HERE.parent))
+try:
+    import numpy as np
+    import cv2  # noqa: F401  (eyes_pupil's)
+    import compare
+    import eyes_model
+except ImportError as e:
+    compare, NO_DEPS = None, str(e)
+
+PPD = 20.0               # screen pixels per degree, as fake_ft_gaze.py
+K = 4.0                  # pupil pixels per degree
+T0, WALL = 1000.0, 1.79e9
+SLIP_AT, SLIP = 14.0, (8.0, 0.0)   # s after T0, px: the headset slips this much (both eyes)
+# (s after T0 of the click, yaw, pitch, nudged)
+TRIALS = [(3, 0, 0, False), (5, 10, 0, False), (7, -15, 5, True), (9, 5, -10, False), (11, 25, 0, False),
+          (13, -5, 8, False),
+          (16, 8, 3, False),                                  # 2 s after the slip: the jump isn't in yet
+          (24, -10, -5, False), (27, 12, 6, False), (30, 0, -12, False), (33, -18, 0, False)]
+
+
+def hit(hy, hp):
+    return {"s": 2, "x": 500 - PPD * hy, "y": 400 - PPD * hp, "j": [-PPD, 0.0, 0.0, -PPD], "dpp": 1 / PPD}
+
+
+def pupil(e, hy, hp, t):
+    """Where eye e's pupil is for a gaze, t s after T0."""
+    sx = SLIP[0] if t >= SLIP_AT else 0.0
+    return (256 + 10 * e + K * hy + sx, 200 - K * hp)
+
+
+def mid(e, t):
+    sx = SLIP[0] if t >= SLIP_AT else 0.0
+    return (256 + 10 * e + sx, 230.0)
+
+
+def looking(t):
+    """The target being looked at: the next one to be clicked."""
+    for tc, hy, hp, _ in TRIALS:
+        if t <= T0 + tc:
+            return hy, hp
+    return TRIALS[-1][1:3]
+
+
+def make_run(d):
+    (d / "start").mkdir(parents=True)
+    dots = [{"truth": (hy, hp), "eye": [{"pupil": np.array(pupil(e, hy, hp, 0)), "mid": np.array(mid(e, 0))}
+                                        for e in (0, 1)]}
+            for hy in (-20, -10, 0, 10, 20) for hp in (-15, -7, 0, 7, 15)]
+    cal = eyes_model.Calibration.fit(dots, {"made": "M", "feature": "pupil", "dots": len(dots)})
+    cal.save(d / "start" / "eyes-calibration.json")
+    with open(d / "start" / "eyes-calibration-dots.jsonl", "w") as f:
+        for k in dots:
+            f.write(json.dumps({"made": "M", "truth": k["truth"]}) + "\n")
+    blank = {"base": [0, 0], "ref": None, "meas": []}
+    (d / "start" / "eyes-state.json").write_text(json.dumps({"calibration": "M", "shift": {"0": blank, "1": blank}}))
+    eyes = {"right": {"reseat": False, "jump": False, "jump_px": [0, 0]},
+            "left": {"reseat": False, "jump": False, "jump_px": [0, 0]}}
+    (d / "run.json").write_text(json.dumps({
+        "scenario": {"id": "synthetic"}, "screen": {"w": 1000, "h": 800, "dpr": 1.0}, "look": [0.35, 0.05],
+        "start": {"time": WALL, "t_raw": T0, "eyes": {"feature": "pupil", "eyes": eyes}}, "last_lesson": WALL - 500}))
+    with open(d / "gaze.jsonl", "w") as g, open(d / "own.jsonl", "w") as o:
+        n = 0
+        while n / 90 < TRIALS[-1][0] + 1:
+            t = T0 + n / 90
+            hy, hp = looking(t)
+            src = {"mmap1": {"hy": hy, "hp": hp + 1, "hit": hit(hy, hp + 1)},
+                   "left": {"hy": hy + 1, "hp": hp + 1, "hit": hit(hy + 1, hp + 1)},
+                   "right": {"hy": hy - 1, "hp": hp + 1, "hit": hit(hy - 1, hp + 1)},
+                   "own": {"hy": hy + 0.5, "hp": hp, "hit": hit(hy + 0.5, hp),
+                           "eyes": [[hy + 0.5, hp], [hy + 0.5, hp]]}}
+            g.write(json.dumps({"t": t, "head": {"yaw": 0, "pitch": 0, "hit": hit(0, 0)}, "src": src}) + "\n")
+            for e in (0, 1):
+                px, py = pupil(e, hy, hp, t - T0)
+                mx, my = mid(e, t - T0)
+                o.write(json.dumps({"t": t + 0.001 * e, "cam": e, "x": px, "y": py, "a": 20, "b": 18, "angle": 0,
+                                    "glints": [[mx, my - 10], [mx, my + 10]]}) + "\n")
+            n += 1
+    with open(d / "trials.jsonl", "w") as f, open(d / "lessons.jsonl", "w") as ls:
+        for i, (tc, hy, hp, nudged) in enumerate(TRIALS):
+            x, y = 500 - PPD * hy, 400 - PPD * hp
+            f.write(json.dumps({"kind": "trial", "i": i, "u": x / 1000, "v": y / 800, "x": x, "y": y,
+                                "click": {"x": x + 3, "y": y, "t": T0 + tc, "button": "left"},
+                                "truth": {"x": x, "y": y, "from": "target"}}) + "\n")
+            if nudged:
+                ls.write(json.dumps({"time": WALL + tc + 0.02, "kind": "own", "raw": [hy + 0.5, hp], "true": [hy, hp],
+                                     "look_t": T0 + tc - 0.05}) + "\n")
+
+
+class CompareTest(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        if compare is None:
+            raise unittest.SkipTest(f"needs numpy and OpenCV: {NO_DEPS}")
+        cls.tmp = tempfile.TemporaryDirectory()
+        cls.dir = Path(cls.tmp.name) / "run"
+        make_run(cls.dir)
+        cls.bench_run = compare.Run(cls.dir)
+        cls.recs = compare.score(cls.bench_run)
+
+    @classmethod
+    def tearDownClass(cls):
+        cls.tmp.cleanup()
+
+    def errs(self, name):
+        return [r["err"][name] for r in self.recs]
+
+    def test_trials_and_truth(self):
+        self.assertEqual(len(self.recs), len(TRIALS))
+        self.assertEqual(self.bench_run.screen, 2)
+        for r, (_, hy, hp, nudged) in zip(self.recs, TRIALS):
+            self.assertAlmostEqual(r["truth"][0], hy, places=6)
+            self.assertAlmostEqual(r["truth"][1], hp, places=6)
+            self.assertEqual(r["nudged"], nudged)
+        # 25 degrees left is the screen's left edge here (x 0)
+        self.assertEqual([r["where"] for r in self.recs], ["edge" if t[1] == 25 else "inside" for t in TRIALS])
+        self.assertAlmostEqual(self.recs[0]["since"], 3 + 500, delta=0.01)
+
+    def test_live(self):
+        for r, (_, hy, hp, _) in zip(self.recs, TRIALS):
+            self.assertAlmostEqual(r["err"]["steamvr"], compare.angle((hy, hp + 1), (hy, hp)), places=4)
+            self.assertAlmostEqual(r["err"]["steamvr-eyes"], r["err"]["steamvr"], places=6)
+            self.assertAlmostEqual(r["err"]["own-live"], compare.angle((hy + 0.5, hp), (hy, hp)), places=4)
+            fused = (hy + 0.35, hp + 0.3)
+            self.assertAlmostEqual(r["err"]["fuse"], compare.angle(fused, (hy, hp)), places=4)
+        # steamvr+clicks learns the 1 degree from the first click on
+        self.assertTrue(all(e < 1e-6 for e in self.errs("steamvr+clicks")[1:]))
+
+    def test_pointer(self):
+        for r, (_, hy, hp, nudged) in zip(self.recs, TRIALS):
+            want = compare.angle((hy + 0.5, hp), (hy, hp)) if nudged else 3 / PPD
+            self.assertAlmostEqual(r["err"]["pointer"], want, delta=0.02)
+            if nudged:
+                self.assertAlmostEqual(r["check"], 3 / PPD, delta=0.02)  # the click vs the lesson's truth
+
+    def test_replays(self):
+        before = [k for k, t in enumerate(TRIALS) if t[0] < SLIP_AT]
+        late = [k for k, t in enumerate(TRIALS) if t[0] > SLIP_AT + 8]
+        slip_deg = SLIP[0] / K
+        for name in ("own-replay", "no-jump", "both-jump", "edge-guard", "every-click", "bridge"):
+            e = self.errs(name)
+            self.assertTrue(all(e[k] < 0.05 for k in before), (name, e))
+        # The slip: the glint jump puts it right some seconds later; without it, it stays 2 degrees
+        # off until a click (there are none after it here); every click puts it right at once.
+        for name in ("own-replay", "both-jump", "edge-guard", "bridge"):
+            e = self.errs(name)
+            self.assertTrue(all(e[k] < 0.1 for k in late), (name, e))
+        nj = self.errs("no-jump")
+        self.assertTrue(all(abs(nj[k] - slip_deg) < 0.15 for k in late), nj)
+        ec = self.errs("every-click")
+        self.assertTrue(all(ec[k] < 0.1 for k in late), ec)
+        k16 = next(k for k, t in enumerate(TRIALS) if t[0] == 16)   # 2 s after: nothing has caught it yet
+        self.assertAlmostEqual(self.errs("own-replay")[k16], slip_deg, delta=0.15)
+        # The live gaze here is made up, 0.5 degrees from the replay's
+        for k in before:
+            self.assertAlmostEqual(self.recs[k]["replay_vs_live"], compare.angle((TRIALS[k][1] + 0.5, TRIALS[k][2]),
+                                                                                 TRIALS[k][1:3]), delta=0.05)
+
+    def test_edge_guard(self):
+        cal = eyes_model.Calibration.load(self.dir / "start" / "eyes-calibration.json")
+        box = compare.calibration_box(self.bench_run, cal)
+        self.assertEqual(box, (-20, 20, -15, 15))
+        clicks = [(T0 + 5, T0 + 4.9, (10.0, 0.0)), (T0 + 11, T0 + 10.9, (25.0, 0.0))]
+        rp = compare.Replay(self.bench_run, self.bench_run.frames(), cal, {}, None, clicks, guard=True, box=box)
+        rp()
+        self.assertEqual((rp.taught, rp.skipped), (1, 1))
+
+    def test_screen_scale(self):
+        """A window 800 x 640 on the 1000 x 800 screen with Qt's pixel ratio 2 (on the Frame: 2752 wide
+        on 3440 pixels): the window's pixels are scaled by the sizes in run.json, not by the ratio."""
+        self.assertIn("no screen sizes", " ".join(self.bench_run.notes))   # this run has none: the ratio (1)
+        d = Path(self.tmp.name) / "scaled"
+        d.mkdir()
+        for f in self.dir.iterdir():
+            if f.is_file():
+                (d / f.name).write_bytes(f.read_bytes())
+        (d / "start").symlink_to(self.dir / "start")
+        info = json.loads((d / "run.json").read_text())
+        info.update(screen={"w": 800, "h": 640, "dpr": 2.0}, screens={"2": [1000, 800]})
+        (d / "run.json").write_text(json.dumps(info))
+        rows = []
+        for line in (d / "trials.jsonl").read_text().splitlines():
+            r = json.loads(line)
+            for k in ("click", "truth"):
+                r[k]["x"], r[k]["y"] = 0.8 * r[k]["x"], 0.8 * r[k]["y"]
+            r["x"], r["y"] = r["truth"]["x"], r["truth"]["y"]
+            rows.append(json.dumps(r))
+        (d / "trials.jsonl").write_text("\n".join(rows) + "\n")
+        run = compare.Run(d)
+        self.assertEqual((run.sx, run.sy), (1.25, 1.25))
+        self.assertEqual(run.notes, [])
+        for r, (_, hy, hp, _) in zip(compare.score(run), TRIALS):
+            self.assertAlmostEqual(r["truth"][0], hy, places=6)
+            self.assertAlmostEqual(r["truth"][1], hp, places=6)
+
+    def test_panel_run(self):
+        """A run on the headset's panel: the truth is the target's direction (no screen), there's
+        no pointer, and the nudges' lessons (none would come from the bench) aren't matched."""
+        d = Path(self.tmp.name) / "panel"
+        d.mkdir()
+        for f in self.dir.iterdir():
+            if f.is_file():
+                (d / f.name).write_bytes(f.read_bytes())
+        (d / "start").symlink_to(self.dir / "start")
+        info = json.loads((d / "run.json").read_text())
+        del info["screen"]
+        info.update(display="panel", look=[0.4, 0.1])
+        (d / "run.json").write_text(json.dumps(info))
+        with open(d / "trials.jsonl", "w") as f:
+            for i, (tc, hy, hp, _) in enumerate(TRIALS):
+                f.write(json.dumps({"kind": "trial", "i": i, "u": 0.0 if hy == 25 else 0.5, "v": 0.5, "yaw": hy, "pitch": hp,
+                                    "click": {"t": T0 + tc, "by": "button"},
+                                    "truth": {"yaw": hy, "pitch": hp, "from": "target"}}) + "\n")
+        run = compare.Run(d)
+        self.assertTrue(run.panel)
+        self.assertEqual(run.notes, [])
+        recs = compare.score(run)
+        self.assertEqual(len(recs), len(TRIALS))
+        for r, (_, hy, hp, _) in zip(recs, TRIALS):
+            self.assertEqual(r["truth"], (hy, hp))
+            self.assertFalse(r["nudged"])
+            self.assertNotIn("pointer", r["err"])
+            self.assertAlmostEqual(r["err"]["steamvr"], compare.angle((hy, hp + 1), (hy, hp)), places=4)
+            self.assertAlmostEqual(r["err"]["own-live"], compare.angle((hy + 0.5, hp), (hy, hp)), places=4)
+        self.assertEqual([r["where"] for r in recs], ["edge" if t[1] == 25 else "inside" for t in TRIALS])
+
+    def test_fusions(self):
+        """Fitted on the other clicks, SteamVR's affine takes out its 1 degree; the fusion is
+        halfway between that and ours (both our eyes are found), ours-else-steam is ours."""
+        recs = compare.score(self.bench_run)
+        compare.cross(recs)
+        before = [k for k, t in enumerate(TRIALS) if t[0] < SLIP_AT]
+        for k, (r, (_, hy, hp, _)) in enumerate(zip(recs, TRIALS)):
+            self.assertLess(r["err"]["steamvr-affine-loo"], 1e-6)
+            self.assertAlmostEqual(r["err"]["fusion-loo"], compare.angle((hy + 0.25, hp), (hy, hp)), places=4)
+            self.assertAlmostEqual(r["err"]["ours-else-steam-loo"], compare.angle((hy + 0.5, hp), (hy, hp)), places=4)
+            self.assertNotIn("steamvr-affine-xrun", r["err"])   # one run: no other runs to fit on
+            if k in before:   # the replay's eyes are right until the slip: so is the fusion with them
+                self.assertLess(r["err"]["fusion-loo:own-replay"], 0.05)
+        # Steady pupil sizes: the size guards refuse nothing
+        for name in ("guard-both", "guard-long", "no-guard"):
+            self.assertEqual(self.errs(name), self.errs("own-replay"), name)
+
+    def test_fusion_from_start(self):
+        """A run started with a fusion (start/gazed-fusion.json), and --fit-fusion's."""
+        with tempfile.TemporaryDirectory() as tmp:
+            d = Path(tmp) / "fused"
+            shutil.copytree(self.dir, d)
+            f = compare.Fusion()
+            f.fit([(hy, hp + 1, hy, hp) for hy in (-20, 0, 20) for hp in (-10, 0, 10)], {"how": "test"})
+            f.save(d / "start" / "gazed-fusion.json")
+            recs = compare.score(compare.Run(d))
+            for r, (_, hy, hp, _) in zip(recs, TRIALS):
+                self.assertLess(r["err"]["steamvr-affine"], 1e-6)
+                self.assertLess(r["err"]["steamvr-affine+clicks"], 1e-6)
+                self.assertAlmostEqual(r["err"]["fusion"], compare.angle((hy + 0.25, hp), (hy, hp)), places=4)
+            out = Path(tmp) / "fusion.json"
+            compare.fit_fusion(recs, out, ["fused"])
+            g = compare.Fusion.load(out)
+            self.assertTrue(g.ready())
+            self.assertEqual((g.info["how"], g.info["dots"]), ("bench", len(TRIALS)))
+            dy, dp = g.steam.get(5.0, 5.0, g.MODE)
+            self.assertAlmostEqual(dy, 0.0, places=3)
+            self.assertAlmostEqual(dp, -1.0, places=2)
+
+    def test_main(self):
+        out = Path(self.tmp.name) / "recs.json"
+        sys.argv = ["compare.py", str(self.dir.parent), "--json", str(out)]
+        compare.main()
+        self.assertEqual(len(json.loads(out.read_text())), len(TRIALS))
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/build.sh

@@ -13,7 +13,8 @@ g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -
   -o build/ft-gaze ft-gaze.cpp $OPENVR_LIBS -lpthread
 stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
 [ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
-g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
+g++ -std=c++17 -O2 -ffunction-sections -fdata-sections -Wl,--gc-sections -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
   -o build/ft-gazepanel panel/ft-gazepanel.cpp $OPENVR_LIBS \
   $(pkg-config --libs gbm libdrm) -lpthread
+strip --strip-unneeded build/ft-gaze build/ft-gazepanel
 echo "built build/ft-gaze build/ft-gazepanel"'

gaze/eye-rate.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""Keep SteamVR's eye cameras at 90 fps, for gaze mode.
+
+The Frame's SteamVR driver runs the eye cameras at 15 to 90 fps (steamvr.eyeTrackingRateMin
+and eyeTrackingRateMax, in drivers/frame_hmd/resources/frame_hmd_additional.vrsettings),
+and asks for the rate once, as SteamVR starts (vrserver.txt: "cv: Request eye tracking
+framerate"). With its defaults it asks for 15 (eyetracking.txt: "Set framerate 15"). SteamVR's
+own tracker copes, but our tracker gets 8 or 9 frames in a calibration dot's 0.6 s, where it
+needs 15, so it can never calibrate, and the gaze pointer would move 15 times a second.
+With steamvr.gazeDepEyeToHeadEnable on, SteamVR asks for 90 (seen 2026-10-10: a headset with
+it on asks for 90 at every start; a user's with it off, at 15, calibrated once it was on).
+SteamVR's settings have no switch for it.
+
+  gaze/eye-rate.py on      # turn it on (gaze/run.sh install); it counts from SteamVR's next start
+  gaze/eye-rate.py off     # turn it off again, if this turned it on (gaze/run.sh uninstall)
+  gaze/eye-rate.py status  # the setting, and the rate the eye cameras run at now
+
+With SteamVR running it goes through vrcmd, since SteamVR keeps its settings in memory and
+writes over the file; without it, it edits SteamVR's settings file.
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+HOME = Path.home()
+SECTION, KEY = "steamvr", "gazeDepEyeToHeadEnable"
+STEAMVR_BIN = "/opt/steamvr/bin/linuxarm64"
+# The first that exists is SteamVR's (as gaze/gazecal.py reads them).
+STEAMVR_SETTINGS = (HOME / ".config/openvr/config/steamvr.vrsettings",
+                    HOME / ".steam/steam/config/steamvr.vrsettings")
+# There when this turned the setting on (it was off), so `off` leaves a setting someone chose alone.
+MARK = HOME / ".local/state/frametop/gaze/steamvr-eye-rate"
+EYE_LOG = HOME / ".local/share/Steam/logs/eyetracking.txt"
+# A terminal in the desktop has the desktop's own config folder, where SteamVR's path registry
+# isn't (see scripts/update-check.py): vrcmd needs the real one.
+ENV = dict(os.environ, XDG_CONFIG_HOME=str(HOME / ".config"), LD_LIBRARY_PATH=STEAMVR_BIN)
+
+
+def steamvr_up():
+    return subprocess.run(["pgrep", "-x", "vrserver"], capture_output=True).returncode == 0
+
+
+def vrcmd(*args):
+    p = subprocess.run([f"{STEAMVR_BIN}/vrcmd", *args], capture_output=True, text=True, timeout=15, env=ENV)
+    return p.stdout
+
+
+def settings_file():
+    return next((p for p in STEAMVR_SETTINGS if p.exists()), STEAMVR_SETTINGS[0])
+
+
+def get():
+    """The setting: True, False, or None when SteamVR isn't running and its file doesn't say
+    (then the driver's default, off)."""
+    if steamvr_up():
+        # vrcmd prints "steamvr.gazeDepEyeToHeadEnable=true"
+        m = re.search(rf"^{SECTION}\.{KEY}=(\w+)", vrcmd("--settings-bool", f"{SECTION}.{KEY}"), re.M)
+        return None if not m else m.group(1) in ("true", "1")
+    try:
+        value = json.loads(settings_file().read_text()).get(SECTION, {}).get(KEY)
+    except (OSError, ValueError, AttributeError):
+        return None
+    return value if isinstance(value, bool) else None
+
+
+def put(on):
+    if steamvr_up():
+        # 1 or 0: vrcmd reads "true" as 0.
+        vrcmd("--set-settings-bool", f"{SECTION}.{KEY}", "1" if on else "0")
+        return get() is on
+    path = settings_file()
+    try:
+        d = json.loads(path.read_text())
+    except FileNotFoundError:
+        d = {}
+    except (OSError, ValueError) as e:
+        print(f"can't read {path}: {e}", file=sys.stderr)
+        return False
+    d.setdefault(SECTION, {})[KEY] = on
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(".frametop-tmp")
+    tmp.write_text(json.dumps(d, indent=3) + "\n")
+    tmp.replace(path)
+    return True
+
+
+def camera_rate():
+    """The rate the eye cameras run at since the eye tracker last started: the last "Set
+    framerate" after its last start, or 90 when it set none (it starts at 90). None without
+    the log or a start in it."""
+    try:
+        lines = EYE_LOG.read_text(errors="replace").splitlines()
+    except OSError:
+        return None
+    starts = [i for i, line in enumerate(lines) if " startup with PID" in line]
+    if not starts:
+        return None
+    rates = re.findall(r"CStereoAdspCams: Set framerate ([\d.]+)", "\n".join(lines[starts[-1]:]))
+    return float(rates[-1]) if rates else 90.0
+
+
+def main():
+    what = sys.argv[1] if len(sys.argv) == 2 else ""
+    if what == "on":
+        if get():
+            print(f"SteamVR's eye cameras: {SECTION}.{KEY} is on already, so they run at 90 fps")
+            return 0
+        if not put(True):
+            print(f"couldn't turn on {SECTION}.{KEY}: the eye cameras may stay at 15 fps, and our eye "
+                  "tracker can't calibrate at that. Try again with: gaze/eye-rate.py on", file=sys.stderr)
+            return 1
+        MARK.parent.mkdir(parents=True, exist_ok=True)
+        MARK.write_text(f"{SECTION}.{KEY} was off; Frametop turned it on\n")
+        print(f"SteamVR's eye cameras: turned on {SECTION}.{KEY}, so they run at 90 fps, not 15, "
+              "once SteamVR restarts")
+        return 0
+    if what == "off":
+        if not MARK.exists():
+            print(f"SteamVR's eye cameras: left as they are (Frametop didn't turn on {SECTION}.{KEY})")
+            return 0
+        if not put(False):
+            print(f"couldn't turn off {SECTION}.{KEY}. Try again with: gaze/eye-rate.py off", file=sys.stderr)
+            return 1
+        MARK.unlink()
+        print(f"SteamVR's eye cameras: turned {SECTION}.{KEY} off again (back to 15 fps once SteamVR restarts)")
+        return 0
+    if what == "status":
+        value = get()
+        rate = camera_rate()
+        print(f"{SECTION}.{KEY}: {'on' if value else 'off'}" + (" (set by Frametop)" if MARK.exists() else ""))
+        print("eye cameras: " + (f"{rate:g} fps since the eye tracker started" if rate else "unknown (no eyetracking.txt)"))
+        return 0
+    print("usage: gaze/eye-rate.py on|off|status", file=sys.stderr)
+    return 2
+
+
+if __name__ == "__main__":
+    sys.exit(main())

gaze/fit-reference.json

@@ -0,0 +1,46 @@
+{
+ "schema": 1,
+ "made": "2026-10-14",
+ "source": "the maintainer's headset: 2 of 3 own-tracker calibrations (2026-10-08, 2026-10-09), the third fitted worse than 4 px",
+ "faces": 1,
+ "ipd": 0.0651,
+ "cameras_follow_lenses": null,
+ "eyes": {
+  "0": {
+   "centre": [
+    341.04,
+    221.4
+   ],
+   "jacobian": [
+    [
+     1.2749,
+     -0.4221
+    ],
+    [
+     -0.4783,
+     -1.3388
+    ]
+   ],
+   "fits": 2,
+   "fit_err": 2.339
+  },
+  "1": {
+   "centre": [
+    334.74,
+    173.53
+   ],
+   "jacobian": [
+    [
+     -1.3415,
+     -0.5307
+    ],
+    [
+     -0.4822,
+     1.4794
+    ]
+   ],
+   "fits": 2,
+   "fit_err": 2.146
+  }
+ }
+}

gaze/fitcheck.py

@@ -51,7 +51,10 @@ GUIDE = [
 
 
 class FitCheck:
-    def __init__(self):
+    def __init__(self, ignore=None):
+        # The eye SteamVR's tracker ignores (0 left, 1 right) with Track Dominant Eye Only on
+        # (gazecal.tracked_eye), or None: its losses say nothing about the fit.
+        self.ignore = ignore
         self.reset()
 
     def reset(self):
@@ -99,7 +102,8 @@ class FitCheck:
             if q and (eye.get("new") or [1, 1])[k]:
                 self.q[k].append(q[k])
         closed = [opens[k] < CLOSED for k in (0, 1)]
-        if all(closed) or all(self.lost):
+        judged = [k for k in (0, 1) if k != self.ignore]
+        if all(closed[k] for k in judged) or all(self.lost[k] for k in judged):
             return  # a blink: says nothing about the fit
         key = (math.floor(hy / CELL), math.floor(hp / CELL))
         for k in (0, 1):
@@ -143,6 +147,8 @@ class FitCheck:
     # --- Summaries ---
 
     def status(self, k):
+        if k == self.ignore:
+            return "not tracked", (0.6, 0.6, 0.6)
         if not self.samples:
             return "no data", (0.6, 0.6, 0.6)
         if self.lost[k]:
@@ -174,8 +180,13 @@ class FitCheck:
             return ["Look around slowly (the screen's corners, then down at your keyboard, up, left and right) "
                     "or press Enter for a guided check."]
         out = []
+        if self.ignore is not None:
+            out.append(f"SteamVR tracks only your {EYES[1 - self.ignore].lower()} (Track Dominant Eye Only in "
+                       f"SteamVR's settings), so your {EYES[self.ignore].lower()} doesn't count here.")
         bad = {}
         for k in (0, 1):
+            if k == self.ignore:
+                continue
             for key, words, _ in REGIONS:
                 share = self.region_share(k, key)
                 if share is not None and share >= 0.15:
@@ -197,7 +208,7 @@ class FitCheck:
                                "face, not one eye's fit.")
                 continue
             k = next(iter(eyes))
-            other = self.region_share(1 - k, key)
+            other = self.region_share(1 - k, key) if 1 - k != self.ignore else None
             vs = f", the {EYES[1 - k].lower()} {other:.0%}" if other is not None else ""
             line = f"{EYES[k]}: lost {eyes[k]:.0%} of the time looking {words}{vs}."
             if key == "down":
@@ -211,10 +222,12 @@ class FitCheck:
                          "eyes.")
             out.append(line)
         s0, s1 = self.signal(0), self.signal(1)
-        if s0 is not None and s1 is not None and abs(s0 - s1) > 0.25:
+        if self.ignore is None and s0 is not None and s1 is not None and abs(s0 - s1) > 0.25:
             k = 0 if s0 < s1 else 1
             out.append(f"The tracker is less sure of your {EYES[k].lower()} even when it has it "
                        f"(signal {min(s0, s1):.0%} against {max(s0, s1):.0%}).")
+        if self.ignore is not None and len(out) == 1:
+            out.append(f"Your {EYES[1 - self.ignore].lower()} is tracked everywhere you've looked so far.")
         if not out:
             out.append("Both eyes are tracked everywhere you've looked so far.")
         return out

gaze/fitcoach.py

@@ -0,0 +1,521 @@
+#!/usr/bin/env python3
+"""fitcoach: how the headset sits on this face, from where our tracker sees each eye, and what
+to do about it (gaze/rec/DESIGN.md, "Fit coach"). The eye recorder runs it before each round;
+the gaze service's fit check, put-on check and slip notice use it too (gaze/gazecheck.py).
+
+It takes per-frame pupil finds (gaze/tracker/eyes_pupil.find_pupil: centre, axes, glints) for
+each camera (0 the right eye, 1 the left), keeps the last second, and compares the medians
+with a reference: where a well-fitted eye sits in that camera's image while looking straight
+ahead, and how the pupil moves per degree of gaze (the reference's Jacobian, fitted on
+calibration dots). Offsets go through that Jacobian into degrees as if the eye had turned, so
+the cameras' orientations don't matter (camera 1's image is turned 180 degrees from camera 0's:
+looking left moves its pupil the other way) and both eyes' readings can be added up:
+
+  both eyes appear to look up       the eyes sit high in the headset: it's too low    -> push it up
+  both appear to look left          the headset sits off to the right                 -> slide it left
+  the eyes appear to look apart     the eyes are wider apart than the lenses           -> IPD wider
+  one appears to look higher        the headset is tilted                              -> level it
+
+The IPD reading only becomes a hint once we know the eye cameras move with the lenses
+(HEADSET-TESTS, 2026-10-14): if they're fixed to the housing, the offset also depends on the IPD
+setting, and that needs a reference per setting. Until then it's reported, not advised.
+
+A baseline is the same reading saved from one person's good fit. Compared with a baseline the
+glint pair's spacing says whether the eye is nearer the camera than then (the eye relief: on
+the maintainer's headset it moved 18 % between two seatings, and 4 % within one), and the put-on
+check says how far the headset moved since. The reference starts as the maintainer's
+(gaze/fit-reference.json); its hints are marked provisional until it's built from more faces.
+
+Measured on the maintainer's headset (2026-10-14, three calibrations of 27 dots): straight-ahead
+pupils moved 15-20 px between seatings, at about 1.3 px per degree; so OK within OK_DEG and
+adjust past ADJUST_DEG, both provisional.
+
+Usage (on the Frame, in the dev container, with frametop-eyegrab running):
+  python3 gaze/fitcoach.py live [--baseline]   print the reading twice a second; --baseline saves
+                                               the last one as your baseline on Ctrl-C
+  python3 gaze/fitcoach.py reference [DOTS...] fit a reference from calibration-dots.jsonl files
+"""
+import json
+import math
+import os
+import statistics
+import sys
+import threading
+import time
+from collections import deque
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+REFERENCE = os.path.join(HERE, "fit-reference.json")
+BASELINE = os.path.expanduser("~/.local/state/frametop/gaze/fit-baseline.json")
+DOTS = os.path.expanduser("~/.local/state/frametop/gaze/eyes/calibration-dots.jsonl")
+EYES = ("right", "left")       # camera 0, camera 1
+WINDOW = 1.0                   # s of finds per reading
+MIN_FINDS = 10                 # finds in the window before an eye is judged
+MIN_FOUND = 0.5                # share of frames with a pupil: under this, the eye is lost
+OK_DEG = 5.0                   # every term within this: the fit is good (provisional)
+ADJUST_DEG = 8.0               # a term past this: say what to do (between the two: keep the last state)
+TILT_DEG = 4.0                 # half the eyes' vertical difference
+RELIEF_TOL = 0.08              # glint spacing this much off the baseline: nearer or further
+MANY_GLINTS = 5                # a find with this many glints has stray reflections. Without glasses
+                               # (three maintainer captures, 2026-10-14) 3+ glints came in up to 37 % of
+                               # finds and 5+ in up to 12 %: a guess until the Glasses bounty's data
+EXTRA_GLINTS = 0.5             # share of finds with MANY_GLINTS: reflections (glasses, a smudge)
+MOVED_DEG = 5.0                # put-on check: this far from the baseline, the headset moved; the slip
+                               # notice: ft-eyes' glint jump this far
+
+
+# ------------------------------------------------------------------------------------------
+# Readings
+
+def glint_spacing(p):
+    """The LED glint pair's spacing in pixels, or None (eyes_pupil.glint_pair's rule)."""
+    pair = _glint_pair(p)
+    if not pair:
+        return None
+    (x0, y0), (x1, y1) = pair
+    return math.hypot(x1 - x0, y1 - y0)
+
+
+def _glint_pair(p):
+    """eyes_pupil.glint_pair without numpy: the vertical-ish pair nearest the pupil centre."""
+    g, best = p.get("glints") or [], None
+    for i in range(len(g)):
+        for j in range(i + 1, len(g)):
+            dx, dy = g[j][0] - g[i][0], g[j][1] - g[i][1]
+            d = math.hypot(dx, dy)
+            if not 5 <= d <= 45 or abs(dy) < 2 * abs(dx):
+                continue
+            cost = math.hypot((g[i][0] + g[j][0]) / 2 - p["x"], (g[i][1] + g[j][1]) / 2 - p["y"])
+            if best is None or cost < best[0]:
+                best = (cost, (g[i], g[j]) if dy > 0 else (g[j], g[i]))
+    return best[1] if best else None
+
+
+class Observations:
+    """The last WINDOW seconds of pupil finds per camera. add() from any thread."""
+
+    def __init__(self, window=WINDOW):
+        self.window = window
+        self._rows = {0: deque(), 1: deque()}   # (t, x, y, a, spacing or None, extra glints) or (t, None)
+        self._lock = threading.Lock()
+
+    def add(self, cam, t, p):
+        if p is None:
+            row = (t, None)
+        else:
+            row = (t, float(p["x"]), float(p["y"]), float(p["a"]), glint_spacing(p), len(p.get("glints") or []) >= MANY_GLINTS)
+        with self._lock:
+            rows = self._rows[cam]
+            rows.append(row)
+            while rows and rows[0][0] < t - self.window:
+                rows.popleft()
+
+    def summary(self, now=None):
+        """{camera: {"n", "found", "x", "y", "a", "spacing", "extra"}}: medians of the finds,
+        found the share of frames with a pupil, extra the share with MANY_GLINTS or more. A camera with
+        no frames in the window: {"n": 0}."""
+        out = {}
+        with self._lock:
+            for cam, rows in self._rows.items():
+                rs = [r for r in rows if now is None or r[0] >= now - self.window]
+                finds = [r for r in rs if r[1] is not None]
+                if not rs:
+                    out[cam] = {"n": 0}
+                    continue
+                sp = [r[4] for r in finds if r[4] is not None]
+                out[cam] = {"n": len(rs), "finds": len(finds), "found": len(finds) / len(rs),
+                            "x": statistics.median(r[1] for r in finds) if finds else None,
+                            "y": statistics.median(r[2] for r in finds) if finds else None,
+                            "a": statistics.median(r[3] for r in finds) if finds else None,
+                            "spacing": statistics.median(sp) if len(sp) >= 3 else None,
+                            "extra": sum(r[5] for r in finds) / len(finds) if finds else 0.0}
+        return out
+
+
+# ------------------------------------------------------------------------------------------
+# The reference and the baseline
+
+def _solve3(a, b):
+    """x for a 3x3 system a x = b (Gaussian elimination with pivoting), or None if singular."""
+    m = [row[:] + [v] for row, v in zip(a, b)]
+    for i in range(3):
+        k = max(range(i, 3), key=lambda r: abs(m[r][i]))
+        if abs(m[k][i]) < 1e-12:
+            return None
+        m[i], m[k] = m[k], m[i]
+        for r in range(3):
+            if r != i:
+                f = m[r][i] / m[i][i]
+                m[r] = [x - f * y for x, y in zip(m[r], m[i])]
+    return [m[i][3] / m[i][i] for i in range(3)]
+
+
+def fit_linear(truths, values):
+    """value = c0 + c1 yaw + c2 pitch by least squares: [c0, c1, c2], or None."""
+    rows = [(1.0, float(t[0]), float(t[1])) for t in truths]
+    ata = [[sum(r[i] * r[j] for r in rows) for j in range(3)] for i in range(3)]
+    atb = [sum(r[i] * v for r, v in zip(rows, values)) for i in range(3)]
+    return _solve3(ata, atb)
+
+
+class Reference:
+    """Per camera: the straight-ahead pupil centre (px) and the Jacobian [[dx/dyaw, dx/dpitch],
+    [dy/dyaw, dy/dpitch]] (px per degree); optionally the glint spacing, the IPD it was made
+    at (m), whether the eye cameras move with the lenses, where it came from and from how many faces."""
+
+    def __init__(self, eyes, ipd=None, source="", faces=1, cameras_follow_lenses=None, made=""):
+        self.eyes, self.ipd, self.source, self.faces = eyes, ipd, source, faces
+        self.cameras_follow_lenses, self.made = cameras_follow_lenses, made
+
+    @property
+    def provisional(self):
+        return self.faces < 3
+
+    @classmethod
+    def from_dots(cls, rows, source="", ipd=None, max_err=4.0):
+        """From calibration-dots.jsonl rows ({"made", "truth": [yaw, pitch], "eye": {"0": {"pupil": [x, y]}}}).
+        Each calibration ("made") is fitted on its own; those that fit worse than max_err px
+        (median) are left out, and the rest averaged."""
+        fits = {0: [], 1: []}
+        for made in sorted({r.get("made", "") for r in rows}):
+            for cam in (0, 1):
+                rs = [r for r in rows if r.get("made", "") == made and (r.get("eye") or {}).get(str(cam))]
+                if len(rs) < 6:
+                    continue
+                truths = [r["truth"] for r in rs]
+                px = [r["eye"][str(cam)]["pupil"] for r in rs]
+                cx, cy = fit_linear(truths, [p[0] for p in px]), fit_linear(truths, [p[1] for p in px])
+                if cx is None or cy is None:
+                    continue
+                err = statistics.median(math.hypot(cx[0] + cx[1] * t[0] + cx[2] * t[1] - p[0],
+                                                   cy[0] + cy[1] * t[0] + cy[2] * t[1] - p[1]) for t, p in zip(truths, px))
+                if err <= max_err:
+                    fits[cam].append((cx, cy, err))
+        eyes = {}
+        for cam, fs in fits.items():
+            if not fs:
+                continue
+            avg = lambda k, i: sum(f[k][i] for f in fs) / len(fs)   # noqa: E731
+            eyes[cam] = {"centre": [avg(0, 0), avg(1, 0)], "jacobian": [[avg(0, 1), avg(0, 2)], [avg(1, 1), avg(1, 2)]],
+                         "fits": len(fs), "fit_err": statistics.median(f[2] for f in fs)}
+        return cls(eyes, ipd=ipd, source=source, made=time.strftime("%Y-%m-%d"))
+
+    def degrees(self, cam, dx, dy):
+        """A pupil offset in px as the gaze change it would be: (yaw, pitch) degrees."""
+        (a, b), (c, d) = self.eyes[cam]["jacobian"]
+        det = a * d - b * c
+        if abs(det) < 1e-9:
+            return 0.0, 0.0
+        return (d * dx - b * dy) / det, (-c * dx + a * dy) / det
+
+    def to_json(self):
+        eyes = {str(k): {"centre": [round(v, 2) for v in e["centre"]],
+                         "jacobian": [[round(v, 4) for v in row] for row in e["jacobian"]],
+                         **{k2: (round(v2, 3) if isinstance(v2, float) else v2) for k2, v2 in e.items()
+                            if k2 not in ("centre", "jacobian")}}
+                for k, e in sorted(self.eyes.items())}
+        return {"schema": 1, "made": self.made, "source": self.source, "faces": self.faces, "ipd": self.ipd,
+                "cameras_follow_lenses": self.cameras_follow_lenses,
+                "eyes": eyes}
+
+    @classmethod
+    def from_json(cls, d):
+        return cls({int(k): v for k, v in d.get("eyes", {}).items()}, ipd=d.get("ipd"), source=d.get("source", ""),
+                   faces=d.get("faces", 1), cameras_follow_lenses=d.get("cameras_follow_lenses"), made=d.get("made", ""))
+
+    @classmethod
+    def load(cls, path=REFERENCE):
+        try:
+            with open(path) as f:
+                return cls.from_json(json.load(f))
+        except (OSError, ValueError):
+            return None
+
+    def save(self, path=REFERENCE):
+        _write_json(path, self.to_json())
+
+
+def make_baseline(summary, ipd=None):
+    """A baseline from a good reading: per camera the centre and glint spacing, and the IPD."""
+    eyes = {str(c): {"centre": [s["x"], s["y"]], "spacing": s.get("spacing"), "a": s.get("a")}
+            for c, s in summary.items() if s.get("x") is not None}
+    return {"schema": 1, "made": time.strftime("%Y-%m-%d %H:%M:%S"), "ipd": ipd, "eyes": eyes}
+
+
+def load_baseline(path=BASELINE):
+    try:
+        with open(path) as f:
+            return json.load(f)
+    except (OSError, ValueError):
+        return None
+
+
+def save_baseline(baseline, path=None):
+    """Keep a good fit as the person's (the gaze service's fit check, the eye recorder's fitment)."""
+    _write_json(path or BASELINE, baseline)
+
+
+def _write_json(path, obj):
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tmp = path + ".tmp"
+    with open(tmp, "w") as f:
+        json.dump(obj, f, indent=1)
+        f.write("\n")
+    os.replace(tmp, path)
+
+
+# ------------------------------------------------------------------------------------------
+# The assessment
+
+HINTS = {
+    "up": "Push the headset up a little (the top strap).",
+    "down": "Let the headset down a little.",
+    "left": "Slide the headset a little to your left.",
+    "right": "Slide the headset a little to your right.",
+    "ipd-wider": "Turn the IPD wheel a little wider.",
+    "ipd-narrower": "Turn the IPD wheel a little narrower.",
+    "tilt-left-up": "Level the headset: lift its left side a little.",
+    "tilt-right-up": "Level the headset: lift its right side a little.",
+    "nearer": "The headset sits closer to your eyes than your saved fit: loosen the back strap a little.",
+    "further": "The headset sits further from your eyes than your saved fit: tighten the back strap a little.",
+    "reflections": "Stray reflections in the {eye} eye's camera: tilt your glasses a little, or wipe the lens.",
+    "lost": "The {eye} eye is hard to see: lashes touching the lens, or the headset too low on that side?",
+    "ipd-changed": "The IPD setting changed since your saved fit ({was:.1f} mm, now {now:.1f} mm).",
+}
+
+
+def assess(summary, ref, ipd=None, baseline=None, previous=None):
+    """The reading: {"state": "ok" | "adjust" | "lost" | "waiting", "eyes": {camera: {...}},
+    "terms": {"up", "left", "ipd", "tilt"} in degrees, "hints": [{"key", "text", "amount"}],
+    "provisional", "against": "baseline" | "reference"}.
+
+    Against a baseline (the person's own good fit) when there is one, else the reference.
+    previous (an earlier assess() result) keeps the state between OK_DEG and ADJUST_DEG."""
+    eyes, hints, offsets = {}, [], {}
+    centres = {}
+    if baseline:
+        centres = {int(k): v["centre"] for k, v in baseline.get("eyes", {}).items()}
+    for cam, s in summary.items():
+        e = {"name": EYES[cam], "found": round(s.get("found", 0.0), 3), "seen": bool(s.get("n"))}
+        if not s.get("n") or s.get("finds", 0) < MIN_FINDS:
+            e["state"] = "waiting" if not s.get("n") else "lost"
+        elif s["found"] < MIN_FOUND:
+            e["state"] = "lost"
+        else:
+            e["state"] = "found"
+            centre = centres.get(cam) or (ref.eyes.get(cam) or {}).get("centre") if ref else centres.get(cam)
+            if centre and ref and cam in ref.eyes:
+                dx, dy = s["x"] - centre[0], s["y"] - centre[1]
+                offsets[cam] = ref.degrees(cam, dx, dy)
+                e["offset_px"] = [round(dx, 1), round(dy, 1)]
+                e["offset_deg"] = [round(v, 2) for v in offsets[cam]]
+            if s.get("extra", 0.0) > EXTRA_GLINTS:
+                hints.append({"key": "reflections", "text": HINTS["reflections"].format(eye=EYES[cam]), "amount": s["extra"]})
+            if baseline:
+                b = baseline.get("eyes", {}).get(str(cam)) or {}
+                if b.get("spacing") and s.get("spacing"):
+                    rel = s["spacing"] / b["spacing"] - 1.0
+                    e["relief"] = round(rel, 3)
+        eyes[cam] = e
+    lost = [c for c, e in eyes.items() if e["state"] == "lost"]
+    for c in lost:
+        hints.append({"key": "lost", "text": HINTS["lost"].format(eye=EYES[c]), "amount": 1.0 - eyes[c]["found"]})
+    terms = {}
+    if offsets:
+        yaws = {c: v[0] for c, v in offsets.items()}
+        pitches = {c: v[1] for c, v in offsets.items()}
+        terms["up"] = sum(pitches.values()) / len(pitches)          # + : the eyes sit high: the headset is low
+        terms["left"] = sum(yaws.values()) / len(yaws)              # + : the eyes sit left: the headset is right
+        if len(offsets) == 2:
+            terms["ipd"] = (yaws[1] - yaws[0]) / 2                  # + : the eyes are wider apart than the lenses
+            terms["tilt"] = (pitches[1] - pitches[0]) / 2           # + : the left eye sits higher: its side is low
+    t = terms
+    if abs(t.get("up", 0)) > ADJUST_DEG:
+        hints.append(_hint("up" if t["up"] > 0 else "down", t["up"]))
+    if abs(t.get("left", 0)) > ADJUST_DEG:
+        hints.append(_hint("left" if t["left"] > 0 else "right", t["left"]))
+    ipd_advice = ref is not None and ref.cameras_follow_lenses is True
+    if ipd_advice and abs(t.get("ipd", 0)) > ADJUST_DEG:
+        hints.append(_hint("ipd-wider" if t["ipd"] > 0 else "ipd-narrower", t["ipd"]))
+    if abs(t.get("tilt", 0)) > TILT_DEG:
+        hints.append(_hint("tilt-left-up" if t["tilt"] > 0 else "tilt-right-up", t["tilt"]))
+    reliefs = [e["relief"] for e in eyes.values() if "relief" in e]
+    if reliefs:
+        rel = sum(reliefs) / len(reliefs)
+        terms["relief"] = rel
+        if abs(rel) > RELIEF_TOL:
+            hints.append({"key": "nearer" if rel > 0 else "further", "text": HINTS["nearer" if rel > 0 else "further"],
+                          "amount": rel})
+    if baseline and ipd and baseline.get("ipd") and abs(ipd - baseline["ipd"]) >= 0.001:
+        hints.insert(0, {"key": "ipd-changed", "amount": ipd - baseline["ipd"],
+                         "text": HINTS["ipd-changed"].format(was=baseline["ipd"] * 1000, now=ipd * 1000)})
+    # The state, with hysteresis on the position terms.
+    big = max([abs(t.get(k, 0.0)) for k in ("up", "left")] + ([abs(t.get("ipd", 0.0))] if ipd_advice else []), default=0.0)
+    if any(e["state"] == "waiting" for e in eyes.values()) or not eyes:
+        state = "waiting"
+    elif lost:
+        state = "lost"
+    elif not offsets:
+        state = "waiting"
+    elif big > ADJUST_DEG or abs(t.get("tilt", 0)) > TILT_DEG or abs(t.get("relief", 0)) > RELIEF_TOL:
+        state = "adjust"
+    elif big <= OK_DEG:
+        state = "ok"
+    else:
+        state = (previous or {}).get("state") if (previous or {}).get("state") in ("ok", "adjust") else "adjust"
+    if state == "adjust" and not hints:
+        # Between OK_DEG and ADJUST_DEG, and not OK before: name the biggest term.
+        names = {"up": ("up", "down"), "left": ("left", "right")}
+        if ipd_advice:
+            names["ipd"] = ("ipd-wider", "ipd-narrower")
+        term = max((k for k in names if k in t), key=lambda k: abs(t[k]), default=None)
+        if term:
+            hints.append(_hint(names[term][0] if t[term] > 0 else names[term][1], t[term]))
+    hints.sort(key=lambda h: (h["key"] not in ("ipd-changed", "lost"), -abs(h["amount"])))
+    return {"state": state, "eyes": eyes, "terms": {k: round(v, 3) for k, v in terms.items()}, "hints": hints,
+            "provisional": bool(ref is None or ref.provisional), "against": "baseline" if centres else "reference",
+            "ipd": ipd, "ipd_advice": ipd_advice}
+
+
+def _hint(key, amount):
+    return {"key": key, "text": HINTS[key], "amount": amount}
+
+
+def moved(reading):
+    """Put-on check: the headset sits more than MOVED_DEG from the baseline (or changed IPD)."""
+    t = reading.get("terms", {})
+    return reading.get("against") == "baseline" and (
+        max(abs(t.get("up", 0.0)), abs(t.get("left", 0.0))) > MOVED_DEG or
+        any(h["key"] == "ipd-changed" for h in reading.get("hints", [])))
+
+
+def position_hint(terms, limit=MOVED_DEG):
+    """The biggest position term (up, left) past limit as a hint ("up", "down", "left" or
+    "right"), or None."""
+    names = {"up": ("up", "down"), "left": ("left", "right")}
+    term = max((k for k in names if abs(terms.get(k, 0.0)) > limit), key=lambda k: abs(terms[k]), default=None)
+    if term is None:
+        return None
+    return _hint(names[term][0] if terms[term] > 0 else names[term][1], terms[term])
+
+
+def slip_terms(jumps, ref):
+    """ft-eyes' glint jumps ({camera: (dx, dy)}, pixels: how far each pupil moved in its image
+    since the last click, the headset having slipped) as the coach's position terms, {"up",
+    "left"} in degrees, through the reference's Jacobian as assess() does; {} without any."""
+    offsets = [ref.degrees(cam, dx, dy) for cam, (dx, dy) in jumps.items() if ref and cam in ref.eyes]
+    if not offsets:
+        return {}
+    return {"up": sum(o[1] for o in offsets) / len(offsets), "left": sum(o[0] for o in offsets) / len(offsets)}
+
+
+# ------------------------------------------------------------------------------------------
+# Live finds from the shared frames (numpy and OpenCV: the dev container, or a venv with them)
+
+class LiveFinder:
+    """Runs the pupil finder on each camera's newest frame, at most rate times a second, and adds
+    the finds to obs. About 0.4 ms a frame while it follows the pupil, 2 ms when it searches."""
+
+    def __init__(self, obs, rate=30.0, share_path=None, want_path=None, log=None):
+        sys.path.insert(0, os.path.join(HERE, "tracker"))
+        import eyes_pupil   # noqa: E402  (numpy, OpenCV)
+        import eyes_share   # noqa: E402
+        import numpy as np  # noqa: E402
+        self._pupil, self._share_mod, self._np = eyes_pupil, eyes_share, np
+        self.obs, self.period = obs, 1.0 / rate
+        self.share_path, self.want_path = share_path, want_path
+        self.log = log or (lambda line: None)
+        self.error = None
+        self._stop = threading.Event()
+        self._thread = threading.Thread(target=self._run, name="fitcoach-finder", daemon=True)
+
+    def start(self):
+        self._thread.start()
+        return self
+
+    def stop(self):
+        self._stop.set()
+        self._thread.join(2)
+
+    def _run(self):
+        share = self._share_mod.open_share(10, self.share_path, self.want_path)
+        if share is None:
+            self.error = "no eye camera frames: is frametop-eyegrab.service running?"
+            self.log("fit coach: " + self.error)
+            return
+        last, near, touched = {0: -1, 1: -1}, {0: None, 1: None}, 0.0
+        np = self._np
+        while not self._stop.is_set():
+            t0 = time.monotonic()
+            for cam in (0, 1):
+                n = share.count(cam)
+                if n == last[cam] or n == 0:
+                    continue
+                last[cam] = n
+                f = share.frame(cam, n - 1)
+                if f is None:
+                    continue
+                img = np.frombuffer(f[2], np.uint8).reshape(self._share_mod.H, self._share_mod.W)
+                p = self._pupil.find_pupil(img, near[cam])
+                near[cam] = p
+                self.obs.add(cam, f[0], p)
+            if t0 - touched >= 1.0:
+                touched = t0
+                self._share_mod.touch_want(self.want_path)
+                if share.replaced():
+                    share.close()
+                    share = self._share_mod.open_share(10, self.share_path, self.want_path)
+                    if share is None:
+                        self.error = "the eye camera frames stopped"
+                        return
+            self._stop.wait(max(0.0, self.period - (time.monotonic() - t0)))
+        share.close()
+
+
+# ------------------------------------------------------------------------------------------
+
+def _print_reading(r):
+    eyes = "; ".join("%s %s found %.0f%%%s" % (e["name"], e["state"], 100 * e["found"],
+                                               " off %+.1f,%+.1f deg" % tuple(e["offset_deg"]) if "offset_deg" in e else "")
+                     for _c, e in sorted(r["eyes"].items()))
+    terms = " ".join("%s %+.1f" % kv for kv in sorted(r["terms"].items()))
+    print("%-7s %s | %s%s" % (r["state"], eyes, terms, " (provisional)" if r["provisional"] else ""))
+    for h in r["hints"]:
+        print("        -> " + h["text"])
+
+
+def main(argv):
+    if not argv or argv[0] in ("-h", "--help"):
+        sys.exit(__doc__)
+    if argv[0] == "reference":
+        paths = argv[1:] or [DOTS]
+        rows = []
+        for p in paths:
+            with open(p) as f:
+                rows += [json.loads(line) for line in f if line.strip()]
+        ref = Reference.from_dots(rows, source="calibration dots: " + ", ".join(os.path.basename(p) for p in paths))
+        print(json.dumps(ref.to_json(), indent=1))
+        return
+    if argv[0] == "live":
+        ref = Reference.load()
+        baseline = load_baseline()
+        obs = Observations()
+        finder = LiveFinder(obs, log=print).start()
+        last = None
+        try:
+            while finder.error is None:
+                time.sleep(0.5)
+                last = assess(obs.summary(), ref, baseline=baseline, previous=last)
+                _print_reading(last)
+        except KeyboardInterrupt:
+            pass
+        finder.stop()
+        if "--baseline" in argv and last and last["state"] in ("ok", "adjust"):
+            _write_json(BASELINE, make_baseline(obs.summary()))
+            print("saved the baseline: " + BASELINE)
+        return
+    sys.exit(__doc__)
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

gaze/ft-gaze.cpp

@@ -14,10 +14,12 @@
 // about 1.5 KB, 130 KB a second through podman's relay.
 //
 //   {"t":<sample time, CLOCK_MONOTONIC_RAW s>,"age":<ms old when read>,"n":<sample counter>,
-//    "head":{"yaw":..,"pitch":..,"hit":HIT},          head forward ray (for head nudging)
+//    "head":{"yaw":..,"pitch":..,"q":Q,"hit":HIT},    head forward ray (for head nudging)
 //    "src":{"action":SRC,"mmap1":SRC,"mmap2":SRC,"left":SRC,"right":SRC,"own":SRC},"eye":EYE}
-//   SRC = {"hy":..,"hp":..,"hit":HIT} or {"ok":0}    hy/hp: gaze direction relative to the
-//                                                     head, degrees (yaw +left, pitch +up)
+//   SRC = {"hy":..,"hp":..,"hq":Q,"hit":HIT} or {"ok":0}  hy/hp: gaze direction relative to
+//                                                     the head, degrees (yaw +left, pitch +up);
+//                                                     hq: the head pose it's relative to (the
+//                                                     one at the source's own sample time)
 //         mmap1 adds "open":[l,r] (probably eye openness, 0 in a blink) and "dist" (vergence
 //         distance, m); both mmap sets add "lr", the angle between the eyes (deg), which
 //         jumps when the tracker loses an eye, and "eyes":[[hy,hp],[hy,hp]], each eye's own
@@ -39,6 +41,10 @@
 //         variance (about 2e-5 on a clear view of the eye, rising as the lid or lashes get
 //         in the way), "new" whether it changed since the last sample (it freezes while the
 //         tracker can't see that eye, and in blinks). "eye" is null without the mmap.
+//   Q = [w,x,y,z]   a head pose's rotation as a unit quaternion (standing universe), so a
+//         head-relative direction can be turned into the room and back: the gaze service's
+//         fixation lock holds the gaze still in the room, where a look at a screen stays put
+//         while the head turns (gazecal.GazeLock).
 //   HIT = {"s":<screen>,"x":..,"y":..,"j":[dx/dhy,dy/dhy,dx/dhp,dy/dhp],"dpp":<deg per px>}
 //         or null. x, y are pixels on that screen; j is pixels per degree of head-relative
 //         yaw and pitch there, so a correction in degrees can be turned into pixels and back.
@@ -98,10 +104,11 @@ double NowRaw() {
 }
 
 // --- eye-server.mmap (packed, unaligned: read with memcpy) ---
-// The SteamOS 0.4.x beta (SteamVR 2.18.2) moved every field from the timestamp on by 5
-// bytes (measured 2026-10-04 with the ftdiag scan: timestamp 0x157 -> 0x15c, the vectors
-// moved with it; the counter at 0x38 kept its place). Which layout is live is detected at
-// runtime (EyeFile::Detect), so one binary serves both generations.
+// SteamOS 0.4 (SteamVR 2.18.2; first on the 0.4.3 beta, the same on 0.4.5, the release)
+// moved every field from the timestamp on by 5 bytes (measured 2026-10-04 with the ftdiag
+// scan: timestamp 0x157 -> 0x15c, the vectors moved with it; the counter at 0x38 kept its
+// place). Which layout is live is detected at runtime (EyeFile::Detect), so one binary
+// serves both generations.
 constexpr size_t kCounter = 0x38;  // u32, one per sample
 constexpr size_t kTime = 0x157;    // f64, CLOCK_MONOTONIC_RAW seconds
 constexpr size_t kLeft1 = 0x15f, kRight1 = 0x16b;  // set 1: unit vectors, head space
@@ -115,11 +122,11 @@ constexpr size_t kVar1 = 0x177, kVar2 = 0x1b3;
 // x, y, right x, y). An eye's pair stops changing while the tracker can't see it.
 constexpr size_t kMeas = 0x1d3;
 constexpr size_t kNeed = 0x1f3 + 5;  // enough for either layout
-constexpr size_t kShifts[] = {0, 5};  // the layouts EyeFile::Detect knows: stable, 0.4.x beta
+constexpr size_t kShifts[] = {0, 5};  // the layouts EyeFile::Detect knows: SteamOS 0.3, 0.4
 
 struct EyeFile {
-    // Everything from the timestamp on is read at base + shift: 0 on stable, 5 on the
-    // 0.4.x beta (see the constants above). known once Detect has seen that layout's
+    // Everything from the timestamp on is read at base + shift: 0 on SteamOS 0.3, 5 on 0.4
+    // (see the constants above). known once Detect has seen that layout's
     // timestamp tick; before that, reading would yield garbage that still passes
     // ReadSample's check.
     size_t shift = 0;
@@ -467,13 +474,38 @@ std::string HitJson(const std::vector<Screen> &screens, const vr::HmdMatrix34_t
     return buf;
 }
 
+// Q: a pose's rotation as a unit quaternion [w,x,y,z] (v' = R v, R the pose's 3x3 part).
+std::string QuatJson(const vr::HmdMatrix34_t &p) {
+    const auto &m = p.m;
+    double w, x, y, z;
+    const double tr = m[0][0] + m[1][1] + m[2][2];
+    if (tr > 0) {
+        const double s = 2 * std::sqrt(tr + 1);  // 4w
+        w = s / 4, x = (m[2][1] - m[1][2]) / s, y = (m[0][2] - m[2][0]) / s, z = (m[1][0] - m[0][1]) / s;
+    } else if (m[0][0] > m[1][1] && m[0][0] > m[2][2]) {
+        const double s = 2 * std::sqrt(1 + m[0][0] - m[1][1] - m[2][2]);  // 4x
+        w = (m[2][1] - m[1][2]) / s, x = s / 4, y = (m[0][1] + m[1][0]) / s, z = (m[0][2] + m[2][0]) / s;
+    } else if (m[1][1] > m[2][2]) {
+        const double s = 2 * std::sqrt(1 + m[1][1] - m[0][0] - m[2][2]);  // 4y
+        w = (m[0][2] - m[2][0]) / s, x = (m[0][1] + m[1][0]) / s, y = s / 4, z = (m[1][2] + m[2][1]) / s;
+    } else {
+        const double s = 2 * std::sqrt(1 + m[2][2] - m[0][0] - m[1][1]);  // 4z
+        w = (m[1][0] - m[0][1]) / s, x = (m[0][2] + m[2][0]) / s, y = (m[1][2] + m[2][1]) / s, z = s / 4;
+    }
+    const double n = std::sqrt(w * w + x * x + y * y + z * z);
+    if (n < 1e-9) return "[1,0,0,0]";
+    char buf[80];
+    std::snprintf(buf, sizeof buf, "[%.6f,%.6f,%.6f,%.6f]", w / n, x / n, y / n, z / n);
+    return buf;
+}
+
 std::string SrcJson(const std::vector<Screen> &screens, const vr::HmdMatrix34_t &head, Vec3 dHead,
                     const std::string &extra = "") {
     double yaw, pitch;
     Angles(Normalize(dHead), yaw, pitch);
     char buf[96];
     std::snprintf(buf, sizeof buf, "{\"hy\":%.4f,\"hp\":%.4f,", yaw, pitch);
-    return buf + extra + "\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
+    return buf + extra + "\"hq\":" + QuatJson(head) + ",\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
 }
 
 // Sources (--sources, "sources LIST" on stdin): a bit each.
@@ -673,7 +705,7 @@ int main(int argc, char **argv) {
         if (haveMmap && !eyes.known && (eyes.Detecting() || (writing && now >= nextLayoutCheck))) {
             const EyeFile::Detection d = eyes.Detect(now);
             if (d == EyeFile::kFound) {
-                std::fprintf(stderr, "ft-gaze: eye-server.mmap layout: %s\n", eyes.shift ? "beta (+5)" : "stable");
+                std::fprintf(stderr, "ft-gaze: eye-server.mmap layout: %s\n", eyes.shift ? "SteamOS 0.4 (+5)" : "SteamOS 0.3");
             } else if (d == EyeFile::kNone) {
                 nextLayoutCheck = now + 1.0;
                 if (!missSince) missSince = now;
@@ -797,11 +829,12 @@ int main(int argc, char **argv) {
             const Vec3 f = Rotate(headNow, {0, 0, -1});
             yaw = std::atan2(-f.x, -f.z) * 180 / M_PI;
             pitch = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
-            std::printf("{\"t\":%.5f,\"age\":%.1f,\"n\":%u,\"head\":{\"yaw\":%.4f,\"pitch\":%.4f,\"hit\":%s},"
+            std::printf("{\"t\":%.5f,\"age\":%.1f,\"n\":%u,\"head\":{\"yaw\":%.4f,\"pitch\":%.4f,\"q\":%s,\"hit\":%s},"
                         "\"src\":{\"action\":%s,\"mmap1\":%s,\"mmap2\":%s,\"left\":%s,\"right\":%s,\"own\":%s},"
                         "\"eye\":%s}\n",
-                        s.t, (now - s.t) * 1000, s.n, yaw, pitch, HitJson(list, headNow, 0, 0).c_str(), action.c_str(),
-                        m1.c_str(), m2.c_str(), left.c_str(), right.c_str(), own.c_str(), eye.c_str());
+                        s.t, (now - s.t) * 1000, s.n, yaw, pitch, QuatJson(headNow).c_str(),
+                        HitJson(list, headNow, 0, 0).c_str(), action.c_str(), m1.c_str(), m2.c_str(), left.c_str(),
+                        right.c_str(), own.c_str(), eye.c_str());
             if (std::fflush(stdout) != 0) break;  // the reader went away
         }
 

gaze/ft-gazed

@@ -10,9 +10,15 @@ again when the file changes:
                            grabber, and ft-eyes' Python in this checkout), else SteamVR's; it
                            switches when ours is installed or removed.
   GAZE_EYE=auto|left|right the eye bias (gazecal.EyeWeights): auto weights each eye by how far
-                           off it was at your recent nudges; left or right counts that eye twice
-                           as much as the other. Either eye alone carries the gaze when the
-                           other isn't seen.
+                           off it was at your recent nudges (since our tracker's last
+                           calibration: a new one starts them over); left or right counts that
+                           eye twice as much as the other. Either eye alone carries the gaze
+                           when the other isn't seen.
+  GAZE_FUSION=off|on       with our tracker: SteamVR's gaze too (gazecal.Fusion, fusion.json).
+                           SteamVR's two eyes' mean, corrected by an affine fitted at our
+                           calibration's dots, is mixed half and half with ours where both our
+                           eyes are found, and used alone where ours has one or none. off (the
+                           default) until the bench says so (gaze/bench/compare.py's fusion).
 
 Runs ft-gaze (in the dev container), and for every eye tracker sample (90 Hz):
 
@@ -27,8 +33,15 @@ Runs ft-gaze (in the dev container), and for every eye tracker sample (90 Hz):
      ft-gazeprobe (calibration.json, reloaded when the probe changes it) plus what the
      pointer's corrections have taught that eye since (LiveCorrection, saved in
      pointer-lessons.json), then weighted by the eye bias. A lost or closed eye drops out.
-     Our own tracker keeps its own calibration, so its eyes are used as they come;
-  3. smooths it with a fixation lock (the running mean of the current fixation, 1 degree);
+     Our own tracker keeps its own calibration, so its eyes are used as they come, and with
+     GAZE_FUSION=on (and a fusion.json) mixed with SteamVR's (above). When one of its eyes
+     drops out or comes back, gazecal.EyeBridge eases the step between the two eyes' gaze
+     and the one eye's over about 0.3 s, instead of a jump of about a degree. SteamVR's eyes count
+     only when both are open and found, and not while SteamVR tracks one eye only;
+  3. smooths it with a fixation lock (the running mean of the current fixation, 1 degree), on
+     where the gaze points in the room, so a look at a screen stays put while the head turns
+     (gazecal.GazeLock: ft-gaze gives each sample's head pose). With our tracker, an eye whose
+     lids came down stays out for BLINK_SETTLE after they're up again;
   4. sends it to the pointer helper: "gz <yaw> <pitch> <raw yaw> <raw pitch>", head-relative
      degrees (yaw +left, pitch +up). The helper uses it only in gaze mode.
 
@@ -65,7 +78,10 @@ relearn the offset.
 Our own tracker (gaze/tracker/ft-eyes) runs here too, in the dev container, while
 GAZE_TRACKER=own and the gaze is in use (below), or the gaze probe asks for it ("eyes SECONDS",
 a lease the probe renews). It reads the eye-camera frames the root service frametop-eyegrab
-copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs.
+copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs. Which eye net it runs
+(EYES_MODEL: models/catalog.py) is read at its start, so ft-eyes restarts when that changes; starting
+it also runs `models/ft-models auto eyes` in the background (the catalog from GitHub, and the net
+EYES_MODEL needs if it isn't downloaded), and ft-eyes moves to a net that brings.
 
 Idle: ft-gaze and our own tracker run only while the gaze is in use: gaze mode on with someone
 wearing the headset (the pointer helper says both: "gaze ? headset" -> "ok on|off worn|away"), a
@@ -77,7 +93,8 @@ tracker's next click after it starts again re-seats it, as after the headset was
 gaze mode on after a while opens the quick check.
 
 ft-gaze prints only the sources this uses ("sources LIST" on its stdin, see wanted_sources):
-own and mmap1 with our tracker, left, right and mmap1 with SteamVR's eyes, and every source
+own and mmap1 with our tracker (and left, right with the fusion), left, right and mmap1 with
+SteamVR's eyes, and every source
 while a check or the calibration runs or is asked for, since those record them all. So
 SteamVR's gaze action, the one source that calls into vrserver, is read only then (or with
 --source action).
@@ -95,7 +112,7 @@ Control socket: abstract unix datagram "@ft_gazed":
                                    POINTER_GAZE_NUDGE_MAX, so the quick check (gazecheck.py)
   status                           reply: one JSON object
   forget                           drop what the lessons taught (the calibration stays)
-  reload                           read calibration.json and the settings again
+  reload                           read calibration.json, fusion.json and the settings again
   eyes <seconds>                   keep our own tracker running that much longer (at most 120),
                                    whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
   wake <seconds>                   keep the gaze running (not idle) that much longer (at most 120),
@@ -106,7 +123,8 @@ Control socket: abstract unix datagram "@ft_gazed":
   fivecheck                        the five-dot check now (it otherwise follows a quick check
                                    that didn't fix the tracker)
   calaccept | calquit              from the pointer helper while the panel is up: take this dot
-                                   now (a left click, Meta+J) | close it (a right click, Meta+K)
+                                   now (a left click, Enter, a pinch, Meta+J, or a gaze
+                                   precision or gaze drag press) | close it (a right click, Meta+K)
 
 Options: --source action|mmap1|mmap2 (the older one-source path with that source, whatever
 the settings say; set 2 was a little quieter in the probe, but loses the pointer whenever
@@ -130,10 +148,13 @@ from collections import deque
 from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).resolve().parent))
-from gazecal import (DEFAULT_MODEL, EYE_FOUND, EYE_LOST, MODELS, STATE, Correction, EyeFallback,  # noqa: E402
-                     EyeWeights, Fixation, LiveCorrection, SteamEyeLog)
+from gazecal import (DEFAULT_MODEL, EYE_FOUND, EYE_LOST, FUSION, MODELS, STATE, Correction, EyeBridge,  # noqa: E402
+                     EyeFallback, EyeWeights, Fusion, GazeLock, LiveCorrection, SteamEyeLog, TrackedEye)
 from gazecheck import Checks  # noqa: E402
 
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "models"))
+import catalog  # noqa: E402  (which eye net EYES_MODEL means: models/catalog.py)
+
 REPO = Path(__file__).resolve().parents[1]
 HELPER = REPO / "gaze" / "build" / "ft-gaze"
 IN_BOX = REPO / "scripts" / "in-box"  # runs a program in the container it was built in
@@ -159,11 +180,14 @@ LOOK = 0.3        # seconds of samples before that moment make the look (the pro
 RETRY = 3.0       # seconds before starting ft-gaze again
 EYES_RETRY = 10.0  # seconds before starting ft-eyes again after it stopped on its own
 EYES_LEASE_MAX = 120.0
+MODELS_CLI = REPO / "models" / "ft-models"  # "auto eyes": the eye catalog, and the net EYES_MODEL needs
 SETTLE = 0.3      # seconds after an eye is found again before the fallback learns from it
 IDLE_AFTER = 30.0  # seconds after the gaze was last in use before ft-gaze and our tracker stop
 WAKE_SETTLE = 15.0  # seconds after waking that the tracker may take to start sending
 WAKE_MAX = 120.0
 KEYBOARD_PITCH = -20.0  # degrees: gaze under this, on no screen, is a look at the keyboard
+BLINK_SETTLE = 0.1  # seconds our tracker's eye stays out after its lids came down: the pupil the
+                    # lid cuts finds wrong for a few frames past SteamVR's openness (2026-10-17)
 
 
 class PointerLessons(LiveCorrection):
@@ -182,9 +206,15 @@ def log(msg):
     print(f"ft-gazed: {msg}", file=sys.stderr, flush=True)
 
 
+def venv_built(python):
+    """Is a build/venv's python there? It's for the container: in a release its link points at the
+    image's Python (/usr/bin/python3.14), which the host doesn't have, so the link itself counts."""
+    return os.path.lexists(python)
+
+
 def own_installed():
     """Is our own tracker installed: its frame grabber, and ft-eyes' Python here?"""
-    return all(p.exists() for p in EYEGRAB) and EYES_PYTHON.exists()
+    return all(p.exists() for p in EYEGRAB) and venv_built(EYES_PYTHON)
 
 
 def read_settings():
@@ -210,6 +240,18 @@ def read_settings():
     return tracker, setting, bias if bias in BIASES else "auto", nudge
 
 
+def read_fusion():
+    """GAZE_FUSION from frametop.conf: True for on (off, the default, for anything else)."""
+    try:
+        for line in CONF.read_text().splitlines():
+            k, _, v = line.split("#", 1)[0].partition("=")
+            if k.strip() == "GAZE_FUSION":
+                return v.strip().lower() == "on"
+    except OSError:
+        pass
+    return False
+
+
 def mtime(path):
     try:
         return path.stat().st_mtime
@@ -218,6 +260,8 @@ def mtime(path):
 
 
 class Service:
+    fusion_on, fusion = False, None   # (set in __init__; here too for tests that build one by parts)
+
     def __init__(self, source, verbose, to=POINTER):
         self.override, self.verbose, self.to = source, verbose, to
         self.source = source or "mmap1"  # the older path's source
@@ -229,20 +273,30 @@ class Service:
         self.cal_mtime = None
         self.lives = {name: PointerLessons() for name in SOURCES}
         self.weights = {t: EyeWeights(self.bias) for t in TRACKERS}
+        self.own_made = None  # when our tracker's calibration was made, as its misses knew it
         self.dirty = False
+        self.fusion_on = read_fusion()
+        self.fusion, self.fusion_mtime = None, None
+        self.load_fusion()
         self.load_calibration()
         self.load_lessons()
         self.steam = SteamEyeLog()
         self.steam.poll()
         self.refit()
-        self.fix = Fixation(radius=1.0)
+        self.fix = GazeLock(radius=1.0)
         self.opens = (deque(maxlen=90), deque(maxlen=90))  # left, right
+        self.low_until = [0.0, 0.0]  # sample time until which each eye counts as closed (BLINK_SETTLE)
+        self.bridge = EyeBridge()    # our tracker: no jump when an eye drops out or comes back
         self.vergence = deque(maxlen=90)
         self.fallback = EyeFallback()
+        # SteamVR's tracker following one eye only (gazecal.tracked_eye): 0 left, 1 right, or None.
+        self.tracked = TrackedEye()
+        self.one_eye = self.tracked()
         self.lost = [False, False]
         self.bad_at = [0.0, 0.0]    # sample time an eye was last lost or closed
         self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "one_eye_used": 0, "lost_left": 0,
-                       "lost_right": 0, "looking_down": 0, "dropped": 0, "lessons_taken": 0, "refused": 0}
+                       "lost_right": 0, "looking_down": 0, "dropped": 0, "lessons_taken": 0, "refused": 0,
+                       "fused": 0, "steam_only": 0}
         self.last_sample = 0.0
         self.last = None
         self.last_kind = None
@@ -253,6 +307,8 @@ class Service:
         self.eyes_proc = None  # ft-eyes, while it runs
         self.eyes_until = 0.0  # the probe's lease (monotonic time)
         self.eyes_restart_at = 0.0
+        self.eyes_model = self.eyes_model_now()  # what EYES_MODEL means now: ft-eyes restarts when it changes
+        self.models_proc = None  # ft-models auto eyes, while it runs
         self.awake = False      # ft-gaze (and our tracker) run: the gaze is in use (see the top)
         self.idle_at = 0.0      # idle from then, unless it's in use again first
         self.woke_at = 0.0
@@ -283,23 +339,78 @@ class Service:
             return "source"
         if self.tracker == "own":
             return "own"
-        return "eyes" if all(self.models[e].samples for e in SIDES) else "source"
+        # With one eye tracked, that eye's own calibration is enough.
+        sides = SIDES if self.one_eye is None else (SIDES[self.one_eye],)
+        return "eyes" if all(self.models[e].samples for e in sides) else "source"
+
+    @property
+    def fusing(self):
+        """Our tracker's gaze mixed with SteamVR's (GAZE_FUSION=on, a fitted fusion.json)."""
+        return self.kind == "own" and self.fusion_on and self.fusion is not None and self.fusion.ready()
 
     # --- Settings, calibration and lessons ---
 
+    @staticmethod
+    def eyes_model_now():
+        """(EYES_MODEL, the folder it resolves to): ft-eyes reads it at start."""
+        try:
+            r = catalog.resolve("eyes")
+            return r["setting"], r["folder"]
+        except Exception as e:  # noqa: BLE001 (a broken catalog: ft-eyes falls back to the pupil finder)
+            return f"error {e}", None
+
+    def check_eyes_model(self):
+        now = self.eyes_model_now()
+        if now != self.eyes_model:
+            log(f"eye model {now[0]}" + (f" ({now[1]})" if now[1] else "") + (": restarting ft-eyes" if self.eyes_proc else ""))
+            self.eyes_model = now
+            if self.eyes_proc:
+                self.stop_eyes()
+                self.eyes_restart_at = 0.0
+
+    def models_auto(self):
+        """The eye catalog from GitHub (at most every 15 minutes) and the net EYES_MODEL needs, if it's
+        missing, in the background: ft-eyes starts with what's here, and moves to the new one after."""
+        if self.models_proc and self.models_proc.poll() is None:
+            return
+        try:
+            self.models_proc = subprocess.Popen([sys.executable, str(MODELS_CLI), "auto", "eyes"], stdin=subprocess.DEVNULL,
+                                                stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                                                start_new_session=True)
+        except OSError as e:
+            log(f"couldn't check for eye models: {e}")
+
     def load_settings(self):
         self.conf_mtime = mtime(CONF)
+        self.check_eyes_model()
         tracker, self.tracker_setting, bias, self.nudge_max = read_settings()
+        fusion_on = read_fusion()
+        if fusion_on != self.fusion_on:
+            self.fusion_on = fusion_on
+            log(f"fusion {'on' if fusion_on else 'off'}" + ("" if not fusion_on or (self.fusion and self.fusion.ready())
+                                                            else f": no fitted {FUSION} yet (our tracker's calibration makes one)"))
+            self.fix.reset()
         if (tracker, bias) != (self.tracker, self.bias):
             auto = ""
             if self.tracker_setting == "auto":
                 auto = " (auto: ours is installed)" if tracker == "own" else " (auto: ours isn't installed)"
+                if tracker == "steam" and EYEGRAB[1].exists() and not EYEGRAB[0].exists():
+                    auto = (f" (auto: ours lost {EYEGRAB[0]}, which SteamOS updates delete unless it's kept;"
+                            " reinstall it with gaze/tracker/install.sh)")
             log(f"tracker {tracker}{auto}, eye bias {bias}" + (f" (--source {self.override} wins)" if self.override else ""))
             self.tracker, self.bias = tracker, bias
             for w in self.weights.values():
                 w.bias = bias
             self.fix.reset()
 
+    def load_fusion(self):
+        self.fusion_mtime = mtime(FUSION)
+        self.fusion = Fusion.load(FUSION)
+        if self.fusion:
+            log(f"fusion: SteamVR's affine from {self.fusion.info.get('dots', 0)} dots "
+                f"({self.fusion.info.get('how', '?')}, {self.fusion.info.get('made', '?')}), "
+                + ("on" if self.fusion_on else "off (GAZE_FUSION)"))
+
     def load_calibration(self):
         try:
             mt = CALIBRATION.stat().st_mtime
@@ -328,12 +439,14 @@ class Service:
         for t, misses in (d.get("misses") or {}).items():
             if t in self.weights:
                 self.weights[t] = EyeWeights(self.bias, misses)
+        self.own_made = d.get("own_made")
         log(", ".join(f"{n} {len(self.lives[n].samples)}" for n in (self.source,) + SIDES) + " lessons")
 
     def save_lessons(self):
         tmp = LESSONS.with_suffix(".tmp")
         tmp.write_text(json.dumps({"version": 2, "sources": {n: lv.samples for n, lv in self.lives.items() if lv.samples},
-                                   "misses": {t: w.misses for t, w in self.weights.items()}}))
+                                   "misses": {t: w.misses for t, w in self.weights.items()},
+                                   "own_made": self.own_made}))
         tmp.replace(LESSONS)
         self.dirty = False
 
@@ -479,7 +592,7 @@ class Service:
             return "all"
         kind = self.kind
         if kind == "own":
-            return "own,mmap1"
+            return "own,mmap1,left,right" if self.fusing else "own,mmap1"
         if kind == "eyes":
             return "left,right,mmap1"
         return ",".join(dict.fromkeys((self.source, "mmap1", "mmap2")))  # mmap2: each eye, for the fallback
@@ -536,16 +649,21 @@ class Service:
         self.proc = None
         self.mmap_unknown = False
 
+    def own_installed(self):
+        return own_installed()  # (for the checks: the fit coach needs ft-eyes)
+
     def eyes_wanted(self):
         return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
 
     def start_eyes(self):
         """Our own tracker, in the container, with build/venv's numpy and OpenCV. Like
         ft-gaze, it quits when its stdin closes."""
-        if not EYES_PYTHON.exists():
+        if not venv_built(EYES_PYTHON):
             log(f"ft-eyes isn't built: run {REPO}/gaze/tracker/build.sh")
             self.eyes_restart_at = time.monotonic() + 30
             return
+        self.models_auto()
+        self.eyes_model = self.eyes_model_now()
         env = dict(os.environ)
         env["XDG_RUNTIME_DIR"] = f"/run/user/{os.getuid()}"
         self.eyes_proc = subprocess.Popen([str(IN_BOX), str(EYES_PYTHON), str(EYES_PROG), "-v", "--watch-stdin"],
@@ -640,11 +758,17 @@ class Service:
             for k in (0, 1):
                 self.lost[k] = unc[k] > (EYE_FOUND if self.lost[k] else EYE_LOST)
             if not down:
-                self.counts["lost_left"] += self.lost[0]
-                self.counts["lost_right"] += self.lost[1]
+                self.counts["lost_left"] += self.lost[0] and self.one_eye != 1
+                self.counts["lost_right"] += self.lost[1] and self.one_eye != 0
         return low
 
     def on_sample(self, s):
+        one = self.tracked()
+        if one != self.one_eye:
+            log("SteamVR now tracks both eyes" if one is None else
+                f"SteamVR tracks the {SIDES[one]} eye only (Track Dominant Eye Only): going by that eye")
+            self.one_eye = one
+            self.fix.reset()
         self.checks.on_sample(s)
         kind = self.kind
         if kind != self.last_kind:
@@ -665,13 +789,15 @@ class Service:
             if "hy" not in src:
                 return
             eyes = [tuple(e) if e else None for e in (src.get("eyes") or [None, None])]
-            hp, hit = src["hp"], src.get("hit")
+            hp, hit, then = src["hp"], src.get("hit"), src.get("hq")
         else:
             per = [s["src"].get(name) or {} for name in SIDES]
             eyes = [(p["hy"], p["hp"]) if "hy" in p else None for p in per]
             if not any(eyes):
                 return
-            hp, hit = next(e[1] for e in eyes if e), m1.get("hit")
+            one = self.one_eye
+            hp, hit = (eyes[one] if one is not None and eyes[one] else next(e for e in eyes if e))[1], m1.get("hit")
+            then = next((p["hq"] for p in per if "hq" in p), None)
         self.counts["samples"] += 1
         self.last_sample = time.monotonic()
         down = hp < KEYBOARD_PITCH and not hit
@@ -679,9 +805,26 @@ class Service:
         if down:
             self.counts["looking_down"] += 1
             return
+        if own and self.one_eye is not None:
+            # SteamVR judges only that eye's openness, and a blink closes both: ours still
+            # sees the other, so it stays in.
+            low[1 - self.one_eye] = low[self.one_eye]
+        if own:
+            for k in (0, 1):
+                if low[k]:
+                    self.low_until[k] = s["t"] + BLINK_SETTLE
+                elif s["t"] < self.low_until[k]:
+                    low[k] = True
         # Our tracker finds the pupils itself; SteamVR's openness still marks the blinks.
         bad = [eyes[k] is None or low[k] or (not own and self.lost[k]) for k in (0, 1)]
-        if all(bad):
+        if not own and self.one_eye is not None:
+            bad[1 - self.one_eye] = True  # SteamVR ignores that eye
+        steam = None
+        if own and self.fusing and self.one_eye is None:
+            per = [s["src"].get(name) or {} for name in SIDES]
+            steam = self.fusion.steam_gaze(*[(p["hy"], p["hp"]) if "hy" in p and not low[k] and not self.lost[k]
+                                             else None for k, p in enumerate(per)])
+        if all(bad) and steam is None:
             self.counts["blinks"] += 1
             return
         if any(bad):
@@ -695,8 +838,15 @@ class Service:
             for name, e in zip(SIDES, seen):
                 c = self.correction(name, *e) if e else None
                 corrected.append((e[0] + c[0], e[1] + c[1]) if e else None)
-        gy, gp = self.weights["own" if own else "steam"].combine(corrected)
-        fy, fp = self.fix(gy, gp, s["t"], 1.0)
+        g = self.weights["own" if own else "steam"].combine(corrected)
+        if own:
+            g = self.bridge(s["t"], seen, g)
+        if steam is not None:
+            n = sum(e is not None for e in seen)
+            self.counts["fused" if n == 2 else "steam_only"] += 1
+            g = self.fusion.combine(g, n, steam)
+        gy, gp = g
+        fy, fp = self.fix(gy, gp, s["t"], then, (s.get("head") or {}).get("q"))
         self.send(s["t"], fy, fp, fy, fp, seen)
 
     def on_source_sample(self, s):
@@ -715,11 +865,27 @@ class Service:
                 self.bad_at[k] = s["t"]  # the fallback doesn't learn from these either
             return
         bad = [low[k] or self.lost[k] for k in (0, 1)]
+        hy, hp = src["hy"], src["hp"]
+        eyes = (s["src"].get("mmap2") or {}).get("eyes")
+        one = self.one_eye
+        if one is not None:
+            # SteamVR's combined gaze already follows that eye alone; set 2's averages the
+            # ignored one in, so mmap2 takes the eye's own reading. No fallback to learn.
+            if bad[one]:
+                self.counts["blinks"] += 1
+                return
+            if self.source == "mmap2":
+                if not eyes:
+                    self.counts["dropped"] += 1
+                    return
+                hy, hp = eyes[one]
+            fy, fp = self.fix(hy, hp, s["t"], src.get("hq"), (s.get("head") or {}).get("q"))
+            cy, cp = self.correction(self.source, fy, fp)
+            self.send(s["t"], fy + cy, fp + cp, fy, fp, None)
+            return
         if all(bad):
             self.counts["blinks"] += 1
             return
-        hy, hp = src["hy"], src["hp"]
-        eyes = (s["src"].get("mmap2") or {}).get("eyes")
         for k in (0, 1):
             if bad[k]:
                 self.bad_at[k] = s["t"]
@@ -747,8 +913,8 @@ class Service:
             if eyes and s["t"] - max(self.bad_at) > SETTLE:
                 for k in (0, 1):
                     self.fallback.update(k, eyes[k][0], eyes[k][1], hy, hp)
-        # The fixation lock works in degrees here (1 degree per "pixel").
-        fy, fp = self.fix(hy, hp, s["t"], 1.0)
+        # The fixation lock works in degrees here, on the gaze's direction in the room.
+        fy, fp = self.fix(hy, hp, s["t"], src.get("hq"), (s.get("head") or {}).get("q"))
         cy, cp = self.correction(self.source, fy, fp)
         self.send(s["t"], fy + cy, fp + cp, fy, fp, None)
 
@@ -810,6 +976,7 @@ class Service:
                     reply = "error bad seconds"
             elif words[:1] == ["reload"]:
                 self.load_settings()
+                self.load_fusion()
                 self.load_calibration()
                 self.refit()
                 reply = "ok"
@@ -833,9 +1000,29 @@ class Service:
                     self.own, self.own_at = json.loads(data), time.monotonic()
                 except ValueError:
                     pass
+                else:
+                    self.note_own_calibration()
             else:
                 log(f"our tracker: {data}")
 
+    def note_own_calibration(self):
+        """Our tracker's status names a calibration other than the one its eye misses were
+        measured on: they start over, and the eyes count alike until each has EyeWeights.MIN
+        again. The misses say how far each eye's old mapping was off, and a new calibration
+        replaces both mappings (it can drop an eye, or be someone else's). Live 2026-10-18 a
+        calibration's own dots fed the right eye misses of the right-only one before it, and the
+        balance after it was that one's. Whatever made the calibration: the panel, the probe."""
+        made = (self.own.get("calibration") or {}).get("made")
+        if not made or made == self.own_made:
+            return
+        w = self.weights["own"]
+        if self.own_made is not None and any(w.misses):
+            log(f"our tracker's calibration from {made}: the eye weights start over "
+                f"(were {', '.join(f'{v:.2f}' for v in w.weights())} from {', '.join(str(len(m)) for m in w.misses)} misses)")
+            self.weights["own"] = EyeWeights(self.bias)
+        self.own_made = made
+        self.dirty = True
+
     def status(self):
         kind = self.kind
         tracker = "own" if kind == "own" else "steam"
@@ -857,7 +1044,13 @@ class Service:
             cal = own.get("calibration") or {}
             st.update(calibration_samples=cal.get("dots", 0), calibration_made=cal.get("made"),
                       lessons=max(len(m) for m in w.misses), own_running=bool(own),
-                      own_reseat=any(e.get("reseat") for e in own.get("eyes", {}).values()))
+                      own_reseat=any(e.get("reseat") for e in own.get("eyes", {}).values()),
+                      size_guard=own.get("size_guard"), size_guard_counts=own.get("size_guard_counts"),
+                      own_mapped=own.get("mapped"))
+        f = self.fusion
+        st["fusion"] = {"setting": "on" if self.fusion_on else "off", "in_use": self.fusing,
+                        "fitted": bool(f and f.ready()), "dots": f.info.get("dots", 0) if f else 0,
+                        "how": f.info.get("how") if f else None, "made": f.info.get("made") if f else None}
         st.update(awake=self.awake, idle=None if self.awake else self.idle_reason())
         st.update(eyes_process=self.eyes_proc is not None, eyegrab=EYES_CAMS.exists(),
                   tracker_setting=self.tracker_setting, own_installed=own_installed())
@@ -876,9 +1069,14 @@ class Service:
         if mtime(CALIBRATION) != self.cal_mtime:
             self.load_calibration()
             self.refit()
+        if mtime(FUSION) != self.fusion_mtime:
+            self.load_fusion()
         if mtime(CONF) != self.conf_mtime or (self.tracker_setting == "auto"
                                               and own_installed() != (self.tracker == "own")):
             self.load_settings()
+        if self.models_proc and self.models_proc.poll() is not None:
+            self.models_proc = None
+            self.check_eyes_model()   # a net it fetched: ft-eyes moves to it
         self.update_awake()
         want = self.eyes_wanted()
         if want and not self.eyes_proc and time.monotonic() >= self.eyes_restart_at:
@@ -907,6 +1105,8 @@ class Service:
                     self.on_control()
                 elif key.data == "checks":
                     self.checks.on_readable()
+                elif key.data == "eyes_reply":
+                    self.checks.on_eyes_reply(key.fileobj)
                 elif key.data == "panel" and self.checks.panel_proc:
                     self.checks.read_panel()
                 elif key.data == "own":

gaze/gazecal.py

@@ -7,6 +7,7 @@ the tracker has lost the other), EyeWeights (how much each eye counts), and Stea
 reports them.
 """
 
+import json
 import math
 import os
 import statistics
@@ -97,6 +98,99 @@ class Fixation:
         return self.sum[0] / self.count, self.sum[1] / self.count
 
 
+def direction(yaw, pitch):
+    """A head-relative (yaw, pitch) in degrees as a unit vector, as ft-gaze's Direction: -Z
+    forward, positive yaw turns left (about +Y), positive pitch looks up."""
+    y, p = math.radians(yaw), math.radians(pitch)
+    return (-math.sin(y) * math.cos(p), math.sin(p), -math.cos(y) * math.cos(p))
+
+
+def angles(v):
+    """(yaw, pitch) in degrees of a direction, as ft-gaze's Angles."""
+    n = math.sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]) or 1.0
+    return math.degrees(math.atan2(-v[0], -v[2])), math.degrees(math.asin(max(-1.0, min(1.0, v[1] / n))))
+
+
+def rotate(q, v, inverse=False):
+    """v turned by the unit quaternion q = [w, x, y, z] (ft-gaze's "hq" and the head's "q": head
+    space into the room), or back into head space with inverse."""
+    w, x, y, z = q
+    if inverse:
+        x, y, z = -x, -y, -z
+    # v + 2w (u x v) + 2 u x (u x v), u = (x, y, z)
+    cx, cy, cz = y * v[2] - z * v[1], z * v[0] - x * v[2], x * v[1] - y * v[0]
+    dx, dy, dz = y * cz - z * cy, z * cx - x * cz, x * cy - y * cx
+    return (v[0] + 2 * (w * cx + dx), v[1] + 2 * (w * cy + dy), v[2] + 2 * (w * cz + dz))
+
+
+class GazeLock:
+    """The fixation lock (Fixation, above) on where the gaze points in the room.
+
+    A look at something that stays put (a screen) stays put in the room while the head turns:
+    the eyes turn the other way, so the head-relative gaze slides. Locked in head-relative
+    degrees, the pointer rode along with the head until the slide passed the radius, then
+    snapped back, a jump every few hundred milliseconds; replaying two practice captures, during
+    still looks with the head moving 5-30 degrees a second that was 4 jumps over 1 degree a
+    second, and 13 in the one second it turned faster. Locked in the room it's 1 to 2 (2026-10-17).
+
+    Each sample's gaze is turned into the room with the head pose of its own sample time
+    (`then`, ft-gaze's "hq"), locked there, and turned back into the head with the newest pose
+    (`now`, the head's "q"), the closest one to the pose the pointer helper draws it with.
+    Without poses (an older ft-gaze) it's the head-relative lock."""
+
+    def __init__(self, radius=1.0):
+        self.radius = radius
+        self.head = Fixation(radius)
+        self.reset()
+
+    def reset(self):
+        self.head.reset()
+        self.sum = [0.0, 0.0, 0.0]
+        self.count = 0
+        self.outside = []
+        self.last_t = None
+
+    def __call__(self, yaw, pitch, t, then=None, now=None):
+        """The locked gaze, head-relative (yaw, pitch) degrees, for a head-relative sample at t."""
+        if not then or not now:
+            if self.count:
+                self.reset()
+            return self.head(yaw, pitch, t, 1.0)
+        if self.last_t is None and self.head.count:
+            self.head.reset()
+        m = self.lock(rotate(then, direction(yaw, pitch)), t)
+        return angles(rotate(now, m, inverse=True))
+
+    @staticmethod
+    def mean(s):
+        n = math.sqrt(s[0] * s[0] + s[1] * s[1] + s[2] * s[2]) or 1.0
+        return (s[0] / n, s[1] / n, s[2] / n)
+
+    def lock(self, v, t):
+        """Fixation's rules on room directions (unit vectors)."""
+        if self.last_t is not None and (t <= self.last_t or t - self.last_t > 0.5):
+            self.reset()
+        self.last_t = t
+        if self.count:
+            m = self.mean(self.sum)
+            dot = max(-1.0, min(1.0, m[0] * v[0] + m[1] * v[1] + m[2] * v[2]))
+            if math.degrees(math.acos(dot)) > self.radius:
+                self.outside.append(v)
+                if len(self.outside) < 2:
+                    return m  # one stray sample: probably noise
+                self.sum = [sum(p[k] for p in self.outside) for k in range(3)]
+                self.count = len(self.outside)
+                self.outside = []
+                return self.mean(self.sum)
+        self.outside = []
+        if self.count >= 90:  # the last second or so: a slow drift still gets followed
+            self.sum = [s * 89 / 90 for s in self.sum]
+            self.count = 89
+        self.sum = [self.sum[k] + v[k] for k in range(3)]
+        self.count += 1
+        return self.mean(self.sum)
+
+
 MODELS = ["none", "offset", "affine", "affine+grid", "quadratic", "quadratic+grid"]
 DEFAULT_MODEL = "quadratic"
 
@@ -397,6 +491,99 @@ class LiveCorrection:
 EYE_LOST = 0.004
 EYE_FOUND = 0.0025
 
+# SteamOS 0.4's "Track Dominant Eye Only" (SteamVR's settings, General, advanced): SteamVR's
+# tracker ignores the other eye, for someone whose eyes don't look at the same spot. Its
+# settings: steamvr.eyeTrackingDominantEyeOnly, and steamvr.dominantEye (0 left, 1 right,
+# SteamVR's default). Frametop then goes by that eye alone: a calibration that waits for both
+# eyes would never take a dot, and the other eye's reading isn't where the person looks.
+STEAMVR_SETTINGS = (Path.home() / ".config" / "openvr" / "config" / "steamvr.vrsettings",
+                    Path.home() / ".steam" / "steam" / "config" / "steamvr.vrsettings")
+
+
+def tracked_eye(paths=STEAMVR_SETTINGS):
+    """The one eye SteamVR's tracker follows (0 left, 1 right), or None for both. The first
+    of the settings files that exists counts (SteamVR keeps only settings changed from its
+    defaults, so a missing key is the default)."""
+    for path in paths:
+        try:
+            text = Path(path).read_text()
+        except OSError:
+            continue
+        try:
+            steamvr = json.loads(text).get("steamvr")
+        except (ValueError, AttributeError):
+            return None
+        if not isinstance(steamvr, dict) or steamvr.get("eyeTrackingDominantEyeOnly") is not True:
+            return None
+        return 0 if steamvr.get("dominantEye", 1) == 0 else 1
+    return None
+
+
+class TrackedEye:
+    """tracked_eye(), read again when SteamVR's settings file changes (looked at no more than
+    every CHECK seconds), since the setting can change while a service runs."""
+
+    CHECK = 2.0
+
+    def __init__(self, paths=STEAMVR_SETTINGS):
+        self.paths = paths
+        self.eye = None
+        self.stamp = None
+        self.checked = None
+
+    def __call__(self, now=None):
+        now = time.monotonic() if now is None else now
+        if self.checked is not None and now - self.checked < self.CHECK:
+            return self.eye
+        self.checked = now
+        stamp = []
+        for path in self.paths:
+            try:
+                st = os.stat(path)
+                stamp.append((st.st_mtime_ns, st.st_size))
+            except OSError:
+                stamp.append(None)
+        if stamp != self.stamp:
+            self.stamp = stamp
+            self.eye = tracked_eye(self.paths)
+        return self.eye
+
+
+class EyeBridge:
+    """Our tracker's eyes, each with its own calibration, miss in their own ways, so the gaze
+    jumped whenever one eye dropped out or came back (a median 0.3-0.85 degrees in the 10-16
+    edges round): enough to break the fixation lock. While both are seen, this averages each
+    eye's offset from their combined gaze over about LEARN seconds; with one eye, the gaze is
+    that eye plus its offset, fading out over about FADE seconds, since the offset belongs to
+    where the eyes looked then.
+
+    `__call__` takes and gives head-relative degrees (yaw, pitch)."""
+
+    LEARN = 0.3
+    FADE = 0.3
+
+    def __init__(self):
+        self.off = [None, None]   # per eye: (yaw, pitch) from it to the combined gaze
+        self.at = None            # sample time both were last seen
+
+    def __call__(self, t, eyes, combined):
+        """The gaze at sample time t (s), from `combined`, the eye-weighted gaze of `eyes`
+        [(yaw, pitch) or None, (yaw, pitch) or None]."""
+        seen = [k for k in (0, 1) if eyes[k] is not None]
+        if len(seen) == 2:
+            a = 1.0 if self.at is None else 1.0 - math.exp(-max(t - self.at, 0.0) / self.LEARN)
+            for k in seen:
+                d = (combined[0] - eyes[k][0], combined[1] - eyes[k][1])
+                o = self.off[k]
+                self.off[k] = d if o is None else (o[0] + a * (d[0] - o[0]), o[1] + a * (d[1] - o[1]))
+            self.at = t
+            return combined
+        if len(seen) == 1 and self.at is not None and self.off[seen[0]] is not None:
+            f = math.exp(-max(t - self.at, 0.0) / self.FADE)
+            o = self.off[seen[0]]
+            return combined[0] + f * o[0], combined[1] + f * o[1]
+        return combined
+
 
 class EyeFallback:
     """The gaze from one eye, while the tracker has lost the other.
@@ -488,7 +675,10 @@ class EyeWeights:
     alike until then. Each miss is measured before its lesson teaches anything, so each is a
     fresh test. On SteamVR's own test (2026-09-29) its calibration dots said the left eye was
     the better one and new spots said the right, so the misses come from lessons, not the fit.
-    An eye that isn't seen (None) drops out, and the other carries the gaze alone."""
+    An eye that isn't seen (None) drops out, and the other carries the gaze alone. Our
+    tracker's start over with each new calibration of it (ft-gazed's note_own_calibration).
+    Not from the new fit's per-eye spread: on 2026-10-18's replays (frame-eyes
+    one_eye_replay.py) spread and equal weights came out the same, 2.29 degrees median."""
 
     LEAN = 2.0
     KEEP = 20
@@ -531,6 +721,126 @@ class EyeWeights:
         return (sum(wk * e[0] for wk, e in zip(w, seen)) / total, sum(wk * e[1] for wk, e in zip(w, seen)) / total)
 
 
+FUSION = STATE / "fusion.json"
+
+
+class Fusion:
+    """Our tracker and SteamVR's together (ft-gazed's GAZE_FUSION=on). On two Gaze Bench runs
+    (2026-10-17, 25 clicks on a head-locked panel out to 37 degrees) the two failed on
+    different looks: ours at its misfinds and with an eye lost, SteamVR with a steady bend
+    across the view. Ours was 2.01 degrees off at the median (6.47 at the 90th percentile),
+    SteamVR's two eyes' mean 2.67 (5.72). An affine correction fitted on the other clicks took
+    SteamVR's to 1.78 (4.26); ours where both its eyes were found, mixed half and half with
+    that, and SteamVR's alone where ours had one eye, to 1.51 (3.10). Fitted on the other run
+    only, 1.41 (4.59) (gaze/bench/compare.py's steamvr-affine-loo, fusion-loo, fusion-xrun).
+
+    So: SteamVR's eyes' mean, corrected by an affine fitted at the calibration's dots (gazecheck
+    records SteamVR's eyes at each of ours, and fits this after ours: fusion.json); then ours
+    mixed in at SHARE where both our eyes are found, SteamVR's alone where we have one, and ours
+    alone where SteamVR has no gaze.
+
+    click() adds an offset on top: the median of what the clicks since the headset went on said
+    was still off (SteamVR starts its eye model over then, and the error may move). ft-gazed
+    doesn't feed it yet: within one wear on those runs it made SteamVR's worse (2.15 to 2.72 at
+    the median, fitted on the other run), and across wears it's untested (compare.py's +clicks)."""
+
+    MODE = "affine"
+    RIDGE = 0.05    # Correction.fit's: on the gain and tilt, so a lopsided set of dots can't bend it far
+    SHARE = 0.5     # our tracker's share when both its eyes are found
+    MIN = 5         # dots for a fit
+    KEEP = 5        # clicks the offset on top comes from
+    STALE = 8.0     # degrees: a click this far off is a look elsewhere, not SteamVR's error
+
+    def __init__(self):
+        self.steam = Correction()
+        self.info = {}
+        self.points = []
+        self.clicks = []   # (wear time, dy, dp): what was still off at a click
+        self.wear = None   # the headset's current wear (SteamEyeLog.worn): clicks from others don't count
+
+    def ready(self):
+        return self.steam.samples >= self.MIN
+
+    def fit(self, points, info=None):
+        """points: (yaw, pitch, true yaw, true pitch), SteamVR's eyes' mean at a dot and the dot."""
+        self.points = [list(map(float, p)) for p in points]
+        self.steam.fit([(hy, hp, ty - hy, tp - hp) for hy, hp, ty, tp in self.points], self.MODE, ridge=self.RIDGE)
+        self.info = dict(info or {}, dots=len(self.points), fitted_at=time.time())
+        self.clicks = []
+
+    def misses(self):
+        """How far off the affine leaves each of the points it was fitted on (degrees)."""
+        out = []
+        for hy, hp, ty, tp in self.points:
+            dy, dp = self.steam.get(hy, hp, self.MODE)
+            out.append(math.hypot(ty - hy - dy, tp - hp - dp))
+        return out
+
+    @staticmethod
+    def mean(left, right):
+        if not left or not right:
+            return None
+        return (left[0] + right[0]) / 2, (left[1] + right[1]) / 2
+
+    def offset(self):
+        keep = [c for c in self.clicks if c[0] == self.wear][-self.KEEP:]
+        if not keep:
+            return 0.0, 0.0
+        return statistics.median(c[1] for c in keep), statistics.median(c[2] for c in keep)
+
+    def steam_gaze(self, left, right, clicks=True):
+        """SteamVR's corrected gaze from its two eyes' readings, or None without both."""
+        m = self.mean(left, right)
+        if m is None or not self.ready():
+            return None
+        dy, dp = self.steam.get(m[0], m[1], self.MODE)
+        oy, op = self.offset() if clicks else (0.0, 0.0)
+        return m[0] + dy + oy, m[1] + dp + op
+
+    def click(self, left, right, thy, thp):
+        """A look at (thy, thp) with SteamVR's eyes there: how far off the correction still was
+        joins the offset on top. Returns that miss (degrees), or None."""
+        g = self.steam_gaze(left, right, clicks=False)
+        if g is None:
+            return None
+        dy, dp = thy - g[0], thp - g[1]
+        miss = math.hypot(dy, dp)
+        if miss <= self.STALE:
+            self.clicks = (self.clicks + [(self.wear, dy, dp)])[-4 * self.KEEP:]
+        return miss
+
+    def combine(self, own, own_eyes, steam, share=None):
+        """own: ours, combined from own_eyes of its eyes (0-2); steam: steam_gaze(). Either may be None."""
+        share = self.SHARE if share is None else share
+        if steam is None:
+            return own
+        if own is None or own_eyes < 2:
+            return steam
+        return share * own[0] + (1 - share) * steam[0], share * own[1] + (1 - share) * steam[1]
+
+    def to_json(self):
+        return {"version": 1, "steam": self.steam.to_json(), "mode": self.MODE, "points": self.points, **self.info}
+
+    def from_json(self, d):
+        self.steam.from_json(d.get("steam") or {})
+        self.points = d.get("points") or []
+        self.info = {k: v for k, v in d.items() if k not in ("version", "steam", "mode", "points")}
+        return self
+
+    @classmethod
+    def load(cls, path=FUSION):
+        try:
+            return cls().from_json(json.loads(Path(path).read_text()))
+        except (OSError, ValueError):
+            return None
+
+    def save(self, path=FUSION):
+        path = Path(path)
+        tmp = path.with_suffix(".tmp")
+        tmp.write_text(json.dumps(self.to_json(), indent=1))
+        tmp.replace(path)
+
+
 class SteamEyeLog:
     """Follows SteamVR's eye tracking log (read only) for what moves the raw gaze under a
     calibration.
@@ -640,7 +950,7 @@ def cross_validate(points, mode):
     return errs
 
 
-def steady_samples(samples, vergence_jump=1.5, why=None):
+def steady_samples(samples, vergence_jump=1.5, why=None, eye=None):
     """The samples of one look at one spot where the tracker had both eyes: none in a blink
     (openness under half its median over the samples), none where it had lost an eye (its
     variance over EYE_LOST), and none where the angle between the eyes' directions (`lr`, the
@@ -648,28 +958,37 @@ def steady_samples(samples, vergence_jump=1.5, why=None):
     vergence itself depends on distance (about 2.8 degrees for a screen 1.3 m away, a
     fraction of one far off), so only a jump away from what it was during this look means
     the tracker lost an eye. Without the mmap there's nothing to judge by: all are kept.
+    `eye` (0 left, 1 right; see tracked_eye) judges that eye alone: the other one's loss,
+    openness and the vergence don't count.
     `why`, a dict, gets how many were dropped for each reason: "lost_left", "lost_right",
     "lost_both", "blink" and "vergence" (each sample once, for the first that applies)."""
     if why is None:
         why = {}
+
+    def openness(o):
+        return o[eye] if eye is not None else min(o)
     # Openness: a blink is a sharp drop from what it was during this look. Not a fixed
     # level: looking down, the upper lids come down with the eyes, and in bright light you
     # squint, so the reading can stay under 0.5 for the whole look while the tracker follows
     # the eyes fine (a calibration dot at the bottom of the bright round failed that way).
-    opens = [min(o) for o in ((smp["src"].get("mmap1") or {}).get("open") for smp in samples) if o]
+    opens = [openness(o) for o in ((smp["src"].get("mmap1") or {}).get("open") for smp in samples) if o]
     floor = max(0.12, 0.5 * statistics.median(opens)) if len(opens) >= 5 else 0.12
     seen = []
     for smp in samples:
         m1 = smp["src"].get("mmap1") or {}
         o = m1.get("open")
         lost = [u > EYE_LOST for u in m1.get("unc") or [0, 0]]
+        if eye is not None:
+            lost[1 - eye] = False
         # A lost eye's openness reads 0 too, so a lost eye is named before a blink.
         key = ("lost_both" if all(lost) else "lost_left" if lost[0] else "lost_right") if any(lost) else \
-            "blink" if o and min(o) < floor else None
+            "blink" if o and openness(o) < floor else None
         if key:
             why[key] = why.get(key, 0) + 1
         else:
             seen.append(smp)
+    if eye is not None:
+        return seen
 
     def vergence(smp):
         return (smp["src"].get("mmap1") or {}).get("lr", (smp["src"].get("mmap2") or {}).get("lr"))

gaze/gazecheck.py

@@ -30,8 +30,30 @@ directions: look at each one.
   fit     the headset fit check (on "fitcheck", Check headset fit on the Gaze page): live, a
           card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
           seen) and hints, from the gaze probe's Headset fit (gaze/fitcheck.py), while you
-          adjust the headset. A left click or Meta+J runs its guided check (dots, then looks
+          adjust the headset. A left click, Enter, a pinch or Meta+J runs its guided check (dots, then looks
           down, up, left and right); a right click or Meta+K closes it, as does FIT_TIMEOUT.
+          With our tracker installed, its first hint line is the fit coach's (gaze/fitcoach.py):
+          where the headset sits, from where ft-eyes sees each pupil and glint (ft-eyes "fit",
+          run for the check whichever tracker is in use). It's judged only while you look
+          straight ahead (COACH_CENTRE), since the pupils move with the gaze. A good reading
+          (the coach's "ok") is kept as your saved fit when the check closes (fitcoach.BASELINE,
+          with SteamVR's IPD setting from the panel): the put-on check compares with it.
+
+The fit watch (GAZE_FIT_WATCH=1 in frametop.conf, off by default until it's been tried; read
+again when the file changes) adds two things, with our tracker installed, its reference
+(gaze/fit-reference.json) and a saved fit:
+  put-on  the quick check that opens when the headset goes on first reads the fit coach while
+          you look at its dot, against your saved fit, and holds the dot meanwhile (at most
+          PUTON_READ). If the headset sits more than fitcoach.MOVED_DEG from it, or the IPD
+          setting changed, the panel widens (five's size) and says which way to move it, live,
+          until it's back; then the dot is taken as usual. A left click, Enter, a pinch or Meta+J takes it as
+          it sits, and after PUTON_TIMEOUT it's taken anyway. So the click teaches our tracker
+          the headset where you meant it to be, not where it landed.
+  slip    while gaze mode is on with our tracker, ft-eyes' glint jump (the headset slipped
+          since the last click; "jump_px" in its status) past fitcoach.MOVED_DEG for SLIP_HOLD
+          shows a small note low in view for NOTE_SECONDS (the panel's "note" strip: "Your
+          headset slipped" and which way to push it), at most once every NOTE_COOLDOWN, and
+          never over a check. It goes sooner once the slip is gone.
 
 A check asked for while the gaze service idles (quickcal, calibrate, fitcheck) wakes it and
 waits until the tracker sends, at most ft-gazed's WAKE_SETTLE; then it opens, or logs why not.
@@ -39,26 +61,52 @@ waits until the tracker sends, at most ft-gazed's WAKE_SETTLE; then it opens, or
 The quick check's dot captures itself: from CHECK_SETTLE after it shows (the eyes getting
 there), once the gaze has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and
 capture time). It's the gaze holding still that counts, not where the tracker puts it, so it
-works however far off the tracker is; a left click or Meta+J (the pointer helper's
-"calaccept") takes it now. The full calibration's and five's dots wait for that click: you
+works however far off the tracker is; a left click, Enter, an index pinch with hand tracking,
+or Meta+J (the pointer helper's "calaccept") takes it now. The full calibration's and five's dots wait for that click: you
 click when you're looking at the dot (the user asked for that: a steady gaze isn't always on
 the dot), and the gaze held still up to then is taken (ACCEPT_SPREAD). They wait as long as
 it takes, up to CLICK_IDLE. A dot not taken says why in the panel's note line (reject_reason:
 gazecal.steady_samples' drop counts for SteamVR's tracker, ft-eyes' reply for ours), as does a
 click with nothing taken after ACCEPT_WAIT, and a failed calibration names its most common
-reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
-renewed every second; the helper shows it again by itself when that stops).
+reason there and in the status. A right click or Meta+K ("calquit") closes the panel. A press
+mapped to gaze precision or gaze drag counts as the left click (pointer/helper/calpanel.h);
+Enter is the input relay's ("calkey 1" to the helper while the panel is up), and a pinch the
+helper's own, read from ft-hands whether or not hand interaction is on. The
+pointer hides meanwhile ("calpanel 1", renewed every second; the helper shows it again by itself
+when that stops).
 
 Our tracker's first calibration: before it has one, ft-eyes publishes no gaze (it maps pupils
 to a gaze only with a calibration), so there's no gaze to hold still. Its calibration runs
 anyway ("blind"): someone in the headset (SteamVR's tracker sees an eye) and ft-eyes answering
 are enough to start it, each dot stands in for the gaze, and a click takes the CHECK_WINDOW up
-to it. ft-eyes then checks that each pupil was seen and held still in that window (calib-point)
-and says why not. Without this, a fresh install could never calibrate our tracker.
+to it. ft-eyes then checks that the pupils were seen and held still in that window (calib-point:
+one eye is enough, and the dot is kept for that eye) and says why not. Without this, a fresh
+install could never calibrate our tracker. A full calibration goes blind the same way whenever
+ours sends no gaze for OWN_GONE: its calibration maps one eye, and that's the one you can't
+show it now. Live 2026-10-18 a calibration with the left eye closed mapped only the right, and
+the next one, with the right eye closed, took no dot: every click waited for a gaze that
+couldn't come ("Waiting: the eye tracker isn't sending a gaze").
+
+Looked away (our tracker's full calibration): where SteamVR saw both eyes at a dot, their mean more
+than LOOKED_AWAY from it, past their usual offset in this calibration (the median over the dots
+it took, from AWAY_OFFSET_MIN of them), and the dot isn't taken: "you looked away from the dot"
+(looked_away; ft-eyes isn't asked). Live 2026-10-18 the palm menu opened the Steam menu
+mid-calibration, and three dots were taken 17-35 degrees from where the user looked. A third
+dot in a row like that is SteamVR being off, not you: it's taken, and the check stops for that
+calibration (AWAY_GIVE_UP).
+
+The service doesn't wait for ft-eyes' answers to calib-point and calib-fit (ask_eyes): the dot
+shows its ring full, and further clicks do nothing until the answer comes, or its deadline
+passes; while it fits, a right click doesn't close the panel either. Until 2026-10-09 it waited, up to 3 s a dot and 10 s for the fit, so the gaze stopped,
+the helper's panel lease ran out (the pointer came back, and a click went to the desktop
+behind the panel), and an answer over EYES_GONE closed the calibration as the headset coming off.
 
 What a capture teaches:
   our tracker   quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
-                calib-start, a calib-point for each dot, calib-fit (its calibration)
+                calib-start, a calib-point for each dot, calib-fit (its calibration); and
+                SteamVR's two eyes at each dot ft-eyes took, where SteamVR saw both: after the
+                fit, SteamVR's affine for ft-gazed's fusion (gazecal.Fusion) is fitted on
+                them and saved to fusion.json (kept as it was with under Fusion.MIN of them)
   SteamVR's     quick and five: a lesson for each eye, like the pointer's; full: each source's
                 calibration fitted from the dots (gazecal.Correction, the probe's way), saved
                 to calibration.json, and the lessons start over on top of it
@@ -75,10 +123,12 @@ import statistics
 import subprocess
 import sys
 import time
+from collections import deque
 from pathlib import Path
 
+import fitcoach
 from fitcheck import MIN_REGION, FitCheck, wrap
-from gazecal import DEFAULT_MODEL, EYE_LOST, STATE, steady_samples
+from gazecal import DEFAULT_MODEL, EYE_LOST, FUSION, STATE, Fusion, steady_samples
 
 REPO = Path(__file__).resolve().parents[1]
 PANEL_PROG = REPO / "gaze" / "build" / "ft-gazepanel"
@@ -96,6 +146,11 @@ CHECK_WINDOW = 0.6
 CHECK_SPREAD = 1.0     # degrees
 ACCEPT_SPREAD = 2.5    # degrees: a capture asked for (calaccept) takes this much
 CLICK_IDLE = 120.0     # seconds a dot of five or full waits for its click; then the check closes
+LOOKED_AWAY = 15.0     # degrees: SteamVR's eyes this far from a full calibration's dot (our tracker's), past
+                       # their usual offset in it, and the dot isn't taken: you looked away
+AWAY_OFFSET_MIN = 3    # dots taken with SteamVR's eyes before their median offset counts
+AWAY_GIVE_UP = 2       # dots in a row turned down as looked away: at a third, SteamVR is off, not you;
+                       # that one's taken and the check stops
 QUICK_TIMEOUT = 6.0
 QUICK_COOLDOWN = 120.0
 DON_DELAY = 3.0        # seconds of eyes after AWAY_MIN without: the headset went on
@@ -113,6 +168,15 @@ FULL_RETRY = 10.0      # seconds before an automatic calibration that failed to
 FIT_TIMEOUT = 300.0    # seconds the fit check stays up
 FIT_EVERY = 0.5        # seconds between its cards' updates (each is a new picture for the panel)
 FIT_HINT_WIDTH = 95    # characters a hint line holds in the panel
+COACH_CENTRE = 8.0     # degrees: the fit coach judges only while the gaze stays this near straight ahead
+COACH_LEASE = 5.0      # seconds of ft-eyes asked for at a time, while the fit check is up
+PUTON_REASON = "the headset went on"
+PUTON_READ = 4.0       # seconds the put-on check holds the dot for a straight-ahead reading
+PUTON_TIMEOUT = 30.0   # seconds it says what to move before the dot is taken as it sits
+PUTON_GONE = 5.0       # seconds without eyes that close it meanwhile (adjusting loses them)
+SLIP_HOLD = 2.0        # seconds a slip lasts before its note
+NOTE_SECONDS = 8.0
+NOTE_COOLDOWN = 120.0
 
 
 def log(msg):
@@ -147,6 +211,7 @@ STEAM_REASONS = {"lost_left": ("left eye lost", "SteamVR lost your left eye"),
                  "vergence": ("eyes disagreed", "SteamVR's two eyes disagreed")}
 FIT_HINT = "check the headset fit"
 ACCEPT_WAIT = 1.5      # seconds after a click with no capture before the panel says what it waits for
+OWN_GONE = 0.5         # seconds without our tracker's gaze before a full calibration goes blind (see the top)
 
 
 def reject_reason(reply, why):
@@ -158,13 +223,18 @@ def reject_reason(reply, why):
         key = max(why, key=why.get)
         return *STEAM_REASONS[key], key.startswith("lost")
     words = reply.removeprefix("fail ").split()
-    # ft-eyes: "fail the left eye was seen in only 3 frames", "fail the left eye moved (4.2 px)"
+    # ft-eyes: "fail the left eye moved (4.2 px)", "fail neither eye was seen enough (right 3
+    # frames, left 0 frames)"; one eye seen is enough for a dot (and its answer is "ok ...")
+    if reply.startswith("fail neither eye was seen"):
+        return "eyes not seen", "our tracker saw neither eye well enough", True
     if len(words) >= 3 and words[0] == "the" and words[2] == "eye":
         eye = words[1]
         if "seen" in words:
             return f"{eye} eye not seen", f"our tracker saw your {eye} eye in only {words[-2]} frames", True
         if "moved" in words:
             return f"{eye} eye moved", f"your {eye} eye moved while you looked", False
+    if reply.startswith("looked away"):  # gazecheck's own (looked_away): SteamVR's eyes were elsewhere
+        return "looked away", "you looked away from the dot (by SteamVR's eyes)", False
     if not reply:
         return "no answer", "our tracker didn't answer", False
     if reply.startswith("our tracker isn't running"):
@@ -260,6 +330,12 @@ class Checks:
         self.panel_restart_at = 0.0
         self.screens_shown = None
         self.last_progress = 0.0
+        self.asking = None       # (socket, deadline, done): a command to ft-eyes waiting for its reply
+        self.watch = (None, False)  # (frametop.conf's mtime, GAZE_FIT_WATCH on)
+        self.slip_ref = None     # the fit coach's reference, for the slip note (loaded once)
+        self.slip_since = None   # a slip past MOVED_DEG since then
+        self.note_until = None   # the slip note shows until then
+        self.note_next = 0.0     # and not again before
 
     @property
     def active(self):
@@ -345,6 +421,48 @@ class Checks:
                 if was != (on, headset):
                     self.svc.update_awake()
 
+    def ask_eyes(self, command, timeout, done):
+        """A command to ft-eyes whose reply comes later: done(reply) runs from on_eyes_reply, or with
+        "" after `timeout` (tick), as ask() gives without one. ask() held the whole service up to 3 s
+        a dot (calib-point) and 10 s at the end (calib-fit): the gaze stopped, and the helper's 3 s
+        calpanel lease ran out, so the pointer came back and a click went to the desktop behind the
+        panel. Each command has its own socket, so a late reply can't be taken for the next one."""
+        self.drop_ask()
+        s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
+        try:
+            s.bind("")
+            s.sendto(command.encode(), EYES)
+        except OSError:
+            s.close()
+            done("")
+            return
+        self.sel.register(s, selectors.EVENT_READ, "eyes_reply")
+        self.asking = (s, time.monotonic() + timeout, done)
+
+    def on_eyes_reply(self, sock):
+        if not self.asking or self.asking[0] is not sock:
+            return  # dropped already (closed, or timed out in this loop)
+        try:
+            reply = sock.recv(4096).decode("utf-8", "replace")
+        except BlockingIOError:
+            return
+        except OSError:
+            reply = ""
+        done = self.asking[2]
+        self.drop_ask()
+        done(reply)
+
+    def drop_ask(self):
+        if not self.asking:
+            return
+        s = self.asking[0]
+        self.asking = None
+        try:
+            self.sel.unregister(s)
+        except (KeyError, ValueError):
+            pass
+        s.close()
+
     # --- State ---
 
     def calibrated(self):
@@ -439,6 +557,34 @@ class Checks:
             return
         self.start("quick", reason)
 
+    def fit_watch(self):
+        """GAZE_FIT_WATCH=1 in frametop.conf: the put-on check and the slip note (see the top)."""
+        try:
+            m = CONF.stat().st_mtime_ns
+        except OSError:
+            m = None
+        if m != self.watch[0]:
+            on = False
+            try:
+                for line in CONF.read_text().splitlines():
+                    k, _, v = line.split("#", 1)[0].partition("=")
+                    if k.strip() == "GAZE_FIT_WATCH":
+                        on = v.strip() == "1"
+            except OSError:
+                pass
+            if on != self.watch[1]:
+                log(f"fit watch {'on' if on else 'off'}")
+            self.watch = (m, on)
+        return self.watch[1]
+
+    def panel_ipd(self):
+        """SteamVR's IPD setting (metres) from the panel, or None."""
+        words = ask(PANEL, "ipd", 0.3).split()
+        try:
+            return float(words[1]) if words[:1] == ["ok"] and float(words[1]) > 0 else None
+        except (IndexError, ValueError):
+            return None
+
     # --- Running a check ---
 
     def start(self, kind, reason):
@@ -464,9 +610,13 @@ class Checks:
         self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
                       "started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
                       "skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": "",
-                      "blind": blind}
+                      "blind": blind, "fusion": [], "away_dots": [], "away_off": False, "steam_offsets": [],
+                      "own_at": now, "run_blind": blind, "went_blind": False}
         log(f"{kind} check: {reason}" + (" (our tracker's first: no gaze yet, so each click takes the look "
                                          "up to it)" if blind else ""))
+        self.note_until = None   # the check's panel replaces the slip note
+        if kind == "quick" and reason == PUTON_REASON:
+            self.start_puton(now)
         if kind == "full":
             st = ask(SCREENS, "state", 0.5).split()
             if len(st) >= 3 and st[0] == "ok":
@@ -487,19 +637,219 @@ class Checks:
         if time.monotonic() - self.sample_at > 2:
             return "error the headset is off or the eye tracker isn't sending"
         now = time.monotonic()
+        ignore = None if self.svc.one_eye is None else 1 - self.svc.one_eye  # the eye SteamVR ignores
         self.check = {"kind": "fit", "reason": reason, "own": False, "dots": [], "i": 0, "started": now, "shown": now,
                       "run": [], "accept": False, "done_at": None, "tries": 0, "skipped": 0, "captured": 0,
-                      "points": {}, "fit": FitCheck(), "drawn": {}, "drawn_at": 0.0, "step": None}
+                      "points": {}, "fit": FitCheck(ignore=ignore), "drawn": {}, "drawn_at": 0.0, "step": None,
+                      "coach": self.new_coach()}
         log(f"fit check: {reason}")
+        self.note_until = None
+        if self.check["coach"]:
+            self.check["coach"]["ipd"] = self.panel_ipd()
         self.to_helper("calpanel 1")
         self.to_panel("show fit")
         self.to_panel("title Headset fit: adjust the headset while you watch")
-        self.to_panel("text Left click or Meta+J: guided check · Right click or Meta+K: done")
+        self.to_panel("text Left click, Enter or pinch: guided check · Right click or Meta+K: done")
         return "ok"
 
+    def new_coach(self):
+        """The fit coach's part of the fit check, or None without our tracker (or its reference)."""
+        if not self.svc.own_installed():
+            return None
+        ref = fitcoach.Reference.load()
+        if ref is None:
+            log(f"fit coach: no reference ({fitcoach.REFERENCE})")
+            return None
+        return {"ref": ref, "baseline": fitcoach.load_baseline(), "reading": None, "summary": None,
+                "looks": deque(), "centred": False, "logged": None, "ipd": None, "good": None}
+
+    def coach_feed(self, s, now):
+        k = self.check["coach"]
+        m1 = s["src"].get("mmap1") or {}
+        if k is None or "hy" not in m1:
+            return
+        k["looks"].append((now, m1["hy"], m1["hp"]))
+        while k["looks"] and k["looks"][0][0] < now - fitcoach.WINDOW:
+            k["looks"].popleft()
+
+    def coach_tick(self, now):
+        """Keeps ft-eyes running and asks it for the last second of finds; judged when centred."""
+        k = self.check["coach"]
+        if k is None:
+            return
+        svc = self.svc
+        svc.eyes_until = max(svc.eyes_until, now + COACH_LEASE)
+        looks = k["looks"]
+        k["centred"] = (len(looks) >= 10 and looks[-1][0] - looks[0][0] > 0.8 * fitcoach.WINDOW
+                        and all(math.hypot(y, p) <= COACH_CENTRE for _, y, p in looks))
+        if self.asking:
+            if now >= self.asking[1]:
+                self.drop_ask()
+            return
+        if not svc.eyes_proc:
+            k["summary"] = None
+            return
+        centred = k["centred"]
+
+        def got(reply):
+            try:
+                summary = {int(cam): v for cam, v in json.loads(reply).items()}
+            except (ValueError, AttributeError):
+                k["summary"] = None
+                return
+            k["summary"] = summary
+            if centred and self.check and self.check.get("coach") is k:
+                k["reading"] = fitcoach.assess(summary, k["ref"], ipd=k["ipd"], baseline=k["baseline"],
+                                               previous=k["reading"])
+                if k["reading"]["state"] == "ok":
+                    k["good"] = summary   # (the fit check keeps the last as your saved fit)
+        self.ask_eyes("fit", 0.4, got)
+
+    def coach_lines(self):
+        """The fit coach's hints for the panel (first), or [] without it."""
+        c = self.check
+        k = c.get("coach")
+        if k is None:
+            return []
+        if k["summary"] is None:
+            return ["Headset position: starting our eye tracker..."]
+        if not k["centred"]:
+            if k["reading"] is None:
+                return ["Headset position: look straight ahead for a moment."]
+            return [f"Headset position (when you last looked straight ahead): {self.coach_words(k['reading'])}"]
+        return [f"Headset position: {self.coach_words(k['reading'])}"] if k["reading"] else []
+
+    @staticmethod
+    def coach_words(r):
+        if r["state"] == "waiting":
+            return "waiting for both eyes."
+        hints = " ".join(h["text"] for h in r["hints"][:2])
+        if r["state"] == "ok":
+            words = "good." + (f" {hints}" if hints else "")
+        else:
+            words = hints or "a little off."
+        if r["provisional"] and r["against"] == "reference":
+            words += " (A first guess: measured on one face so far.)"
+        return words
+
+    # --- The put-on check (the fit watch: see the top) ---
+
+    def start_puton(self, now):
+        c = self.check
+        if not self.fit_watch():
+            return
+        k = self.new_coach()
+        if k is None or not k["baseline"]:
+            if k is not None:
+                log("put-on check: no saved fit yet (Check headset fit saves a good one)")
+            return
+        k["ipd"] = self.panel_ipd()
+        c["coach"] = k
+        c["puton"] = {"state": "reading", "until": now + PUTON_READ, "since": now, "lines": None}
+        log("put-on check: reading the fit against your saved one")
+
+    def puton_holds(self, c):
+        """The put-on check holds the quick check's dot (reading, or the headset moved)."""
+        return (c.get("puton") or {}).get("state") in ("reading", "moved")
+
+    def puton_tick(self, now):
+        c = self.check
+        pu, k = c["puton"], c["coach"]
+        if not self.puton_holds(c):
+            return   # (no more asks: the quick check skips samples while one waits)
+        self.coach_tick(now)
+        if c["accept"]:
+            self.puton_done("the dot taken as the headset sits (a click)")
+            return
+        r = k["reading"] if k["centred"] and k["summary"] is not None else None
+        if r is not None and r["state"] == "waiting":
+            r = None
+        if pu["state"] == "reading":
+            if r is None:
+                if now > pu["until"]:
+                    self.puton_done("no straight-ahead reading in time; the dot is taken as usual")
+                return
+            if not fitcoach.moved(r):
+                self.puton_done("the headset sits as your saved fit")
+                return
+            pu["state"], pu["since"] = "moved", now
+            log("put-on check: the headset moved since your saved fit: terms "
+                + ", ".join(f"{n} {v:+.1f}" for n, v in r["terms"].items()))
+            self.to_panel("show five")   # wider: the hints don't fit quick's square
+            self.to_panel("title The headset sits differently from your saved fit")
+            self.show_dot()
+            self.puton_lines(r)
+            return
+        if r is not None and not fitcoach.moved(r):
+            self.note("Good: it's back where it was")
+            self.puton_done("the headset is back as your saved fit")
+            return
+        if r is not None:
+            self.puton_lines(r)
+        if now - pu["since"] > PUTON_TIMEOUT:
+            self.puton_done(f"not back after {PUTON_TIMEOUT:.0f} s: the dot is taken as the headset sits")
+
+    def puton_lines(self, r):
+        """What to move, in the panel: the position hint at the bottom, a changed IPD over it."""
+        pu = self.check["puton"]
+        ipd = next((h["text"] for h in r["hints"] if h["key"] == "ipd-changed"), "")
+        pos = fitcoach.position_hint(r["terms"])
+        lines = (pos["text"] if pos else "Move the headset back where it sat, then look at the dot", ipd)
+        if lines != pu["lines"]:
+            pu["lines"] = lines
+            self.to_panel(f"text {lines[0]}")
+            self.note(lines[1])
+
+    def puton_done(self, why):
+        c = self.check
+        now = time.monotonic()
+        c["puton"]["state"], c["puton"]["why"] = "done", why
+        log(f"put-on check: {why}")
+        c["run"], c["started"], c["shown"] = [], now, now   # the quick check's own time starts now
+        if c["puton"]["lines"]:
+            self.to_panel("text Look at the dot")
+
+    # --- The slip note (the fit watch: see the top) ---
+
+    def slip_watch(self, now):
+        svc = self.svc
+        own = svc.own if now - svc.own_at < 3 else {}
+        on = (svc.kind == "own" and svc.eyes_proc is not None and own and self.gaze_on
+              and self.headset is not False and not self.check and self.fit_watch())
+        hint = None
+        if on:
+            if self.slip_ref is None:
+                self.slip_ref = fitcoach.Reference.load() or False
+            jumps = {}
+            for cam, name in enumerate(fitcoach.EYES):
+                j = (own.get("eyes") or {}).get(name, {}).get("jump_px")
+                if j and any(j):
+                    jumps[cam] = j
+            hint = fitcoach.position_hint(fitcoach.slip_terms(jumps, self.slip_ref or None))
+        if hint is None:
+            self.slip_since = None
+            if self.note_until is not None and not self.check:
+                self.to_panel("hide")
+            self.note_until = None
+            return
+        if self.note_until is not None:
+            if now >= self.note_until:
+                self.to_panel("hide")
+                self.note_until = None
+            return
+        if self.slip_since is None:
+            self.slip_since = now
+        if now - self.slip_since >= SLIP_HOLD and now >= self.note_next:
+            self.to_panel("show note")
+            self.to_panel("title Your headset slipped")
+            self.to_panel(f"text {hint['text']}")
+            self.note_until, self.note_next = now + NOTE_SECONDS, now + NOTE_COOLDOWN
+            log(f"slip note: {hint['text']} ({hint['amount']:+.1f} deg)")
+
     def fit_tick(self, now):
         c = self.check
         fit = c["fit"]
+        self.coach_tick(now)
         # The guided check: its dots at head-relative directions in the panel (the probe's
         # screen fractions, spread over the middle of the panel), and its looks as the title.
         step = fit.guide_step(now)
@@ -529,9 +879,10 @@ class Checks:
                 drawn[k] = cmd
                 self.to_panel(cmd)
         if fit.have_eye_data and fit.samples < 3 * MIN_REGION:
-            hints = ["Look around slowly: up, down, left and right. Or left click (Meta+J) for a guided check."]
+            hints = ["Look around slowly: up, down, left and right. Or left click, press Enter or pinch for a guided check."]
         else:
             hints = fit.hints()
+        hints = self.coach_lines() + hints
         lines = [line for h in hints for line in wrap(h, FIT_HINT_WIDTH)][:8]
         cmd = "hints " + "|".join(lines)
         if drawn.get("hints") != cmd:
@@ -544,9 +895,9 @@ class Checks:
         if c["kind"] == "full":
             self.to_panel(f"bg {ROUND_BG[rnd]}")
             self.to_panel(f"title Gaze calibration: {ROUND_NAMES[rnd]} round, {rnd + 1} of 3")
-            self.to_panel("text Look at the dot and click (left click or Meta+J). Right click or Meta+K: stop")
+            self.to_panel("text Look at the dot, then left click, press Enter or pinch. Right click or Meta+K: stop")
         elif c["kind"] == "five":
-            self.to_panel(f"text Look at the dot and click ({c['i'] + 1} of {len(c['dots'])})")
+            self.to_panel(f"text Look at the dot, then click, Enter or pinch ({c['i'] + 1} of {len(c['dots'])})")
         else:
             self.to_panel("text Look at the dot")
         self.to_panel(f"dot {yaw:.3f} {pitch:.3f} look")
@@ -559,22 +910,37 @@ class Checks:
         if self.pending:
             self.run_pending()
         unc = (s["src"].get("mmap1") or {}).get("unc")
-        if unc and min(unc) <= EYE_LOST:
+        one = self.svc.one_eye
+        if unc and (min(unc) if one is None else unc[one]) <= EYE_LOST:
             self.seen_at = time.monotonic()
             if self.away and self.back_since is None:
                 self.back_since = self.seen_at
         c = self.check
         if c and c["kind"] == "fit":
             c["fit"].feed(s, self.sample_at)
+            self.coach_feed(s, self.sample_at)
             return
-        if not c or c["done_at"]:
-            return
+        if c and self.puton_holds(c):
+            self.coach_feed(s, self.sample_at)
+        if not c or c["done_at"] or self.asking:
+            return  # (asking: ft-eyes has this dot's look, or the fit)
         if c["own"]:
             src = s["src"].get("own") or {}
-            if c["blind"]:
-                # Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
-                # gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
-                # the pupils held still (see the top).
+            if "hy" in src:
+                c["own_at"] = self.sample_at
+            # Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
+            # gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
+            # the pupils held still (see the top). A full calibration does the same while ours
+            # sends no gaze: its calibration maps only the eye you've closed.
+            blind = c["blind"] or (c["kind"] == "full" and self.sample_at - c["own_at"] > OWN_GONE)
+            if blind != c["run_blind"]:
+                c["run"].clear()  # the dot standing in and our gaze aren't one look
+                c["run_blind"] = blind
+                if blind and not c["went_blind"]:
+                    c["went_blind"] = True
+                    log(f"{c['kind']} check: our tracker sends no gaze (its calibration's eyes aren't seen): "
+                        "each click takes the look up to it")
+            if blind:
                 yaw, pitch, _ = c["dots"][c["i"]]
                 src = {"hy": yaw, "hp": pitch}
         else:
@@ -587,7 +953,7 @@ class Checks:
             return
         now = time.monotonic()
         c["gaze_at"] = now
-        if now < c["shown"] + CHECK_SETTLE:
+        if now < c["shown"] + CHECK_SETTLE or self.puton_holds(c):
             return
         g = (src["hy"], src["hp"])
         run = c["run"]
@@ -635,10 +1001,28 @@ class Checks:
             miss = [math.hypot(yaw - e[0], pitch - e[1]) if e else None for e in eyes]
             rec.update(eyes=eyes, miss=miss)
             t0, t1 = samples[0]["t"], samples[-1]["t"]
+            if c["kind"] == "full" and svc.one_eye is None:
+                # SteamVR's eyes at the dot too, for the fusion (finish): each one's median over
+                # the steady samples, as its own calibration takes them.
+                steady = steady_samples(samples)
+                lr = []
+                for name in ("left", "right"):
+                    pts = [(smp["src"][name]["hy"], smp["src"][name]["hp"]) for smp in steady
+                           if "hy" in (smp["src"].get(name) or {})]
+                    lr.append((statistics.median(p[0] for p in pts), statistics.median(p[1] for p in pts))
+                              if len(pts) >= 15 else None)
+                if all(lr):
+                    rec["steam"] = lr
+                    if self.looked_away(rec):
+                        self.captured(rec, False)
+                        return
             if c["kind"] == "full":
-                reply = ask(EYES, f"calib-point {t0:.6f} {t1:.6f} {yaw:.4f} {pitch:.4f}", 3.0)
-                rec["reply"] = reply
-                ok = reply.startswith("ok")
+                # ft-eyes checks the pupils held still: its answer takes the dot or not (point_done).
+                # The ring shows it full meanwhile, so the click is seen to have landed.
+                self.to_panel(f"dot {yaw:.3f} {pitch:.3f} capture 1.00")
+                self.ask_eyes(f"calib-point {t0:.6f} {t1:.6f} {yaw:.4f} {pitch:.4f}", 3.0,
+                              lambda reply: self.point_done(rec, miss, reply))
+                return
             else:
                 try:
                     svc.eyes_sock.sendto(f"click {t1:.6f} {yaw:.4f} {pitch:.4f}".encode(), EYES)
@@ -648,7 +1032,8 @@ class Checks:
                 svc.weights["own"].add(miss)
         else:
             why = {}
-            steady = steady_samples(samples, why=why)
+            one = svc.one_eye
+            steady = steady_samples(samples, why=why, eye=one)
             rec["dropped"] = why
             reads = {}
             for name in ("action", "mmap1", "mmap2", "left", "right"):
@@ -656,11 +1041,20 @@ class Checks:
                        if "hy" in (smp["src"].get(name) or {})]
                 if len(pts) >= 15:
                     reads[name] = (statistics.median(p[0] for p in pts), statistics.median(p[1] for p in pts))
+            if one is not None:
+                # SteamVR tracks one eye (gazecal.tracked_eye): the other's reading isn't where
+                # you look, and set 2's average has it in, so mmap2 is that eye's own (as ft-gazed
+                # sends it then).
+                reads.pop(("left", "right")[1 - one], None)
+                reads.pop("mmap2", None)
+                if ("left", "right")[one] in reads:
+                    reads["mmap2"] = reads[("left", "right")[one]]
             rec["reads"] = reads
-            main = ("left", "right") if svc.kind == "eyes" else (svc.source,)
+            main = (("left", "right") if one is None else (("left", "right")[one],)) if svc.kind == "eyes" else (svc.source,)
             if not all(n in reads for n in main):
                 ok = False
-                rec["reply"] = f"only {len(steady)} of {len(samples)} samples had both eyes"
+                rec["reply"] = f"only {len(steady)} of {len(samples)} samples had " + (
+                    "both eyes" if one is None else f"your {('left', 'right')[one]} eye")
             elif c["kind"] == "full":
                 for name, (hy, hp) in reads.items():
                     c["points"].setdefault(name, []).append((hy, hp, yaw - hy, pitch - hp))
@@ -680,9 +1074,52 @@ class Checks:
                     svc.lives[name].add({"time": time.time(), "hy": hy, "hp": hp, "dy": yaw - hy, "dp": pitch - hp,
                                          "wy": 1.0, "wp": 1.0, "how": "check"}, svc.models[name], svc.mode)
                 rec["miss"] = miss
-                if svc.kind == "eyes":
+                if svc.kind == "eyes" and len(miss) == 2:  # (one eye tracked: nothing to weigh)
                     svc.weights["steam"].add(miss)
                 svc.dirty = True
+        self.captured(rec, ok)
+
+    def looked_away(self, rec):
+        """A full calibration's dot for our tracker, where SteamVR saw both eyes: whether they say
+        you looked away (LOOKED_AWAY past their usual offset in this calibration). The dot isn't
+        taken then (ft-eyes isn't asked): one taken there teaches our tracker the wrong place.
+        Past AWAY_GIVE_UP dots in a row turned down, it's SteamVR that's off: the check stops."""
+        c = self.check
+        (my, mp), (ty, tp) = Fusion.mean(*rec["steam"]), rec["true"]
+        dy, dp = ty - my, tp - mp
+        offs = c["steam_offsets"]
+        if len(offs) >= AWAY_OFFSET_MIN:
+            dy -= statistics.median(o[0] for o in offs)
+            dp -= statistics.median(o[1] for o in offs)
+        rec["away"] = round(math.hypot(dy, dp), 2)
+        away = rec["away"] > LOOKED_AWAY and not c["away_off"]
+        if away and c["i"] not in c["away_dots"] and len(c["away_dots"]) >= AWAY_GIVE_UP:
+            c["away_off"], away = True, False
+            log(f"full check: SteamVR's eyes {rec['away']:.0f} deg off at {AWAY_GIVE_UP + 1} dots in a row: "
+                "taken as SteamVR being off, the looked-away check is off for this calibration")
+        if not away:
+            c["away_dots"] = []
+            offs.append((ty - my, tp - mp))
+            return False
+        if c["i"] not in c["away_dots"]:
+            c["away_dots"].append(c["i"])
+        rec["reply"] = f"looked away ({rec['away']:.0f} deg by SteamVR)"
+        return True
+
+    def point_done(self, rec, miss, reply):
+        """ft-eyes' answer to a full calibration dot's calib-point ("" without one)."""
+        rec["reply"] = reply
+        ok = reply.startswith("ok")
+        if ok:
+            self.svc.weights["own"].add(miss)
+            if rec.get("steam"):
+                self.check["fusion"].append((*Fusion.mean(*rec["steam"]), *rec["true"]))
+        self.captured(rec, ok)
+
+    def captured(self, rec, ok):
+        """A capture is over: the dot is taken, tried again, or skipped."""
+        c = self.check
+        yaw, pitch, _ = c["dots"][c["i"]]
         if not ok:
             short, long, fit = reject_reason(rec.get("reply", "") if c["own"] else None, rec.get("dropped"))
             rec["reason"] = long
@@ -761,8 +1198,17 @@ class Checks:
             return
         self.full_failed = None
         if c["own"]:
-            reply = ask(EYES, "calib-fit", 10.0)
-            log(f"calibration ({c['captured']} of {n} dots): our tracker says {reply or 'nothing'}")
+            def fitted(reply):
+                log(f"calibration ({c['captured']} of {n} dots): our tracker says {reply or 'nothing'}")
+                if reply.startswith("ok"):
+                    self.fit_fusion(c["fusion"])
+                self.close()
+
+            self.to_panel("text Saving the calibration")
+            c["done_at"] = None  # (tick would advance past the last dot again)
+            c["fitting"] = True
+            self.ask_eyes("calib-fit", 10.0, fitted)
+            return
         else:
             mode = svc.mode if svc.mode != "none" else DEFAULT_MODEL
             for name, pts in c["points"].items():
@@ -780,11 +1226,39 @@ class Checks:
             log(f"calibration ({c['captured']} of {n} dots): {mode}, saved")
         self.close()
 
+    def fit_fusion(self, pts):
+        """SteamVR's affine for ft-gazed's fusion, from its eyes at the dots ours took."""
+        if len(pts) < Fusion.MIN:
+            log(f"fusion: SteamVR saw both eyes at only {len(pts)} of the dots; {FUSION.name} stays as it was")
+            return
+        f = Fusion()
+        f.fit(pts, {"how": "calibration", "made": time.strftime("%Y-%m-%d %H:%M:%S"), "wear": self.svc.steam.worn()})
+        try:
+            f.save(FUSION)
+        except OSError as e:
+            log(f"fusion: couldn't save {FUSION}: {e}")
+            return
+        self.svc.load_fusion()
+        log(f"fusion: SteamVR's affine from {len(pts)} dots, {statistics.median(f.misses()):.2f} deg left at the "
+            "median, saved")
+
     def close(self, why=None):
         if not self.check:
             return
         if why:
             log(f"{self.check['kind']} check closed: {why}")
+        k = self.check.get("coach")
+        if k and k["reading"]:
+            r = k["reading"]
+            log(f"fit coach, last reading: {r['state']} against the {r['against']}, terms "
+                + ", ".join(f"{n} {v:+.1f}" for n, v in r["terms"].items()))
+        if k and k["good"] and self.check["kind"] == "fit":
+            try:
+                fitcoach.save_baseline(fitcoach.make_baseline(k["good"], k["ipd"]))
+                log(f"fit coach: a good fit, saved as yours ({fitcoach.BASELINE})")
+            except OSError as e:
+                log(f"fit coach: can't save the fit: {e}")
+        self.drop_ask()
         self.to_panel("hide")
         self.to_helper("calpanel 0")
         if self.check["kind"] == "full" and self.screens_shown:
@@ -794,8 +1268,8 @@ class Checks:
 
     def quit(self):
         c = self.check
-        if not c:
-            return
+        if not c or c.get("fitting"):
+            return  # (fitting: ft-eyes has every dot; the panel closes once it answers)
         self.close("quit")
         if c["kind"] == "full" and self.calibrated() is False:
             # No calibration still: gaze mode can't work, so it goes off until it's turned on again.
@@ -849,7 +1323,8 @@ class Checks:
         if cmd == "calaccept":
             if self.check and self.check["kind"] == "fit":
                 self.check["fit"].toggle_guide(time.monotonic())
-            elif self.check:
+            elif self.check and (not self.asking or self.puton_holds(self.check)):
+                # (asking: this dot's click landed already; the put-on check's asks are the coach's)
                 if not self.check["accept"]:
                     self.check["accept_at"] = time.monotonic()
                 self.check["accept"] = True
@@ -887,13 +1362,25 @@ class Checks:
             else:
                 self.fit_tick(now)
             return
+        if c.get("puton"):
+            self.puton_tick(now)
+            c = self.check
+            if not c:
+                return
+        if self.asking:
+            # ft-eyes has a dot's look or the fit: wait for its answer, at most to the deadline.
+            if now >= self.asking[1]:
+                done = self.asking[2]
+                self.drop_ask()
+                done("")
+            return
         if c["done_at"] and now >= c["done_at"]:
             if c.get("closing"):
                 self.close()
             else:
                 self.advance()
             return
-        if not self.eyes_seen(EYES_GONE):
+        if not self.eyes_seen(PUTON_GONE if self.puton_holds(c) else EYES_GONE):
             self.close("the headset came off")
             return
         if c["done_at"]:
@@ -905,7 +1392,7 @@ class Checks:
                 self.note("Waiting: the eye tracker isn't sending a gaze" if big else "Waiting: no gaze")
             else:
                 self.note("Waiting for your gaze to hold still on the dot" if big else "Hold your look still")
-        if c["kind"] == "quick" and now - c["started"] > QUICK_TIMEOUT:
+        if c["kind"] == "quick" and now - c["started"] > QUICK_TIMEOUT and not self.puton_holds(c):
             self.close("ignored")
         elif c["kind"] != "quick" and now - c["shown"] > CLICK_IDLE:
             self.close(f"no click on dot {c['i'] + 1} for {CLICK_IDLE:.0f} s")
@@ -942,6 +1429,7 @@ class Checks:
                 self.reseat_seen = True
                 self.auto_quick("our tracker asked for a click")
         self.need_full("gaze mode is on without a calibration")
+        self.slip_watch(now)
 
     def status(self):
         c = self.check
@@ -949,10 +1437,13 @@ class Checks:
               if self.pending else None, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
               "problem": self.problem(),
               "panel": self.panel_proc is not None,
-              "last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}
+              "last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None,
+              "fit_watch": self.watch[1], "slip_note": self.note_until is not None}
         if c:
             st["check"] = {"kind": c["kind"], "dot": c["i"] + 1, "dots": len(c["dots"]), "captured": c["captured"],
                            "skipped": c["skipped"], "reason": c["reason"]}
+            if c.get("puton"):
+                st["check"]["puton"] = c["puton"]["state"]
         return st
 
     def stop(self):

gaze/panel/ft-gazepanel.cpp

@@ -13,11 +13,13 @@
 // size changes with it, and the tracker's error with it); "fit" is FIT_DEG across (4:3),
 // see-through like quick, for the headset fit check: a card per eye (tracked or lost, the
 // tracker's signal, how much of the last 10 s it was seen) and hints. Every dot a check shows
-// must fit its panel: gazecheck.py's PANEL_DEG mirrors these sizes and checks it.
+// must fit its panel: gazecheck.py's PANEL_DEG mirrors these sizes and checks it. "note" is a
+// strip NOTE_DEG across (4:1), see-through, NOTE_PITCH degrees below straight ahead, for the gaze
+// service's slip notice: its title and text lines only, no dot.
 //
 // Control socket: abstract unix datagram "@ft_gazepanel" (--socket NAME); a sender with an
 // address gets "ok" or "error ...":
-//   show quick|five|full|fit     the panel, empty, in front of you
+//   show quick|five|full|fit|note the panel, empty, in front of you (note: below it)
 //   hide
 //   bg <0..1>                    the background's brightness (full)
 //   dot <yaw> <pitch> <state> [<progress 0..1>]
@@ -33,6 +35,48 @@
 //   hints <line>|<line>|...      fit: lines under the cards (empty to clear)
 //   ping
 //
+// The eye recorder's mode (gaze/rec/DESIGN.md, "Panel"): "show rec" is FULL_DEG across with
+// the full calibration's solid background (bg), and the hand panel's lines (hands/rec/panel):
+//   title <text>                 the big line at the top
+//   step <text>                  a small line at the top right, "Round 2 of 5"; rec on|off puts a
+//                                red "Rec" dot by it
+//   text <text>                  the instruction, under the title, wrapped ("|" starts a line)
+//   big <text> / action <text>   large in the middle (the countdown) / a cyan line under it
+//   note <text> / keys <text>    the orange warning line / a faint line along the bottom
+//   countdown <0..1>|off         a thin bar along the bottom: the share of the step's time left
+//   paused on|off                "Paused" over the picture
+// Its dot is a second overlay (OVERLAY.dot), drawn once per state and only moved, so a moving
+// dot is smooth at the headset's frame rate and costs no pictures:
+//   dot <yaw> <pitch> <state> [<progress>]   as above, at the panel's distance
+//   path circle <yaw> <pitch> <radius> <seconds>
+//   path line <yaw0> <pitch0> <yaw1> <pitch1> <seconds>
+//   path eight <yaw> <pitch> <yaw radius> <pitch radius> <seconds>
+//                                the dot moves by itself: round the circle once per seconds
+//                                (from the top, going left), along the line and back (seconds
+//                                each way, at a steady speed), or a figure eight (seconds a
+//                                loop; up and down twice per side to side), until the next dot,
+//                                path, world or "dot 0 0 off". Reply: "ok"
+//   world <x> <y> <z> [<state> [<progress>]]
+//                                a dot fixed in the room: the point is in the head frame now
+//                                (metres, +x right, +y up, -z forward), placed in the room with
+//                                the headset's pose, facing the head as it moves; the same point
+//                                again changes only the state. Reply "ok <room x> <y> <z>"
+//                                (standing universe)
+//   distance <metres>            the panel and its dot that far away (0.25 to 10), the same size
+//                                in degrees (the Depth bounty)
+//   ipd                          "ok <metres>": SteamVR's IPD setting now (--no-vr: 0.0630)
+//   log start <dir> / log stop   append to <dir>/targets.jsonl, poses.jsonl and ipd.jsonl (below)
+//                                until stopped; "log stop" replies "ok <targets> <poses> <ipd>"
+// targets.jsonl, the ground truth: what the dot did, when. A row for each frame the dot moves in
+// (path, world) and for each change (dot, state, background, distance): {"t": when the frame
+// reaches the eyes (estimated: the next vsync after the change, plus SteamVR's vsync to photons),
+// "set": when the panel set it, both CLOCK_MONOTONIC_RAW seconds, the eye frames' clock;
+// "frame": SteamVR's vsync count; "dot": "fixed|path|world|off", "yaw", "pitch" (head-relative
+// degrees, for world from the head's predicted pose), "dist" (metres), "state", "progress",
+// "bg"}. poses.jsonl: the headset's pose 250 times a second, {"t", "hmd": {"m": 3x4 row-major,
+// standing universe, "r": ETrackingResult, "ok"}}. ipd.jsonl: SteamVR's IPD setting at the start
+// and when it changes, {"t", "ipd": metres}, looked at 30 times a second as well as on its event.
+//
 // Each picture goes into the next of three shared buffers (linear DMA-BUFs SteamVR imported
 // once, the size of the biggest panel; the texture bounds show the part in use), and the panel
 // switches to it, as screens/keyboard.cpp does. SetOverlayRaw, which uploads a new texture each
@@ -41,7 +85,10 @@
 // It's only the fallback. A "show" makes the panel visible once its first picture is in.
 //
 // Options: --watch-stdin (quit when stdin closes: the service runs it), --socket NAME,
-// --distance METRES. Runs in the dev container (gaze/build.sh builds it into gaze/build).
+// --overlay KEY (frametop.gazepanel; the eye recorder runs a copy of its own next to the gaze
+// service's), --distance METRES, --no-vr (no SteamVR: for tests, at a made-up 90 Hz vsync; with
+// --dump DIR each picture goes there as a PAM file). Runs in the dev container (gaze/build.sh
+// builds it into gaze/build).
 #include <openvr.h>
 
 #define STB_TRUETYPE_IMPLEMENTATION
@@ -51,6 +98,7 @@
 #include <fcntl.h>
 #include <gbm.h>
 #include <poll.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/un.h>
 #include <unistd.h>
@@ -64,6 +112,7 @@
 #include <cstdlib>
 #include <cstring>
 #include <map>
+#include <mutex>
 #include <string>
 #include <thread>
 #include <vector>
@@ -75,8 +124,22 @@ constexpr double kQuickDeg = 16;      // QUICK_DEG: the one-dot check's square
 constexpr double kFullDeg = 64;       // FULL_DEG: the full calibration's width (4:3)
 constexpr double kFitDeg = 40;        // FIT_DEG: the headset fit check's width (4:3)
 constexpr double kFiveDeg = 40;       // FIVE_DEG: the five-dot check's width (4:3), past its dots
+constexpr double kNoteDeg = 30;       // NOTE_DEG: the slip notice's strip (4:1)
+constexpr double kNotePitch = -16;    // NOTE_PITCH: the strip's centre, degrees below straight ahead
 constexpr int kQuickPx = 320, kFullW = 1024, kFullH = 768, kFitW = 800, kFitH = 600, kFiveW = 800, kFiveH = 600;
+constexpr int kNoteW = 600, kNoteH = 150;
+constexpr double kDotDeg = 1.8;       // the rec mode's dot overlay: the ring's 1.5 degrees, and a margin
+constexpr int kDotPx = 96;
+constexpr int64_t kPosePeriodNs = 4'000'000;  // 250 Hz, as the hand panel
+constexpr int64_t kIpdPeriodNs = 33'000'000;
 std::atomic<bool> g_stop{false};
+std::mutex g_vr;  // around OpenVR calls once the pose log's thread makes them too
+
+int64_t RawNs() {
+    timespec ts;
+    clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+    return int64_t(ts.tv_sec) * 1'000'000'000 + ts.tv_nsec;
+}
 
 // ---------------------------------------------------------------- text (as screens/keyboard.cpp)
 
@@ -135,6 +198,41 @@ std::vector<uint32_t> Codepoints(const std::string &s) {
     return out;
 }
 
+int TextWidth(const std::string &s, int size) {
+    if (!g_fontOk) return 0;
+    int width = 0;
+    for (uint32_t cp : Codepoints(s)) width += GetGlyph(cp, size).advance;
+    return width;
+}
+
+// s as lines no wider than width (the hand panel's): "|" starts a new line, and lines break
+// between words.
+std::vector<std::string> Wrap(const std::string &s, int size, int width) {
+    std::vector<std::string> out;
+    if (s.empty()) return out;
+    for (size_t at = 0; at <= s.size();) {
+        const size_t bar = std::min(s.find('|', at), s.size());
+        const std::string para = s.substr(at, bar - at);
+        std::string line;
+        for (size_t w = 0; w < para.size();) {
+            const size_t space = std::min(para.find(' ', w), para.size());
+            const std::string word = para.substr(w, space - w);
+            w = space + 1;
+            if (word.empty()) continue;
+            const std::string longer = line.empty() ? word : line + " " + word;
+            if (line.empty() || TextWidth(longer, size) <= width) {
+                line = longer;
+            } else {
+                out.push_back(line);
+                line = word;
+            }
+        }
+        out.push_back(line);
+        at = bar + 1;
+    }
+    return out;
+}
+
 // ---------------------------------------------------------------- the picture
 
 struct EyeCard {
@@ -143,7 +241,10 @@ struct EyeCard {
 };
 
 struct Panel {
-    bool full = false, fit = false;
+    bool full = false, fit = false, rec = false, strip = false;  // strip: "show note"
+    std::string step, big, action, keys;  // rec
+    bool recOn = false, paused = false;
+    double countdown = -1;
     EyeCard eyes[2];
     std::vector<std::string> hints;
     int w = kQuickPx, h = kQuickPx;
@@ -253,9 +354,58 @@ void ToPixel(const Panel &p, double yaw, double pitch, double &x, double &y) {
     y = (0.5 - Y / (2 * half) * p.w / p.h) * p.h;
 }
 
+void DrawDot(Panel &p, double x, double y, double pxPerDeg, const std::string &state, double progress, double ink);
+
+// The eye recorder's panel (see the top). Its dot is the other overlay's.
+void DrawRec(Panel &p, double pxPerDeg) {
+    const bool light = p.bg > 0.5;
+    const double faint = light ? 0.2 : 0.75, ink = light ? 0.05 : 0.95;
+    const int titleSize = int(pxPerDeg * 1.5), textSize = int(pxPerDeg * 1.2), small = int(pxPerDeg * 0.9);
+    const int bigSize = int(pxPerDeg * 4.0), margin = int(p.w * 0.05);
+    const int titleY = int(titleSize * 1.2);
+    Text(p, p.title, titleSize, p.w / 2, titleY, ink);
+    if (!p.step.empty()) {
+        const int sw = TextWidth(p.step, small);
+        Text(p, p.step, small, p.w - margin - sw, titleY, faint, true);
+        if (p.recOn) {
+            const int rx = p.w - margin - sw - small;
+            Disc(p, rx, titleY, small * 0.35, 0, 0.9, 0.15, 0.1, 1);
+            Text(p, "Rec", small, rx - small - TextWidth("Rec", small), titleY, 1.0, 0.35, 0.3, true);
+        }
+    } else if (p.recOn) {
+        Disc(p, p.w - margin - small * 0.35, titleY, small * 0.35, 0, 0.9, 0.15, 0.1, 1);
+    }
+    int y = titleY + int(titleSize * 1.4);
+    for (const std::string &line : Wrap(p.text, textSize, int(p.w * 0.8))) {
+        Text(p, line, textSize, p.w / 2, y, faint);
+        y += int(textSize * 1.35);
+    }
+    Text(p, p.big, bigSize, p.w / 2, p.h / 2, ink);
+    if (light) Text(p, p.action, textSize, p.w / 2, int(p.h * 0.68), 0.0, 0.35, 0.45);
+    else Text(p, p.action, textSize, p.w / 2, int(p.h * 0.68), 0.35, 0.85, 1.0);
+    if (light) Text(p, p.note, textSize, p.w / 2, p.h - int(textSize * 2.8), 0.7, 0.12, 0.05);
+    else Text(p, p.note, textSize, p.w / 2, p.h - int(textSize * 2.8), 1.0, 0.62, 0.3);
+    Text(p, p.keys, small, p.w / 2, p.h - int(small * 1.6), faint);
+    if (p.countdown >= 0) {
+        const int bh = std::max(3, int(pxPerDeg * 0.2));
+        Rect(p, 0, p.h - bh, p.w, p.h, ink, ink, ink, 0.12);
+        Rect(p, 0, p.h - bh, int(p.w * std::clamp(p.countdown, 0.0, 1.0)), p.h, 0.35, 0.85, 1.0, 0.9);
+    }
+    if (p.paused) {
+        Rect(p, 0, 0, p.w, p.h, 0, 0, 0, 0.55);
+        Text(p, "Paused", bigSize, p.w / 2, p.h / 2, 1.0);
+    }
+}
+
 void Draw(Panel &p) {
     p.px.assign(size_t(p.w) * p.h * 4, 0);
     const double pxPerDeg = p.w / p.wDeg;
+    if (p.rec) {
+        for (size_t i = 0; i < p.px.size(); i += 4)
+            p.px[i] = p.px[i + 1] = p.px[i + 2] = uint8_t(std::lround(p.bg * 255)), p.px[i + 3] = 255;
+        DrawRec(p, pxPerDeg);
+        return;
+    }
     if (p.full) {
         for (size_t i = 0; i < p.px.size(); i += 4)
             p.px[i] = p.px[i + 1] = p.px[i + 2] = uint8_t(std::lround(p.bg * 255)), p.px[i + 3] = 255;
@@ -272,6 +422,12 @@ void Draw(Panel &p) {
     }
     const bool light = p.full && p.bg > 0.5;  // a dark dot on the bright round
     const double ink = light ? 0.0 : 1.0, faint = light ? 0.2 : 0.75;
+    if (p.strip) {
+        // The slip notice: the title and the hint, together in the middle of the strip.
+        Text(p, p.title, int(pxPerDeg * 1.4), p.w / 2, int(p.h * 0.36), ink);
+        Text(p, p.text, int(pxPerDeg * 1.05), p.w / 2, int(p.h * 0.68), faint);
+        return;
+    }
     const int titleSize = int(pxPerDeg * (p.full ? 1.5 : 1.1)), textSize = int(pxPerDeg * (p.full ? 1.2 : 0.9));
     Text(p, p.title, titleSize, p.w / 2, int(titleSize * 1.2), faint);
     Text(p, p.text, textSize, p.w / 2, p.h - int(textSize * 1.3), faint);
@@ -282,21 +438,26 @@ void Draw(Panel &p) {
     if (!p.dotOn || p.state == "off") return;
     double x, y;
     ToPixel(p, p.dotYaw, p.dotPitch, x, y);
+    DrawDot(p, x, y, pxPerDeg, p.state, p.progress, ink);
+}
+
+// The dot at (x, y): its look for each state.
+void DrawDot(Panel &p, double x, double y, double pxPerDeg, const std::string &state, double progress, double ink) {
     const double core = 0.22 * pxPerDeg;
-    if (p.state == "look") {
+    if (state == "look") {
         // Still, so the panel looks solid and is drawn again only when something changes.
         const double rr = 0.75 * pxPerDeg;
         Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.6);
         Disc(p, x, y, core, 0, ink, ink, ink, 1);
-    } else if (p.state == "capture") {
+    } else if (state == "capture") {
         const double rr = 0.75 * pxPerDeg;
         Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.25);
-        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, 0.3, 0.85, 1.0, 1, std::clamp(p.progress, 0.0, 1.0));
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, 0.3, 0.85, 1.0, 1, std::clamp(progress, 0.0, 1.0));
         Disc(p, x, y, core, 0, ink, ink, ink, 1);
-    } else if (p.state == "done") {
+    } else if (state == "done") {
         Disc(p, x, y, 0.75 * pxPerDeg, 0, 0.25, 0.85, 0.4, 0.9);
         Disc(p, x, y, core, 0, 1, 1, 1, 1);
-    } else if (p.state == "fail") {
+    } else if (state == "fail") {
         Disc(p, x, y, 0.75 * pxPerDeg, 0.6 * pxPerDeg, 0.95, 0.35, 0.3, 0.9);
         Disc(p, x, y, core, 0, ink, ink, ink, 1);
     }
@@ -312,6 +473,7 @@ struct Buffer {
 };
 
 struct Buffers {
+    int w = kMaxW, h = kMaxH;  // the biggest picture they take
     int drm = -1;
     gbm_device *gbm = nullptr;
     Buffer b[3];
@@ -319,14 +481,15 @@ struct Buffers {
     bool ok = false;
 
     bool Make() {
+        std::lock_guard<std::mutex> lk(g_vr);
         drm = open("/dev/dri/renderD128", O_RDWR | O_CLOEXEC);
         if (drm >= 0) gbm = gbm_create_device(drm);
         for (Buffer &x : b) {
             // ABGR8888 is R, G, B, A in memory, like Panel::px.
-            if (gbm) x.bo = gbm_bo_create(gbm, kMaxW, kMaxH, GBM_FORMAT_ABGR8888, GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR);
+            if (gbm) x.bo = gbm_bo_create(gbm, w, h, GBM_FORMAT_ABGR8888, GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR);
             if (!x.bo || (x.fd = gbm_bo_get_fd(x.bo)) < 0) break;
             vr::DmabufAttributes_t a{};
-            a.unWidth = kMaxW, a.unHeight = kMaxH;
+            a.unWidth = w, a.unHeight = h;
             a.unDepth = a.unMipLevels = a.unArrayLayers = a.unSampleCount = 1;
             a.unFormat = DRM_FORMAT_ABGR8888;
             a.ulModifier = DRM_FORMAT_MOD_LINEAR;
@@ -340,12 +503,17 @@ struct Buffers {
         ok = b[2].handle != 0;
         if (!ok) {
             std::fprintf(stderr, "ft-gazepanel: no shared buffers; falling back to SetOverlayRaw (it flickers)\n");
-            Drop();
+            DropLocked();
         }
         return ok;
     }
 
     void Drop() {
+        std::lock_guard<std::mutex> lk(g_vr);
+        DropLocked();
+    }
+
+    void DropLocked() {
         for (Buffer &x : b) {
             if (x.handle) vr::VRIPCResourceManager()->UnrefResource(x.handle);
             if (x.fd >= 0) close(x.fd);
@@ -362,6 +530,7 @@ struct Buffers {
     void Present(vr::IVROverlay *ov, vr::VROverlayHandle_t h, const Panel &p) {
         vr::EVROverlayError e;
         if (!ok) {
+            std::lock_guard<std::mutex> lk(g_vr);
             e = ov->SetOverlayRaw(h, const_cast<uint8_t *>(p.px.data()), uint32_t(p.w), uint32_t(p.h), 4);
         } else {
             Buffer &x = b[next];
@@ -383,7 +552,8 @@ struct Buffers {
                 }
             }
             gbm_bo_unmap(x.bo, mapping);
-            const vr::VRTextureBounds_t bounds{0, 0, float(p.w) / kMaxW, float(p.h) / kMaxH};
+            std::lock_guard<std::mutex> lk(g_vr);
+            const vr::VRTextureBounds_t bounds{0, 0, float(p.w) / w, float(p.h) / this->h};
             ov->SetOverlayTextureBounds(h, &bounds);
             vr::Texture_t tex = {&x.handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma};
             e = ov->SetOverlayTexture(h, &tex);
@@ -393,18 +563,217 @@ struct Buffers {
     }
 };
 
+// ---------------------------------------------------------------- the rec mode's logs (see the top)
+
+void PutFloat(FILE *f, double v) { std::fprintf(f, "%.6g", std::isfinite(v) ? v : 0.0); }
+double Sec(int64_t ns) { return double(ns) / 1e9; }
+
+// A JSON-lines file the main thread writes.
+struct Lines {
+    FILE *f = nullptr;
+    long rows = 0;
+    bool Open(const std::string &path) {
+        Close();
+        if (!(f = std::fopen(path.c_str(), "ae"))) return false;
+        std::setvbuf(f, nullptr, _IOFBF, 1 << 16);
+        rows = 0;
+        return true;
+    }
+    long Close() {
+        if (!f) return rows;
+        if (std::fflush(f) != 0 || std::ferror(f)) std::fprintf(stderr, "ft-gazepanel: writing a log failed\n");
+        std::fclose(f);
+        f = nullptr;
+        return rows;
+    }
+};
+
+// The headset's pose 250 times a second (as the hand panel's, head only), on a thread of its
+// own, so a picture or a command never delays a sample.
+struct PoseLog {
+    FILE *f = nullptr;
+    std::thread th;
+    std::atomic<bool> run{false};
+    std::atomic<long> samples{0};
+
+    bool Start(const std::string &path, bool vr) {
+        Stop();
+        if (!(f = std::fopen(path.c_str(), "ae"))) return false;
+        std::setvbuf(f, nullptr, _IOFBF, 1 << 16);
+        samples = 0;
+        if (vr) run = true, th = std::thread([this] { Run(); });
+        return true;
+    }
+
+    long Stop() {
+        if (!f) return 0;
+        run = false;
+        if (th.joinable()) th.join();
+        if (std::fflush(f) != 0 || std::ferror(f)) std::fprintf(stderr, "ft-gazepanel: writing the pose log failed\n");
+        std::fclose(f);
+        f = nullptr;
+        return samples;
+    }
+
+    void Run() {
+        auto *sys = vr::VRSystem();
+        vr::TrackedDevicePose_t pose{};
+        int64_t next = RawNs(), lastFlush = next;
+        while (run) {
+            int64_t before, after;
+            {
+                std::lock_guard<std::mutex> lk(g_vr);
+                before = RawNs();
+                sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &pose, 1);
+                after = RawNs();
+            }
+            const int64_t t = before + (after - before) / 2;
+            std::fprintf(f, "{\"t\": %.6f, \"hmd\": ", Sec(t));
+            if (!pose.bDeviceIsConnected) {
+                std::fputs("null}\n", f);
+            } else {
+                std::fputs("{\"m\": [", f);
+                for (int k = 0; k < 12; ++k) {
+                    if (k) std::fputs(", ", f);
+                    PutFloat(f, pose.mDeviceToAbsoluteTracking.m[k / 4][k % 4]);
+                }
+                std::fprintf(f, "], \"r\": %d, \"ok\": %s}}\n", int(pose.eTrackingResult), pose.bPoseIsValid ? "true" : "false");
+            }
+            ++samples;
+            if (t - lastFlush >= 1'000'000'000) std::fflush(f), lastFlush = t;
+            next += kPosePeriodNs;
+            const int64_t now = RawNs();
+            if (next < now - kPosePeriodNs) next = now;
+            // Linux has no absolute sleeps on CLOCK_MONOTONIC_RAW (EINVAL at once: the loop spun at
+            // ~160k poses/s), so sleep the gap relative on CLOCK_MONOTONIC.
+            if (const int64_t wait = next - now; wait > 0) {
+                const timespec ts{time_t(wait / 1'000'000'000), long(wait % 1'000'000'000)};
+                clock_nanosleep(CLOCK_MONOTONIC, 0, &ts, nullptr);
+            }
+        }
+    }
+};
+
+// ---------------------------------------------------------------- the rec mode's dot
+
+// A head-relative direction (yaw +left, pitch +up; see ToPixel) as a unit vector in the head
+// frame (+x right, +y up, -z forward), and back.
+void Direction(double yaw, double pitch, double v[3]) {
+    const double yr = yaw * M_PI / 180, pr = pitch * M_PI / 180;
+    v[0] = -std::sin(yr) * std::cos(pr), v[1] = std::sin(pr), v[2] = -std::cos(yr) * std::cos(pr);
+}
+
+void Angles(const double v[3], double &yaw, double &pitch) {
+    yaw = std::atan2(-v[0], -v[2]) * 180 / M_PI;
+    pitch = std::atan2(v[1], std::hypot(v[0], v[2])) * 180 / M_PI;
+}
+
+// An overlay's transform at `at`, its face (+z) turned toward `eye`, upright (the hand panel's).
+vr::HmdMatrix34_t Facing(const double at[3], const double eye[3]) {
+    auto norm = [](double v[3]) {
+        const double n = std::sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]);
+        if (n < 1e-9) return false;
+        v[0] /= n, v[1] /= n, v[2] /= n;
+        return true;
+    };
+    double z[3] = {eye[0] - at[0], eye[1] - at[1], eye[2] - at[2]};
+    if (!norm(z)) z[0] = z[1] = 0, z[2] = 1;
+    double x[3] = {z[2], 0, -z[0]};
+    if (!norm(x)) x[0] = 1, x[1] = x[2] = 0;
+    const double y[3] = {z[1] * x[2] - z[2] * x[1], z[2] * x[0] - z[0] * x[2], z[0] * x[1] - z[1] * x[0]};
+    vr::HmdMatrix34_t m{};
+    for (int r = 0; r < 3; ++r)
+        m.m[r][0] = float(x[r]), m.m[r][1] = float(y[r]), m.m[r][2] = float(z[r]), m.m[r][3] = float(at[r]);
+    return m;
+}
+
+// m's point p in the room -> the device frame (m is device to room, rotation orthonormal).
+void ToDevice(const vr::HmdMatrix34_t &m, const double p[3], double out[3]) {
+    const double d[3] = {p[0] - m.m[0][3], p[1] - m.m[1][3], p[2] - m.m[2][3]};
+    for (int c = 0; c < 3; ++c) out[c] = m.m[0][c] * d[0] + m.m[1][c] * d[1] + m.m[2][c] * d[2];
+}
+
+void ToRoom(const vr::HmdMatrix34_t &m, const double p[3], double out[3]) {
+    for (int r = 0; r < 3; ++r) out[r] = m.m[r][0] * p[0] + m.m[r][1] * p[1] + m.m[r][2] * p[2] + m.m[r][3];
+}
+
+struct Dot {
+    std::string mode = "off";  // off, fixed, path, world
+    double yaw = 0, pitch = 0;
+    std::string state = "look";
+    double progress = 0;
+    // path
+    std::string kind;
+    double a[4] = {0, 0, 0, 0}, seconds = 1;
+    int64_t start = -1;  // the path's first frame's photon time (ns), set on that frame
+    // world
+    double room[3] = {0, 0, 0}, head[3] = {0, 0, 0};  // where it is, and the head-frame point it came from
+    // drawn: what the overlay's picture shows, and whether it's visible
+    std::string drawnState;
+    double drawnProgress = -1, drawnBg = -1;
+    bool shown = false;
+    bool changed = true;  // log a row on the next frame
+};
+
+// Where a path's dot is t seconds after it started.
+void PathAt(const Dot &d, double t, double &yaw, double &pitch) {
+    const double ph = t / std::max(d.seconds, 0.05);
+    if (d.kind == "circle") {
+        yaw = d.a[0] + d.a[2] * std::sin(2 * M_PI * ph), pitch = d.a[1] + d.a[2] * std::cos(2 * M_PI * ph);
+    } else if (d.kind == "line") {
+        double f = std::fmod(ph, 2.0);
+        if (f > 1) f = 2 - f;
+        yaw = d.a[0] + (d.a[2] - d.a[0]) * f, pitch = d.a[1] + (d.a[3] - d.a[1]) * f;
+    } else {  // eight
+        yaw = d.a[0] + d.a[2] * std::sin(2 * M_PI * ph), pitch = d.a[1] + d.a[3] * std::sin(4 * M_PI * ph);
+    }
+}
+
+// --no-vr: a picture as a PAM file (RGBA, straight alpha), as the hand panel's.
+void Dump(const Panel &p, const std::string &path) {
+    if (FILE *f = std::fopen(path.c_str(), "wb")) {
+        std::fprintf(f, "P7\nWIDTH %d\nHEIGHT %d\nDEPTH 4\nMAXVAL 255\nTUPLTYPE RGB_ALPHA\nENDHDR\n", p.w, p.h);
+        std::fwrite(p.px.data(), 1, p.px.size(), f);
+        std::fclose(f);
+    }
+}
+
+// The headset's pose `ahead` seconds from now; --no-vr's is at the room's origin, looking down -z.
+vr::TrackedDevicePose_t HeadPose(bool noVr, float ahead) {
+    vr::TrackedDevicePose_t pose{};
+    if (noVr) {
+        pose.bPoseIsValid = pose.bDeviceIsConnected = true;
+        pose.mDeviceToAbsoluteTracking.m[0][0] = pose.mDeviceToAbsoluteTracking.m[1][1] = pose.mDeviceToAbsoluteTracking.m[2][2] = 1;
+    } else {
+        std::lock_guard<std::mutex> lk(g_vr);
+        vr::VRSystem()->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, ahead, &pose, 1);
+    }
+    return pose;
+}
+
+bool Is(const char *buf, const char *word, const char **rest = nullptr) {
+    const size_t n = std::strlen(word);
+    if (std::strncmp(buf, word, n) || (buf[n] && buf[n] != ' ')) return false;
+    if (rest) *rest = buf[n] ? buf + n + 1 : buf + n;
+    return true;
+}
+
 }  // namespace
 
 int main(int argc, char **argv) {
-    bool watchStdin = false;
-    std::string sockName = "ft_gazepanel";
+    bool watchStdin = false, noVr = false;
+    std::string sockName = "ft_gazepanel", overlayKey = "frametop.gazepanel", dumpDir;
     double distance = 1.5;
     for (int i = 1; i < argc; ++i) {
         if (!std::strcmp(argv[i], "--watch-stdin")) watchStdin = true;
+        else if (!std::strcmp(argv[i], "--no-vr")) noVr = true;
         else if (!std::strcmp(argv[i], "--socket") && i + 1 < argc) sockName = argv[++i];
+        else if (!std::strcmp(argv[i], "--overlay") && i + 1 < argc) overlayKey = argv[++i];
+        else if (!std::strcmp(argv[i], "--dump") && i + 1 < argc) dumpDir = argv[++i];
         else if (!std::strcmp(argv[i], "--distance") && i + 1 < argc) distance = std::clamp(std::atof(argv[++i]), 0.5, 5.0);
         else {
-            std::fprintf(stderr, "usage: %s [--watch-stdin] [--socket NAME] [--distance METRES]\n", argv[0]);
+            std::fprintf(stderr, "usage: %s [--watch-stdin] [--socket NAME] [--overlay KEY] [--distance METRES] "
+                                 "[--no-vr [--dump DIR]]\n", argv[0]);
             return 2;
         }
     }
@@ -427,39 +796,176 @@ int main(int argc, char **argv) {
         return 1;
     }
 
-    // As Frametop's other SteamVR clients: background first, so we never start vrserver.
-    vr::EVRInitError err = vr::VRInitError_None;
-    vr::VR_Init(&err, vr::VRApplication_Background);
-    if (err == vr::VRInitError_None) {
-        vr::VR_Shutdown();
-        vr::VR_Init(&err, vr::VRApplication_Overlay);
+    vr::IVROverlay *ov = nullptr;
+    vr::VROverlayHandle_t h = vr::k_ulOverlayHandleInvalid, dotH = vr::k_ulOverlayHandleInvalid;
+    double period = 1 / 90.0, toPhotons = 0;
+    if (!noVr) {
+        // As Frametop's other SteamVR clients: background first, so we never start vrserver.
+        vr::EVRInitError err = vr::VRInitError_None;
+        vr::VR_Init(&err, vr::VRApplication_Background);
+        if (err == vr::VRInitError_None) {
+            vr::VR_Shutdown();
+            vr::VR_Init(&err, vr::VRApplication_Overlay);
+        }
+        if (err != vr::VRInitError_None) {
+            std::fprintf(stderr, "ft-gazepanel: SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
+            return 1;
+        }
+        ov = vr::VROverlay();
+        if (ov->CreateOverlay(overlayKey.c_str(), "Frametop gaze calibration", &h) != vr::VROverlayError_None ||
+            ov->CreateOverlay((overlayKey + ".dot").c_str(), "Frametop gaze dot", &dotH) != vr::VROverlayError_None) {
+            std::fprintf(stderr, "ft-gazepanel: can't create the overlay (another copy running?)\n");
+            return 1;
+        }
+        ov->SetOverlaySortOrder(h, 250);  // in front of Frametop's screens and the pointer's dot
+        ov->SetOverlaySortOrder(dotH, 251);
+        auto *sys = vr::VRSystem();
+        vr::ETrackedPropertyError pe;
+        const float hz = sys->GetFloatTrackedDeviceProperty(vr::k_unTrackedDeviceIndex_Hmd, vr::Prop_DisplayFrequency_Float, &pe);
+        if (pe == vr::TrackedProp_Success && hz > 30) period = 1.0 / hz;
+        const float ph = sys->GetFloatTrackedDeviceProperty(vr::k_unTrackedDeviceIndex_Hmd, vr::Prop_SecondsFromVsyncToPhotons_Float, &pe);
+        if (pe == vr::TrackedProp_Success && ph >= 0 && ph < 0.1) toPhotons = ph;
     }
-    if (err != vr::VRInitError_None) {
-        std::fprintf(stderr, "ft-gazepanel: SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
-        return 1;
-    }
-    vr::IVROverlay *ov = vr::VROverlay();
-    vr::VROverlayHandle_t h = vr::k_ulOverlayHandleInvalid;
-    if (ov->CreateOverlay("frametop.gazepanel", "Frametop gaze calibration", &h) != vr::VROverlayError_None) {
-        std::fprintf(stderr, "ft-gazepanel: can't create the overlay (another copy running?)\n");
-        return 1;
-    }
-    ov->SetOverlaySortOrder(h, 250);  // in front of Frametop's screens and the pointer's dot
     LoadFont();
-    Buffers buffers;
-    if (buffers.Make()) ov->SetOverlayFlag(h, vr::VROverlayFlags_IsPremultiplied, true);
+    Buffers buffers, dotBuffers;
+    dotBuffers.w = dotBuffers.h = kDotPx;
+    if (!noVr && buffers.Make()) ov->SetOverlayFlag(h, vr::VROverlayFlags_IsPremultiplied, true);
+    if (!noVr && dotBuffers.Make()) ov->SetOverlayFlag(dotH, vr::VROverlayFlags_IsPremultiplied, true);
+    if (!dumpDir.empty()) mkdir(dumpDir.c_str(), 0755);
 
     Panel p;
+    Panel dotPic;  // the rec mode's dot overlay's picture
+    dotPic.w = dotPic.h = kDotPx, dotPic.wDeg = kDotDeg;
+    Dot dot;
+    Lines targets, ipdLog;
+    PoseLog poses;
+    float ipdNow = -1;
+    int64_t ipdAt = 0;
+    int pictures = 0;
     bool visible = false, dirty = false, shown = false;  // shown: SteamVR shows it (after its first picture)
     auto place = [&] {
+        if (noVr) return;
         const double half = std::tan(p.wDeg * M_PI / 360);
         vr::HmdMatrix34_t m{};
-        m.m[0][0] = m.m[1][1] = m.m[2][2] = 1;
-        m.m[2][3] = float(-distance);
+        // Turned down about x by the strip's pitch (none for the others), facing the eyes.
+        const double tilt = p.strip ? kNotePitch * M_PI / 180 : 0, c = std::cos(tilt), sn = std::sin(tilt);
+        m.m[0][0] = 1;
+        m.m[1][1] = float(c), m.m[1][2] = float(-sn), m.m[2][1] = float(sn), m.m[2][2] = float(c);
+        m.m[1][3] = float(distance * sn), m.m[2][3] = float(-distance * c);
+        std::lock_guard<std::mutex> lk(g_vr);
         ov->SetOverlayTransformTrackedDeviceRelative(h, vr::k_unTrackedDeviceIndex_Hmd, &m);
         ov->SetOverlayWidthInMeters(h, float(2 * distance * half));
     };
-    std::fprintf(stderr, "ft-gazepanel running: @%s, %.2f m\n", sockName.c_str(), distance);
+    auto hideDot = [&] {
+        if (dot.shown && !noVr) {
+            std::lock_guard<std::mutex> lk(g_vr);
+            ov->HideOverlay(dotH);
+        }
+        dot.shown = false;
+    };
+    // The next frame's vsync count and when it reaches the eyes (RAW ns).
+    auto nextFrame = [&](int64_t now, uint64_t &frame) {
+        double since = 0;
+        frame = 0;
+        if (!noVr) {
+            float s = 0;
+            std::lock_guard<std::mutex> lk(g_vr);
+            if (vr::VRSystem()->GetTimeSinceLastVsync(&s, &frame)) since = s;
+        } else {
+            const int64_t per = int64_t(period * 1e9);
+            since = double(now % per) / 1e9, frame = uint64_t(now / per);
+        }
+        frame += 1;
+        return now + int64_t((period - std::min(since, period) + toPhotons) * 1e9);
+    };
+    auto ipdLook = [&](int64_t now, const char *why) {
+        if (noVr || !ipdLog.f) return;
+        vr::ETrackedPropertyError pe;
+        float v;
+        {
+            std::lock_guard<std::mutex> lk(g_vr);
+            v = vr::VRSystem()->GetFloatTrackedDeviceProperty(vr::k_unTrackedDeviceIndex_Hmd, vr::Prop_UserIpdMeters_Float, &pe);
+        }
+        ipdAt = now;
+        if (pe != vr::TrackedProp_Success || v == ipdNow) return;
+        ipdNow = v;
+        std::fprintf(ipdLog.f, "{\"t\": %.6f, \"ipd\": %.5f, \"from\": \"%s\"}\n", Sec(now), v, why);
+        ++ipdLog.rows;
+    };
+    // The rec mode's dot, once a frame: where it is at the next frame's photons, the overlay
+    // moved there, and a targets row when it moved or changed.
+    auto dotTick = [&](int64_t now) {
+        if (!p.rec || !visible || dot.mode == "off") {
+            if (dot.changed && targets.f && p.rec) {
+                uint64_t frame;
+                const int64_t t = nextFrame(now, frame);
+                std::fprintf(targets.f, "{\"t\": %.6f, \"set\": %.6f, \"frame\": %llu, \"dot\": \"off\", \"dist\": %.3f, \"bg\": %.3f}\n",
+                             Sec(t), Sec(now), (unsigned long long)frame, distance, p.bg);
+                ++targets.rows;
+            }
+            dot.changed = false;
+            hideDot();
+            return;
+        }
+        uint64_t frame;
+        const int64_t t = nextFrame(now, frame);
+        double yaw = dot.yaw, pitch = dot.pitch, dist = distance;
+        bool moved = false;
+        vr::HmdMatrix34_t m{};
+        bool absolute = false;
+        if (dot.mode == "path") {
+            if (dot.start < 0) dot.start = t;
+            PathAt(dot, Sec(t - dot.start), yaw, pitch);
+            moved = true;
+        }
+        if (dot.mode == "world") {
+            // The head where it will be at the photons, and the dot's direction from it.
+            const vr::TrackedDevicePose_t pose = HeadPose(noVr, float(Sec(t - now)));
+            if (pose.bPoseIsValid) {
+                double v[3];
+                ToDevice(pose.mDeviceToAbsoluteTracking, dot.room, v);
+                dist = std::sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]);
+                if (dist > 1e-6) {
+                    for (double &c : v) c /= dist;
+                    Angles(v, yaw, pitch);
+                }
+                const double eye[3] = {pose.mDeviceToAbsoluteTracking.m[0][3], pose.mDeviceToAbsoluteTracking.m[1][3],
+                                       pose.mDeviceToAbsoluteTracking.m[2][3]};
+                m = Facing(dot.room, eye);
+                absolute = true;
+            }
+            moved = true;
+        } else {
+            double v[3];
+            Direction(yaw, pitch, v);
+            const double at[3] = {v[0] * dist, v[1] * dist, v[2] * dist}, eye[3] = {0, 0, 0};
+            m = Facing(at, eye);
+        }
+        const bool redraw = dot.state != dot.drawnState || dot.progress != dot.drawnProgress || p.bg != dot.drawnBg;
+        if (redraw) {
+            dotPic.px.assign(size_t(dotPic.w) * dotPic.h * 4, 0);
+            DrawDot(dotPic, dotPic.w / 2.0, dotPic.h / 2.0, dotPic.w / dotPic.wDeg, dot.state, dot.progress, p.bg > 0.5 ? 0.0 : 1.0);
+            dot.drawnState = dot.state, dot.drawnProgress = dot.progress, dot.drawnBg = p.bg;
+            if (!noVr) dotBuffers.Present(ov, dotH, dotPic);
+        }
+        if (!noVr && (moved || dot.changed || redraw)) {
+            std::lock_guard<std::mutex> lk(g_vr);
+            if (absolute) ov->SetOverlayTransformAbsolute(dotH, vr::TrackingUniverseStanding, &m);
+            else ov->SetOverlayTransformTrackedDeviceRelative(dotH, vr::k_unTrackedDeviceIndex_Hmd, &m);
+            ov->SetOverlayWidthInMeters(dotH, float(2 * dist * std::tan(kDotDeg * M_PI / 360)));
+            if (!dot.shown) ov->ShowOverlay(dotH);
+        }
+        dot.shown = true;
+        if (targets.f && (moved || dot.changed || redraw)) {
+            std::fprintf(targets.f, "{\"t\": %.6f, \"set\": %.6f, \"frame\": %llu, \"dot\": \"%s\", \"yaw\": %.4f, \"pitch\": %.4f, "
+                                    "\"dist\": %.3f, \"state\": \"%s\", \"progress\": %.3f, \"bg\": %.3f}\n",
+                         Sec(t), Sec(now), (unsigned long long)frame, dot.mode.c_str(), yaw, pitch, dist, dot.state.c_str(),
+                         dot.progress, p.bg);
+            ++targets.rows;
+        }
+        dot.changed = false;
+    };
+    std::fprintf(stderr, "ft-gazepanel running: @%s, %.2f m%s\n", sockName.c_str(), distance, noVr ? ", no SteamVR" : "");
 
     while (!g_stop) {
         char buf[512];
@@ -467,30 +973,118 @@ int main(int argc, char **argv) {
         socklen_t fromLen = sizeof from;
         ssize_t n;
         while ((n = recvfrom(sock, buf, sizeof buf - 1, 0, reinterpret_cast<sockaddr *>(&from), &fromLen)) > 0) {
+            while (n > 0 && (buf[n - 1] == '\n' || buf[n - 1] == '\r')) --n;  // from echo or socat
             buf[n] = 0;
             std::string reply = "ok";
             char word[16] = "", state[16] = "";
             double a = 0, b = 0, c = 0, d = 0, e = 0;
             int rest = 0;
+            const char *arg = nullptr;
             if (!std::strncmp(buf, "show ", 5)) {
+                p.rec = !std::strcmp(buf + 5, "rec");
                 p.full = !std::strcmp(buf + 5, "full");
                 p.fit = !std::strcmp(buf + 5, "fit");
-                const bool five = !std::strcmp(buf + 5, "five");
-                p.w = p.full ? kFullW : p.fit ? kFitW : five ? kFiveW : kQuickPx;
-                p.h = p.full ? kFullH : p.fit ? kFitH : five ? kFiveH : kQuickPx;
-                p.wDeg = p.full ? kFullDeg : p.fit ? kFitDeg : five ? kFiveDeg : kQuickDeg;
+                p.strip = !std::strcmp(buf + 5, "note");
+                const bool five = !std::strcmp(buf + 5, "five"), big = p.full || p.rec;
+                p.w = big ? kFullW : p.fit ? kFitW : five ? kFiveW : p.strip ? kNoteW : kQuickPx;
+                p.h = big ? kFullH : p.fit ? kFitH : five ? kFiveH : p.strip ? kNoteH : kQuickPx;
+                p.wDeg = big ? kFullDeg : p.fit ? kFitDeg : five ? kFiveDeg : p.strip ? kNoteDeg : kQuickDeg;
                 p.title.clear(), p.text.clear(), p.note.clear(), p.dotOn = false, p.state = "off";
                 p.eyes[0] = p.eyes[1] = EyeCard{}, p.hints.clear();
+                p.step.clear(), p.big.clear(), p.action.clear(), p.keys.clear();
+                p.recOn = p.paused = false, p.countdown = -1;
+                dot.mode = "off", dot.changed = true;
                 place();
                 visible = dirty = true;  // shown with its first picture
             } else if (!std::strcmp(buf, "hide")) {
-                ov->HideOverlay(h);
+                if (!noVr) {
+                    std::lock_guard<std::mutex> lk(g_vr);
+                    ov->HideOverlay(h);
+                }
+                hideDot();
                 visible = shown = false;
+                dot.mode = "off", dot.changed = true;
             } else if (std::sscanf(buf, "bg %lf", &a) == 1) {
-                p.bg = std::clamp(a, 0.0, 1.0), dirty = true;
+                p.bg = std::clamp(a, 0.0, 1.0), dirty = true, dot.changed = true;
             } else if (std::sscanf(buf, "dot %lf %lf %15s %lf", &a, &b, state, &c) >= 3) {
                 p.dotYaw = a, p.dotPitch = b, p.state = state, p.progress = c;
-                p.dotOn = std::strcmp(state, "off") != 0, dirty = true;
+                p.dotOn = std::strcmp(state, "off") != 0;
+                if (p.rec) {
+                    dot.mode = p.dotOn ? "fixed" : "off", dot.yaw = a, dot.pitch = b, dot.state = state, dot.progress = c;
+                    dot.changed = true;
+                } else {
+                    dirty = true;
+                }
+            } else if (Is(buf, "path", &arg)) {
+                char kind[16] = "";
+                double v[5] = {0, 0, 0, 0, 0};
+                const int got = std::sscanf(arg, "%15s %lf %lf %lf %lf %lf", kind, &v[0], &v[1], &v[2], &v[3], &v[4]);
+                const int want = !std::strcmp(kind, "circle") ? 5 : (!std::strcmp(kind, "line") || !std::strcmp(kind, "eight")) ? 6 : 0;
+                if (!p.rec) {
+                    reply = "error path is for show rec";
+                } else if (!want || got != want || v[want - 2] <= 0) {
+                    reply = "error path circle|line|eight <numbers> <seconds>";
+                } else {
+                    dot.mode = "path", dot.kind = kind, dot.seconds = v[want - 2], dot.start = -1;
+                    for (int k = 0; k < 4; ++k) dot.a[k] = k < want - 2 ? v[k] : 0;
+                    dot.state = "look", dot.progress = 0, dot.changed = true;
+                }
+            } else if (Is(buf, "world", &arg)) {
+                double v[3];
+                char st[16] = "look";
+                double prog = 0;
+                if (!std::strcmp(arg, "off")) {
+                    if (dot.mode == "world") dot.mode = "off", dot.changed = true;
+                } else if (!p.rec) {
+                    reply = "error world is for show rec";
+                } else if (std::sscanf(arg, "%lf %lf %lf %15s %lf", &v[0], &v[1], &v[2], st, &prog) < 3) {
+                    reply = "error world <x> <y> <z> [<state> [<progress>]]";
+                } else {
+                    const vr::TrackedDevicePose_t pose = HeadPose(noVr, 0);
+                    if (!pose.bPoseIsValid) {
+                        reply = "error no headset pose";
+                    } else {
+                        if (dot.mode != "world" || v[0] != dot.head[0] || v[1] != dot.head[1] || v[2] != dot.head[2])
+                            ToRoom(pose.mDeviceToAbsoluteTracking, v, dot.room);
+                        std::copy(v, v + 3, dot.head);
+                        dot.mode = "world", dot.state = st, dot.progress = prog, dot.changed = true;
+                        char r[96];
+                        std::snprintf(r, sizeof r, "ok %.4f %.4f %.4f", dot.room[0], dot.room[1], dot.room[2]);
+                        reply = r;
+                    }
+                }
+            } else if (std::sscanf(buf, "distance %lf", &a) == 1) {
+                distance = std::clamp(a, 0.25, 10.0), dot.changed = true;
+                place();
+            } else if (Is(buf, "ipd")) {
+                float v = 0.063f;
+                vr::ETrackedPropertyError pe = vr::TrackedProp_Success;
+                if (!noVr) {
+                    std::lock_guard<std::mutex> lk(g_vr);
+                    v = vr::VRSystem()->GetFloatTrackedDeviceProperty(vr::k_unTrackedDeviceIndex_Hmd, vr::Prop_UserIpdMeters_Float, &pe);
+                }
+                char r[48];
+                std::snprintf(r, sizeof r, "ok %.5f", v);
+                reply = pe == vr::TrackedProp_Success ? r : "error no IPD from SteamVR";
+            } else if (Is(buf, "log", &arg)) {
+                if (!std::strncmp(arg, "start ", 6)) {
+                    const std::string dir = arg + 6;
+                    if (!targets.Open(dir + "/targets.jsonl") || !ipdLog.Open(dir + "/ipd.jsonl") ||
+                        !poses.Start(dir + "/poses.jsonl", !noVr)) {
+                        targets.Close(), ipdLog.Close(), poses.Stop();
+                        reply = "error can't write in " + dir;
+                    } else {
+                        ipdNow = -1, dot.changed = true;
+                        ipdLook(RawNs(), "start");
+                    }
+                } else if (!std::strcmp(arg, "stop")) {
+                    char r[96];
+                    const long tr = targets.Close(), ir = ipdLog.Close(), pr = poses.Stop();
+                    std::snprintf(r, sizeof r, "ok %ld %ld %ld", tr, pr, ir);
+                    reply = r;
+                } else {
+                    reply = "error log start <dir> | log stop";
+                }
             } else if (int k; std::sscanf(buf, "eye %d %lf %lf %lf %lf %lf %n", &k, &a, &b, &c, &d, &e, &rest) >= 6 &&
                        rest > 0 && (k == 0 || k == 1)) {
                 p.eyes[k] = EyeCard{buf + rest, a, b, c, d, e}, dirty = true;
@@ -503,12 +1097,26 @@ int main(int argc, char **argv) {
                     at = bar + 1;
                 }
                 dirty = true;
-            } else if (!std::strncmp(buf, "title", 5)) {
-                p.title = buf[5] == ' ' ? buf + 6 : "", dirty = true;
-            } else if (!std::strncmp(buf, "text", 4)) {
-                p.text = buf[4] == ' ' ? buf + 5 : "", dirty = true;
-            } else if (!std::strncmp(buf, "note", 4)) {
-                p.note = buf[4] == ' ' ? buf + 5 : "", dirty = true;
+            } else if (Is(buf, "title", &arg)) {
+                p.title = arg, dirty = true;
+            } else if (Is(buf, "text", &arg)) {
+                p.text = arg, dirty = true;
+            } else if (Is(buf, "note", &arg)) {
+                p.note = arg, dirty = true;
+            } else if (Is(buf, "step", &arg)) {
+                p.step = arg, dirty = true;
+            } else if (Is(buf, "big", &arg)) {
+                p.big = arg, dirty = true;
+            } else if (Is(buf, "action", &arg)) {
+                p.action = arg, dirty = true;
+            } else if (Is(buf, "keys", &arg)) {
+                p.keys = arg, dirty = true;
+            } else if (Is(buf, "rec", &arg) && (!std::strcmp(arg, "on") || !std::strcmp(arg, "off"))) {
+                p.recOn = !std::strcmp(arg, "on"), dirty = true;
+            } else if (Is(buf, "paused", &arg) && (!std::strcmp(arg, "on") || !std::strcmp(arg, "off"))) {
+                p.paused = !std::strcmp(arg, "on"), dirty = true;
+            } else if (Is(buf, "countdown", &arg)) {
+                p.countdown = !std::strcmp(arg, "off") ? -1 : std::clamp(std::atof(arg), 0.0, 1.0), dirty = true;
             } else if (std::sscanf(buf, "%15s", word) == 1 && !std::strcmp(word, "ping")) {
                 reply = visible ? "ok shown" : "ok hidden";
             } else {
@@ -518,26 +1126,57 @@ int main(int argc, char **argv) {
                 sendto(sock, reply.data(), reply.size(), 0, reinterpret_cast<sockaddr *>(&from), fromLen);
             fromLen = sizeof from;
         }
-        vr::VREvent_t ev;
-        while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev))
-            if (ev.eventType == vr::VREvent_Quit) {
-                vr::VRSystem()->AcknowledgeQuit_Exiting();
-                g_stop = true;
+        if (!noVr) {
+            vr::VREvent_t ev;
+            std::unique_lock<std::mutex> lk(g_vr);
+            while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev)) {
+                if (ev.eventType == vr::VREvent_Quit) {
+                    vr::VRSystem()->AcknowledgeQuit_Exiting();
+                    g_stop = true;
+                } else if (ev.eventType == vr::VREvent_IpdChanged && ipdLog.f) {
+                    ipdNow = ev.data.ipd.ipdMeters;
+                    std::fprintf(ipdLog.f, "{\"t\": %.6f, \"ipd\": %.5f, \"from\": \"event\"}\n", Sec(RawNs()), ipdNow);
+                    ++ipdLog.rows;
+                }
             }
+        }
         if (visible && dirty) {
             Draw(p);
-            buffers.Present(ov, h, p);
-            if (!shown) ov->ShowOverlay(h), shown = true;
+            ++pictures;
+            if (!dumpDir.empty()) Dump(p, dumpDir + "/" + std::to_string(pictures) + ".pam");
+            if (!noVr) {
+                buffers.Present(ov, h, p);
+                std::lock_guard<std::mutex> lk(g_vr);
+                if (!shown) ov->ShowOverlay(h);
+            }
+            shown = true;
             dirty = false;
         }
-        // Until a command comes, or 10 ms while shown (SteamVR's events). Hidden, it waits up to a
-        // second: it woke 20 to 30 times a second for nothing, the main cost left with gaze idle.
-        // A closed stdin (--watch-stdin) wakes it too, so quitting doesn't wait.
+        const int64_t now = RawNs();
+        dotTick(now);
+        if (ipdLog.f && now - ipdAt >= kIpdPeriodNs) ipdLook(now, "look");
+        // Until a command comes, or the next frame while the rec mode's dot moves, or 10 ms while
+        // shown (SteamVR's events). Hidden, it waits up to a second: it woke 20 to 30 times a
+        // second for nothing, the main cost left with gaze idle. A closed stdin (--watch-stdin)
+        // wakes it too, so quitting doesn't wait.
+        double wait = visible ? 0.010 : 1.0;
+        if (visible && p.rec && (dot.mode == "path" || dot.mode == "world")) {
+            uint64_t frame;
+            // Just after the next vsync: the compositor has taken this frame's transform.
+            wait = std::max(0.001, Sec(nextFrame(now, frame) - now) - toPhotons + 0.001);
+        }
+        if (ipdLog.f) wait = std::min(wait, Sec(kIpdPeriodNs));
         pollfd fds[2] = {{sock, POLLIN, 0}, {0, POLLIN, 0}};
-        poll(fds, watchStdin ? 2 : 1, visible ? 10 : 1000);
+        const timespec ts{time_t(wait), long((wait - std::floor(wait)) * 1e9)};
+        ppoll(fds, watchStdin ? 2 : 1, &ts, nullptr);
     }
-    ov->DestroyOverlay(h);
-    buffers.Drop();
-    vr::VR_Shutdown();
+    targets.Close(), ipdLog.Close(), poses.Stop();
+    if (!noVr) {
+        std::lock_guard<std::mutex> lk(g_vr);
+        ov->DestroyOverlay(dotH);
+        ov->DestroyOverlay(h);
+    }
+    buffers.Drop(), dotBuffers.Drop();
+    if (!noVr) vr::VR_Shutdown();
     return 0;
 }

gaze/rec/CONSENT.md

@@ -0,0 +1,96 @@
+# Recording your eyes for the Frametop eye dataset
+
+Version: 2026-10-16
+
+Frametop's eye tracking needs a better model than it has: one that keeps working when you look far to the side, blink, squint, wear glasses or makeup, or when the headset moves on your face. To train it, we're collecting recordings of many people's eyes into a dataset that only the maintainers can read. This page explains what the eye recorder records, where it goes, who can see it, and what you agree to if you take part. Please read all of it.
+
+## Who runs this
+
+Frametop is an independent open-source project, maintained by DeeJanuz (<https://github.com/DeeJanuz>). It isn't affiliated with or endorsed by Valve or Hugging Face. Steam and Steam Frame are trademarks of Valve Corporation. This text was written without a lawyer, in good faith; if something in it seems wrong or unclear, please say so before you take part.
+
+You can reach the maintainer through the Frametop repository's issues (<https://github.com/Frametop/frametop/issues>) or the dataset's discussion page on Hugging Face. Both are public.
+
+## Who can take part
+
+- You must be **18 or older**.
+- For now, you can't take part if you live in **Illinois, Texas or Washington** (USA). Those states have laws about biometric data, which includes images of the iris, that need more than this project can provide yet.
+- Take part only if you're comfortable with the maintainers seeing close-up images of your eyes.
+
+## Your eyes are biometric data
+
+The headset's eye cameras film each eye from a few centimetres away. The iris in those images can identify a person, the way a fingerprint can. That's why this dataset works differently from Frametop's hand dataset:
+
+- **The eye images are never public.** Your headset encrypts them before they're uploaded, so that only the maintainers' keys can read them. Hugging Face, other contributors and people who download the dataset get only the encrypted files.
+- **What's trained from them can be public.** Models trained on the dataset can be published and shared. They don't contain your images, though no one can rule out that a model remembers something about the eyes it was trained on.
+
+## What is recorded
+
+While a round runs, the eye recorder saves:
+
+- **Eye images.** Infrared images from the headset's 2 eye cameras, about 90 a second each. They show your eyes, eyelids, eyelashes and the skin around them, and your glasses or contact lenses if you wear them. Encrypted in the export.
+- **Where you looked.** Where each dot was when you looked at it, what Frametop's eye tracker and SteamVR's found in each image (pupil and glint positions, gaze directions), and how the fit check rated the headset's position. These stay readable in the export, so a pull request can be checked before anyone decrypts it.
+- **Motion and settings.** The headset's position and rotation, many times a second, and its IPD setting (the distance between the lenses).
+- **Prompts.** What you were asked to do and when.
+- **Your answers.** Glasses, contacts or inserts, eye makeup, the face pad and strap, and your measured IPD if you gave it. Readable in the export.
+- **Health answers, only if you tick their box.** Eye surgery and conditions such as a turned eye, a droopy eyelid or nystagmus. They're health data, so they're kept in a separate file and encrypted like the images. You can leave them out, and still take part.
+- **Times.** When each round was recorded, with your time zone.
+- **A contributor id.** A random number made on your headset the first time you agree to this page. It isn't made from your name, your Steam account or your headset. It keeps your rounds together and lets you withdraw them.
+
+The recorder doesn't record sound, the headset's outward cameras, your name, your email address or your Steam account.
+
+**Your Hugging Face account.** You upload with your own Hugging Face account, and your pull request shows its username next to your contributor id, publicly. The images in it are encrypted, but anyone can see that this account contributed eye recordings, and read the readable parts listed above. Use an account you're happy to have linked to them.
+
+## What it's used for
+
+- Training and testing eye-tracking models: finding the pupil, the glints and the eyelids, and working out where you're looking, including when the headset sits badly. Mainly for Frametop, and possibly elsewhere (below, "The license").
+- Checking and improving the recorder's fit coach, which helps people put the headset on well.
+- It isn't used to recognise or identify people, and the maintainers won't try.
+
+## Nothing leaves your headset unless you send it
+
+- Recordings stay on your headset, in `~/.local/share/frametop/eyes/contrib`. Nothing is uploaded automatically.
+- Before you share anything, you can look through every recording in the Review page. You can delete any stretch of a take, a whole take or a whole round. Export leaves out what you deleted.
+- You upload the export yourself: from the Upload page, or with Export and upload, which sends it once it's exported. That opens a pull request on the dataset. The maintainer checks it before it becomes part of the dataset, and may decline it. Until it's merged, you can close the pull request yourself.
+
+## Where it goes, and who can read it
+
+- The dataset is hosted on Hugging Face (<https://huggingface.co/datasets/DeeJanuz/frametop-eyes>), whose servers may be outside your country, for example in the USA. It's gated: people must accept its terms to download it, and what they get has the eye images and health answers encrypted.
+- Only the maintainers hold the keys that decrypt them. Today that's DeeJanuz, with a key on each of two of their own computers (one of them trains the models) and a backup copy stored offline. The keys are never uploaded. If a key ever leaked, uploads made to it could be read: the maintainers would say so on the dataset's page and stop using it.
+- Decrypted images are kept only on the maintainers' own computers, for training and checking models, and aren't shared with anyone else or uploaded to other services.
+- Recordings stay in the dataset until they're withdrawn or the dataset is taken down.
+
+## Safety
+
+The rounds ask you to look at dots, including far to the sides, to blink and squint, and sometimes to turn your head or move the headset on your face. Sit where you normally use the headset and stop whenever anything is uncomfortable, or if your eyes get tired or sore. If you have a condition that bright or moving lights affect, such as photosensitive epilepsy, don't take part. You take part at your own risk.
+
+## The license
+
+- **You give the maintainer of Frametop a non-exclusive, worldwide, royalty-free license to use your contribution for any purpose, including commercially.** That includes copying it, changing it, and training, publishing and selling models made from it, in Frametop and elsewhere. It doesn't include publishing your eye images or health answers, or giving them to anyone outside the maintainers. The maintainer today is DeeJanuz. If someone else, or an organization, takes over maintaining Frametop, this license passes to them, with the same limits. It's non-exclusive: you keep any rights you have in your recordings and can do what you like with your own copies. It ends for a recording when you withdraw it, except for models already trained with it.
+- You confirm that you have the right to give this license: the recordings are of your own eyes.
+- There is no payment. The dataset and the eye recorder come with no warranty, and as far as the law allows, the maintainer isn't liable for any loss or damage from taking part.
+
+## Withdrawing
+
+You can withdraw a contribution at any time, without giving a reason:
+
+- **Before it's merged:** close your pull request on Hugging Face. The maintainer deletes its files.
+- **After it's merged:** post on the dataset's discussion page, or in the Frametop repository's issues, with your contributor id (shown in the eye recorder) and which rounds, or "all". Post from the Hugging Face account that uploaded them if you can, so the maintainer can tell it's you; otherwise, say how to check. These requests are public, so don't add anything else that identifies you.
+
+Then, usually within 30 days:
+
+- your recordings are deleted from the dataset and purged from its repository's history, and the maintainers delete their decrypted copies;
+- they're left out of anything trained after that.
+
+What can't be undone: models already trained with them.
+
+## Your rights
+
+Depending on where you live (for example in the EU or the UK, under the GDPR), the law may give you more rights over this data: to get a copy of it, to have it corrected or deleted, to object to its use, and to complain to your data protection authority. Eye images that can identify you and health answers are special kinds of data there, and they're used only because you explicitly agreed to it, here and with the health answers' own tick. You can withdraw that agreement at any time, as above. Withdrawing doesn't make earlier use unlawful. To use any of these rights, contact the maintainer as above.
+
+## Changes to this text
+
+If this text changes, the eye recorder shows the new version and asks you again before your next round. Each upload records the version you agreed to, and recordings you already uploaded stay under that version, unless you agree to a newer one or withdraw them.
+
+## Agreeing
+
+By ticking the three boxes and "Agree and continue", you confirm that you're 18 or older, that you don't live in Illinois, Texas or Washington, and that you agree to the above, including the use of images of your eyes as described. You can still decide not to upload anything.

gaze/rec/DESIGN.md

@@ -0,0 +1,178 @@
+# Eye recorder: design (2026-10-14)
+
+The eye recorder guides a person through recording their eyes with the headset's two eye cameras, at known gaze targets and through the cases where eye tracking breaks. It also fits the headset: it shows whether each eye sits where the cameras see it best, shows the IPD setting, and catches the headset shifting on the face, with instructions to put it back. Recordings stay on the headset until the person uploads them. Only the maintainers can read what's uploaded (below, "Privacy").
+
+It's the hand recorder's sibling (`hands/rec/DESIGN.md`): the same session runner pattern, headset button, review, export and Hugging Face upload, with eye cameras instead of hand cameras. Like the hand recorder it ships in Frametop and, later, as a standalone app for people without Frametop (Frametop/frametop-eye-recorder, Frametop as a submodule).
+
+The data is for training a better eye tracker than `gaze/tracker/` (ft-eyes, a classic pupil and glint finder) and SteamVR's, one that holds up at the edges of view, through blinks and squints, with glasses and makeup, and when the headset moves.
+
+## Decisions (2026-10-14)
+
+| | Decided |
+|---|---|
+| Who reads the recordings | Maintainers only. The eye cameras see the iris, which identifies a person. Trained models can be public; the images never are. |
+| How | Encrypted frames in a gated dataset, `DeeJanuz/frametop-eyes` (below, "Privacy"). |
+| Keys | Two maintainer keys, both recipients of every upload: key 1 on the maintainer's training machine (the 7i), key 2 kept offline in the maintainer's password manager. |
+| Packaging | Backend here in `gaze/rec/`, then a standalone repo with Frametop as a submodule, as the hand recorder does. |
+| Sessions | Short rounds picked from a bounty board (below, "Bounty board"), not one long session. Each round starts with a fit check and 9 dots. |
+| Bounty list | `bounties.json` in the dataset repo, fetched by the app (a copy ships for offline use). The board shows your own done marks and a maintainer-set priority tag per bounty, not community counts. |
+| Fit and slip source | Our tracker, ft-eyes: pupil and glint positions in the camera image. It needs ft-eyegrab, the root service `gaze/tracker/install.sh` installs. |
+| Where the fit coach runs | A fitment step in the recorder before each round, for better recordings, and the put-on and slip check in Frametop's gaze service. |
+| Shared code | The hand recorder's non-hand parts move into a shared module first, and the bounty board is built there too, so the hand recorder can use it next. |
+| License | As hands: contributors give the maintainer a license for any use, including commercial models; the eye images and health answers are never published or shared outside the maintainers (user, 2026-10-14). |
+| Standalone repo | Created after the window and a full round pass in the headset; step 9 comes first (user, 2026-10-14). |
+| Checklist | Glasses/inserts/contacts, eye makeup, measured IPD, and eye surgery or conditions. That last one is health data: a separate opt-in, and its answers are encrypted like the images. |
+
+## Privacy
+
+A private Hugging Face repo can't take contributions: people outside the org can't see it, so they can't open a pull request, and org membership would let every member read every contribution. So the dataset is gated like `DeeJanuz/frametop-hands`, and the export encrypts what identifies a person before upload:
+- Every frame chunk is encrypted with [age](https://age-encryption.org) (pyrage, which has aarch64 wheels: `pyrage-1.4.0-cp310-abi3-manylinux_2_17_aarch64`) to the recipients in `recipients.txt`: the maintainers' public keys: key 1 on the 7i, key 2 offline. The private keys never go in a repo.
+- The health answers go in `health.json.age`, encrypted the same way.
+- Labels, timings, fit readings, the IPD and the rest of session.json stay readable, so `validate.py` and a review can check a pull request before anyone decrypts it.
+- Contributors open pull requests from their own accounts, so `hub.py` and the review tools carry over from hands.
+
+The risk: if a private key leaks, every upload made to it can be read. A new key is one more recipient for new uploads; old uploads stay readable to the old key until they're re-encrypted. Withdrawal means deleting the files and squashing history, as with hands.
+
+## What's measured already
+
+- **The eye cameras** (`gaze/tracker/ft-eyegrab.c`): 512x400 8-bit IR, about 90 fps each when SteamVR runs them at 90 (`gaze/eye-rate.py`; at its default 15 fps a recording is worth little, so the recorder checks the rate first). Camera 0 is the right eye (steep angle), camera 1 the left (head-on). ft-eyegrab copies the frames read-only into `/dev/shm/frametop-eyes-cams` while someone touches `/dev/shm/frametop-eyes-want`.
+- **Size.** Raw, both cameras are 36.9 MB/s, 2.2 GB a minute. zstd level 1 on 200 MB of the `fit1` capture (2026-10-14): 56 % of the size at about 250 MB/s on one core, so compressing while recording costs about 15 % of a core. Level 3: 53 %, 2.3x slower. Level 9: 49 %, 9.5x slower. The recorder uses level 1 while recording: about 1.2 GB per recorded minute, so a session of about 10 recorded minutes (below) is about 12 GB, close to a hand session.
+- **IPD.** SteamVR reports it (2026-10-14, a probe against the live vrserver): `Prop_UserIpdMeters_Float` 0.0651, `Prop_IpdUIRangeMinMeters` 0.057, `Max` 0.071, `Prop_MinimumIpdStepMeters` 0.0015, and the eye-to-head transforms are ±IPD/2. `driver_cv.so` reads the range from the headset's EEPROM. So the recorder can read the IPD setting live and record it with every take. Not yet known: whether the eye cameras move with the lenses when the IPD changes. That decides how the IPD guidance works (below), and it's the first headset test.
+- **ft-eyes** already finds each pupil (centre, ellipse) and glint pair per frame, keeps a per-eye slip estimate from the glints (`eyes_model.SlipTracker`), and resets its shift when frames stop for 3 s (the headset was off). `gaze/fitcheck.py` already keeps per-eye tracked and lost by gaze direction, with hints.
+
+## Parts
+
+| Part | What it does |
+|---|---|
+| `gaze/rec/session.py` (Python, no Qt) | The session runner, like `hands/rec/session.py`: reads a round's script, drives the panel, starts and stops the frame writer per step, reads the live fit for feedback, writes each take. `--dry-run`, `--next-after`, stdin steering and `--button-device` as in hands. |
+| `gaze/rec/rounds/<id>.json` | Each kind of round's steps, targets and timings (below, "Bounty board"). |
+| `gaze/rec/eyerec.py` | The frame writer: reads the shared frames (as `lab/ft-eyes-record` does) and writes zstd-1 chunks, one per second per camera, with an index (below, "Files"). A thread of the session, not a process: ft-eyes-record shows the copy needs no numpy. |
+| `gaze/fitcoach.py` (shared) | The fit coach: from ft-eyes' pupil and glint positions, how the headset sits on this face, and what to do about it (below, "Fit coach"). The recorder, the Gaze page's fit check and the put-on check all use it. |
+| ft-gazepanel `rec` mode | The headset panel. The gaze panel already draws dots at exact head-relative directions, which the ground truth needs, so it gains a `rec` mode with the hand panel's text, step, countdown and paused lines rather than a second panel (below, "Panel"). |
+| `gaze/rec/takes.py`, `validate.py`, `hub.py` | Review, export, checks and upload, from `hands/rec/`'s. The shared parts move into a common module rather than being copied (below, "Shared with the hand recorder"). |
+| `gaze/rec/ft_eyerec.py` + `main.qml` | The desktop window: consent, the bounty board, the checklist and eye camera check, the round with its fit card, review, export, upload. Launched by `ft-eyerec` (the dev container), installed to the app menu by `install.sh`. The standalone app gets a Qt Quick Controls window, as the hand recorder's does. |
+| `gaze/rec/CONSENT.md`, `UPLOAD.md` | The texts. Eye images need a stronger consent than hands (biometric data, who can read it, how to withdraw). They need a review like `frame-hands/notes/legal-review-2026-10-03.md` before uploads open. |
+
+## Fit coach
+
+It reads, per eye, from ft-eyes: the pupil centre, its ellipse, the glint pair (both glints and their spacing), and how often the eye was found over the last second. It compares them with a reference: where a well-fitted eye sits in that camera's image. At first the reference comes from the maintainer's headset; once contributions come in, it comes from their rounds with good nine-dot fits (every round's fitment step records exactly that), per headset model. The coach says which reference it used, and until it's built from more than one face its hints are marked provisional.
+
+What it can tell, and the instruction it gives:
+
+| Reading | Means | Instruction |
+|---|---|---|
+| Both pupils high or low in their images | The headset sits low or high on the face | "Push the headset up" / "Let it down a little", with the top strap |
+| Both pupils left or right | The headset sits off to one side | "Slide the headset right" |
+| Pupils apart or together, compared with the reference at this IPD | IPD set too narrow or wide | "Turn the IPD wheel wider", until both are centred |
+| One pupil higher than the other | The headset is tilted | "Level the headset" |
+| Glint pair spacing larger or smaller than the reference | The eye is nearer or further than usual (eye relief: face pad, straps, glasses) | "Tighten the back strap" / "The headset is pressing close: loosen it" |
+| A glint missing, or extra bright spots | Glasses reflections, or the lens fogged or smudged | "Tilt your glasses" / "Wipe the lenses" |
+| An eye found in under half the frames, looking straight ahead | Lashes on the lens, a droopy lid, or the eye outside the camera's view | The per-eye card says which |
+
+The IPD row depends on the first headset test. If the cameras move with the lenses, a pupil's sideways offset in its image directly says how far that eye is from its lens axis, and the coach turns the IPD wheel's direction into "wider" or "narrower" until both offsets are about zero, at any IPD. If the cameras are fixed, the offset depends on the IPD setting too, and the coach needs the reference at each IPD step, built from rounds recorded at different settings (there's no IPD sweep: the setting can't be changed with the headset on).
+
+**Put-on and shift check.** After a good fit, the coach keeps that fit as the person's baseline (`~/.local/state/frametop/gaze/fit-baseline.json`: per eye, the pupil looking straight ahead and the glint pair, with the IPD setting). Then:
+- When the headset goes back on (frames resume after ft-eyes' 3 s gap), it shows the fit card for a few seconds while the person looks ahead. If the pupils sit more than a threshold (start with 1.5 mm equivalent, tuned from data) from the baseline, it says which way to move the headset, live, until it's back. If the IPD setting changed, it says so first.
+- While in use, ft-eyes' glint slip estimate already tracks small movement. A shift past the threshold that stays for 2 s gets a quiet notice ("Your headset slipped down: push it up, or look at the dot to recalibrate"), not a modal.
+In the recorder the fit coach runs as each round's fitment step, and the shift warning runs during the round. In Frametop the put-on and slip check becomes part of the gaze service, off by default until it's been tried.
+
+## Bounty board
+
+People pick what to record from a board of bounties. Each bounty is one round of about 5 minutes, in step mode: the headset's right button starts each step (press = next or pause, two presses = redo, hold = stop), as in the hand recorder. Every round starts the same way:
+1. **Fitment** (not recorded as a take, but its readings go into fit.json): the fit coach until the headset sits well (below, "Fit coach"). Skippable after a warning, and the round records that it was skipped. (The coach is off for now, 2026-10-16: then it only asks for a comfortable fit.) It shows the IPD setting, read from SteamVR when the round starts. The setting can't be changed with the headset on (2026-10-16), so there's no IPD step: a wrong setting means stopping, setting it, and starting again. Each take's ipd.jsonl still logs it.
+2. **Nine dots** (recorded): a 3x3 grid to 20 degrees on the mid background. It's the round's own reference, so each round can be labelled alone, even after a re-seat.
+3. **The bounty's steps.**
+
+How to build a new one, and the rules a round script must follow (beeps for anything done with the eyes shut, cue words as labels, timing, reach, comfort): `rounds/README.md`.
+
+Every round uploads on its own. The board marks the bounties you've done, by your contributor id in `profile.json`, with how many times. You can do one again.
+
+The launch bounties:
+
+| Bounty | What | Edge case it covers |
+|---|---|---|
+| View edges | Dots at the panel's edges and corners: 30 degrees to the sides, 23 up and down, about 35 in the corners (the rec panel is 64 degrees across; every round's dots are checked against it) | Eyes turned far: the lid, the glints lost, the pupil foreshortened |
+| Blinks and lids | Blink on a beep, close both eyes on a beep and open them on the next (sounds.py: the eyes are shut, so words can't cue them), wink each eye, squint, eyes wide, half-closed, looking down with lids lowered | Closed and partly closed eyes; one eye missing |
+| Headset slips | Nudge the headset up, down, left, right, tilt it, tip its front down, pull it off the face, on a beep, holding it through dots, letting go on a low beep; then take it off and put it back on; dots after each | Headset shifting on the face: the put-on check's data |
+| Glasses | A 5x5 grid and the edges with glasses or inserts on (offered when the checklist says so) | Lens reflections, the glasses' frame in view |
+| Jumps and pursuit | A dot jumping between random spots (0.5 to 1.5 s each), then moving on circles, lines and a figure eight at two speeds | Saccades, overshoot, smooth pursuit, the tracker's lag |
+| Depth | Dots at 0.5 m, 1.5 m and 5 m in turn (the panel's distance changes) | Vergence; the eyes' angle with distance |
+| Light changes | The background steps dark to bright and back while looking at a dot, then a 5x5 grid at each of three backgrounds | Pupil dilation and constriction; error against pupil size |
+| Face moves | Smile, raise eyebrows, frown, yawn, talk (read a line aloud: the microphone isn't recorded), chew | The face moving the headset and the lids |
+| Head turning | A dot fixed in the room while turning the head (head poses logged) | The eyes counter-rotating (VOR), headset jiggle |
+| Reading | Lines of text to read, then a page of icons to look over | Natural use |
+
+Later candidates, not at launch: eye makeup (the grid and blinks with mascara or eyeliner; the checklist still records makeup on every round), long wear (the grid after 30+ minutes in the headset).
+
+### bounties.json
+
+In the dataset repo, fetched when the board opens (a copy in the app for offline use, used when the fetch fails):
+
+```json
+{"schema": 1, "updated": "2026-10-14",
+ "bounties": [{"id": "edges", "title": "View edges", "blurb": "...", "minutes": 5,
+               "priority": "high|normal|low|retired", "script": "edges", "needs": {"glasses": true}}]}
+```
+
+`script` names a round in the app's `rounds/` folder: a new kind of round needs a release, but the list, wording and priorities don't. `needs` hides a bounty unless the checklist matches. A bounty id the app doesn't know is shown as "needs an update".
+
+The board is in the shared recorder module, so the hand recorder's v2 sections can become bounties later.
+
+### Checklist
+
+Asked once and kept in profile.json, editable before each round, all optional, none identifying: glasses (none, glasses, prescription inserts, contacts), eye makeup today (none, mascara, eyeliner, lash extensions), the face pad and strap in use, and a measured IPD if they know it (from an optician). Separately, behind its own consent tick: eye surgery or conditions (LASIK or other surgery, a lazy or turned eye, a droopy lid, nystagmus, other). Those answers go into `health.json`, which the export encrypts. Eye colour isn't asked: the IR images show what matters about it.
+
+## Panel
+
+ft-gazepanel gains `show rec`: a large head-locked panel (FULL_DEG, 64 degrees, with a solid background whose brightness the script sets) that keeps every existing command (`dot`, `bg`, `title`, `text`, `note`, `eye`, `hints`) and adds what the hand panel has: `step`, `big` (countdown), `action`, `keys`, `rec on|off`, `paused on|off`. New for the ground truth:
+- `path <kind> <params> <seconds>`: the panel moves a dot itself (circle, line, figure eight), so it's smooth at the headset's frame rate, not at the socket's.
+- `distance <metres>`: moves the panel for the Depth bounty.
+- `world <x> <y> <z>`: a dot fixed in the room (the Head turning bounty), as the hand panel's touch target.
+- Every picture the panel shows with a dot logs that dot's direction and the frame's time to `targets.jsonl` (CLOCK_MONOTONIC_RAW, the eye frames' clock). The ground truth is what was on screen when, not what was asked.
+
+## Files
+
+```
+~/.local/share/frametop/eyes/contrib/
+  profile.json                  consent, contributor id, checklist, bounties done
+  rounds/<YYYYMMDD-HHMMSS>-<bounty>/
+    session.json                the bounty, checklist, versions, takes, eye camera rate, IPD at start
+    fit.json                    the fit coach's readings: the fitment step, and any slip warnings
+    health.json                 the opt-in health answers (encrypted in the export)
+    takes/<NN>-<step group>/    numbered in order: NN-dots (the nine dots), then the bounty's
+      cam0/000000.zst ...       right eye: one zstd chunk per second, frames back to back (512x400 u8)
+      cam1/000000.zst ...       left eye
+      index.tsv                 per frame: camera, chunk, offset in it, slot, CLOCK_MONOTONIC_RAW time
+      targets.jsonl             what the panel showed, when (direction, distance, state, background)
+      prompts.jsonl             what the person was asked to do, when (as hands)
+      poses.jsonl               head poses, 250 Hz (as hands; needed for Head turning)
+      ipd.jsonl                 the IPD setting at the start and when it changes
+      steamvr.jsonl             SteamVR's gaze from ft-gaze, per sample (a baseline, not a label)
+      own.jsonl                 ft-eyes' pupils, glints and gaze (a baseline, not a label)
+      take.json                 status, deleted ranges, parts, clock samples, as hands
+  exports/<session>/
+```
+
+The export recompresses nothing (the chunks are zstd already), drops deleted ranges by rewriting the chunks they touch, encrypts each chunk and `health.json` to `recipients.txt` (`.age`), and writes the manifest and `SHA256SUMS` over the encrypted files. It never includes `/persist` files: unlike hands, nothing here needs the unit's calibration, and the eye cameras' isn't in `/persist` anyway (2026-10-14: `device_config.json` has the displays' transforms only).
+
+## Shared with the hand recorder
+
+Much of `hands/rec/` isn't about hands: the headset button reader, profile and consent, take bookkeeping (deleted ranges, redo, parts, clock samples), the Hugging Face login and upload, validate's manifest and checksum checks, and most of the window. These live in `recorder/` at the top of the tree, with the bounty board, and both recorders use them: `common.py`, `units.py`, `button.py`, `bounties.py`, `files.py` (ranges, checksums, sizes), `checks.py` (validate's report, SHA256SUMS, typed manifest lookups), `hub.py` (login, errors, upload records, the pull request flow), `age.py` (the export's encryption) and `window.py` (the window backend's shared half: consent, disk, session controls, export, login and upload; each recorder's `Backend` subclasses it). The hand recorder's modules keep their names on top of them, and its tests pass. The export queue, with Export and upload sending a round by itself once it's exported (2026-10-17), is `window.py`'s too, so both windows have it: `hands/rec/DESIGN.md` (Upload) has how it works.
+
+Uploads: `gaze/rec/hub.py` sends nothing to the real dataset until `CONSENT.md` and `UPLOAD.md` exist and aren't drafts. `FT_EYEREC_ALLOW_UPLOAD=1` opens rehearsals, and only to another dataset (`FT_EYEREC_DATASET`). Rehearsed 2026-10-14 against the private `DeeJanuz/frametop-eyes-test` with a round of random bytes: uploaded, downloaded, checked, and decrypted with both keys by age 1.1.1 on the 7i.
+
+## Build order
+
+1. **Headset test 1:** do the eye cameras move with the lenses when the IPD changes? The setting can't be changed with the headset on, so compare nine-dot rounds recorded at two settings (ft-eyes' pupil x at the centre dot). Decides the IPD guidance.
+2. Pull the shared recorder code out of `hands/rec/` (hand recorder tests pass).
+3. `eyerec.py` (frame writer) and its replay test against a capture.
+4. `fitcoach.py` with readings from a capture, and the fit card on the existing fit panel.
+5. ft-gazepanel `rec` mode, `path`, `distance`, target logging.
+6. `session.py`, the nine-dot opener and the first rounds, run with `--dry-run`, then on a replay. The bounty board and `bounties.json`.
+7. Review, export with encryption (two recipients), `validate.py`, `hub.py` against a test dataset.
+8. The window, then the standalone repo.
+9. The put-on check in the gaze service. Built 2026-10-14 behind `GAZE_FIT_WATCH=1`: the quick check holds its dot while the coach reads the fit against the saved one (the fit check and the recorder's fitment save a good fit), and the slip note is a strip the gaze panel draws (`show note`) from ft-eyes' `jump_px`. Thresholds are the coach's 5 degrees, not 1.5 mm yet; tuned from the headset tests.
+
+Before uploads open: the consent texts and a review like `frame-hands/notes/legal-review-2026-10-03.md`, with the health opt-in in it.
+
+Not in scope here: the training. The dataset format is designed so that frame-eyes or a later trainer can label pupils from the targets and `own.jsonl`, but that's its own project.

gaze/rec/UPLOAD.md

@@ -0,0 +1,10 @@
+### About uploading
+
+- **What's sent:** the export in `@EXPORT_PATH@` (@EXPORT_SIZE@), into `contributions/@CONTRIBUTOR@/@ROUND@` in [@DATASET@](https://huggingface.co/datasets/@DATASET@). Upload first checks that every file is complete and matches its checksum, and that the eye images and health answers are encrypted.
+- **Who can read it:** the eye images and health answers are encrypted on this headset to the maintainers' keys. Hugging Face and anyone who downloads the dataset get only the encrypted files. The rest (where the dots were, what the trackers found, the fit readings and your checklist answers) stays readable.
+- **Your account:** the pull request comes from your Hugging Face account, and anyone can see its username next to your contributor id, `@CONTRIBUTOR@`. The login stays saved on this headset, so you only log in once.
+- **The dataset's terms:** the first time, open the dataset's page and accept its terms. Until you have, Upload stops with "accept the dataset's terms first".
+- **Once your pull request's link shows, plug in the headset and leave it plugged in until this page says Uploaded.** A round is several gigabytes, so this can take a while. You can take the headset off: the Eye Recorder keeps it awake until the upload is done. Keep the Eye Recorder open, since closing it stops the upload.
+- **If it stops partway** (Cancel, or the network drops), press Upload again. It carries on in the same pull request, and files already sent aren't sent twice.
+- **The maintainer's review:** they check that the files are complete and decrypt correctly before merging. You can follow it and answer questions on the pull request's page. Once it's merged, you can delete the round and its export here to free the space.
+- **Withdrawing:** see "Withdrawing" in the consent text. You'll need your contributor id, `@CONTRIBUTOR@`.

gaze/rec/bounties.json

@@ -0,0 +1,13 @@
+{"schema": 1, "updated": "2026-10-14",
+ "bounties": [
+  {"id": "edges", "title": "View edges", "blurb": "Dots far out to the sides and corners: eyes turned as far as they go.", "minutes": 5, "priority": "high", "script": "edges"},
+  {"id": "blinks", "title": "Blinks and lids", "blurb": "Blinks, winks, squints and half-closed eyes.", "minutes": 5, "priority": "high", "script": "blinks"},
+  {"id": "slips", "title": "Headset slips", "blurb": "Nudge the headset on your face, take it off and on: dots after each.", "minutes": 6, "priority": "high", "script": "slips"},
+  {"id": "glasses", "title": "Glasses", "blurb": "The grid and the edges with glasses or inserts on.", "minutes": 5, "priority": "high", "script": "glasses", "needs": {"glasses": true}},
+  {"id": "jumps", "title": "Jumps and pursuit", "blurb": "A jumping dot, then one gliding on circles, lines and a figure eight.", "minutes": 5, "priority": "normal", "script": "jumps"},
+  {"id": "depth", "title": "Depth", "blurb": "Dots near, mid and far.", "minutes": 4, "priority": "normal", "script": "depth"},
+  {"id": "light", "title": "Light changes", "blurb": "Dark to bright and back: pupils changing size.", "minutes": 5, "priority": "normal", "script": "light"},
+  {"id": "face", "title": "Face moves", "blurb": "Smile, frown, yawn, talk and chew while looking at a dot.", "minutes": 4, "priority": "normal", "script": "face"},
+  {"id": "head", "title": "Head turning", "blurb": "A dot fixed in the room while you turn your head.", "minutes": 4, "priority": "normal", "script": "head"},
+  {"id": "reading", "title": "Reading", "blurb": "Read a few paragraphs and look around freely.", "minutes": 4, "priority": "low", "script": "reading"}
+ ]}

gaze/rec/eyerec.py

@@ -0,0 +1,326 @@
+#!/usr/bin/env python3
+"""The eye recorder's frame writer (DESIGN.md: "Files"): every frame of both eye cameras, from
+the frames ft-eyegrab shares (gaze/tracker/eyes_share.py), into a take's folder as zstd chunks.
+
+  cam0/000000.zst, 000001.zst, ...   the right eye (camera 0): up to CHUNK_FRAMES frames each
+  cam1/...                           the left eye (camera 1)
+  index.tsv                          a header line, then one line per frame:
+                                     camera, chunk, frame in the chunk, the tracker's slot, time
+                                     (CLOCK_MONOTONIC_RAW seconds, the share's)
+
+A chunk is the frames back to back, 512x400 bytes each, compressed with zstd level 1 (56 % of the
+size at about 250 MB/s on one core, measured 2026-10-14; both cameras at 90 fps are 36.9 MB/s
+raw). A chunk ends when it's full or the recording pauses, so a pause or a deleted range never
+splits one. The index lines of a chunk are written once the chunk is on disk, so the index never
+names a chunk that isn't there; they're in chunk order, not time order across the cameras.
+
+One thread copies frames out of the share as they come (it holds only 8 per camera, 89 ms);
+another compresses and writes, so a slow chunk never makes the copy fall behind. If the writer
+falls MAX_QUEUED chunks behind, the next chunk is dropped and counted rather than held in memory.
+
+Plain Python, standard library: zstd is Python 3.14's compression.zstd (the dev container's), or
+zstandard, or the zstd command (SteamOS's host Python 3.12 has neither module).
+
+From the command line, for testing (FT_EYES_CAMS points it at a replay, lab/ft-eyes-replay):
+  python3 gaze/rec/eyerec.py DIR [SECONDS]
+  python3 gaze/rec/eyerec.py --read DIR      the frames' count and times, decompressed
+"""
+import os
+import queue
+import shutil
+import subprocess
+import sys
+import threading
+import time
+from collections import deque
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(os.path.dirname(HERE), "tracker"))
+import eyes_share  # noqa: E402  (gaze/tracker/eyes_share.py)
+
+W, H = eyes_share.W, eyes_share.H
+FRAME = W * H
+CAMERAS = (0, 1)               # 0 the right eye, 1 the left (gaze/tracker/README: camera 0 is the steep one)
+CHUNK_FRAMES = 90              # a second at 90 fps
+LEVEL = 1
+MAX_QUEUED = 6                 # chunks waiting for the writer (about 18 MB each) before one is dropped
+MIN_FREE = 3e9                 # bytes: stop before the disk fills
+RATE_WINDOW = 2.0              # s of frame times for each camera's rate
+SHARE_WAIT = 10.0              # s to wait for the share at the start, or after it was replaced
+POLL = 0.002
+INDEX_HEADER = "# camera\tchunk\tframe\tslot\tt_raw\n"
+
+
+def _compressor():
+    """(compress(bytes) -> bytes, decompress(bytes) -> bytes, name)."""
+    try:
+        from compression import zstd   # Python 3.14
+        return (lambda b: zstd.compress(b, level=LEVEL)), zstd.decompress, "compression.zstd"
+    except ImportError:
+        pass
+    try:
+        import zstandard
+        c, d = zstandard.ZstdCompressor(level=LEVEL), zstandard.ZstdDecompressor()
+        return c.compress, (lambda b: d.decompress(b, max_output_size=CHUNK_FRAMES * FRAME)), "zstandard"
+    except ImportError:
+        pass
+    exe = shutil.which("zstd") or ("/usr/bin/zstd" if os.path.exists("/usr/bin/zstd") else None)
+    if not exe:
+        return None, None, None
+
+    def run(args, data):
+        return subprocess.run([exe, *args], input=data, capture_output=True, check=True).stdout
+    return (lambda b: run(["-q", "-%d" % LEVEL, "-c"], b)), (lambda b: run(["-q", "-d", "-c"], b)), exe
+
+
+compress, decompress, ZSTD = _compressor()
+
+
+class Writer:
+    """Records the eye cameras into take_dir while recording is on.
+
+    start() starts the threads (not recording yet); record(True) and record(False) start and
+    pause; stop() ends the open chunks, waits for the writer and returns stats(). error is set
+    (and recording stops) when the disk gets full or the share is gone for good."""
+
+    def __init__(self, take_dir, share_path=None, want_path=None, log=None, min_free=MIN_FREE,
+                 chunk_frames=CHUNK_FRAMES):
+        if compress is None:
+            raise RuntimeError("no zstd: neither compression.zstd, zstandard nor the zstd command")
+        self.dir, self.share_path, self.want_path = take_dir, share_path, want_path
+        self.log = log or (lambda line: None)
+        self.min_free, self.chunk_frames = min_free, chunk_frames
+        self.error = None
+        self._on = False
+        self._lock = threading.Lock()
+        self._stop = threading.Event()
+        self._queue = queue.Queue()
+        self._open = {c: [] for c in CAMERAS}      # the chunk being filled: [(t, slot, bytes)]
+        self._next = {c: 0 for c in CAMERAS}       # the next chunk's number
+        self._frames = {c: 0 for c in CAMERAS}     # frames written (in chunks on disk)
+        self._dropped = {c: 0 for c in CAMERAS}    # frames lost: overwritten in the share, or a dropped chunk
+        self._times = {c: deque() for c in CAMERAS}
+        self._last = {c: None for c in CAMERAS}
+        self._bytes = 0
+        self._busy = 0.0                           # s spent compressing and writing
+        for c in CAMERAS:
+            os.makedirs(os.path.join(take_dir, "cam%d" % c), exist_ok=True)
+        self._index = open(os.path.join(take_dir, "index.tsv"), "a")
+        if self._index.tell() == 0:
+            self._index.write(INDEX_HEADER)
+        # Numbering goes on after any chunks already there (a take resumed after a pause in another run).
+        for c in CAMERAS:
+            names = [n for n in os.listdir(os.path.join(take_dir, "cam%d" % c)) if n.endswith(".zst")]
+            self._next[c] = max((int(n[:-4]) + 1 for n in names if n[:-4].isdigit()), default=0)
+        self._reader = threading.Thread(target=self._read, name="eyerec-read", daemon=True)
+        self._writer = threading.Thread(target=self._write, name="eyerec-write", daemon=True)
+
+    def start(self):
+        self._reader.start()
+        self._writer.start()
+        return self
+
+    def record(self, on):
+        with self._lock:
+            if self._on and not on:
+                for c in CAMERAS:
+                    self._end_chunk(c)
+            self._on = bool(on) and self.error is None
+
+    @property
+    def recording(self):
+        return self._on
+
+    def stop(self):
+        self.record(False)
+        self._stop.set()
+        self._reader.join(5)
+        self._queue.put(None)
+        self._writer.join(60)
+        self._index.close()
+        return self.stats()
+
+    def stats(self):
+        """{"frames": [right, left], "dropped": [...], "fps": [...], "bytes": on disk, "busy": s,
+        "queued": chunks waiting, "last_t": [the newest frame's time or None], "error"}"""
+        with self._lock:
+            fps = []
+            for c in CAMERAS:
+                ts = self._times[c]
+                fps.append(round((len(ts) - 1) / (ts[-1] - ts[0]), 1) if len(ts) > 1 and ts[-1] > ts[0] else 0.0)
+            return {"frames": [self._frames[c] for c in CAMERAS], "dropped": [self._dropped[c] for c in CAMERAS],
+                    "fps": fps, "bytes": self._bytes, "busy": round(self._busy, 3), "queued": self._queue.qsize(),
+                    "last_t": [self._last[c] for c in CAMERAS], "error": self.error}
+
+    # -- the copy
+
+    def _read(self):
+        share = eyes_share.open_share(SHARE_WAIT, self.share_path, self.want_path)
+        if share is None:
+            self._fail("no eye camera frames (%s): is frametop-eyegrab.service running?" % (self.share_path or eyes_share.CAMS))
+            return
+        seen = {c: share.count(c) for c in CAMERAS}
+        touched = checked = time.monotonic()
+        while not self._stop.is_set():
+            got = False
+            for c in CAMERAS:
+                n = share.count(c)
+                if n <= seen[c]:   # nothing new (and never step back: that would copy frames twice)
+                    continue
+                if n - seen[c] > share.slots:   # fell behind: those are gone
+                    with self._lock:
+                        if self._on:
+                            self._dropped[c] += n - seen[c] - share.slots
+                    seen[c] = n - share.slots
+                for i in range(seen[c], n):
+                    f = share.frame(c, i)
+                    got = True
+                    with self._lock:
+                        if f is None:
+                            if self._on:
+                                self._dropped[c] += 1
+                            continue
+                        self._seen(c, f[0])
+                        if self._on:
+                            self._open[c].append(f)
+                            if len(self._open[c]) >= self.chunk_frames:
+                                self._end_chunk(c)
+                seen[c] = n
+            now = time.monotonic()
+            if now - touched >= 1.0:
+                touched = now
+                eyes_share.touch_want(self.want_path)
+            if now - checked >= 2.0:
+                checked = now
+                if self._on and shutil.disk_usage(self.dir).free < self.min_free:
+                    self._fail("under %.1f GB free on the disk" % (self.min_free / 1e9))
+                if share.replaced():   # ft-eyegrab restarted (SteamVR did): follow it
+                    self.log("eye frames: the share was replaced; following it")
+                    share.close()
+                    share = eyes_share.open_share(SHARE_WAIT, self.share_path, self.want_path)
+                    if share is None:
+                        self._fail("the eye camera frames stopped (frametop-eyegrab.service)")
+                        return
+                    seen = {c: share.count(c) for c in CAMERAS}
+            if not got:
+                time.sleep(POLL)
+        share.close()
+
+    def _seen(self, c, t):
+        ts = self._times[c]
+        ts.append(t)
+        while ts and ts[0] < t - RATE_WINDOW:
+            ts.popleft()
+        self._last[c] = t
+
+    def _end_chunk(self, c):
+        """Hand the open chunk to the writer (with the lock held)."""
+        frames = self._open[c]
+        if not frames:
+            return
+        self._open[c] = []
+        if self._queue.qsize() >= MAX_QUEUED:
+            self._dropped[c] += len(frames)
+            self.log("eye frames: the writer is behind; dropped %d frames of camera %d" % (len(frames), c))
+            return
+        self._queue.put((c, self._next[c], frames))
+        self._next[c] += 1
+
+    def _fail(self, why):
+        with self._lock:
+            if self.error is None:
+                self.error = why
+                self.log("eye frames: " + why)
+            if self._on:
+                for c in CAMERAS:
+                    self._end_chunk(c)
+            self._on = False
+
+    # -- the writer
+
+    def _write(self):
+        while True:
+            item = self._queue.get()
+            if item is None:
+                return
+            c, chunk, frames = item
+            t0 = time.perf_counter()
+            path = os.path.join(self.dir, "cam%d" % c, "%06d.zst" % chunk)
+            try:
+                data = compress(b"".join(f[2] for f in frames))
+                with open(path + ".tmp", "wb") as f:
+                    f.write(data)
+                os.replace(path + ".tmp", path)
+            except (OSError, subprocess.CalledProcessError) as e:
+                self._fail("can't write %s: %s" % (path, e))
+                with self._lock:
+                    self._dropped[c] += len(frames)
+                continue
+            self._index.write("".join("%d\t%d\t%d\t%d\t%.6f\n" % (c, chunk, i, f[1], f[0]) for i, f in enumerate(frames)))
+            self._index.flush()
+            with self._lock:
+                self._frames[c] += len(frames)
+                self._bytes += len(data)
+                self._busy += time.perf_counter() - t0
+
+
+def read_index(take_dir):
+    """index.tsv as [(camera, chunk, frame, slot, t), ...] in its order."""
+    out = []
+    try:
+        with open(os.path.join(take_dir, "index.tsv")) as f:
+            for line in f:
+                if line.startswith("#") or not line.strip():
+                    continue
+                c, chunk, i, slot, t = line.split("\t")
+                out.append((int(c), int(chunk), int(i), int(slot), float(t)))
+    except FileNotFoundError:
+        pass
+    return out
+
+
+def read_chunk(take_dir, cam, chunk):
+    """The frames of a chunk, as a list of 512x400 byte strings."""
+    with open(os.path.join(take_dir, "cam%d" % cam, "%06d.zst" % chunk), "rb") as f:
+        raw = decompress(f.read())
+    if len(raw) % FRAME:
+        raise ValueError("cam%d/%06d.zst: %d bytes isn't whole frames" % (cam, chunk, len(raw)))
+    return [raw[i:i + FRAME] for i in range(0, len(raw), FRAME)]
+
+
+def iter_frames(take_dir):
+    """Every frame in index order: (camera, t, slot, bytes)."""
+    cache = (None, None)
+    for c, chunk, i, slot, t in read_index(take_dir):
+        if cache[0] != (c, chunk):
+            cache = ((c, chunk), read_chunk(take_dir, c, chunk))
+        yield c, t, slot, cache[1][i]
+
+
+def main(argv):
+    if not argv or argv[0] in ("-h", "--help"):
+        sys.exit(__doc__)
+    if argv[0] == "--read":
+        n, ts = [0, 0], [[], []]
+        for c, t, _slot, _data in iter_frames(argv[1]):
+            n[c] += 1
+            ts[c].append(t)
+        for c in CAMERAS:
+            span = max(ts[c]) - min(ts[c]) if len(ts[c]) > 1 else 0.0
+            print("camera %d: %d frames over %.2f s" % (c, n[c], span))
+        return
+    secs = float(argv[1]) if len(argv) > 1 else 10.0
+    w = Writer(argv[0], log=lambda line: print(line, file=sys.stderr)).start()
+    w.record(True)
+    end = time.monotonic() + secs
+    try:
+        while time.monotonic() < end and w.error is None:
+            time.sleep(1)
+            print(w.stats(), file=sys.stderr)
+    except KeyboardInterrupt:
+        pass
+    print(w.stop())
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

gaze/rec/ft-eyerec

@@ -0,0 +1,24 @@
+#!/bin/bash
+# Launch the Frametop Eye Recorder from a Plasma session on the Frame host (its app menu entry,
+# gaze/rec/install.sh).
+# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there; a
+# release's image has the recorders' cryptography and huggingface_hub in /opt/frametop/venv, which
+# distrobox leaves off the front of PATH, so it goes first here).
+# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
+# the container's cgroup; the Frametop session runs on a private bus from
+# dbus-run-session). The session's Wayland socket and bus go to the app itself.
+# Options go to ft_eyerec.py: --base DIR, --page NAME, and --dry-run (--hub-dry-run, --ignore-eyes) for testing.
+here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
+wl=${WAYLAND_DISPLAY:-wayland-0}
+case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
+session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
+export XDG_RUNTIME_DIR=/run/user/$(id -u)
+export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
+# From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
+# host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
+cd "$HOME" || exit 1
+exec "$here/../../scripts/in-box" env PATH="/opt/frametop/venv/bin:/opt/frametop/bin:$PATH:/usr/sbin:/sbin" \
+  WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+  XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
+  QT_QPA_PLATFORM="wayland;xcb" \
+  python3 "$here/ft_eyerec.py" "$@"

gaze/rec/ft-eyerec.desktop

@@ -0,0 +1,9 @@
+[Desktop Entry]
+Type=Application
+Name=Frametop Eye Recorder
+GenericName=Record your eyes for the eye dataset
+Comment=Fit the headset, then record, review and export eye recordings for Frametop's eye dataset
+Exec=@REPO@/gaze/rec/ft-eyerec
+Icon=camera-video
+Categories=Utility;
+Keywords=eyes;eye tracking;gaze;dataset;record;frametop;

gaze/rec/ft_eyerec.py

@@ -0,0 +1,830 @@
+#!/usr/bin/env python3
+"""Frametop Eye Recorder: record your eyes for the eye dataset (gaze/rec/DESIGN.md).
+
+A Kirigami (QML) app with a Python backend, in the dev container, as the hand recorder
+(hands/rec/ft_handrec.py); what the two windows share is recorder/window.py:
+  - Welcome: the consent text (CONSENT.md), the first time and again when its version changes.
+    Agreeing writes profile.json with a random contributor id.
+  - Bounties: the bounty board (recorder/bounties.py): what to record, the priority the
+    maintainers gave it, and how often you've done it. Refresh fetches the dataset's board
+    (hub.py board). Picking one goes on to the checklist.
+  - Before you start: the checklist (glasses, eye makeup, face pad and strap, a measured IPD),
+    kept in profile.json; the health questions behind their own tick, kept in BASE/health.json
+    and encrypted in the export; free space; the eye camera check (frames from ft-eyegrab at
+    90 fps, with the fix when SteamVR runs them at 15). Start hands the round to session.py.
+  - Round: the runner's live status, the fit card while the headset is fitted, Next, Pause,
+    Redo, Skip and Stop (Space, P, R, S and Esc while the window has focus; the button on the
+    headset's right side too). The prompts and dots appear in the headset.
+  - Review: rounds, their takes, and a viewer of both eye cameras a frame at a time, where
+    ranges, takes and rounds can be deleted (takes.py).
+  - Export: encrypt what's kept into exports/<round>/ at nice 19 (takes.py).
+  - Upload: the Hugging Face login, and Upload: hub.py checks the export (validate.py) and
+    uploads it as a pull request. Nothing reaches the real dataset until CONSENT.md and
+    UPLOAD.md are reviewed (hub.py's gate); --hub-dry-run does everything but the network calls.
+Everything lives under ~/.local/share/frametop/eyes/contrib (--base). Launch with
+gaze/rec/ft-eyerec (host wrapper).
+"""
+import argparse
+import bisect
+import datetime
+import json
+import os
+import signal
+import subprocess
+import sys
+import threading
+import time
+
+from PySide6.QtCore import Property, Qt, QTimer, QUrl, Signal, Slot
+from PySide6.QtGui import QColor, QFont, QGuiApplication, QIcon, QImage, QPainter
+from PySide6.QtQml import QQmlApplicationEngine
+from PySide6.QtQuick import QQuickImageProvider
+from PySide6.QtQuickControls2 import QQuickStyle
+import shiboken6
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.dirname(HERE)
+REPO = os.path.dirname(GAZE)
+sys.path.insert(0, HERE)
+sys.path.insert(0, REPO)
+import eyerec  # noqa: E402  (next to this file; it imports gaze/tracker/eyes_share.py)
+import eyes_share  # noqa: E402
+import hub  # noqa: E402
+import takes  # noqa: E402
+from recorder import bounties  # noqa: E402
+from recorder.common import host_command, write_json  # noqa: E402
+from recorder.files import human_bytes, read_json  # noqa: E402
+from recorder.window import RecorderBackend  # noqa: E402
+
+HUB_PATH = os.path.join(HERE, "hub.py")
+EYE_RATE = os.path.join(GAZE, "eye-rate.py")
+AWAKE_UNIT = "frametop-eyerec-awake.service"
+# A round of about 5 recorded minutes is about 6 GB (1.2 GB a minute: DESIGN.md, "Size"), and
+# its export as much again.
+ROUND_BYTES = 12 * 1000 ** 3
+ACTIVE_STATES = ("starting", "fitment", "ready", "countdown", "running", "paused")
+EYE_FPS_OK = 60          # under this the cameras run at SteamVR's 15 fps: recordings are worth little
+EYE_CHECK_S = 1.5        # how long the eye camera check counts frames
+# After the eye rate fix's SteamVR restart: check the cameras this often, for at most this long.
+RECHECK_S = 3
+RECHECK_FOR_S = 120
+FRAME_HEIGHT = 400       # the viewer's height for each camera (the cameras' own)
+THUMB_HEIGHT = 96
+# The checklist's choices (DESIGN.md, "Checklist"); the keys are what profile.json keeps.
+GLASSES = [("none", "No glasses"), ("glasses", "Glasses"), ("inserts", "Prescription lens inserts"),
+           ("contacts", "Contact lenses")]
+MAKEUP = [("mascara", "Mascara"), ("eyeliner", "Eyeliner"), ("lashes", "Lash extensions")]
+PADS = [("", "Rather not say"), ("stock", "The face pad it came with"), ("other", "Another face pad")]
+STRAPS = [("", "Rather not say"), ("stock", "The strap it came with"), ("other", "Another strap")]
+# The health questions: only with their own tick, and only ever in health.json(.age).
+HEALTH = [("surgery", "LASIK or other eye surgery"), ("strabismus", "A lazy or turned eye"),
+          ("ptosis", "A droopy eyelid"), ("nystagmus", "Nystagmus (eyes that move by themselves)"),
+          ("other", "Another eye condition")]
+GROUP_TITLES = {"ipd": "IPD sweep", "dots": "Nine dots"}
+STATUS_TEXT = {"running": "interrupted", "error": "ended with an error", "stopped": "stopped early"}
+
+
+def round_label(rid, title=""):
+    """20261014-120000-edges -> 2026-10-14 12:00 · View edges."""
+    try:
+        when = datetime.datetime.strptime(rid[:15], "%Y%m%d-%H%M%S").strftime("%Y-%m-%d %H:%M")
+    except ValueError:
+        return rid
+    rest = rid[16:]
+    n = rest.rsplit("-", 1)[1] if "-" in rest and rest.rsplit("-", 1)[1].isdigit() else ""
+    return f"{when} · {title or rest}" + (f" ({n})" if n else "")
+
+
+def take_title(group):
+    return GROUP_TITLES.get(group) or group.replace("-", " ").replace("_", " ").capitalize()
+
+
+def clock_text(seconds):
+    return f"{int(seconds // 60)}:{seconds % 60:04.1f}"
+
+
+def eye_check(share_path=None, seconds=EYE_CHECK_S):
+    """Are the eye cameras' frames coming, and how fast? {"state": ok|slow|none, "fps": [right,
+    left], "text"}. Counts frames in ft-eyegrab's share (read-only; it touches the want file,
+    which wakes ft-eyegrab, as every reader does)."""
+    share = eyes_share.open_share(3.0, share_path)
+    if share is None:
+        return {"state": "none", "fps": [0, 0],
+                "text": "No eye camera frames: frametop-eyegrab.service isn't running. Frametop's gaze setup "
+                        "installs it (gaze/tracker/install.sh), and SteamVR has to be running."}
+    try:
+        time.sleep(0.5)    # ft-eyegrab starts copying once it sees the want file
+        eyes_share.touch_want()
+        a, t0 = [share.count(c) for c in eyerec.CAMERAS], time.monotonic()
+        time.sleep(seconds)
+        b, t1 = [share.count(c) for c in eyerec.CAMERAS], time.monotonic()
+    finally:
+        share.close()
+    fps = [round((y - x) / (t1 - t0), 1) for x, y in zip(a, b)]
+    if max(fps) <= 0:
+        return {"state": "none", "fps": fps,
+                "text": "The eye cameras aren't sending frames. Is SteamVR running? If it is, restarting it "
+                        "starts the eye cameras again."}
+    if min(fps) < EYE_FPS_OK:
+        return {"state": "slow", "fps": fps,
+                "text": f"The eye cameras run at {min(fps):.0f} frames a second; recordings need 90. SteamVR runs "
+                        "them at 90 with a setting Frametop can turn on, from SteamVR's next start."}
+    return {"state": "ok", "fps": fps, "text": ""}
+
+
+class Timeline:
+    """A take's frames for the viewer: the camera with more frames sets the timeline, and the
+    other camera's frame nearest in time goes beside it. Chunks are decompressed when needed
+    and the last few kept."""
+    KEEP = 4
+
+    def __init__(self, take_dir, clock):
+        self.dir = take_dir
+        self.clock = clock or []
+        frames = {c: [] for c in eyerec.CAMERAS}
+        for c, chunk, i, _slot, t in eyerec.read_index(take_dir):
+            if c in frames:
+                frames[c].append((t, chunk, i))
+        for c in frames:
+            frames[c].sort()
+        self.main = max(eyerec.CAMERAS, key=lambda c: (len(frames[c]), -c))
+        self.frames = frames
+        self.times = {c: [f[0] for f in frames[c]] for c in frames}
+        self._chunks = {}
+        self._lock = threading.Lock()
+
+    def __len__(self):
+        return len(self.frames[self.main])
+
+    def time(self, i):
+        return self.frames[self.main][i][0]
+
+    def mono_ns(self, t_raw):
+        """A frame's RAW seconds as CLOCK_MONOTONIC ns (take.json's clock samples, turned round)."""
+        raw = int(round(t_raw * 1e9))
+        guess = raw - takes.raw_offset_ns(raw, self.clock)
+        return raw - takes.raw_offset_ns(guess, self.clock)
+
+    def _chunk(self, cam, chunk):
+        with self._lock:
+            key = (cam, chunk)
+            if key in self._chunks:
+                self._chunks[key] = self._chunks.pop(key)   # the newest at the end
+                return self._chunks[key]
+        frames = eyerec.read_chunk(self.dir, cam, chunk)
+        with self._lock:
+            self._chunks[key] = frames
+            while len(self._chunks) > self.KEEP:
+                self._chunks.pop(next(iter(self._chunks)))
+        return frames
+
+    def pair(self, i):
+        """{camera: bytes} at timeline index i: the main camera's frame and the other's nearest."""
+        out = {}
+        t, chunk, k = self.frames[self.main][i]
+        out[self.main] = self._chunk(self.main, chunk)[k]
+        for c in eyerec.CAMERAS:
+            if c == self.main or not self.frames[c]:
+                continue
+            j = bisect.bisect_left(self.times[c], t)
+            j = min(range(max(0, j - 1), min(len(self.times[c]), j + 1)), key=lambda n: abs(self.times[c][n] - t))
+            _, chunk, k = self.frames[c][j]
+            out[c] = self._chunk(c, chunk)[k]
+        return out
+
+
+class Timelines:
+    """Timelines by take, made again when the take's index.tsv or take.json changes."""
+
+    def __init__(self, store):
+        self.store = store
+        self._made = {}
+        self._lock = threading.Lock()
+
+    def get(self, rnd, take):
+        d = self.store.take_dir(rnd, take)
+        stamp = tuple(os.path.getmtime(os.path.join(d, n)) if os.path.exists(os.path.join(d, n)) else 0
+                      for n in ("index.tsv", "take.json"))
+        with self._lock:
+            got = self._made.get((rnd, take))
+            if got and got[0] == stamp:
+                return got[1]
+        tl = Timeline(d, read_json(os.path.join(d, "take.json")).get("clock"))
+        with self._lock:
+            self._made[(rnd, take)] = (stamp, tl)
+            if len(self._made) > 8:
+                self._made.pop(next(iter(self._made)))
+        return tl
+
+
+def grey_image(data):
+    img = QImage(data, eyerec.W, eyerec.H, eyerec.W, QImage.Format_Grayscale8)
+    return img.copy()
+
+
+class EyeProvider(QQuickImageProvider):
+    """image://eyes/pair/ROUND/TAKE/INDEX: both eye cameras at a timeline index, the right eye
+    (camera 0) first. image://eyes/thumb/ROUND/TAKE: the main camera mid-take, small. Anything
+    after "?" is ignored (it makes QML load the image again)."""
+
+    def __init__(self, timelines):
+        super().__init__(QQuickImageProvider.ImageType.Image)
+        self.timelines = timelines
+
+    def requestImage(self, ident, size, requested):
+        parts = ident.split("?", 1)[0].split("/")
+        try:
+            if parts[0] == "thumb" and len(parts) == 3:
+                tl = self.timelines.get(parts[1], parts[2])
+                if not len(tl):
+                    raise ValueError("no frames")
+                data = tl.pair(len(tl) // 2)[tl.main]
+                return grey_image(data).scaledToHeight(THUMB_HEIGHT, Qt.SmoothTransformation)
+            if parts[0] == "pair" and len(parts) == 4:
+                return self._pair(self.timelines.get(parts[1], parts[2]), int(parts[3]),
+                                  requested.height() if requested.height() > 0 else FRAME_HEIGHT)
+        except (OSError, ValueError, IndexError, KeyError) as e:
+            print(f"ft-eyerec: image {ident}: {e}", file=sys.stderr)
+        img = QImage(4, 3, QImage.Format_Grayscale8)
+        img.fill(0)
+        return img
+
+    def _pair(self, tl, i, height):
+        frames = tl.pair(i)
+        gap = 8
+        scaled = [(c, grey_image(frames[c]).scaledToHeight(height, Qt.SmoothTransformation))
+                  for c in eyerec.CAMERAS if c in frames]
+        out = QImage(sum(s.width() for _, s in scaled) + gap * (len(scaled) - 1), height, QImage.Format_RGB32)
+        out.fill(QColor(30, 30, 30))
+        p = QPainter(out)
+        font = QFont()
+        font.setPixelSize(max(12, height // 22))
+        p.setFont(font)
+        x = 0
+        for c, img in scaled:
+            p.drawImage(x, 0, img)
+            p.setPen(QColor(255, 200, 80))
+            p.drawText(x + 6, 6 + font.pixelSize(), ("Right eye" if c == 0 else "Left eye") + f" (camera {c})")
+            x += img.width() + gap
+        p.end()
+        return out
+
+
+class Backend(RecorderBackend):
+    """The eye recorder's window backend: recorder/window.py's, plus the bounty board, the
+    checklist and health answers, the eye camera check, the round and the review."""
+    APP = "Frametop Eye Recorder"
+    AWAKE_UNIT = AWAKE_UNIT
+    AWAKE_WHY = "Exporting or uploading eye recordings"
+    HUB_PATH = HUB_PATH
+    CONSENT_PATH = hub.CONSENT_PATH
+    UPLOAD_PATH = hub.UPLOAD_PATH
+    ACTIVE_STATES = ACTIVE_STATES
+    ROUND_BYTES = ROUND_BYTES
+    PROFILE_KEEP = ("checklist", "bounties", "eyes_ipd")
+
+    # A property here can't notify with the base's signals (PySide6 6.11 leaves it without
+    # one), so these relay them.
+    profileRelay = Signal()
+    sessionsRelay = Signal()
+    boardChanged = Signal()
+    healthChanged = Signal()
+    eyesChanged = Signal()
+    _eyesArrived = Signal(dict)
+    _boardArrived = Signal(dict)
+
+    def __init__(self, store, session_options=None, hub_dry_run=False, ignore_eyes=False, timelines=None):
+        super().__init__(store, hub, hub_dry_run)
+        self.profileChanged.connect(self.profileRelay)
+        self.sessionsChanged.connect(self.sessionsRelay)
+        self.profileChanged.connect(self.boardChanged)    # the board's done marks and needs
+        self._session_options = session_options or {}  # Session options: dry_run, speed, cams, button
+        self._session_mod = None
+        self._session_error = ""
+        self._timelines = timelines or Timelines(store)
+        self._ignore_eyes = ignore_eyes
+        self._eyes = {}
+        self._eyes_busy = False
+        self._restarting = False
+        self._board = {"state": "", "text": ""}
+        self._board_busy = False
+        self._eyesArrived.connect(self._on_eyes)
+        self._boardArrived.connect(self._on_board)
+
+    # --- the session runner, imported when first needed
+    def _runner(self):
+        if self._session_mod is None and not self._session_error:
+            try:
+                import session as mod
+                self._session_mod = mod
+            except Exception as e:  # missing, or broken: say so, the rest of the app still works
+                self._session_error = f"The session runner (gaze/rec/session.py) can't be loaded: {e}"
+        return self._session_mod
+
+    def _runner_error(self):
+        self._runner()
+        return self._session_error
+
+    def _texts_draft(self):
+        return not hub.texts_ready()
+
+    def _upload_allowed(self):
+        try:
+            return hub.upload_allowed(hub.dataset_id())
+        except hub.HubError:
+            return False
+
+    def _upload_values(self, session, path):
+        return {"ROUND": session}
+
+    def _session_label(self, session):
+        return round_label(session)
+
+    @Property(str, constant=True)
+    def toolVersion(self):
+        return takes.tool_version()
+
+    # --- consent
+    @Slot(bool, bool, bool)
+    def acceptConsent(self, adult, region, agree):
+        """Write profile.json (recorder/window.py); its checklist and bounties stay."""
+        optional = self.store.profile().get("optional")
+        self._write_consent(adult, region, agree, optional if isinstance(optional, dict) else {})
+
+    # --- the bounty board
+    def _checklist(self):
+        c = self.store.profile().get("checklist")
+        return c if isinstance(c, dict) else {}
+
+    def _board_now(self):
+        mod = self._runner()
+        return mod.load_board(self.store.base) if mod else None
+
+    @Property("QVariantList", notify=boardChanged)
+    def bounties(self):
+        """The board's bounties (bounties.view), with "lastText" for the last round."""
+        mod = self._runner()
+        if not mod:
+            return []
+        rows = bounties.view(self._board_now(), self.store.profile(), self._checklist(), mod.have_round)
+        for b in rows:
+            b["lastText"] = round_label(b["last"]).split(" · ")[0] if b["last"] else ""
+            b["minutesText"] = f"about {b['minutes']:g} min" if b["minutes"] else ""
+        return rows
+
+    @Property(str, notify=boardChanged)
+    def boardUpdated(self):
+        board = self._board_now()
+        return board["updated"] if board else ""
+
+    @Property("QVariantMap", notify=boardChanged)
+    def boardFetch(self):
+        """The last fetch: {"state": ""|fetching|ok|error, "text"}."""
+        return self._board
+
+    @Slot()
+    def refreshBoard(self):
+        """hub.py board in a child process: the dataset's board into board-cache.json."""
+        if self._board_busy or self._hub_dry_run:
+            return
+        self._board_busy = True
+        self._board = {"state": "fetching", "text": "Getting the newest board"}
+        self.boardChanged.emit()
+        argv = self._hub_argv("board", "--json")
+
+        def run():
+            try:
+                r = subprocess.run(argv, capture_output=True, text=True, timeout=60)
+                lines = [json.loads(x) for x in r.stdout.splitlines() if x.startswith("{")]
+                result = lines[-1] if lines else {"error": {"text": (r.stderr.strip().splitlines() or ["hub.py failed"])[-1]}}
+            except (OSError, ValueError, subprocess.TimeoutExpired) as e:
+                result = {"error": {"text": f"The board didn't come: {e}"}}
+            self._boardArrived.emit(result)
+        self._thread(run)
+
+    def _on_board(self, result):
+        self._board_busy = False
+        if "done" in result:
+            self._board = {"state": "ok", "text": f"The newest board, of {result['done'].get('updated', '?')}"}
+        else:
+            self._board = {"state": "error", "text": (result.get("error") or {}).get("text", "The board didn't come")
+                           + " The board that came with the recorder is used."}
+        self.boardChanged.emit()
+
+    @Slot(str, result=str)
+    def planText(self, bounty_id):
+        """How long a bounty's round takes (session.plan_seconds), with the fit check before it."""
+        mod = self._runner()
+        b = next((b for b in self.bounties if b["id"] == bounty_id), None)
+        if not mod or not b:
+            return ""
+        try:
+            plan = mod.build_plan(mod.load_round(mod.round_path(b["script"])), 0)
+        except (OSError, ValueError) as e:
+            return f"Couldn't read the round: {e}"
+        return (f"{len(plan)} steps, about {mod.plan_seconds(plan) / 60:.0f} min recorded, after putting the "
+                "headset on")
+
+    # --- the checklist and health answers
+    @Property("QVariantMap", notify=profileRelay)
+    def checklist(self):
+        c = self._checklist()
+        return {"glasses": c.get("glasses") if c.get("glasses") in dict(GLASSES) else "none",
+                "makeup": [m for m in c.get("makeup") or [] if m in dict(MAKEUP)],
+                "pad": c.get("pad") if c.get("pad") in dict(PADS) else "",
+                "strap": c.get("strap") if c.get("strap") in dict(STRAPS) else "",
+                "ipd_mm": c.get("ipd_mm") if isinstance(c.get("ipd_mm"), (int, float)) else 0}
+
+    @Property("QVariantList", constant=True)
+    def glassesChoices(self):
+        return [{"value": k, "text": v} for k, v in GLASSES]
+
+    @Property("QVariantList", constant=True)
+    def makeupChoices(self):
+        return [{"value": k, "text": v} for k, v in MAKEUP]
+
+    @Property("QVariantList", constant=True)
+    def padChoices(self):
+        return [{"value": k, "text": v} for k, v in PADS]
+
+    @Property("QVariantList", constant=True)
+    def strapChoices(self):
+        return [{"value": k, "text": v} for k, v in STRAPS]
+
+    @Property("QVariantList", constant=True)
+    def healthChoices(self):
+        return [{"value": k, "text": v} for k, v in HEALTH]
+
+    @Slot("QVariantMap")
+    def saveChecklist(self, answers):
+        """Keep the checklist in profile.json: only the known answers. ipd_mm: 0 when not known,
+        else 50 to 80 mm."""
+        a = dict(answers)
+        try:
+            ipd = round(float(a.get("ipd_mm") or 0), 1)
+        except (TypeError, ValueError):
+            ipd = 0.0
+        checklist = {"glasses": a.get("glasses") if a.get("glasses") in dict(GLASSES) else "none",
+                     "makeup": sorted({m for m in a.get("makeup") or [] if m in dict(MAKEUP)}),
+                     "pad": a.get("pad") if a.get("pad") in dict(PADS) else "",
+                     "strap": a.get("strap") if a.get("strap") in dict(STRAPS) else "",
+                     "ipd_mm": ipd if 50 <= ipd <= 80 else None}
+        profile = self.store.profile()
+        if not profile or profile.get("checklist") == checklist:
+            return
+        profile["checklist"] = checklist
+        write_json(self.store.profile_path, profile)
+        self.profileChanged.emit()
+
+    def _health_path(self):
+        return os.path.join(self.store.base, "health.json")
+
+    def _health(self):
+        h = read_json(self._health_path())
+        return h if h.get("opt_in") is True else {}
+
+    @Property(bool, notify=healthChanged)
+    def healthOptIn(self):
+        return bool(self._health())
+
+    @Property("QVariantList", notify=healthChanged)
+    def healthAnswers(self):
+        return [k for k in self._health().get("conditions") or [] if k in dict(HEALTH)]
+
+    @Slot(bool, "QVariantList")
+    def saveHealth(self, opt_in, conditions):
+        """The health answers, in BASE/health.json (only this user can read it), never in
+        profile.json; each round gets a copy, which its export encrypts. Opting out deletes them."""
+        path = self._health_path()
+        if not opt_in:
+            try:
+                os.remove(path)
+            except FileNotFoundError:
+                pass
+        else:
+            data = {"schema": 1, "opt_in": True, "conditions": sorted({c for c in conditions if c in dict(HEALTH)})}
+            if read_json(path) == data:
+                return
+            fd = os.open(path + ".tmp", os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
+            with os.fdopen(fd, "w") as f:
+                json.dump(data, f, indent=1)
+                f.write("\n")
+            os.replace(path + ".tmp", path)
+        self.healthChanged.emit()
+
+    # --- the eye camera check
+    @Property(str, notify=eyesChanged)
+    def eyeState(self):
+        """ok, slow, none; checking while it runs; "" before the first check."""
+        if self._eyes_busy and not self._eyes:
+            return "checking"
+        return self._eyes.get("state", "")
+
+    @Property(str, notify=eyesChanged)
+    def eyeText(self):
+        return self._eyes.get("text", "")
+
+    @Property(str, notify=eyesChanged)
+    def eyeFpsText(self):
+        fps = self._eyes.get("fps") or []
+        return " · ".join(f"{name} {f:.0f} fps" for name, f in zip(("right", "left"), fps))
+
+    @Property(bool, notify=eyesChanged)
+    def eyeBusy(self):
+        return self._eyes_busy
+
+    @Property(bool, notify=eyesChanged)
+    def restartingSteamVR(self):
+        return self._restarting
+
+    @Property(bool, constant=True)
+    def eyesIgnored(self):
+        return self._ignore_eyes or bool(self._session_options.get("dry_run"))
+
+    @Property(bool, notify=eyesChanged)
+    def eyesBlockStart(self):
+        return not self.eyesIgnored and self._eyes.get("state") != "ok"
+
+    def _check_now(self):
+        if self.eyesIgnored:
+            return {"state": "ok", "fps": [], "text": ""}
+        try:
+            return eye_check(self._session_options.get("cams"))
+        except Exception as e:
+            return {"state": "none", "fps": [], "text": f"The eye camera check failed: {e}"}
+
+    @Slot()
+    def checkEyes(self):
+        if self._eyes_busy or self.sessionActive:
+            return
+        self._eyes_busy = True
+        self.eyesChanged.emit()
+        self._thread(lambda: self._eyesArrived.emit(self._check_now()))
+
+    def _on_eyes(self, result):
+        self._eyes = result
+        self._eyes_busy = self._restarting
+        self.eyesChanged.emit()
+
+    @Slot(bool)
+    def restartSteamVR(self, fast):
+        """fast: first turn on SteamVR's 90 fps eye cameras (gaze/eye-rate.py on, on the host).
+        Then systemctl --user restart --no-block steamvr.service (the job runs in systemd, so it
+        finishes even when this window closes with the Frametop desktop), and the eye check every
+        few seconds until the frames come at 90."""
+        if self._restarting or self.sessionActive:
+            return
+        self._restarting = self._eyes_busy = True
+        self.eyesChanged.emit()
+
+        def run():
+            result = None
+            try:
+                if fast:
+                    r = subprocess.run(host_command("python3", EYE_RATE, "on"), capture_output=True, text=True,
+                                       timeout=60)
+                    if r.returncode != 0:
+                        raise RuntimeError((r.stderr or r.stdout).strip() or "eye-rate.py: exit %d" % r.returncode)
+                r = subprocess.run(host_command("systemctl", "--user", "restart", "--no-block", "steamvr.service"),
+                                   capture_output=True, text=True, timeout=60)
+                if r.returncode != 0:
+                    raise RuntimeError((r.stderr or r.stdout).strip() or "exit %d" % r.returncode)
+                end = time.monotonic() + RECHECK_FOR_S
+                time.sleep(RECHECK_S * 3)
+                while time.monotonic() < end:
+                    result = self._check_now()
+                    if result.get("state") == "ok":
+                        break
+                    time.sleep(RECHECK_S)
+            except Exception as e:
+                result = {"state": "none", "fps": [], "text": f"Couldn't restart SteamVR: {e}"}
+            self._restarting = False
+            self._eyesArrived.emit(result or self._check_now())
+        self._thread(run)
+
+    # --- the round
+    @Slot(str, result=bool)
+    def startSession(self, bounty_id):
+        if self.sessionActive:
+            return False
+        mod = self._runner()
+        if not mod:
+            self.message.emit(self._session_error, True)
+            return False
+        if self.needsConsent:
+            self.message.emit("Agree to the consent text first (Welcome page)", True)
+            return False
+        entry = next((b for b in self.bounties if b["id"] == bounty_id), None)
+        if not entry:
+            self.message.emit("Pick a bounty first", True)
+            return False
+        if entry["available"]:
+            self.message.emit(entry["available"], True)
+            return False
+        if not self.eyesIgnored:
+            self._eyes = self._check_now()   # fresh: it takes two seconds
+            self.eyesChanged.emit()
+            if self._eyes.get("state") != "ok":
+                self.message.emit(self._eyes.get("text", "The eye cameras aren't ready"), True)
+                return False
+        board = self._board_now() or {}
+        try:
+            self._session = mod.Session(self.store.base, entry["script"], bounty=dict(entry, updated=board.get("updated")),
+                                        profile=self.store.profile(), checklist=self._checklist(), health=self._health(),
+                                        on_status=lambda s: self._statusArrived.emit(dict(s)),
+                                        profile_path=self.store.profile_path, **self._session_options)
+        except Exception as e:
+            self.message.emit(f"Couldn't set up the round: {e}", True)
+            return False
+        self._session_id = ""
+        self._status = {"state": "starting", "bounty": bounty_id}
+        self.statusChanged.emit()
+        session = self._session
+
+        def run():
+            try:
+                session.start()
+            except Exception as e:
+                self._statusArrived.emit({"state": "error", "error": str(e)})
+        self._thread(run)
+        return True
+
+    def _on_status(self, status):
+        super()._on_status(status)
+        if status.get("state") == "done":
+            self.profileChanged.emit()    # the bounty is marked done, and the IPD kept
+
+    # --- review
+    def _session_rows(self):
+        out = []
+        for r in self.store.rounds():
+            r["label"] = round_label(r["id"], r["title"])
+            r["sizeText"] = human_bytes(r["bytes"])
+            r["exportText"] = human_bytes(r["export_bytes"]) if r["exported"] else ""
+            r["active"] = self.sessionActive and r["id"] == self._session_id
+            r["statusText"] = "" if r["active"] else STATUS_TEXT.get(r["status"], "")
+            out.append(r)
+        return out
+
+    @Slot(str, result="QVariantList")
+    def takeList(self, rnd):
+        try:
+            rows = self.store.takes(rnd)
+        except ValueError:
+            return []
+        for t in rows:
+            t["title"] = take_title(t["group"])
+            t["durationText"] = f"{int(t['duration_s'] // 60)}:{int(t['duration_s'] % 60):02d}"
+            t["sizeText"] = human_bytes(t["bytes"])
+            # Per eye, as the viewer steps (the camera with more frames)
+            t["frameCount"] = max(t["frames"], default=0)
+            t["deletedCount"] = max(t["deleted_frames"], default=0)
+            t.pop("ranges")
+        return rows
+
+    @Slot(str, str, result="QVariantMap")
+    def takeInfo(self, rnd, take):
+        """For the viewer: {count, title, ranges: [[first, last] timeline indexes]}."""
+        try:
+            tl = self._timelines.get(rnd, take)
+            ranges = self.store.ranges(rnd, take)
+            meta = self.store.take_meta(rnd, take)
+        except (OSError, ValueError):
+            return {"count": 0, "title": take, "ranges": []}
+        marks = []
+        monos = [tl.mono_ns(t) for t in tl.times[tl.main]]
+        for a, b in ranges:
+            first, last = bisect.bisect_left(monos, a), bisect.bisect_right(monos, b) - 1
+            marks.append([first, last] if first <= last else [-1, -1])
+        return {"count": len(tl), "title": take_title(meta.get("group", take.split("-", 1)[-1])), "ranges": marks}
+
+    @Slot(str, str, int, result=str)
+    def frameTime(self, rnd, take, i):
+        """Frame i's time from the take's first, m:ss.s."""
+        try:
+            tl = self._timelines.get(rnd, take)
+            return clock_text(tl.time(i) - tl.time(0))
+        except (OSError, ValueError, IndexError):
+            return ""
+
+    @Slot(str, str, int, int)
+    def deleteRange(self, rnd, take, first, last):
+        """Leave frames first..last (timeline indexes, either order) out of the export: both
+        cameras', by time."""
+        if self._active_guard(rnd):
+            return
+        tl = self._timelines.get(rnd, take)
+        first, last = sorted((max(0, first), min(len(tl) - 1, last)))
+        if first > last:
+            return
+        # Half a millisecond either side: index.tsv keeps whole microseconds, and the clocks'
+        # conversion rounds; the frames are 11 ms apart.
+        self.store.delete_range(rnd, take, tl.mono_ns(tl.time(first)) - 500000, tl.mono_ns(tl.time(last)) + 500000)
+        self.message.emit(f"Frames {first + 1} to {last + 1} won't be exported", False)
+        self.sessionsChanged.emit()
+
+    @Slot(str, str, int)
+    def restoreRange(self, rnd, take, k):
+        self.store.restore_range(rnd, take, k)
+        self.message.emit("Range restored", False)
+        self.sessionsChanged.emit()
+
+    @Slot(str, str)
+    def deleteTake(self, rnd, take):
+        if self._active_guard(rnd):
+            return
+        try:
+            self.store.delete_take(rnd, take)
+            self.message.emit(f"Deleted {take}", False)
+        except (OSError, ValueError) as e:
+            self.message.emit(f"Couldn't delete {take}: {e}", True)
+        self.refreshSessions()
+
+    @Slot(str)
+    def deleteSession(self, rnd):
+        if self._active_guard(rnd) or self._busy_guard(rnd):
+            return
+        try:
+            self.store.delete_round(rnd)
+            self.message.emit(f"Deleted the round of {round_label(rnd)}", False)
+        except (OSError, ValueError) as e:
+            self.message.emit(f"Couldn't delete the round: {e}", True)
+        self.refreshSessions()
+
+    # --- export
+    @Property(str, constant=True)
+    def exportProblem(self):
+        """Why nothing can be exported here, or ""."""
+        if eyerec.compress is None:
+            return "zstd isn't installed in the dev container, so nothing can be exported. Run setup/dev-container.sh."
+        try:
+            import cryptography  # noqa: F401  (recorder/age.py's)
+        except ImportError:
+            return ("The cryptography library isn't installed in the dev container, so nothing can be encrypted "
+                    "for export. Run setup/dev-container.sh.")
+        return ""
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Frametop Eye Recorder")
+    ap.add_argument("--base", default=takes.DEFAULT_BASE, help="where profile.json, rounds/ and exports/ go")
+    ap.add_argument("--page", default="",
+                    help="open on this page: welcome, bounties, checklist, session, review, export, upload")
+    ap.add_argument("--dry-run", action="store_true",
+                    help="test: rounds start no processes, make up the panel's replies and print its commands")
+    ap.add_argument("--speed", type=float, default=1.0, help="test, with --dry-run: run rounds this much faster")
+    ap.add_argument("--cams", help="test: read eye frames from this share (lab/ft-eyes-replay's)")
+    ap.add_argument("--no-headset-button", action="store_true",
+                    help="rounds don't read the headset's button (Next, pause, resume)")
+    ap.add_argument("--hub-dry-run", action="store_true",
+                    help="test: Upload checks the export and says what it would send, with no network calls")
+    ap.add_argument("--ignore-eyes", action="store_true",
+                    help="start rounds even if the eye camera check finds no frames, or slow ones")
+    ap.add_argument("--qml", default=os.path.join(HERE, "main.qml"),
+                    help="the window (default: this folder's main.qml, which needs Kirigami)")
+    ap.add_argument("--style", default="org.kde.desktop", help="the Qt Quick Controls style")
+    a, qt_args = ap.parse_known_args()
+    app = QGuiApplication([sys.argv[0]] + qt_args)
+    app.setApplicationName("ft-eyerec")
+    app.setApplicationDisplayName("Frametop Eye Recorder")
+    app.setDesktopFileName("ft-eyerec")
+    if not QIcon.themeName():
+        QIcon.setThemeName("breeze")
+    QQuickStyle.setStyle(a.style)
+    store = takes.Store(a.base)
+    timelines = Timelines(store)
+    engine = QQmlApplicationEngine()
+    engine.addImageProvider("eyes", EyeProvider(timelines))
+    options = {"dry_run": True, "speed": a.speed} if a.dry_run else {}
+    if a.cams:
+        options["cams"] = a.cams
+    if a.no_headset_button or a.dry_run:
+        options["button"] = False
+    backend = Backend(store, options, hub_dry_run=a.hub_dry_run, ignore_eyes=a.ignore_eyes, timelines=timelines)
+    app.aboutToQuit.connect(backend.shutdown)
+    mod = None if a.dry_run else backend._runner()
+    if mod:   # Frametop's gaze service, if a round that was killed left it paused
+        backend._thread(lambda: mod.resume_units(os.path.join(mod.run_dir(), mod.PAUSED_UNITS_FILE)))
+    engine.rootContext().setContextProperty("backend", backend)
+    engine.rootContext().setContextProperty("startPage", a.page)
+    engine.load(QUrl.fromLocalFile(os.path.abspath(a.qml)))
+    if not engine.rootObjects():
+        sys.exit(1)
+
+    # SIGTERM and Ctrl+C quit as closing does, so a running round still stops cleanly. Python
+    # runs signal handlers between bytecodes: the timer gives it some while Qt waits.
+    def on_signal(*_):
+        engine.rootObjects()[0].setProperty("quitting", True)
+        app.quit()
+    signal.signal(signal.SIGTERM, on_signal)
+    signal.signal(signal.SIGINT, on_signal)
+    tick = QTimer(interval=500, timeout=lambda: None)
+    tick.start()
+    code = app.exec()
+    # The window goes before the backend (as the hand recorder: bindings would run against it).
+    shiboken6.delete(engine)
+    sys.exit(code)
+
+
+if __name__ == "__main__":
+    main()

gaze/rec/hub.py

@@ -0,0 +1,280 @@
+#!/usr/bin/env python3
+"""Upload an eye recorder export to the eye dataset on Hugging Face.
+
+As the hand recorder's hub.py (the shared part is recorder/hub.py): the window runs it as a child
+process, it uses huggingface_hub's saved login (`hub.py login` is the browser's device code), and
+nobody types a token anywhere. An upload:
+  1. checks the export with validate.py (against recipients.txt's keys), and stops on errors;
+  2. stops if this export (same SHA256SUMS) was uploaded before, unless --again;
+  3. stops unless CONSENT.md and UPLOAD.md are here and neither is a draft. FT_EYEREC_ALLOW_UPLOAD=1
+     opens it for a rehearsal, and only to another dataset than the real one (FT_EYEREC_DATASET):
+     the real one takes nothing before the texts are reviewed; and it stops for an export made under
+     a draft consent text (agree to the current one, then export again);
+  4. checks the login and access to the dataset, opens the pull request first, uploads the folder
+     to contributions/<contributor>/<round> in it, and records it under "uploads" in session.json.
+--dry-run does all of it except the network calls and the records, and says what it would upload.
+
+The images and health answers are encrypted already (takes.py's export): nothing here can read
+them, and the dataset's readers need one of its keys.
+
+`hub.py board` fetches the bounty board (bounties.json) from the dataset into BASE/board-cache.json,
+with the saved login when there is one (the dataset is gated): the window's board then uses it
+when it's newer than the one shipped here (recorder/bounties.py checks what it may say).
+
+usage: hub.py [--base DIR] whoami [--json]
+       hub.py [--base DIR] login [--json]
+       hub.py [--base DIR] board [--json]
+       hub.py [--base DIR] upload ROUND [--dry-run] [--again] [--json]
+With --json, each line of output is one JSON object, as the hand recorder's hub.py.
+"""
+import argparse
+import json
+import os
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))
+import takes  # noqa: E402  (next to this file)
+from recorder import bounties  # noqa: E402
+from recorder import hub as rhub  # noqa: E402
+from recorder.common import write_json  # noqa: E402
+from recorder.hub import HubError, export_sha, is_draft, now_iso  # noqa: E402,F401
+
+HF_DATASET = "DeeJanuz/frametop-eyes"
+DATASET_ENV = "FT_EYEREC_DATASET"
+ALLOW_ENV = "FT_EYEREC_ALLOW_UPLOAD"
+CONSENT_PATH = os.path.join(HERE, "CONSENT.md")
+UPLOAD_PATH = os.path.join(HERE, "UPLOAD.md")
+
+
+def dataset_id():
+    """HF_DATASET, or FT_EYEREC_DATASET when set."""
+    return rhub.dataset_id(DATASET_ENV, HF_DATASET)
+
+
+def texts_ready():
+    return all(os.path.isfile(p) and not is_draft(p) for p in (CONSENT_PATH, UPLOAD_PATH))
+
+
+def upload_allowed(repo):
+    """Real uploads: once the texts are reviewed, or for a rehearsal against another dataset."""
+    return texts_ready() or (os.environ.get(ALLOW_ENV) == "1" and repo != HF_DATASET)
+
+
+def meta_path(store, rnd):
+    return os.path.join(store.round_dir(rnd), "session.json")
+
+
+def describe(summary, report, sha, nbytes):
+    """(commit message, commit description) from validate's summary of the manifest."""
+    s = summary
+    message = f"Eyes: {s.get('bounty')} round {s.get('round')} from {s.get('contributor')}"
+    lines = ["Contribution to the Frametop eye dataset, uploaded from the eye recorder. The eye images "
+             "and any health answers are encrypted to the dataset's keys.", "",
+             f"- Round: {s.get('round')}",
+             f"- Bounty: {s.get('bounty')}",
+             f"- Contributor: {s.get('contributor')}",
+             f"- Takes: {s.get('takes')} ({s.get('minutes')} minutes, {s.get('frames')} frames)",
+             f"- Glasses: {s.get('glasses') or 'not given'}",
+             f"- Health answers: {'included, encrypted' if s.get('health') else 'none'}",
+             f"- Consent version: {s.get('consent_version')}",
+             f"- Tool: {s.get('tool')}",
+             f"- Size: {takes.human_bytes(nbytes)}",
+             f"- SHA256 of SHA256SUMS: {sha}", "",
+             "Checked with gaze/rec/validate.py: no errors" + (f", {len(report.warnings)} warnings:"
+                                                                if report.warnings else ".")]
+    lines += [f"- {w}" for w in report.warnings]
+    return message, "\n".join(lines) + "\n"
+
+
+def _hf():
+    return rhub.hf_module("run setup/dev-container.sh (or: pip install --user huggingface_hub)")
+
+
+def whoami():
+    return rhub.whoami(_hf(), dataset_id())
+
+
+def login(progress=None):
+    _hf()
+    return rhub.login(dataset_id(), progress, lambda: whoami(), "update the dev container (setup/dev-container.sh).")
+
+
+def board_url(repo):
+    return f"{rhub.dataset_url(repo)}/resolve/main/bounties.json"
+
+
+def fetch_board(base):
+    """The dataset's bounties.json into base/board-cache.json (session.load_board reads it).
+    Returns {"updated", "bounties"}; raises HubError."""
+    repo = dataset_id()
+    token = None
+    try:
+        token = _hf().get_token()
+    except (HubError, AttributeError):
+        pass   # no huggingface_hub, or an old one: try without a login
+    try:
+        board = bounties.fetch(board_url(repo), os.path.join(base, "board-cache.json"), token)
+    except ValueError as e:
+        raise HubError("hub", f"The dataset's bounty board isn't valid: {e}") from None
+    except OSError as e:
+        status = getattr(e, "code", None)
+        if status in (401, 403):
+            raise HubError("terms", "Log in and accept the dataset's terms to get its newest bounty board.",
+                           rhub.dataset_url(repo)) from None
+        if status == 404:
+            raise HubError("not-found", f"{repo} has no bounty board yet.") from None
+        raise HubError("network", f"Couldn't fetch the bounty board ({type(e).__name__}: {e}).") from None
+    return {"updated": board["updated"], "bounties": len(board["bounties"])}
+
+
+def upload(store, rnd, dry_run=False, again=False, progress=None, log=None):
+    """Upload exports/<round> (the steps in this file's docstring). Returns the result; raises HubError."""
+    tell = progress or (lambda phase, text, fraction=None, **extra: None)
+    say = log or (lambda text: None)
+    import validate
+    repo = dataset_id()
+    try:
+        export = store.export_dir(rnd)
+    except ValueError as e:
+        raise HubError("no-export", str(e)) from None
+    if not os.path.isfile(os.path.join(export, "manifest.json")):
+        raise HubError("no-export", f"No export of round {rnd}: export it first")
+
+    tell("check", "Checking the export", 0.0)
+    report = validate.validate(export, progress=lambda f, text: tell("check", text, f))
+    if not report.ok:
+        raise HubError("invalid", f"The export has {len(report.errors)} problems, so it can't be uploaded. Export "
+                                  "the round again; if that doesn't help, report it.", errors=report.errors)
+    summary = report.summary
+    sha = export_sha(export)
+    before = rhub.previous_upload(meta_path(store, rnd), sha)
+    if before and not again:
+        raise HubError("duplicate", f"This export was uploaded already, on {before.get('uploaded', '?')}: "
+                                    f"{before.get('pr_url', '')}", before.get("pr_url", ""), extra={"previous": before})
+    if not dry_run and not upload_allowed(repo):
+        raise HubError("closed", "Eye contributions aren't open yet: the consent and upload texts are waiting for a "
+                                 f"legal review. (A rehearsal: {ALLOW_ENV}=1 with {DATASET_ENV} set to a test dataset.)")
+    if repo == HF_DATASET and str(summary.get("consent_version") or "").endswith("draft"):
+        raise HubError("draft-consent", "This export was made when you'd agreed to a draft of the consent text. "
+                                        "Agree to the current one on the Welcome page, then export the round again.")
+    path_in_repo = f"contributions/{summary.get('contributor', '')}/{rnd}"
+    message, description = describe(summary, report, sha, report.bytes)
+    files = []
+    for root, _, names in os.walk(export):
+        files += [os.path.relpath(os.path.join(root, n), export) for n in names]
+    plan = {"repo": repo, "repo_type": "dataset", "folder_path": export, "path_in_repo": path_in_repo,
+            "create_pr": True, "commit_message": message, "commit_description": description,
+            "files": len(files), "bytes": report.bytes, "warnings": report.warnings}
+
+    if dry_run:
+        say(f"dry run: would check the login (whoami) and access to {repo} (auth_check)")
+        say(f"dry run: would upload_folder {len(files)} files, {takes.human_bytes(report.bytes)}, "
+            f"from {export} to datasets/{repo}/{path_in_repo}, as a pull request")
+        for rel in sorted(files):
+            say(f"  {rel}  {takes.human_bytes(os.path.getsize(os.path.join(export, rel)))}")
+        say(f"dry run: commit message: {message}")
+        say("dry run: commit description:\n" + description.rstrip())
+        record = {"repo": repo, "pr_url": "", "uploaded": now_iso(), "export_sha": sha, "status": "done"}
+        say(f"dry run: would record in session.json's uploads: {json.dumps(record)}")
+        tell("done", "Dry run: nothing was uploaded", 1.0)
+        return dict(plan, dry_run=True, pr_url="", record=record)
+
+    pr, who, record = rhub.push(_hf(), repo, export, path_in_repo, message, description, meta_path(store, rnd), sha,
+                                len(files), report.bytes, tell, write_json, takes.human_bytes)
+    tell("done", "Uploaded", 1.0)
+    return dict(plan, dry_run=False, pr_url=record["pr_url"], pr_num=pr.num, user=who.get("name", ""), record=record)
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Upload an eye recorder export to the eye dataset on Hugging Face.")
+    ap.add_argument("--base", default=takes.DEFAULT_BASE)
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    w = sub.add_parser("whoami", help="show the saved Hugging Face login")
+    w.add_argument("--json", action="store_true")
+    li = sub.add_parser("login", help="log in to Hugging Face in the browser")
+    li.add_argument("--json", action="store_true")
+    b = sub.add_parser("board", help="fetch the dataset's bounty board")
+    b.add_argument("--json", action="store_true")
+    u = sub.add_parser("upload", help="upload exports/ROUND as a pull request")
+    u.add_argument("round")
+    u.add_argument("--dry-run", action="store_true", help="no network: say what would be uploaded")
+    u.add_argument("--again", action="store_true", help="upload even if this export was uploaded before")
+    u.add_argument("--json", action="store_true", help="JSON lines, for the window")
+    a = ap.parse_args()
+
+    def emit(obj):
+        print(json.dumps(obj), flush=True)
+
+    try:
+        if a.cmd == "whoami":
+            who = whoami()
+            if a.json:
+                emit({"done": who})
+            else:
+                print(f"logged in as {who['name']} (token role: {who['role'] or 'unknown'})")
+            return 0
+        if a.cmd == "board":
+            got = fetch_board(os.path.abspath(os.path.expanduser(a.base)))
+            if a.json:
+                emit({"done": got})
+            else:
+                print(f"bounty board of {got['updated']}: {got['bounties']} bounties")
+            return 0
+        if a.cmd == "login":
+            def code(phase, text, **extra):
+                if a.json:
+                    emit(dict(extra, phase=phase, text=text))
+                else:
+                    print(f"Open {extra['url']} and approve the code {extra['code']}. Waiting...", flush=True)
+            who = login(code)
+            if a.json:
+                emit({"done": who})
+            else:
+                print(f"logged in as {who['name']}")
+            return 0
+        if a.json:
+            def progress(phase, text, fraction=None, **extra):
+                emit(dict(extra, phase=phase, text=text, fraction=fraction))
+
+            def log(text):
+                emit({"log": text})
+        else:
+            last = {}
+
+            def progress(phase, text, fraction=None, **extra):
+                if extra.get("pr_url"):
+                    print(f"pull request: {extra['pr_url']}", flush=True)
+                line = text if fraction is None else f"{fraction * 100:5.1f}% {text}"
+                if (phase, text) != last.get("key") or fraction in (0.0, 1.0):
+                    print(line, file=sys.stderr, flush=True)
+                last["key"] = (phase, text)
+
+            def log(text):
+                print(text, flush=True)
+        if hasattr(os, "setpriority"):
+            try:
+                os.setpriority(os.PRIO_PROCESS, 0, 19)   # validation reads and hashes everything: stay out of VR's way
+            except OSError:
+                pass
+        result = upload(takes.Store(a.base), a.round, dry_run=a.dry_run, again=a.again, progress=progress, log=log)
+        if a.json:
+            emit({"done": result})
+        elif result["dry_run"]:
+            print(f"dry run done: {result['files']} files would go to datasets/{result['repo']}/{result['path_in_repo']}")
+        else:
+            print(f"uploaded: {result['pr_url']}")
+        return 0
+    except HubError as e:
+        if a.json:
+            emit({"error": e.as_dict()})
+        else:
+            print(f"error ({e.kind}): {e.text}", file=sys.stderr)
+            for line in e.errors:
+                print(f"  {line}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

gaze/rec/install.sh

@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+# Install the eye recorder on a Frame that has Frametop (get.sh --experimental): bring the dev
+# container's packages up to date, build the gaze tools the round uses (gaze/build.sh: ft-gaze and
+# the headset panel ft-gazepanel; gaze/tracker/build.sh: our tracker ft-eyes, which the fit coach
+# runs) and add "Frametop Eye Recorder" to the app menu (for Frametop's desktop, as the hand
+# recorder's). It doesn't change Frametop's gaze setup; the window offers SteamVR's 90 fps eye
+# cameras (gaze/eye-rate.py) when they're slower.
+# A release (FRAME_RELEASE) brings the gaze tools built in its image (pack/Containerfile), so there it
+# only adds the menu entry; its install.sh offers this.
+# The recorder reads the eye cameras through our eye tracker's frame grabber
+# (frametop-eyegrab.service, gaze/tracker/install.sh): without it, the window says so.
+# Usage: gaze/rec/install.sh            install or update
+#        gaze/rec/install.sh uninstall  remove the menu entry. Recordings stay where they are.
+set -euo pipefail
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
+. "$root/scripts/_env.sh"
+entry='~/.local/share/applications/frametop-eyerec.desktop'
+
+case ${1:-install} in
+  install)
+    if [ "$FRAME_RELEASE" = 1 ]; then
+      echo "== the menu entry"
+    else
+      echo "== 1/3 dev container packages"
+      "$root/setup/dev-container.sh"
+      echo "== 2/3 the gaze tools: ft-gaze, ft-gazepanel and ft-eyes"
+      "$root/gaze/build.sh"
+      "$root/gaze/tracker/build.sh"
+      echo "== 3/3 the menu entry"
+    fi
+    fill_template "$root/gaze/rec/ft-eyerec.desktop" |
+      on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
+    echo
+    echo "Installed. On Frametop's desktop, open \"Frametop Eye Recorder\" from the app menu."
+    echo "Recordings go to ~/.local/share/frametop/eyes/contrib."
+    on_frame 'systemctl is-enabled -q frametop-eyegrab.service 2>/dev/null' ||
+      echo "It needs our eye tracker's frame grabber, which isn't installed: gaze/tracker/install.sh"
+    ;;
+  uninstall)
+    on_frame "rm -f $entry"
+    echo "Removed the menu entry."
+    echo "Your recordings are still in ~/.local/share/frametop/eyes/contrib:"
+    echo "delete that folder to remove them."
+    ;;
+  *) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
+esac

gaze/rec/recipients.txt

@@ -0,0 +1,6 @@
+# The eye dataset's recipients (gaze/rec/DESIGN.md "Files"): exports encrypt each chunk and health.json
+# to all of them (recorder/age.py). Key 1 is kept on the 7i, key 2 offline,
+# key 3 on dody (the GPU training machine, which decrypts pull requests itself).
+age1a0tc98pj2s7gdfgrv4gam6ah0qv69ymf3t2cc25hcwepmats69hst35z2e
+age1r0zlq7kcyg4dz4y3vkl9j36jw3zc6ly6reddh9ag5kfy6aw0fy6s5hepat
+age1e5vwgc03s9mk3dj5j39lcgxg6qyvkml77qh7g8svygar49r2kd7svldt70

gaze/rec/rounds/README.md

@@ -0,0 +1,152 @@
+# Building an eye bounty
+
+A bounty is two things:
+
+- **A board entry** in `gaze/rec/bounties.json`: id, title, blurb, minutes, priority, the script it runs,
+  and what it needs from the checklist (`recorder/bounties.py` has the format).
+- **A round script**, `gaze/rec/rounds/<script>.json`: the steps (`session.py`'s docstring has the
+  format of every step kind).
+
+Scripts ship with the recorder. The dataset's copy of the board (`DeeJanuz/frametop-eyes`,
+`bounties.json`) can only describe, reorder, reprioritise or retire bounties, so a new bounty reaches
+people only with a Frametop release that carries its script.
+
+`load_round` enforces the rules marked **(checked)**: a script that breaks one doesn't load, and the
+tests fail.
+
+## What the round is for
+
+Every recorded second becomes training data, labelled by where the dot was. So:
+
+- **Know what each step labels.** In most steps the eyes are on a dot, and the dot's position is the
+  label. Where they aren't (eyes shut, reading, looking around freely), the cue words or the step's
+  kind say so, and training leaves those frames out or treats them as their own case.
+- **One edge case per bounty.** Say in the board's blurb and the DESIGN.md table which case it covers
+  and why the trackers fail there.
+- **Rounds open the same way.** The recorder adds putting the headset on and the nine dots before every
+  round. Don't repeat them. The group `dots` is theirs **(checked)**.
+
+## What the person sees and hears, step by step
+
+A round moves through the same states every step. Each one has its own way of telling the person what
+to do and when, and the step's text and cues have to fit it.
+
+| State | Recorded | What they do | How they know when |
+|---|---|---|---|
+| Ready screen | no | read the text, press Next | the title, the text, the keys line |
+| Countdown 3-2-1 | yes | look at the dot it shows | the numbers |
+| Eyes on dots | yes | look at each dot | its ring fills, then the dot moves; the bar along the bottom |
+| Eyes shut | yes | close, then open | beeps (below) |
+| Moving the headset | yes | push, slide or tilt it by hand, hold, let go | a cue word and a beep each way (below) |
+| Head or reading | yes | the one action the text says, until the bar runs out | the bar along the bottom |
+
+- **The ready screen says it all, the action first.** It's the only time they read: what to do, what
+  starts it ("when it says Push up and beeps"), and what ends it ("until its ring fills", "until the bar
+  runs out", "until it says Let go"). A step with no ready screen (`"ready": false`) can only add to that
+  in a few words, as they're doing something already.
+- **Nothing starts in the countdown.** It points the eyes at the first dot. A one-off action (a blink, a
+  push) starts on a cue word after it, never on "go" from the numbers.
+- **One-off actions get a cue word; ongoing ones get the bar.** Turning the head, reading or looking
+  around run until the bar runs out. A push, a blink or closing the eyes happens on its cue.
+- **Redo does a chain again.** R or two presses during a step start it again, from the first step of its
+  chain; at a ready screen they go back to the step before. Make a chain one thing to do over: a nudge,
+  the dots done holding it, and letting go.
+
+## Cues the person can't see get a beep
+
+The panel can't cue someone whose eyes are shut. A cue that asks for that plays a beep
+(`gaze/rec/sounds.py`) as its third field: `["Close your eyes", 2.5, "close"]`.
+
+- **Blink:** a cue whose word asks for a blink plays `blink` **(checked)**.
+- **Both eyes shut:** a cue asking to close or shut both eyes plays `close` **(checked)**, and the next
+  cue plays `open` **(checked)**, so the person hears when to open them. A step never ends with the eyes
+  shut.
+- **Say it in the step's text**, which is read before Next: "Close your eyes when you hear a beep, and
+  then open your eyes when you hear the beep again." The words still show on the panel too.
+- **One eye or half-closed** (winks, squints, half-closed lids) keep words alone: the open eye reads
+  them. They may beep too.
+- **No voice.** Beeps only; generated speech sounds odd in the headset. A new kind of cue sound goes
+  into `sounds.SOUNDS` and the tests.
+
+## Moving the headset by hand
+
+A hand on the headset can't press Next, and the moment it moves is what the data is about. So:
+
+- **A cue word and a beep start it:** a cue word starting with push, pull, slide, tilt, tip or nudge plays
+  `go` **(checked)**: `["Push up", 2.5, "go"]`. Say the direction in the word, the same way each time
+  ("to your left", "left side up").
+- **Eyes first:** a cue on the dot comes before it, so there's a still moment to compare with
+  **(checked)**: `["Look at the dot", 1.5]`.
+- **Letting go has its own cue and beep:** "Let go" plays `done` **(checked)** and shows 2 s or more
+  **(checked)**, long enough to put the headset back and look at the dot again.
+- **In the same chain:** the `done` cue comes before the next ready screen, a new group or a reseat
+  **(checked)**, so a redo does the push, the dots and the letting go again.
+- **Steps done while holding say so:** their text starts "Keep holding" **(checked)**.
+- **Say exactly how, in the ready screen's text:** where the hand goes (the same grip each time), which way
+  it moves, how far, what stays still ("Keep it level", "Its middle stays where it is"), and that the head
+  and eyes don't move. "Push the headset up" alone leaves people guessing.
+- **A few millimetres, one hand, gently.** Say it in the round's intro, with the kinds of move to come, and
+  that a little blur while it's moved is fine. Never press it into the face.
+
+## Cue words are labels
+
+Training reads the cue words in `prompts.jsonl` (`frame-eyes/train/data/ingest.py`, `shut_cams`):
+
+- A word with **blink, close, shut, wink or yawn** marks the eyes as shut while it shows. **Left** or
+  **right** limits that to one eye, and **half** means the lids are partly open, so the frames keep
+  their labels.
+- Use those words only for that. "Open", "Look", "Relax" and "Normal" are the words for going back.
+- Keep words short (40 characters at most): they show big on the panel.
+
+## Timing
+
+- **Dots:** a hold of 1.2 s or more after a settle of about 0.6 s (the defaults). The eyes take that
+  long to land on a dot and stay.
+- **Shut eyes:** 2 s or more closed, then 2 s or more open before anything else, so the eyes settle back
+  on the dot.
+- **Blinks on a beep:** uneven gaps, 1.2 to 2 s, so the beep can't be predicted.
+- **Length:** about 4 to 6 minutes recorded. Check with `python3 gaze/rec/session.py <script> --plan`, and
+  put that, rounded, in the board's `minutes`.
+
+## Where things can go
+
+- **Dots and paths stay on the panel** **(checked)**: about 30 degrees to the sides, 23 up and down, less
+  in the corners (`off_panel`).
+- **Distance:** 0.25 to 10 m **(checked)**. 1.5 m unless distance is what the bounty is about.
+- **Head still,** unless the bounty is about moving it: say "Keep your head still" in the text of dot steps.
+
+## Takes
+
+- Steps of the same `group` share one take, and a group's steps must follow each other **(checked)**.
+  Group steps that a reviewer would delete together.
+- A `reseat` step isn't recorded, and a round with one gets no shift check.
+- Every step waits for Next unless it has `"ready": false`. Only chain steps that follow on naturally.
+
+## Comfort, safety and privacy
+
+- Comfortable eye turns only: "as far as is comfortable", never "as far as you can".
+- No flashing: brightness changes are steps of a second or more (`light` levels), never a flicker.
+- Nothing that asks for sound, the outward cameras, or anything about the person beyond the checklist.
+  Talking is fine (it moves the face), but the microphone is never recorded.
+- Text to read is neutral and made up: no names, places or questions about the person.
+
+## Writing the text
+
+- Plain words, one instruction per step, the action first: "Look at each dot until its ring fills."
+- `|` starts a new line on the panel, `||` leaves a blank line.
+- Say what to do, not why. The why goes in the round's `intro` or the board's blurb.
+
+## Before it ships
+
+1. `python3 gaze/rec/session.py <script> --plan`: the steps and their length.
+2. The tests, in the dev container: `cd gaze/rec && for t in tests/test_*.py; do python3 $t; done`.
+   `test_shipped_rounds_load` loads every script; update its count.
+3. A dry run: `python3 gaze/rec/session.py <script> --dry-run --speed 20 --next-after 0.1`.
+4. A replay end to end: `frame-eyes/tools/eyerec-dryrun <bounty>` (record, export, validate), and the
+   export through `eyes_hub.py check` and `ingest.py` on dody. Delete those rounds afterwards: their
+   labels are wrong.
+5. A real round in the headset, with a note in `HEADSET-TESTS.md`: is every cue clear, and is the
+   length right?
+6. Add the bounty to `gaze/rec/bounties.json` and a row to DESIGN.md's bounty table. After a release
+   carries the script, put it on the dataset's board too (`frame-eyes/dataset-repo/bounties.json`,
+   uploaded to `DeeJanuz/frametop-eyes`).

gaze/rec/rounds/blinks.json

@@ -0,0 +1,245 @@
+{
+ "schema": 1,
+ "id": "blinks",
+ "title": "Blinks and lids",
+ "intro": "Blinks, closed and half-closed eyes, some of them on a beep. Keep looking at the dot whenever your eyes are open.",
+ "steps": [
+  {
+   "id": "blink-cue",
+   "kind": "prompt",
+   "text": "Look at the dot and blink once on each beep.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Blink",
+     1.6,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.3,
+     "blink"
+    ],
+    [
+     "Blink",
+     2.0,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.5,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.8,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.2,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.7,
+     "blink"
+    ],
+    [
+     "Blink",
+     1.5,
+     "blink"
+    ]
+   ]
+  },
+  {
+   "id": "closed",
+   "kind": "prompt",
+   "text": "Close your eyes when you hear a beep, and then open your eyes when you hear the beep again.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look",
+     2
+    ],
+    [
+     "Close your eyes",
+     2.5,
+     "close"
+    ],
+    [
+     "Open your eyes",
+     2,
+     "open"
+    ],
+    [
+     "Close your eyes",
+     2.5,
+     "close"
+    ],
+    [
+     "Open your eyes",
+     2,
+     "open"
+    ]
+   ]
+  },
+  {
+   "id": "winks",
+   "kind": "prompt",
+   "text": "Wink each eye in turn when it says so (as well as you can).",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look",
+     1.5
+    ],
+    [
+     "Close your left eye",
+     2
+    ],
+    [
+     "Open",
+     1.5
+    ],
+    [
+     "Close your right eye",
+     2
+    ],
+    [
+     "Open",
+     1.5
+    ],
+    [
+     "Close your left eye",
+     2
+    ],
+    [
+     "Open",
+     1.5
+    ],
+    [
+     "Close your right eye",
+     2
+    ],
+    [
+     "Open",
+     1.5
+    ]
+   ]
+  },
+  {
+   "id": "squint",
+   "kind": "prompt",
+   "text": "Keep looking at the dot while you change how open your eyes are.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Normal",
+     2
+    ],
+    [
+     "Squint",
+     3
+    ],
+    [
+     "Normal",
+     2
+    ],
+    [
+     "Eyes wide open",
+     3
+    ],
+    [
+     "Normal",
+     2
+    ],
+    [
+     "Half closed",
+     3
+    ],
+    [
+     "Normal",
+     2
+    ]
+   ]
+  },
+  {
+   "id": "lids-down",
+   "kind": "dots",
+   "text": "Look down at each dot. Let your lids drop as they do when you look down.",
+   "dots": [
+    [
+     0,
+     -12
+    ],
+    [
+     0,
+     -18
+    ],
+    [
+     12,
+     -18
+    ],
+    [
+     -12,
+     -18
+    ],
+    [
+     0,
+     -23
+    ],
+    [
+     18,
+     -20
+    ],
+    [
+     -18,
+     -20
+    ]
+   ],
+   "hold": 1.5,
+   "shuffle": true
+  },
+  {
+   "id": "blink-dots",
+   "kind": "dots",
+   "text": "Look at each dot and blink once while its ring fills.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     10
+    ],
+    [
+     -15,
+     10
+    ],
+    [
+     15,
+     -10
+    ],
+    [
+     -15,
+     -10
+    ]
+   ],
+   "hold": 2.0
+  }
+ ]
+}

gaze/rec/rounds/depth.json

@@ -0,0 +1,89 @@
+{
+ "schema": 1,
+ "id": "depth",
+ "title": "Depth",
+ "intro": "Dots near and far. The panel moves closer and further away; keep the dot sharp.",
+ "steps": [
+  {
+   "id": "near",
+   "kind": "dots",
+   "text": "The panel is close now. Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    3,
+    3,
+    15,
+    15
+   ],
+   "hold": 1.5,
+   "shuffle": true,
+   "distance": 0.5
+  },
+  {
+   "id": "mid",
+   "kind": "dots",
+   "text": "The panel is at arm's length and a bit. Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    3,
+    3,
+    15,
+    15
+   ],
+   "hold": 1.5,
+   "shuffle": true,
+   "distance": 1.5
+  },
+  {
+   "id": "far",
+   "kind": "dots",
+   "text": "The panel is far now. Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    3,
+    3,
+    15,
+    15
+   ],
+   "hold": 1.5,
+   "shuffle": true,
+   "distance": 5.0
+  },
+  {
+   "id": "near-far",
+   "kind": "prompt",
+   "text": "Keep looking at the dot as it comes near and goes far.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Far",
+     2.5
+    ],
+    [
+     "Near",
+     2.5
+    ],
+    [
+     "Far",
+     2.5
+    ],
+    [
+     "Near",
+     2.5
+    ],
+    [
+     "Far",
+     2.5
+    ],
+    [
+     "Near",
+     2.5
+    ]
+   ],
+   "cue_distance": {
+    "Far": 5.0,
+    "Near": 0.5
+   }
+  }
+ ]
+}

gaze/rec/rounds/edges.json

@@ -0,0 +1,129 @@
+{
+ "schema": 1,
+ "id": "edges",
+ "title": "View edges",
+ "intro": "Dots out at the edges of the panel and in its corners: turn only your eyes to them, as far as is comfortable.",
+ "steps": [
+  {
+   "id": "edges",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.|Some are far out: turn your eyes, not your head.",
+   "dots": [
+    [
+     24,
+     0
+    ],
+    [
+     -24,
+     0
+    ],
+    [
+     30,
+     0
+    ],
+    [
+     -30,
+     0
+    ],
+    [
+     0,
+     18
+    ],
+    [
+     0,
+     -18
+    ],
+    [
+     0,
+     23
+    ],
+    [
+     0,
+     -23
+    ],
+    [
+     30,
+     20
+    ],
+    [
+     -30,
+     20
+    ],
+    [
+     30,
+     -20
+    ],
+    [
+     -30,
+     -20
+    ],
+    [
+     22,
+     16
+    ],
+    [
+     -22,
+     16
+    ],
+    [
+     22,
+     -16
+    ],
+    [
+     -22,
+     -16
+    ]
+   ],
+   "hold": 1.2,
+   "shuffle": true
+  },
+  {
+   "id": "edges-sweep",
+   "kind": "path",
+   "text": "Follow the dot with your eyes as it goes round, far out.",
+   "path": "circle 0 0 22 8",
+   "seconds": 16
+  },
+  {
+   "id": "edges-again",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     30,
+     0
+    ],
+    [
+     -30,
+     0
+    ],
+    [
+     0,
+     23
+    ],
+    [
+     0,
+     -23
+    ],
+    [
+     30,
+     20
+    ],
+    [
+     -30,
+     20
+    ],
+    [
+     30,
+     -20
+    ],
+    [
+     -30,
+     -20
+    ]
+   ],
+   "hold": 1.2,
+   "shuffle": true
+  }
+ ]
+}

gaze/rec/rounds/face.json

@@ -0,0 +1,81 @@
+{
+ "schema": 1,
+ "id": "face",
+ "title": "Face moves",
+ "intro": "Faces move the headset and the lids. Keep looking at the dot while you make each face. The microphone isn't recorded.",
+ "steps": [
+  {
+   "id": "faces",
+   "kind": "prompt",
+   "text": "Keep looking at the dot and do what it says.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Smile",
+     3
+    ],
+    [
+     "Relax",
+     2
+    ],
+    [
+     "Raise your eyebrows",
+     3
+    ],
+    [
+     "Relax",
+     2
+    ],
+    [
+     "Frown",
+     3
+    ],
+    [
+     "Relax",
+     2
+    ],
+    [
+     "Yawn (or pretend to)",
+     4
+    ],
+    [
+     "Relax",
+     2
+    ],
+    [
+     "Chew",
+     4
+    ],
+    [
+     "Relax",
+     2
+    ]
+   ]
+  },
+  {
+   "id": "talk",
+   "kind": "prompt",
+   "text": "Read this aloud while looking at the dot as much as you can:|\"The quick brown fox jumps over the lazy dog, then naps in the warm afternoon sun.\"",
+   "dot": [
+    0,
+    10
+   ],
+   "seconds": 10
+  },
+  {
+   "id": "faces-dots",
+   "kind": "dots",
+   "text": "Smile and keep smiling. Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    3,
+    3,
+    15,
+    15
+   ],
+   "hold": 1.0
+  }
+ ]
+}

gaze/rec/rounds/glasses.json

@@ -0,0 +1,69 @@
+{
+ "schema": 1,
+ "id": "glasses",
+ "title": "Glasses",
+ "intro": "With your glasses or inserts on. Wear them as you usually do in the headset.",
+ "steps": [
+  {
+   "id": "grid",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    5,
+    5,
+    24,
+    18
+   ],
+   "hold": 1.2,
+   "shuffle": true
+  },
+  {
+   "id": "edges",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.|Some are far out: turn your eyes, not your head.",
+   "dots": [
+    [
+     30,
+     0
+    ],
+    [
+     -30,
+     0
+    ],
+    [
+     0,
+     23
+    ],
+    [
+     0,
+     -23
+    ],
+    [
+     30,
+     20
+    ],
+    [
+     -30,
+     20
+    ],
+    [
+     30,
+     -20
+    ],
+    [
+     -30,
+     -20
+    ]
+   ],
+   "hold": 1.2,
+   "shuffle": true
+  },
+  {
+   "id": "follow",
+   "kind": "path",
+   "text": "Follow the dot with your eyes.",
+   "path": "eight 0 0 22 14 6",
+   "seconds": 12
+  }
+ ]
+}

gaze/rec/rounds/head.json

@@ -0,0 +1,68 @@
+{
+ "schema": 1,
+ "id": "head",
+ "title": "Head turning",
+ "intro": "A dot that stays put in the room while you turn your head. Keep your eyes on it.",
+ "steps": [
+  {
+   "id": "turn-sides",
+   "kind": "world",
+   "group": "head",
+   "text": "Keep looking at the dot. Turn your head slowly left and right.",
+   "point": [
+    0,
+    0,
+    -1.5
+   ],
+   "seconds": 15
+  },
+  {
+   "id": "turn-updown",
+   "kind": "world",
+   "group": "head",
+   "text": "Keep looking at the dot. Nod slowly up and down.",
+   "point": [
+    0,
+    0,
+    -1.5
+   ],
+   "seconds": 12
+  },
+  {
+   "id": "tilt",
+   "kind": "world",
+   "group": "head",
+   "text": "Keep looking at the dot. Tilt your head side to side.",
+   "point": [
+    0,
+    0,
+    -1.5
+   ],
+   "seconds": 10
+  },
+  {
+   "id": "shake",
+   "kind": "world",
+   "group": "head",
+   "text": "Keep looking at the dot. Shake your head a little, quickly.",
+   "point": [
+    0,
+    0,
+    -1.5
+   ],
+   "seconds": 8
+  },
+  {
+   "id": "walk",
+   "kind": "world",
+   "group": "head",
+   "text": "If you have room: take a step to each side while looking at the dot.",
+   "point": [
+    0,
+    0,
+    -2.0
+   ],
+   "seconds": 12
+  }
+ ]
+}

gaze/rec/rounds/jumps.json

@@ -0,0 +1,78 @@
+{
+ "schema": 1,
+ "id": "jumps",
+ "title": "Jumps and pursuit",
+ "intro": "A dot that jumps around, then one that glides. Follow it with your eyes, as quickly as you can, without moving your head.",
+ "steps": [
+  {
+   "id": "jumps",
+   "kind": "jumps",
+   "text": "Look at the dot wherever it jumps.",
+   "count": 40,
+   "range": [
+    24,
+    18
+   ],
+   "each": [
+    0.5,
+    1.5
+   ]
+  },
+  {
+   "id": "circle-slow",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot with your eyes.",
+   "path": "circle 0 0 15 6",
+   "seconds": 12
+  },
+  {
+   "id": "circle-fast",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot with your eyes. It's faster now.",
+   "path": "circle 0 0 15 2.5",
+   "seconds": 10
+  },
+  {
+   "id": "line-slow",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot from side to side.",
+   "path": "line -24 0 24 0 3",
+   "seconds": 12
+  },
+  {
+   "id": "line-fast",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot from side to side, faster.",
+   "path": "line -24 0 24 0 1.2",
+   "seconds": 8
+  },
+  {
+   "id": "line-up",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot up and down.",
+   "path": "line 0 -18 0 18 2.5",
+   "seconds": 10
+  },
+  {
+   "id": "eight",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot along the figure eight.",
+   "path": "eight 0 0 22 12 6",
+   "seconds": 12
+  },
+  {
+   "id": "eight-fast",
+   "kind": "path",
+   "group": "pursuit",
+   "text": "Follow the dot along the figure eight, faster.",
+   "path": "eight 0 0 22 12 3",
+   "seconds": 9
+  }
+ ]
+}

gaze/rec/rounds/light.json

@@ -0,0 +1,89 @@
+{
+ "schema": 1,
+ "id": "light",
+ "title": "Light changes",
+ "intro": "The panel goes dark and bright. Your pupils change size with it: that's what this records.",
+ "steps": [
+  {
+   "id": "steps",
+   "kind": "light",
+   "text": "Keep looking at the dot while the light changes.",
+   "dot": [
+    0,
+    0
+   ],
+   "levels": [
+    [
+     0.03,
+     4
+    ],
+    [
+     0.9,
+     4
+    ],
+    [
+     0.03,
+     4
+    ],
+    [
+     0.33,
+     4
+    ],
+    [
+     0.9,
+     4
+    ],
+    [
+     0.33,
+     4
+    ],
+    [
+     0.03,
+     4
+    ]
+   ]
+  },
+  {
+   "id": "grid-dark",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    5,
+    5,
+    24,
+    18
+   ],
+   "hold": 1.0,
+   "shuffle": true,
+   "bg": 0.03
+  },
+  {
+   "id": "grid-mid",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    5,
+    5,
+    24,
+    18
+   ],
+   "hold": 1.0,
+   "shuffle": true,
+   "bg": 0.33
+  },
+  {
+   "id": "grid-bright",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    5,
+    5,
+    24,
+    18
+   ],
+   "hold": 1.0,
+   "shuffle": true,
+   "bg": 0.9
+  }
+ ]
+}

gaze/rec/rounds/reading.json

@@ -0,0 +1,31 @@
+{
+ "schema": 1,
+ "id": "reading",
+ "title": "Reading",
+ "intro": "Reading and looking around, as in everyday use.",
+ "steps": [
+  {
+   "id": "read-1",
+   "kind": "prompt",
+   "group": "reading",
+   "dot": null,
+   "seconds": 20,
+   "text": "Read this at your own pace.||The lighthouse keeper climbed the spiral stairs every evening, counting each of the one hundred and twelve steps. At the top, he polished the great lens until it shone, then lit the lamp and watched its beam sweep across the dark water. Ships far out at sea would see the flash and know where the rocks were."
+  },
+  {
+   "id": "read-2",
+   "kind": "prompt",
+   "group": "reading",
+   "dot": null,
+   "seconds": 20,
+   "text": "And this one.||In the morning the market filled with stalls of oranges, peppers and fresh bread. A boy on a bicycle wove between the crates, ringing his bell. Somewhere a radio played an old song, and an old woman hummed along while she weighed out a bag of cherries for a customer who couldn't decide."
+  },
+  {
+   "id": "free",
+   "kind": "prompt",
+   "dot": null,
+   "seconds": 15,
+   "text": "Look around the panel freely: the corners, the edges, wherever you like."
+  }
+ ]
+}

gaze/rec/rounds/slips.json

@@ -0,0 +1,509 @@
+{
+ "schema": 1,
+ "id": "slips",
+ "title": "Headset slips",
+ "intro": "You'll move the headset on your face on purpose, seven ways: up, down, to your left, to your right, tilted, tipped forward and away from your face. Each time it's one hand on the front and a few millimetres, no more; each step says how. The picture may blur a little while it's moved: that's fine. Then you take it off and put it back on.",
+ "steps": [
+  {
+   "id": "nudge-up",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Push up and beeps:|Hold the front with one hand, thumb under the bottom edge, and push the headset straight up a few millimetres, so it sits higher on your face. Keep it level: both sides go up together.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Push up",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-up",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-up",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-down",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Pull down and beeps:|Hold the front with one hand, thumb under the bottom edge, and pull the headset straight down a few millimetres, the way a loose headset sags. Keep it level: both sides go down together.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Pull down",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-down",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-down",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-left",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Slide to your left and beeps:|Hold the front with one hand, thumb under the bottom edge, and slide the headset a few millimetres to your left, across your face. Keep it level and at the same height.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Slide to your left",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-left",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-left",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-right",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Slide to your right and beeps:|Hold the front with one hand, thumb under the bottom edge, and slide the headset a few millimetres to your right, across your face. Keep it level and at the same height.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Slide to your right",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-right",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-right",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-tilt",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Tilt: left side up and beeps:|Hold the front with one hand, thumb under the bottom edge, and turn your wrist so the headset's left side goes up a few millimetres and its right side down, so it sits crooked on your face. Its middle stays where it is.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Tilt: left side up",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-tilt",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-tilt",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-tip",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Tip the front down and beeps:|Hold the front with one hand, thumb under the bottom edge, and tip the front down a little, the way a heavy headset droops: its bottom edge goes down a few millimetres and its top stays against your forehead. Keep it level side to side.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Tip the front down",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-tip",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-tip",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "nudge-away",
+   "kind": "prompt",
+   "group": "nudges",
+   "text": "Look at the dot. When it says Pull away from your face and beeps:|Hold the front with one hand, thumb under the bottom edge, and pull the headset a few millimetres straight out, away from your face, so there's a small gap at your cheeks. Gently: the strap pulls back. Keep it level.|Only the headset moves: your head stays still and your eyes stay on the dot. Hold it there until it says Let go and beeps low.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Look at the dot",
+     1.5
+    ],
+    [
+     "Pull away from your face",
+     2.5,
+     "go"
+    ]
+   ]
+  },
+  {
+   "id": "after-away",
+   "kind": "dots",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding the headset there. Look at each dot until its ring fills. Keep your head still.",
+   "dots": [
+    [
+     0,
+     0
+    ],
+    [
+     15,
+     12
+    ],
+    [
+     -15,
+     -12
+    ],
+    [
+     15,
+     -12
+    ],
+    [
+     -15,
+     12
+    ]
+   ],
+   "hold": 1.0
+  },
+  {
+   "id": "back-away",
+   "kind": "prompt",
+   "group": "nudges",
+   "ready": false,
+   "text": "Keep holding it until it says Let go. Then put the headset back as it was, and look at the dot.",
+   "dot": [
+    0,
+    0
+   ],
+   "cues": [
+    [
+     "Let go, put it back",
+     3,
+     "done"
+    ]
+   ]
+  },
+  {
+   "id": "reseat",
+   "kind": "reseat",
+   "text": "Take the headset off, put it back on as you usually do, then press the button."
+  },
+  {
+   "id": "after-reseat",
+   "kind": "dots",
+   "text": "Look at each dot until its ring fills. Keep your head still.",
+   "grid": [
+    3,
+    3,
+    20,
+    20
+   ],
+   "hold": 1.2,
+   "shuffle": true
+  }
+ ]
+}

gaze/rec/sounds.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+"""The eye recorder's sound cues: plain beeps for steps done with the eyes shut, where the panel's
+words can't be seen, and for moving the headset by hand, where the timing matters. A round script names one as a cue's third field (session.py: "cues":
+[[word, seconds, sound]...]); it plays as the cue starts, and prompts.jsonl's "cue" line says which.
+
+  blink   a short high beep: blink once
+  close   a beep: close your eyes
+  open    the same beep again: open them
+  go      a higher beep: move the headset now (the cue word says how)
+  done    a low, longer one: let go of it
+
+No voice, no sound files: the beeps are made here (a sine with a short fade, so it doesn't click)
+into the run folder, and played with PipeWire's pw-play on the default output (the headset's
+speakers). A beep starts some tens of ms after its cue line (pw-play's start and its 20 ms
+latency); a person's reaction to it takes several times that.
+
+  sounds.py              play each one, a second apart
+  sounds.py NAME...      play those
+"""
+import math
+import os
+import shutil
+import struct
+import subprocess
+import sys
+import time
+import wave
+
+RATE = 48000
+VOLUME = 0.6        # pw-play's stream volume; the beep itself is at half scale
+# name: (pitch Hz, seconds)
+SOUNDS = {
+    "blink": (1000.0, 0.09),
+    "close": (660.0, 0.18),
+    "open": (660.0, 0.18),
+    "go": (880.0, 0.14),
+    "done": (440.0, 0.3),
+}
+FADE_S = 0.005
+
+
+def _dir():
+    d = os.path.join(os.environ.get("XDG_RUNTIME_DIR") or "/run/user/%d" % os.getuid(), "frametop", "eyerec-sounds")
+    os.makedirs(d, exist_ok=True)
+    return d
+
+
+def wav_path(name):
+    """The sound's WAV file, made the first time (and again when its pitch or length changed)."""
+    hz, secs = SOUNDS[name]
+    path = os.path.join(_dir(), "%s-%d-%d.wav" % (name, hz, secs * 1000))
+    if not os.path.exists(path):
+        n, fade = int(RATE * secs), int(RATE * FADE_S)
+        frames = bytearray()
+        for i in range(n):
+            env = min(1.0, i / fade, (n - 1 - i) / fade)
+            frames += struct.pack("<h", int(0.5 * 32767 * env * math.sin(2 * math.pi * hz * i / RATE)))
+        with wave.open(path + ".tmp", "wb") as w:
+            w.setnchannels(1)
+            w.setsampwidth(2)
+            w.setframerate(RATE)
+            w.writeframes(bytes(frames))
+        os.replace(path + ".tmp", path)
+    return path
+
+
+class Player:
+    """Plays sounds without waiting for them. Off (dry runs, tests): it only says what it would play.
+    Without pw-play, or when a sound fails, it logs that once and the round goes on silent."""
+
+    def __init__(self, on=True, log=None):
+        self.log = log or (lambda text: None)
+        self.pw_play = shutil.which("pw-play") if on else None
+        self.procs = []
+        self.warned = False
+        if on and not self.pw_play:
+            self._warn("no pw-play: the sound cues are off")
+        if self.pw_play:
+            for name in SOUNDS:     # made now, not while a step runs
+                wav_path(name)
+
+    def _warn(self, text):
+        if not self.warned:
+            self.warned = True
+            self.log(text)
+
+    def play(self, name):
+        if name not in SOUNDS:
+            raise ValueError("no sound %r" % name)
+        if not self.pw_play:
+            return False
+        self.procs = [p for p in self.procs if p.poll() is None]
+        env = dict(os.environ, XDG_RUNTIME_DIR=os.environ.get("XDG_RUNTIME_DIR") or "/run/user/%d" % os.getuid())
+        try:
+            self.procs.append(subprocess.Popen(
+                [self.pw_play, "--latency", "20ms", "--volume", str(VOLUME), "--media-role", "Notification",
+                 wav_path(name)], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                env=env))
+            return True
+        except OSError as e:
+            self._warn("can't play the sound cues: %s" % e)
+            return False
+
+    def close(self):
+        for p in self.procs:
+            try:
+                p.wait(timeout=1)
+            except subprocess.TimeoutExpired:
+                p.kill()
+        self.procs = []
+
+
+def main(argv):
+    names = argv or list(SOUNDS)
+    bad = [n for n in names if n not in SOUNDS]
+    if bad:
+        sys.exit("no sound %s; there are: %s" % (", ".join(bad), ", ".join(SOUNDS)))
+    p = Player(log=lambda text: print(text, file=sys.stderr))
+    for i, name in enumerate(names):
+        if i:
+            time.sleep(1.0)
+        print(name)
+        p.play(name)
+    p.close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

gaze/rec/takes.py

@@ -0,0 +1,472 @@
+#!/usr/bin/env python3
+"""The eye recorder's rounds and takes on disk: listing, review, deleting, and the encrypted export.
+
+The layout is DESIGN.md's "Files":
+  BASE/profile.json
+  BASE/rounds/<YYYYMMDD-HHMMSS>-<bounty>/session.json, fit.json, health.json, session.log
+  BASE/rounds/<round>/takes/<NN>-<group>/cam0/*.zst, cam1/*.zst, index.tsv (eyerec.py),
+      targets.jsonl, poses.jsonl, ipd.jsonl, own.jsonl, steamvr.jsonl, prompts.jsonl, take.json
+  BASE/exports/<round>/
+
+Deleted ranges live in take.json ("deleted": [[from_ns, to_ns], ...]) on CLOCK_MONOTONIC, the
+clock of prompts.jsonl, which the window's timeline shows. The frames and the other logs are on
+CLOCK_MONOTONIC_RAW seconds; take.json's clock samples ([mono_ns, raw - mono ns]) turn one into
+the other. The files keep every frame until export leaves the deleted ones out.
+
+The export (exports/<round>/, built in <round>.partial and moved when it's whole):
+  manifest.json                 the round, its profile and session, the recipients, each take
+  fit.json                      the fit coach's readings, as recorded
+  health.json.age               the opt-in health answers, encrypted (when there are any)
+  takes/<id>/take.json          as recorded, plus "export": the deleted ranges in RAW seconds
+  takes/<id>/index.tsv          the kept frames; a chunk keeps its number, frames renumbered in it
+  takes/<id>/cam0/NNNNNN.zst.age ...   each chunk encrypted to every recipient (recorder/age.py)
+  takes/<id>/targets.jsonl, ipd.jsonl, prompts.jsonl   whole: they say what was shown and asked
+  takes/<id>/own.jsonl, steamvr.jsonl, poses.jsonl     rows in deleted ranges left out
+  SHA256SUMS                    over every file above, the encrypted ones as they are
+A chunk none of whose frames are deleted is encrypted as it is; one with some deleted frames is
+decompressed, the rest compressed again (zstd 1, as eyerec.py) and encrypted; one with none left
+is dropped. Every chunk is decompressed once to check it holds the frames index.tsv says; a
+chunk that doesn't, or one index.tsv doesn't name (the recorder was killed before the index
+line), is left out and counted in the manifest. session.log isn't exported (it names paths).
+
+Needs the cryptography library to export (recorder/age.py). The rest is standard library.
+
+usage: takes.py [--base DIR] list | takes ROUND | export ROUND
+       takes.py unpack EXPORT --identity KEYFILE --out DIR    decrypt an export (the dataset's side)
+"""
+import argparse
+import datetime
+import json
+import os
+import re
+import shutil
+import sys
+import threading
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+REPO = os.path.dirname(os.path.dirname(HERE))
+sys.path.insert(0, REPO)
+sys.path.insert(0, HERE)
+import eyerec  # noqa: E402  (gaze/rec/eyerec.py: the chunks' format)
+from recorder import age  # noqa: E402
+from recorder.common import tree_version, write_json  # noqa: E402
+from recorder.files import (Cancelled, human_bytes, in_ranges, merge_ranges, read_json, sha256_file,  # noqa: E402,F401
+                            tree_bytes, tree_stat, write_sums)
+
+DEFAULT_BASE = os.path.expanduser("~/.local/share/frametop/eyes/contrib")
+RECIPIENTS = os.path.join(HERE, "recipients.txt")
+ROUND_RE = re.compile(r"^\d{8}-\d{6}-[a-z0-9][a-z0-9-]{0,39}$")   # session.py: <time>-<bounty>[-N]
+TAKE_RE = re.compile(r"^\d{2,}-[a-z0-9][a-z0-9_-]{0,63}$")
+CHUNK_RE = re.compile(r"^(\d{6})\.zst$")
+EXPORT_SCHEMA = 1
+WHOLE_LOGS = ("targets.jsonl", "ipd.jsonl", "prompts.jsonl")
+CUT_LOGS = ("own.jsonl", "steamvr.jsonl", "poses.jsonl")   # "t" in RAW seconds
+GAP_S = 0.5            # frames further apart than this are a pause, not recorded time
+
+
+def tool_version():
+    return "ft-eyerec " + (tree_version(REPO) or "unknown")
+
+
+# ---------------------------------------------------------------- clocks
+
+def raw_offset_ns(t_ns, clock):
+    """RAW - MONOTONIC at t_ns (MONOTONIC), from take.json's clock samples: linear between the
+    two around it, the nearest outside them, 0 without any."""
+    pts = sorted((int(m), int(d)) for m, d in clock or [] if isinstance(m, (int, float)) and isinstance(d, (int, float)))
+    if not pts:
+        return 0
+    if t_ns <= pts[0][0]:
+        return pts[0][1]
+    for (m0, d0), (m1, d1) in zip(pts, pts[1:]):
+        if t_ns <= m1:
+            return d0 + (d1 - d0) * (t_ns - m0) // max(1, m1 - m0)
+    return pts[-1][1]
+
+
+def raw_ranges(ranges, clock):
+    """Deleted ranges (MONOTONIC ns) as [[from, to], ...] in RAW seconds."""
+    return [[(a + raw_offset_ns(a, clock)) / 1e9, (b + raw_offset_ns(b, clock)) / 1e9] for a, b in merge_ranges(ranges)]
+
+
+def recorded_seconds(times):
+    """Time spent recording: the gaps under GAP_S between sorted frame times."""
+    total = 0.0
+    for a, b in zip(times, times[1:]):
+        if b - a < GAP_S:
+            total += b - a
+    return total
+
+
+def cut_jsonl(src, dst, ranges):
+    """Copy a jsonl log, leaving out rows whose "t" (RAW seconds) is in ranges. Lines that don't
+    parse are copied as they are (validate.py reports them)."""
+    kept = cut = 0
+    with open(src, encoding="utf-8", errors="replace") as f, open(dst, "w", encoding="utf-8", newline="\n") as out:
+        for line in f:
+            if not line.strip():
+                continue
+            try:
+                t = json.loads(line).get("t")
+            except (ValueError, AttributeError):
+                t = None
+            if isinstance(t, (int, float)) and in_ranges(t, ranges):
+                cut += 1
+                continue
+            out.write(line if line.endswith("\n") else line + "\n")
+            kept += 1
+    return kept, cut
+
+
+# ---------------------------------------------------------------- the store
+
+class Store:
+    def __init__(self, base=DEFAULT_BASE):
+        self.base = os.path.abspath(os.path.expanduser(base))
+        self.rounds_dir = os.path.join(self.base, "rounds")
+        self.exports_dir = os.path.join(self.base, "exports")
+        self.profile_path = os.path.join(self.base, "profile.json")
+
+    # --- paths, checked: ids come from the window, so nothing may climb out of the base
+    def round_dir(self, rnd):
+        if not ROUND_RE.match(rnd or ""):
+            raise ValueError(f"not a round id: {rnd!r}")
+        return os.path.join(self.rounds_dir, rnd)
+
+    def take_dir(self, rnd, take):
+        if not TAKE_RE.match(take or ""):
+            raise ValueError(f"not a take id: {take!r}")
+        return os.path.join(self.round_dir(rnd), "takes", take)
+
+    def export_dir(self, rnd):
+        self.round_dir(rnd)  # checks the id
+        return os.path.join(self.exports_dir, rnd)
+
+    def profile(self):
+        return read_json(self.profile_path)
+
+    # --- listing
+    def rounds(self):
+        """Newest first: [{id, bounty, title, started, status, dry_run, takes, bytes, exported,
+        export_stale (changed since), export_bytes}]."""
+        out = []
+        try:
+            names = os.listdir(self.rounds_dir)
+        except OSError:
+            return out
+        for rid in names:
+            path = os.path.join(self.rounds_dir, rid)
+            if not ROUND_RE.match(rid) or not os.path.isdir(path):
+                continue
+            meta = read_json(os.path.join(path, "session.json"))
+            size, changed = tree_stat(path)
+            manifest = os.path.join(self.export_dir(rid), "manifest.json")
+            exported = os.path.getmtime(manifest) if os.path.isfile(manifest) else 0.0
+            out.append({"id": rid, "bounty": meta.get("bounty", ""), "title": meta.get("title", ""),
+                        "started": meta.get("started", ""), "status": meta.get("status", ""),
+                        "dry_run": bool(meta.get("dry_run")), "takes": len(self.take_ids(rid)), "bytes": size,
+                        "exported": bool(exported), "export_stale": bool(exported) and changed > exported,
+                        "export_bytes": tree_bytes(self.export_dir(rid)) if exported else 0})
+        return sorted(out, key=lambda r: r["id"], reverse=True)
+
+    def take_ids(self, rnd):
+        d = os.path.join(self.round_dir(rnd), "takes")
+        try:
+            names = os.listdir(d)
+        except OSError:
+            return []
+        return sorted(n for n in names if TAKE_RE.match(n) and os.path.isdir(os.path.join(d, n)))
+
+    def takes(self, rnd):
+        """[{id, group, status, frames: [right, left], deleted_frames: [..], ranges, duration_s,
+        bytes, started_ns, ended_ns}]."""
+        out = []
+        for tid in self.take_ids(rnd):
+            path = self.take_dir(rnd, tid)
+            meta = read_json(os.path.join(path, "take.json"))
+            ranges = merge_ranges(meta.get("deleted") or [])
+            raw = raw_ranges(ranges, meta.get("clock"))
+            times = {c: [] for c in eyerec.CAMERAS}
+            for c, _chunk, _i, _slot, t in eyerec.read_index(path):
+                if c in times:
+                    times[c].append(t)
+            for c in times:
+                times[c].sort()
+            out.append({"id": tid, "group": meta.get("group", tid.split("-", 1)[-1]),
+                        "status": meta.get("status", "recording"),
+                        "frames": [len(times[c]) for c in eyerec.CAMERAS],
+                        "deleted_frames": [sum(1 for t in times[c] if in_ranges(t, raw)) for c in eyerec.CAMERAS],
+                        "ranges": ranges, "duration_s": max(recorded_seconds(times[c]) for c in eyerec.CAMERAS),
+                        "bytes": sum(tree_bytes(os.path.join(path, "cam%d" % c)) for c in eyerec.CAMERAS),
+                        "started_ns": meta.get("started_ns"), "ended_ns": meta.get("ended_ns")})
+        return out
+
+    def take_meta(self, rnd, take):
+        return read_json(os.path.join(self.take_dir(rnd, take), "take.json"))
+
+    # --- deleted ranges
+    def ranges(self, rnd, take):
+        return merge_ranges(self.take_meta(rnd, take).get("deleted") or [])
+
+    def _set_ranges(self, rnd, take, ranges):
+        path = os.path.join(self.take_dir(rnd, take), "take.json")
+        meta = read_json(path)
+        if not meta:
+            raise RuntimeError(f"no take {rnd}/{take}")
+        meta["deleted"] = merge_ranges(ranges)
+        write_json(path, meta)
+        return meta["deleted"]
+
+    def delete_range(self, rnd, take, from_ns, to_ns):
+        """Leave the frames from from_ns to to_ns (CLOCK_MONOTONIC, inclusive) out of the export."""
+        return self._set_ranges(rnd, take, self.ranges(rnd, take) + [[int(from_ns), int(to_ns)]])
+
+    def restore_range(self, rnd, take, i):
+        """Take deleted range i (in ranges()' order) back."""
+        ranges = self.ranges(rnd, take)
+        if 0 <= i < len(ranges):
+            del ranges[i]
+        return self._set_ranges(rnd, take, ranges)
+
+    # --- deleting files
+    def delete_take(self, rnd, take):
+        path = self.take_dir(rnd, take)
+        if os.path.isdir(path):
+            shutil.rmtree(path)
+        meta_path = os.path.join(self.round_dir(rnd), "session.json")
+        meta = read_json(meta_path)
+        if take in (meta.get("takes") or []):
+            meta["takes"] = [t for t in meta["takes"] if t != take]
+            meta.setdefault("deleted_takes", []).append(take)
+            write_json(meta_path, meta)
+
+    def delete_round(self, rnd):
+        """The round and its export, if any."""
+        for path in (self.round_dir(rnd), self.export_dir(rnd)):
+            if os.path.isdir(path):
+                shutil.rmtree(path)
+
+    def delete_export(self, rnd):
+        path = self.export_dir(rnd)
+        if os.path.isdir(path):
+            shutil.rmtree(path)
+
+    # --- export
+    def export(self, rnd, progress=None, cancel=None, keep_notes=False, low_priority=True, recipients=None):
+        """Write exports/<round>/ (the module's docstring). progress(fraction 0..1, text) is
+        called as it goes; cancel() (or cancel.is_set()) returning true stops it, raising
+        Cancelled. Nothing is left behind on failure or cancel. Returns the export's path.
+        recipients: "age1..." keys, default recipients.txt's. low_priority: run this thread at
+        nice 19 (Linux nice is per thread)."""
+        recipients = list(recipients) if recipients else age.read_recipients(RECIPIENTS)
+        age._crypto()   # fails now, not half way, without the cryptography library
+        cancelled = getattr(cancel, "is_set", cancel) or (lambda: False)
+        report = progress or (lambda fraction, text: None)
+        src = self.round_dir(rnd)
+        if not os.path.isdir(src):
+            raise RuntimeError(f"no round {rnd}")
+        if low_priority:
+            try:
+                os.setpriority(os.PRIO_PROCESS, threading.get_native_id(), 19)
+            except OSError:
+                pass
+        final = self.export_dir(rnd)
+        work = final + ".partial"
+        if os.path.isdir(work):
+            shutil.rmtree(work)
+        os.makedirs(work)
+        try:
+            self._export(rnd, src, work, recipients, report, cancelled, keep_notes)
+            if os.path.isdir(final):
+                shutil.rmtree(final)
+            os.replace(work, final)
+        except BaseException:
+            shutil.rmtree(work, ignore_errors=True)
+            raise
+        report(1.0, "Done")
+        return final
+
+    def _export(self, rnd, src, work, recipients, report, cancelled, keep_notes):
+        profile = self.profile()
+        meta = read_json(os.path.join(src, "session.json"))
+        meta.pop("uploads", None)   # earlier uploads' records (hub.py) aren't part of the contribution
+        takes = self.takes(rnd)
+        total = sum(t["bytes"] for t in takes) or 1
+        done = 0
+        entries = []
+        for t in takes:
+            if cancelled():
+                raise Cancelled()
+            tdir = self.take_dir(rnd, t["id"])
+            out = os.path.join(work, "takes", t["id"])
+            os.makedirs(out)
+            take_meta = read_json(os.path.join(tdir, "take.json"))
+            raw = raw_ranges(t["ranges"], take_meta.get("clock"))
+            logs = {}
+            for name in WHOLE_LOGS:
+                if os.path.isfile(os.path.join(tdir, name)):
+                    shutil.copyfile(os.path.join(tdir, name), os.path.join(out, name))
+            for name in CUT_LOGS:
+                if os.path.isfile(os.path.join(tdir, name)):
+                    kept, cut = cut_jsonl(os.path.join(tdir, name), os.path.join(out, name), raw)
+                    logs[name] = {"rows": kept, "deleted": cut}
+
+            def step(nbytes, title=t["id"]):
+                nonlocal done
+                done += nbytes
+                report(min(done / total, 0.99), f"Encrypting {title}")
+            chunks = self._chunks(tdir, out, raw, recipients, step, cancelled)
+            if take_meta:
+                take_meta["export"] = {"deleted_raw_s": raw}
+                write_json(os.path.join(out, "take.json"), take_meta)
+            entries.append({"id": t["id"], "group": t["group"], "status": t["status"],
+                            "frames": chunks["frames"], "frames_deleted": chunks["deleted"],
+                            "duration_s": round(t["duration_s"], 2), "chunks": chunks["chunks"],
+                            "raw_bytes": chunks["raw_bytes"], "left_out": chunks["left_out"], "logs": logs})
+        if cancelled():
+            raise Cancelled()
+        if os.path.isfile(os.path.join(src, "fit.json")):
+            shutil.copyfile(os.path.join(src, "fit.json"), os.path.join(work, "fit.json"))
+        health = os.path.join(src, "health.json")
+        if os.path.isfile(health):
+            age.encrypt_file(health, os.path.join(work, "health.json.age"), recipients)
+        shown = json.loads(json.dumps(profile))
+        shown.pop("health", None)   # only ever encrypted, in health.json.age
+        if not keep_notes and isinstance(shown.get("optional"), dict):
+            shown["optional"].pop("notes", None)
+        manifest = {"schema": EXPORT_SCHEMA, "tool": tool_version(),
+                    "exported": datetime.datetime.now().astimezone().isoformat(timespec="seconds"),
+                    "round_id": rnd, "bounty": meta.get("bounty", ""),
+                    "contributor": profile.get("contributor", ""),
+                    "consent_version": (profile.get("consent") or {}).get("version", ""),
+                    "encryption": {"format": "age-encryption.org/v1", "recipients": recipients,
+                                   "files": ["takes/*/cam*/*.zst.age"] + (["health.json.age"] if os.path.isfile(health) else [])},
+                    "frame": {"width": eyerec.W, "height": eyerec.H, "format": "u8 grey",
+                              "cameras": {"0": "right eye", "1": "left eye"}},
+                    "profile": shown, "session": meta, "takes": entries}
+        write_json(os.path.join(work, "manifest.json"), manifest)
+        report(0.995, "Writing checksums")
+        write_sums(work, cancelled)
+
+    @staticmethod
+    def _chunks(tdir, out, ranges, recipients, step, cancelled):
+        """Each camera's chunks into out/camN/*.zst.age and the kept frames into out/index.tsv."""
+        rows = {}
+        for c, chunk, i, slot, t in eyerec.read_index(tdir):
+            rows.setdefault((c, chunk), []).append((i, slot, t))
+        on_disk = set()
+        for c in eyerec.CAMERAS:
+            try:
+                names = os.listdir(os.path.join(tdir, "cam%d" % c))
+            except OSError:
+                continue
+            on_disk.update((c, int(m.group(1))) for m in map(CHUNK_RE.match, names) if m)
+        res = {"frames": [0] * len(eyerec.CAMERAS), "deleted": [0] * len(eyerec.CAMERAS), "chunks": 0,
+               "raw_bytes": 0, "left_out": {"not_indexed": 0, "missing": 0, "bad": 0}}
+        res["left_out"]["not_indexed"] = len(on_disk - set(rows))
+        lines = [eyerec.INDEX_HEADER]
+        for c, chunk in sorted(rows):
+            if cancelled():
+                raise Cancelled()
+            frames = sorted(rows[(c, chunk)])
+            if c not in eyerec.CAMERAS:
+                continue
+            if (c, chunk) not in on_disk:
+                res["left_out"]["missing"] += 1
+                continue
+            path = os.path.join(tdir, "cam%d" % c, "%06d.zst" % chunk)
+            with open(path, "rb") as f:
+                packed = f.read()
+            step(len(packed))
+            try:
+                data = eyerec.decompress(packed)
+            except Exception:
+                data = b""
+            if len(data) != len(frames) * eyerec.FRAME or [i for i, _, _ in frames] != list(range(len(frames))):
+                res["left_out"]["bad"] += 1
+                continue
+            kept = [(i, slot, t) for i, slot, t in frames if not in_ranges(t, ranges)]
+            res["deleted"][c] += len(frames) - len(kept)
+            if not kept:
+                continue
+            if len(kept) < len(frames):
+                packed = eyerec.compress(b"".join(data[i * eyerec.FRAME:(i + 1) * eyerec.FRAME] for i, _, _ in kept))
+            os.makedirs(os.path.join(out, "cam%d" % c), exist_ok=True)
+            with open(os.path.join(out, "cam%d" % c, "%06d.zst.age" % chunk), "wb") as f:
+                f.write(age.encrypt_bytes(packed, recipients))
+            lines.extend("%d\t%d\t%d\t%d\t%.6f\n" % (c, chunk, k, slot, t) for k, (_, slot, t) in enumerate(kept))
+            res["frames"][c] += len(kept)
+            res["chunks"] += 1
+            res["raw_bytes"] += len(kept) * eyerec.FRAME
+        with open(os.path.join(out, "index.tsv"), "w") as f:
+            f.writelines(lines)
+        return res
+
+
+# ---------------------------------------------------------------- the dataset's side
+
+def read_identity(path):
+    with open(path) as f:
+        for line in f:
+            if line.strip().startswith("AGE-SECRET-KEY-1"):
+                return line.strip()
+    raise ValueError(f"{path} has no age identity")
+
+
+def unpack(export, identity_path, dest, progress=None):
+    """Check an export's SHA256SUMS, then copy it to dest with every .age file decrypted (its
+    name without .age). The output is a take folder eyerec.py can read. Returns dest."""
+    identity = read_identity(identity_path)
+    report = progress or (lambda fraction, text: None)
+    sums = {}
+    with open(os.path.join(export, "SHA256SUMS")) as f:
+        for line in f:
+            digest, rel = line.rstrip("\n").split("  ", 1)
+            if rel.startswith("/") or ".." in rel.split("/"):
+                raise ValueError(f"SHA256SUMS names {rel!r}")
+            sums[rel] = digest
+    for k, (rel, digest) in enumerate(sorted(sums.items())):
+        src = os.path.join(export, rel)
+        if sha256_file(src) != digest:
+            raise ValueError(f"{rel}: checksum doesn't match")
+        out = os.path.join(dest, rel[:-4] if rel.endswith(".age") else rel)
+        os.makedirs(os.path.dirname(out), exist_ok=True)
+        if rel.endswith(".age"):
+            age.decrypt_file(src, out, identity)
+        else:
+            shutil.copyfile(src, out)
+        report((k + 1) / len(sums), rel)
+    return dest
+
+
+def main():
+    ap = argparse.ArgumentParser(description="List, export and unpack eye recorder rounds.")
+    ap.add_argument("--base", default=DEFAULT_BASE)
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    sub.add_parser("list")
+    sub.add_parser("takes").add_argument("round")
+    sub.add_parser("export").add_argument("round")
+    up = sub.add_parser("unpack", help="check and decrypt an export (needs one of the dataset's keys)")
+    up.add_argument("export")
+    up.add_argument("--identity", required=True, help="an age identity file")
+    up.add_argument("--out", required=True)
+    a = ap.parse_args()
+    store = Store(a.base)
+    if a.cmd == "list":
+        for r in store.rounds():
+            print(f"{r['id']}  {r['status'] or '-'}  {r['takes']} takes  {human_bytes(r['bytes'])}"
+                  + ("  dry run" if r["dry_run"] else "") + ("  exported" if r["exported"] else ""))
+    elif a.cmd == "takes":
+        for t in store.takes(a.round):
+            print(f"{t['id']}  {t['status']}  frames {t['frames']} ({t['deleted_frames']} deleted)  "
+                  f"{t['duration_s']:.1f} s  {human_bytes(t['bytes'])}")
+    elif a.cmd == "unpack":
+        print(unpack(a.export, a.identity, a.out))
+    else:
+        path = store.export(a.round, progress=lambda f, text: print(f"\r{f * 100:5.1f}% {text:40.40}", end="",
+                                                                     flush=True))
+        print(f"\n{path}  {human_bytes(tree_bytes(path))}")
+
+
+if __name__ == "__main__":
+    main()

gaze/rec/tests/test_eyerec.py

@@ -0,0 +1,253 @@
+#!/usr/bin/env python3
+"""The eye recorder's frame writer (gaze/rec/eyerec.py) against a made-up share, laid out as
+ft-eyegrab's (gaze/tracker/eyes_share.py). No eye images: each frame is one byte value
+repeated, from its camera and number, so the round trip can be checked.
+
+  python3 gaze/rec/tests/test_eyerec.py
+"""
+import mmap
+import os
+import shutil
+import struct
+import sys
+import tempfile
+import threading
+import time
+import unittest
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.dirname(HERE))
+import eyerec  # noqa: E402
+
+W, H, SLOTS = eyerec.W, eyerec.H, 8
+ESIZE = 64 + W * H
+
+
+def frame_bytes(cam, n):
+    return bytes([(cam * 128 + n) % 256]) * (W * H)
+
+
+class FakeShare:
+    """Writes frames into a share file at fps per camera, on a thread, as ft-eyegrab --share does."""
+
+    def __init__(self, path, fps=90.0):
+        self.path, self.fps = path, fps
+        self.count = [0, 0]
+        self._stop = threading.Event()
+        self._make()
+        self._thread = threading.Thread(target=self._run, daemon=True)
+
+    def _make(self):
+        size = 64 + 2 * SLOTS * ESIZE
+        tmp = self.path + ".new"
+        fd = os.open(tmp, os.O_RDWR | os.O_CREAT | os.O_TRUNC, 0o600)
+        os.ftruncate(fd, size)
+        self.mm = mmap.mmap(fd, size)
+        os.close(fd)
+        struct.pack_into("<6I2QI", self.mm, 0, 0x31434546, 1, W, H, SLOTS, ESIZE, 0, 0, os.getpid())
+        os.replace(tmp, self.path)
+        self.count = [0, 0]
+
+    def replace(self):
+        """As ft-eyegrab restarting: a new file, the counts from 0."""
+        self._make()
+
+    def start(self):
+        self._thread.start()
+        return self
+
+    def stop(self):
+        self._stop.set()
+        self._thread.join(2)
+
+    def put(self, cam):
+        n = self.count[cam]
+        off = 64 + (cam * SLOTS + n % SLOTS) * ESIZE
+        seq = struct.unpack_from("<Q", self.mm, off)[0]
+        struct.pack_into("<Q", self.mm, off, seq + 1)
+        self.mm[off + 64:off + 64 + W * H] = frame_bytes(cam, n)
+        struct.pack_into("<dQII", self.mm, off + 8, time.clock_gettime(time.CLOCK_MONOTONIC_RAW), n, cam, n % 4 + 4 * cam)
+        struct.pack_into("<Q", self.mm, off, seq + 2)
+        self.count[cam] = n + 1
+        struct.pack_into("<Q", self.mm, 24 + 8 * cam, n + 1)
+
+    def _run(self):
+        period = 1.0 / self.fps
+        nxt = time.monotonic()
+        while not self._stop.is_set():
+            for c in (0, 1):
+                self.put(c)
+            nxt += period
+            time.sleep(max(0.0, nxt - time.monotonic()))
+
+
+class WriterTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-test-")
+        self.share_path = os.path.join(self.tmp, "cams")
+        self.want = os.path.join(self.tmp, "want")
+        self.take = os.path.join(self.tmp, "take")
+        self.lines = []
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def writer(self, **kw):
+        kw.setdefault("min_free", 0)
+        return eyerec.Writer(self.take, share_path=self.share_path, want_path=self.want, log=self.lines.append, **kw)
+
+    def wait_frames(self, w, n, timeout=10):
+        end = time.monotonic() + timeout
+        while time.monotonic() < end:
+            with w._lock:
+                if all(len(w._times[c]) and w._last[c] is not None for c in (0, 1)) and \
+                        sum(len(w._open[c]) for c in (0, 1)) + sum(w._frames.values()) >= n:
+                    return
+            time.sleep(0.02)
+        self.fail("not enough frames")
+
+    def test_round_trip_with_a_pause(self):
+        share = FakeShare(self.share_path).start()
+        try:
+            w = self.writer(chunk_frames=20).start()
+            time.sleep(0.2)                 # frames before recording aren't kept
+            w.record(True)
+            time.sleep(0.6)
+            w.record(False)
+            time.sleep(0.3)                 # nor while paused
+            w.record(True)
+            time.sleep(0.4)
+            stats = w.stop()
+        finally:
+            share.stop()
+        self.assertIsNone(stats["error"])
+        self.assertEqual(stats["dropped"], [0, 0])
+        self.assertGreater(min(stats["frames"]), 50)
+        self.assertGreater(min(stats["fps"]), 60)
+        rows = eyerec.read_index(self.take)
+        self.assertEqual(len(rows), sum(stats["frames"]))
+        got = list(eyerec.iter_frames(self.take))
+        for c in (0, 1):
+            ts = [t for cam, t, _s, _d in got if cam == c]
+            gaps = [b - a for a, b in zip(ts, ts[1:])]
+            self.assertGreater(max(gaps), 0.25, "the pause leaves a gap")
+            self.assertEqual(sorted(ts), ts)
+            # Each frame is what the share had: a byte value from its camera and number, so
+            # consecutive frames of a camera go up by one (mod 256) outside the gap.
+            vals = [d[0] for cam, _t, _s, d in got if cam == c]
+            steps = {(b - a) % 256 for a, b, g in zip(vals, vals[1:], gaps) if g < 0.05}
+            self.assertEqual(steps, {1})
+            self.assertTrue(all(d == bytes([d[0]]) * len(d) for cam, _t, _s, d in got if cam == c))
+        # The pause ended the chunks: none spans it.
+        for c in (0, 1):
+            chunks = {}
+            for cam, chunk, i, _slot, t in rows:
+                if cam == c:
+                    chunks.setdefault(chunk, []).append(t)
+            self.assertTrue(all(max(ts) - min(ts) < 0.25 for ts in chunks.values()))
+        self.assertTrue(os.path.exists(self.want), "ft-eyegrab's want file is touched")
+        # It compresses: a repeated byte is tiny.
+        self.assertLess(stats["bytes"], sum(stats["frames"]) * W * H / 100)
+
+    def test_a_slow_writer_drops_chunks_not_frames_in_the_share(self):
+        share = FakeShare(self.share_path).start()
+        real = eyerec.compress
+
+        def slow(b):
+            time.sleep(0.3)
+            return real(b)
+        try:
+            with mock.patch.object(eyerec, "compress", slow), mock.patch.object(eyerec, "MAX_QUEUED", 1):
+                w = self.writer(chunk_frames=5).start()
+                w.record(True)
+                time.sleep(1.0)
+                stats = w.stop()
+        finally:
+            share.stop()
+        self.assertGreater(sum(stats["dropped"]), 0)
+        self.assertTrue(any("behind" in line for line in self.lines))
+        rows = eyerec.read_index(self.take)
+        self.assertEqual(len(rows), sum(stats["frames"]))
+        for c, chunk, _i, _s, _t in rows:   # every chunk the index names is there
+            self.assertTrue(os.path.exists(os.path.join(self.take, "cam%d" % c, "%06d.zst" % chunk)))
+
+    def test_follows_a_replaced_share(self):
+        share = FakeShare(self.share_path).start()
+        try:
+            w = self.writer(chunk_frames=10).start()
+            w.record(True)
+            time.sleep(0.4)
+            share.replace()                 # ft-eyegrab restarted with SteamVR
+            time.sleep(2.8)                 # the reader looks every 2 s
+            before = w.stats()["frames"]
+            time.sleep(0.5)
+            stats = w.stop()
+        finally:
+            share.stop()
+        self.assertIsNone(stats["error"])
+        self.assertTrue(any("replaced" in line for line in self.lines))
+        self.assertGreater(min(stats["frames"]), min(before))
+
+    def test_no_share(self):
+        with mock.patch.object(eyerec, "SHARE_WAIT", 0.2):
+            w = self.writer().start()
+            w.record(True)
+            time.sleep(0.6)
+            stats = w.stop()
+        self.assertIn("frametop-eyegrab", stats["error"])
+        self.assertFalse(w.recording)
+
+    def test_stops_before_the_disk_fills(self):
+        share = FakeShare(self.share_path).start()
+        try:
+            w = self.writer(min_free=1e18).start()
+            w.record(True)
+            time.sleep(2.5)
+            self.assertFalse(w.recording)
+            stats = w.stop()
+        finally:
+            share.stop()
+        self.assertIn("free on the disk", stats["error"])
+        w.record(True)
+        self.assertFalse(w.recording, "it doesn't start again after an error")
+
+    def test_numbering_goes_on_in_a_take_already_there(self):
+        share = FakeShare(self.share_path).start()
+        try:
+            for _ in range(2):
+                w = self.writer(chunk_frames=10).start()
+                w.record(True)
+                time.sleep(0.3)
+                w.stop()
+        finally:
+            share.stop()
+        rows = eyerec.read_index(self.take)
+        chunks = [chunk for c, chunk, _i, _s, _t in rows if c == 0]
+        self.assertEqual(len(set(chunks)), len(os.listdir(os.path.join(self.take, "cam0"))))
+        with open(os.path.join(self.take, "index.tsv")) as f:
+            self.assertEqual(sum(1 for line in f if line.startswith("#")), 1)
+
+
+@unittest.skipUnless(shutil.which("zstd") or os.path.exists("/usr/bin/zstd"), "no zstd command")
+class CommandFallbackTest(unittest.TestCase):
+    """The zstd command, as on the SteamOS host's Python 3.12."""
+
+    def test_round_trip(self):
+        real_import = __import__
+
+        def no_modules(name, *a, **kw):
+            if name in ("compression", "zstandard"):
+                raise ImportError(name)
+            return real_import(name, *a, **kw)
+        with mock.patch("builtins.__import__", no_modules):
+            c, d, name = eyerec._compressor()
+        self.assertTrue(name.endswith("zstd"))
+        data = frame_bytes(0, 3) + frame_bytes(1, 4)
+        self.assertEqual(d(c(data)), data)
+        if eyerec.decompress is not None:   # the module's and the command's are the same format
+            self.assertEqual(eyerec.decompress(c(data)), data)
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_hub.py

@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+"""The eye recorder's upload (gaze/rec/hub.py) without the network: the dry run, the gate that
+keeps the real dataset closed until the texts are reviewed, upload records, and the pull request
+flow against a fake Hub. The round is test_takes.make_round's random frames.
+
+  python3 gaze/rec/tests/test_hub.py
+"""
+import json
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+import uuid
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+sys.path.insert(0, os.path.join(HERE, ".."))
+import hub  # noqa: E402
+import takes  # noqa: E402
+from test_takes import HAVE, RID, make_round  # noqa: E402
+
+
+class Pr:
+    def __init__(self, num, status="draft"):
+        self.num, self.status, self.is_pull_request = num, status, True
+        self.url = f"https://huggingface.co/datasets/a/b/discussions/{num}"
+
+
+@unittest.skipUnless(HAVE, "needs zstd and the cryptography library (the dev container)")
+class HubTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-hub-")
+        base = os.path.join(self.tmp, "contrib")
+        make_round(base)
+        self.contributor = str(uuid.uuid4())
+        with open(os.path.join(base, "profile.json"), "w") as f:
+            json.dump({"contributor": self.contributor,
+                       "consent": {"version": "test-1", "adult": True, "region_ok": True, "accepted": "2026-10-14"}}, f)
+        self.store = takes.Store(base)
+        meta = hub.meta_path(self.store, RID)
+        with open(meta) as f:
+            m = json.load(f)
+        m.pop("uploads")   # the fixture's, for the export's test
+        with open(meta, "w") as f:
+            json.dump(m, f)
+        self.store.export(RID, low_priority=False)   # to recipients.txt's keys, as validate wants
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def test_dry_run(self):
+        logged = []
+        with mock.patch.dict(os.environ, {hub.DATASET_ENV: "someone/test-eyes"}):
+            result = hub.upload(self.store, RID, dry_run=True, log=logged.append)
+        self.assertTrue(result["dry_run"])
+        self.assertEqual(result["repo"], "someone/test-eyes")
+        self.assertEqual(result["path_in_repo"], f"contributions/{self.contributor}/{RID}")
+        self.assertIn("- Bounty: edges", result["commit_description"])
+        self.assertIn("- Health answers: included, encrypted", result["commit_description"])
+        self.assertTrue(any("health.json.age" in line for line in logged))
+        self.assertEqual(hub.rhub.uploads(hub.meta_path(self.store, RID)), [])
+
+    def test_gate(self):
+        self.assertTrue(hub.texts_ready(), "the texts are reviewed (2026-10-16)")
+        with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "", hub.DATASET_ENV: ""}):
+            self.assertTrue(hub.upload_allowed(hub.dataset_id()))
+        # While a text is a draft: never the real dataset, a rehearsal only with ALLOW_ENV.
+        with mock.patch.object(hub, "texts_ready", return_value=False):
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: ""}):
+                self.assertFalse(hub.upload_allowed(hub.dataset_id()), "never the real dataset before the review")
+                with self.assertRaises(hub.HubError) as cm:
+                    hub.upload(self.store, RID)
+                self.assertEqual(cm.exception.kind, "closed")
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: "a/test"}):
+                self.assertTrue(hub.upload_allowed(hub.dataset_id()))
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "", hub.DATASET_ENV: "a/test"}):
+                self.assertFalse(hub.upload_allowed(hub.dataset_id()))
+
+    def test_draft_consent_blocks(self):
+        # An export made under a draft consent text never goes to the real dataset.
+        prof = os.path.join(self.store.base, "profile.json")
+        with open(prof) as f:
+            p = json.load(f)
+        p["consent"]["version"] = "2026-10-14-draft"
+        with open(prof, "w") as f:
+            json.dump(p, f)
+        shutil.rmtree(self.store.export_dir(RID))
+        self.store.export(RID, low_priority=False)
+        with mock.patch.dict(os.environ, {hub.DATASET_ENV: ""}):
+            with self.assertRaises(hub.HubError) as cm:
+                hub.upload(self.store, RID, dry_run=True)
+        self.assertEqual(cm.exception.kind, "draft-consent")
+
+    def test_invalid_blocks(self):
+        with open(os.path.join(self.store.export_dir(RID), "health.json"), "w") as f:
+            f.write("{}")
+        with self.assertRaises(hub.HubError) as cm:
+            hub.upload(self.store, RID, dry_run=True)
+        self.assertEqual(cm.exception.kind, "invalid")
+
+    def test_pull_request_first(self):
+        calls = []
+
+        class Api:
+            fail = True
+
+            def whoami(self):
+                return {"name": "someone", "auth": {"accessToken": {"role": "write"}}}
+
+            def auth_check(self, repo, repo_type=None):
+                pass
+
+            def create_pull_request(self, repo, title, description=None, repo_type=None):
+                calls.append("create")
+                return Pr(3)
+
+            def get_discussion_details(self, repo, num, repo_type=None):
+                return Pr(num)
+
+            def upload_folder(self, **kw):
+                calls.append(("upload", kw["revision"], kw["path_in_repo"]))
+                if Api.fail:
+                    raise ConnectionResetError("reset")
+
+            def change_discussion_status(self, repo, num, status, repo_type=None):
+                calls.append(("status", num, status))
+
+        env = {hub.DATASET_ENV: "a/b", hub.ALLOW_ENV: "1"}
+        meta = hub.meta_path(self.store, RID)
+        before = os.stat(meta).st_mtime_ns
+        with mock.patch.dict(os.environ, env), mock.patch.object(hub, "_hf", return_value=mock.Mock(HfApi=Api)):
+            with self.assertRaises(hub.HubError) as cm:
+                hub.upload(self.store, RID)
+            self.assertEqual(cm.exception.kind, "network")
+            [rec] = hub.rhub.uploads(meta)
+            self.assertEqual((rec["status"], rec["pr_num"]), ("started", 3))
+            Api.fail = False
+            result = hub.upload(self.store, RID)
+            with self.assertRaises(hub.HubError) as cm:
+                hub.upload(self.store, RID, dry_run=True)
+            self.assertEqual(cm.exception.kind, "duplicate")
+        where = f"contributions/{self.contributor}/{RID}"
+        self.assertEqual(calls, ["create", ("upload", "refs/pr/3", where), ("upload", "refs/pr/3", where),
+                                 ("status", 3, "open")])
+        self.assertEqual(result["pr_num"], 3)
+        self.assertEqual(os.stat(meta).st_mtime_ns, before, "the record doesn't make the export look stale")
+        self.assertFalse(self.store.rounds()[0]["export_stale"])
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_panel.py

@@ -0,0 +1,173 @@
+#!/usr/bin/env python3
+"""The gaze panel's rec mode (gaze/panel/ft-gazepanel.cpp, "show rec") without SteamVR (--no-vr,
+a made-up 90 Hz vsync): its commands, and targets.jsonl, the ground truth of where the dot was
+when. Needs the built panel (gaze/build.sh); skipped without it.
+
+  python3 gaze/rec/tests/test_panel.py
+"""
+import json
+import math
+import os
+import shutil
+import socket
+import subprocess
+import tempfile
+import time
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+PANEL = os.path.join(HERE, "..", "..", "build", "ft-gazepanel")
+IN_BOX = os.path.expanduser("~/.local/bin/distrobox")
+
+
+def command():
+    """The panel's argv: in the dev container on the Frame's host, where it was built."""
+    if os.path.exists("/run/.containerenv") or not os.path.exists(IN_BOX):
+        return [PANEL]
+    return [IN_BOX, "enter", "dev", "--", PANEL]
+
+
+@unittest.skipUnless(os.path.exists(PANEL), "the panel isn't built (gaze/build.sh)")
+class RecPanelTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="gazepanel-test-")
+        self.name = f"ft_gazepanel_test_{os.getpid()}"
+        self.proc = subprocess.Popen(command() + ["--no-vr", "--watch-stdin", "--socket", self.name, "--dump",
+                                                  os.path.join(self.tmp, "pics")],
+                                     stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        self.sock.bind("")
+        self.sock.settimeout(1.0)
+        end = time.monotonic() + 20
+        while True:
+            try:
+                if self.ask("ping").startswith("ok"):
+                    break
+            except OSError:
+                pass
+            if time.monotonic() > end:
+                self.fail("the panel didn't start")
+            time.sleep(0.1)
+
+    def tearDown(self):
+        self.proc.stdin.close()
+        try:
+            self.proc.wait(5)
+        except subprocess.TimeoutExpired:
+            self.proc.kill()
+        self.proc.stderr.close()
+        self.sock.close()
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def ask(self, cmd):
+        self.sock.sendto(cmd.encode(), "\0" + self.name)
+        return self.sock.recv(4096).decode()
+
+    def rows(self):
+        with open(os.path.join(self.tmp, "targets.jsonl")) as f:
+            return [json.loads(line) for line in f]
+
+    def test_rec_lines_and_errors(self):
+        for cmd in ("show rec", "bg 0.3", "title Nine dots", "step Round 1 of 5", "rec on", "text Look at each dot|as it comes",
+                    "big 3", "action Ready? Press the side button", "note Your left eye is hard to see",
+                    "keys Space: go · P: pause", "countdown 0.4", "paused on", "paused off", "countdown off"):
+            self.assertEqual(self.ask(cmd), "ok", cmd)
+        self.assertTrue(self.ask("path spiral 0 0 1 2").startswith("error"))
+        self.assertTrue(self.ask("path circle 0 0 10 0").startswith("error"), "seconds must be positive")
+        self.assertTrue(self.ask("log start /nonexistent/dir").startswith("error"))
+        self.assertEqual(self.ask("ipd"), "ok 0.06300")
+        time.sleep(0.2)
+        self.assertTrue(os.listdir(os.path.join(self.tmp, "pics")), "pictures were drawn")
+        self.ask("show quick")
+        self.assertTrue(self.ask("path circle 0 0 10 2").startswith("error"), "only in rec")
+
+    def test_fixed_dot_logs_its_changes(self):
+        self.ask("show rec")
+        self.assertEqual(self.ask(f"log start {self.tmp}"), "ok")
+        self.ask("dot 10 -5 look")
+        time.sleep(0.15)
+        self.ask("dot 10 -5 capture 0.5")
+        time.sleep(0.15)
+        self.ask("bg 0.8")
+        time.sleep(0.15)
+        self.ask("dot 0 0 off")
+        time.sleep(0.15)
+        reply = self.ask("log stop").split()
+        rows = self.rows()
+        self.assertEqual(int(reply[1]), len(rows))
+        on = [r for r in rows if r["dot"] == "fixed"]
+        self.assertEqual([(r["yaw"], r["pitch"], r["state"]) for r in on],
+                         [(10, -5, "look"), (10, -5, "capture"), (10, -5, "capture")])
+        self.assertEqual(on[-1]["bg"], 0.8)
+        self.assertEqual(rows[-1]["dot"], "off")
+        for r in rows:
+            self.assertGreater(r["t"], r["set"], "the photons come after it's set")
+            self.assertLess(r["t"] - r["set"], 0.03)
+        # The RAW clock, as the eye frames' (this process's CLOCK_MONOTONIC_RAW).
+        self.assertLess(abs(rows[-1]["set"] - time.clock_gettime(time.CLOCK_MONOTONIC_RAW)), 5)
+
+    def test_path_every_frame(self):
+        self.ask("show rec")
+        self.ask(f"log start {self.tmp}")
+        self.assertEqual(self.ask("path circle 2 1 10 1.0"), "ok")
+        time.sleep(1.2)
+        self.ask("dot 0 0 off")
+        self.ask("log stop")
+        rows = [r for r in self.rows() if r["dot"] == "path"]
+        ts = [r["t"] for r in rows]
+        self.assertGreater(len(rows), 80, "about one a frame")
+        gaps = [b - a for a, b in zip(ts, ts[1:])]
+        self.assertTrue(all(g > 0 for g in gaps))
+        self.assertAlmostEqual(sorted(gaps)[len(gaps) // 2], 1 / 90, delta=0.002)
+        self.assertEqual(len({r["frame"] for r in rows}), len(rows), "one row a frame")
+        for r in rows:
+            self.assertAlmostEqual(math.hypot(r["yaw"] - 2, r["pitch"] - 1), 10, delta=1e-3)
+        first = rows[0]
+        self.assertAlmostEqual((first["yaw"], first["pitch"])[1], 11, delta=0.5, msg="it starts at the top")
+        # A lap a second: back near the top a second later, having gone left (yaw up) first.
+        later = min(rows, key=lambda r: abs(r["t"] - first["t"] - 1.0))
+        self.assertAlmostEqual(later["pitch"], 11, delta=0.5)
+        self.assertGreater(rows[5]["yaw"], 2)
+
+    def test_line_and_eight(self):
+        self.ask("show rec")
+        self.ask(f"log start {self.tmp}")
+        self.ask("path line -10 0 10 0 0.5")
+        time.sleep(0.8)
+        self.ask("path eight 0 0 15 8 1")
+        time.sleep(0.6)
+        self.ask("log stop")
+        rows = self.rows()
+        line = [r for r in rows if r["dot"] == "path" and abs(r["pitch"]) < 1e-6 and abs(r["yaw"]) <= 10 + 1e-6]
+        self.assertGreater(len(line), 60)
+        self.assertGreater(max(r["yaw"] for r in line), 9)
+        self.assertLess(min(r["yaw"] for r in line), -9.5)
+        eight = [r for r in rows if r["t"] > line[-1]["t"]]
+        self.assertTrue(all(abs(r["yaw"]) <= 15 + 1e-6 and abs(r["pitch"]) <= 8 + 1e-6 for r in eight if r["dot"] == "path"))
+
+    def test_world_dot_and_distance(self):
+        self.ask("show rec")
+        self.ask(f"log start {self.tmp}")
+        reply = self.ask("world 0.5 0 -1")
+        self.assertTrue(reply.startswith("ok"), reply)
+        self.assertEqual([round(float(v), 3) for v in reply.split()[1:]], [0.5, 0.0, -1.0])
+        time.sleep(0.2)
+        again = self.ask("world 0.5 0 -1 capture 0.5")
+        self.assertEqual(again, reply, "the same point again: only the state changes")
+        time.sleep(0.1)
+        self.ask("dot 0 0 look")
+        self.assertEqual(self.ask("distance 0.5"), "ok")
+        time.sleep(0.1)
+        self.ask("log stop")
+        rows = self.rows()
+        world = [r for r in rows if r["dot"] == "world"]
+        self.assertGreater(len(world), 10)
+        # --no-vr's head is at the origin looking down -z: half a metre right of a metre ahead.
+        self.assertAlmostEqual(world[0]["yaw"], -math.degrees(math.atan2(0.5, 1)), delta=0.01)
+        self.assertAlmostEqual(world[0]["dist"], math.hypot(0.5, 1), delta=0.001)
+        self.assertEqual(rows[-1]["dist"], 0.5)
+        self.assertEqual(rows[-1]["dot"], "fixed")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_qml_backend.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+"""main.qml against ft_eyerec.Backend: every backend.name(...) the window calls is a slot, and
+every backend.name it reads is a property or a slot. A method that lost its @Slot shows up in
+QML only as "is not a function" when its button is pressed (the hand recorder's Export, 2026-10-03).
+Needs PySide6 (the dev container); skipped without it. FT_EYEREC_QML=PATH checks another
+window against the same backend (the standalone Eye Recorder's, in its CI).
+
+  python3 gaze/rec/tests/test_qml_backend.py
+"""
+import os
+import re
+import sys
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+REC = os.path.dirname(HERE)
+sys.path.insert(0, REC)
+QML = os.environ.get("FT_EYEREC_QML") or os.path.join(REC, "main.qml")
+
+
+class QmlBackendTest(unittest.TestCase):
+    def setUp(self):
+        try:
+            import ft_eyerec
+        except ImportError as e:
+            self.skipTest(f"no PySide6: {e}")
+        meta = self.meta = ft_eyerec.Backend.staticMetaObject
+        self.slots = {bytes(meta.method(i).name()).decode() for i in range(meta.methodCount())}
+        self.props = {meta.property(i).name() for i in range(meta.propertyCount())}
+        with open(QML) as f:
+            self.qml = f.read()
+
+    def test_calls_are_slots(self):
+        called = set(re.findall(r"\bbackend\.(\w+)\s*\(", self.qml))
+        self.assertTrue(called)
+        self.assertEqual(sorted(called - self.slots), [], "called from main.qml but not a slot")
+
+    def test_call_arity(self):
+        """Each backend.name(a, b) call passes as many arguments as some slot of that name takes
+        (a decorator left at the old count fails only when the button is pressed)."""
+        arity = {}
+        meta = self.meta
+        for i in range(meta.methodCount()):
+            mm = meta.method(i)
+            arity.setdefault(bytes(mm.name()).decode(), set()).add(mm.parameterCount())
+        bad = []
+        for m in re.finditer(r"\bbackend\.(\w+)\s*\(", self.qml):
+            depth, args, k, seen = 1, 0, m.end(), False
+            while depth and k < len(self.qml):
+                c = self.qml[k]
+                if c in "([{":
+                    depth += 1
+                elif c in ")]}":
+                    depth -= 1
+                elif c == "," and depth == 1:
+                    args += 1
+                elif not c.isspace() and depth >= 1:
+                    seen = True
+                k += 1
+            n = args + 1 if seen else 0
+            if m.group(1) in arity and n not in arity[m.group(1)]:
+                bad.append("%s: %d arguments, slots take %s" % (m.group(1), n, sorted(arity[m.group(1)])))
+        self.assertEqual(bad, [])
+
+    def test_reads_exist(self):
+        read = set(re.findall(r"\bbackend\.(\w+)\b(?!\s*\()", self.qml))
+        self.assertTrue(read)
+        self.assertEqual(sorted(read - self.props - self.slots), [], "read in main.qml but not on Backend")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_session.py

@@ -0,0 +1,471 @@
+#!/usr/bin/env python3
+"""The eye recorder's session runner (gaze/rec/session.py) without the headset: the round
+scripts' checks, the plans, the control sockets' answers, and whole rounds in a dry run.
+
+  python3 gaze/rec/tests/test_session.py
+"""
+import glob
+import json
+import os
+import shutil
+import socket
+import sys
+import tempfile
+import threading
+import time
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+import session  # noqa: E402
+
+
+def write_round(tmp, steps, **extra):
+    d = dict({"schema": 1, "id": "t", "title": "Test", "steps": steps}, **extra)
+    path = os.path.join(tmp, "t.json")
+    with open(path, "w") as f:
+        json.dump(d, f)
+    return path
+
+
+class RoundTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-round-")
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def bad(self, steps, want):
+        with self.assertRaises(ValueError) as e:
+            session.load_round(write_round(self.tmp, steps))
+        self.assertIn(want, str(e.exception))
+
+    def test_shipped_rounds_load(self):
+        paths = sorted(glob.glob(os.path.join(session.ROUNDS_DIR, "*.json")))
+        self.assertEqual(len(paths), 10)
+        for p in paths:
+            r = session.load_round(p)
+            self.assertEqual(r["id"], os.path.basename(p)[:-5])
+            self.assertTrue(r["steps"][0]["ready"], p)
+
+    def test_panel_reach(self):
+        self.assertFalse(session.off_panel(0, 0))
+        self.assertFalse(session.off_panel(30, 20))
+        self.assertTrue(session.off_panel(38, 0))
+        self.assertTrue(session.off_panel(0, 26))
+        self.assertTrue(session.off_panel(30, 23), "corners reach less far up")
+        self.bad([{"id": "a", "kind": "dots", "dots": [[40, 0]]}], "off the panel")
+        self.bad([{"id": "a", "kind": "path", "path": "circle 0 0 26 4", "seconds": 4}], "off the panel")
+        self.bad([{"id": "a", "kind": "jumps", "count": 3, "range": [40, 10], "each": [0.5, 1]}], "off the panel")
+
+    def test_bad_steps(self):
+        self.bad([{"id": "a", "kind": "dance"}], "unknown kind")
+        self.bad([{"id": "A b", "kind": "dots", "dots": [[0, 0]]}], "bad id")
+        self.bad([{"id": "a", "kind": "path", "path": "spiral 0 0 1 2", "seconds": 2}], "bad path")
+        self.bad([{"id": "a", "kind": "path", "path": "circle 0 0 10 0", "seconds": 2}], "positive")
+        self.bad([{"id": "a", "kind": "prompt"}], "seconds")
+        self.bad([{"id": "a", "kind": "world", "point": [0, 0, 1], "seconds": 3}], "ahead")
+        self.bad([{"id": "a", "kind": "dots", "dots": [[0, 0]]}, {"id": "a", "kind": "dots", "dots": [[0, 0]]}], "repeat")
+        self.bad([{"id": "a", "kind": "dots", "group": "g", "dots": [[0, 0]]}, {"id": "b", "kind": "dots", "dots": [[0, 0]]},
+                  {"id": "c", "kind": "dots", "group": "g", "dots": [[0, 0]]}], "comes back")
+        self.bad([{"id": "a", "kind": "dots", "group": "dots", "dots": [[0, 0]]}], "opener")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1, "honk"]]}], "sound: blink")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1, "blink", 2]]}], "cues want")
+        # The eyes-shut rules (rounds/README.md)
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1]]}], 'needs the sound "blink"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Close both eyes", 2], ["Open", 2]]}], 'needs the sound "close"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Shut your eyes", 2, "close"]]}], 'comes a cue with the sound "open"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Close your eyes", 2, "close"], ["Open", 2]]}], '"open"')
+        # Moving the headset by hand
+        look = ["Look at the dot", 1.5]
+        self.bad([{"id": "a", "kind": "prompt", "cues": [look, ["Push up", 2]]}], 'needs the sound "go"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Push up", 2, "go"], ["Let go", 2, "done"]]}],
+                 "the eyes are still first")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [look, ["Push up", 2, "go"], ["Let go", 1, "done"]]}], "2 s or more")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [look, ["Push up", 2, "go"]]}], 'a "done" cue')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [look, ["Push up", 2, "go"]]},
+                  {"id": "b", "kind": "prompt", "cues": [["Let go", 2, "done"]]}], 'a "done" cue')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [look, ["Tilt: left side up", 2, "go"]]},
+                  {"id": "b", "kind": "dots", "ready": False, "group": "a", "dots": [[0, 0]], "text": "Look"},
+                  {"id": "c", "kind": "prompt", "ready": False, "group": "a", "text": "Keep holding",
+                   "cues": [["Let go", 2, "done"]]}], 'starts "Keep holding"')
+        session.load_round(write_round(self.tmp, [
+            {"id": "a", "kind": "prompt", "cues": [look, ["Slide to your left", 2, "go"]]},
+            {"id": "b", "kind": "dots", "ready": False, "group": "a", "dots": [[0, 0]], "text": "Keep holding it"},
+            {"id": "c", "kind": "prompt", "ready": False, "group": "a", "text": "Keep holding",
+             "cues": [["Let go, put it back", 2, "done"]]}]))
+        for word in ("Close your left eye", "Half closed", "Squint", "Open"):
+            session.load_round(write_round(self.tmp, [{"id": "a", "kind": "prompt", "cues": [[word, 1]]}]))
+
+    def test_sound_cues(self):
+        r = session.load_round(write_round(self.tmp, [{"id": "a", "kind": "prompt",
+                                                       "cues": [["Look", 1], ["Blink", 1.5, "blink"]]}]))
+        self.assertEqual(r["steps"][0]["cues"], [["Look", 1.0], ["Blink", 1.5, "blink"]])
+        self.assertEqual(session.step_seconds(r["steps"][0]), 2.5)
+
+    def test_grid_and_defaults(self):
+        r = session.load_round(write_round(self.tmp, [{"id": "a", "kind": "dots", "grid": [3, 2, 10, 5], "ready": False},
+                                                      {"id": "b", "kind": "reseat", "text": "off and on"}]))
+        a = r["steps"][0]
+        self.assertEqual(a["dots"], [[10, 5], [0, 5], [-10, 5], [10, -5], [0, -5], [-10, -5]])
+        self.assertTrue(a["ready"], "the first step always waits")
+        self.assertEqual((a["bg"], a["distance"], a["group"]), (session.MID_BG, session.PANEL_DISTANCE, "a"))
+        self.assertFalse(r["shift_check"], "a round that reseats on purpose has no shift check")
+
+    def test_plan(self):
+        rnd = session.load_round(os.path.join(session.ROUNDS_DIR, "jumps.json"))
+        a = session.build_plan(rnd, 7)
+        self.assertEqual([s["group"] for s in a[:2]], ["dots", "jumps"])
+        nine = a[0]["dots"]
+        self.assertEqual((nine[0], nine[-1], len(nine)), ([0.0, 0.0], [0.0, 0.0], 10))
+        self.assertEqual(sorted(map(tuple, nine[1:-1])), sorted(map(tuple, (d for d in session.grid(3, 3, 20, 20) if d != [0, 0]))))
+        jumps = a[1]
+        self.assertEqual(len(jumps["dots"]), 40)
+        self.assertTrue(all(0.5 <= t <= 1.5 for t in jumps["times"]))
+        self.assertTrue(all(not session.off_panel(*d) for d in jumps["dots"]))
+        self.assertEqual(a, session.build_plan(rnd, 7), "the same seed, the same round")
+        self.assertNotEqual(a[0]["dots"], session.build_plan(rnd, 8)[0]["dots"])
+        self.assertGreater(session.plan_seconds(a), 60)
+
+
+class SocketTest(unittest.TestCase):
+    """The panel and ft-eyes answer every command in order: an answer waited for is the one
+    after the answers of commands sent before it."""
+
+    def setUp(self):
+        self.name = "eyerec_socket_test_%d_%s" % (os.getpid(), self.id().rsplit(".", 1)[-1])
+        self.server = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        self.server.bind("\0" + self.name)
+        self.server.settimeout(0.2)
+        self.delay = 0.02
+        self.drop = set()
+        self.running = True
+        threading.Thread(target=self.serve, daemon=True).start()
+
+    def tearDown(self):
+        self.running = False
+        self.server.close()
+
+    def serve(self):
+        while self.running:
+            try:
+                data, addr = self.server.recvfrom(4096)
+            except OSError:
+                continue
+            text = data.decode()
+            time.sleep(self.delay)
+            if text in self.drop:
+                continue
+            try:
+                self.server.sendto(("ok " + text).encode(), addr)
+            except OSError:
+                pass
+
+    def test_answers_in_order(self):
+        s = session.Socket(self.name)
+        for n in range(5):
+            s.cmd("dot %d" % n)
+        self.assertEqual(s.cmd("ipd", reply=True), "ok ipd")
+        self.assertEqual(s.pending, 0)
+        s.cmd("bg 1")
+        self.assertEqual(s.cmd("log stop", reply=True), "ok log stop")
+        s.close()
+
+    def test_lost_answer(self):
+        s = session.Socket(self.name)
+        self.drop.add("fit")
+        self.assertIsNone(s.cmd("fit", reply=True, timeout=0.2))
+        self.assertEqual(s.cmd("ipd", reply=True), "ok ipd", "counting starts afresh")
+        s.close()
+
+    def test_errors_logged(self):
+        logs = []
+        s = session.Socket(self.name, log=logs.append)
+        s.cmd("x")
+        time.sleep(0.1)
+        s._got("error bad")
+        self.assertEqual(logs, ["%s: error bad" % self.name])
+        s.close()
+
+
+class DryRunTest(unittest.TestCase):
+    """Whole rounds with no processes: the files, the fitment, the controls."""
+
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-dry-")
+        self.out = []
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def run_round(self, script, steer=None, profile=None, **kw):
+        path = os.path.join(self.tmp, "profile.json")
+        with open(path, "w") as f:
+            json.dump(profile or {}, f)
+        states = []
+        kw.setdefault("fit_coach", True)   # the coach's own tests; test_fitment_coach_off has it off
+        s = session.Session(self.tmp, script, profile=profile, dry_run=True, speed=kw.pop("speed", 40),
+                            next_after=kw.pop("next_after", 0.25), profile_path=path, button=False,
+                            on_status=lambda st: states.append(dict(st)), **kw)
+        s.print = self.out.append
+        s.start()
+        if steer:
+            steer(s, states)
+        s.join(60)
+        self.assertFalse(s._thread.is_alive())
+        with open(path) as f:
+            return s, states, json.load(f)
+
+    def files(self, s):
+        with open(os.path.join(s.session_dir, "session.json")) as f:
+            sj = json.load(f)
+        with open(os.path.join(s.session_dir, "fit.json")) as f:
+            fit = json.load(f)
+        return sj, fit
+
+    def prompts(self, s, take):
+        with open(os.path.join(s.session_dir, "takes", take, "prompts.jsonl")) as f:
+            return [json.loads(line) for line in f]
+
+    def test_round_and_profile(self):
+        s, states, profile = self.run_round("depth")
+        self.assertEqual(s.state, "done")
+        sj, fit = self.files(s)
+        # No IPD step: the setting is read and shown at the fitment.
+        self.assertEqual(sj["takes"], ["01-dots", "02-near", "03-mid", "04-far", "05-near-far"])
+        self.assertEqual((sj["status"], sj["fit"], sj["ipd_start"]), ("done", "good", 0.063))
+        self.assertNotIn("ipd_sweep", sj)
+        self.assertIn(session.IPD_TEXT.format(mm=63.0), "\n".join(self.out))
+        self.assertEqual(fit["fitment"]["result"], "good")
+        self.assertIsNotNone(fit["baseline"])
+        self.assertEqual(profile["bounties"]["depth"]["rounds"], 1)
+        self.assertEqual(profile["eyes_ipd"], 0.063)
+        for take in sj["takes"]:
+            with open(os.path.join(s.session_dir, "takes", take, "take.json")) as f:
+                self.assertEqual(json.load(f)["status"], "complete", take)
+        events = [e["event"] for e in self.prompts(s, "02-near")]
+        self.assertEqual(events[:4], ["take", "ready", "rec", "step"])
+        self.assertEqual(events[-3:], ["wait", "rec", "end"])
+        self.assertIn("ft_eyerecpanel: distance 0.500", self.out)
+        self.assertIn("ft_eyerecpanel: distance 5.000", self.out)
+        s2, _, profile = self.run_round("depth", profile=profile)
+        self.assertEqual(self.files(s2)[0]["takes"][0], "01-dots")
+        self.assertEqual(profile["bounties"]["depth"]["rounds"], 2)
+
+    def test_blink_beeps(self):
+        # The blinks round's beeps: each in prompts.jsonl's cue line, played as the cue starts.
+        s, _, _ = self.run_round("blinks", profile={"eyes_ipd": 0.063})
+        self.assertEqual(s.state, "done")
+        played = [line[len("sound: "):] for line in self.out if line.startswith("sound: ")]
+        self.assertEqual(played, ["blink"] * 8 + ["close", "open", "close", "open"])
+        sj, _ = self.files(s)
+        [take] = [t for t in sj["takes"] if t.endswith("blink-cue")]
+        cues = [e for e in self.prompts(s, take) if e["event"] == "cue"]
+        self.assertEqual([(e["word"], e.get("sound")) for e in cues], [("Blink", "blink")] * 8)
+        [take] = [t for t in sj["takes"] if t.endswith("closed")]
+        cues = [e.get("sound") for e in self.prompts(s, take) if e["event"] == "cue"]
+        self.assertEqual(cues, [None, "close", "open", "close", "open"])
+
+    def test_fitment_warns_then_skips(self):
+        # Next before the fit has been good for FIT_GOOD_S: a warning, then the round goes on.
+        s, states, _ = self.run_round("reading", next_after=0.01, speed=20)
+        sj, fit = self.files(s)
+        self.assertEqual((sj["fit"], fit["fitment"]["result"]), ("skipped", "skipped"))
+        self.assertTrue(any(st["note"] == session.FIT_SKIP_NOTE for st in states))
+        self.assertIsNone(fit["baseline"])
+
+    def test_fitment_coach_off(self):
+        # The default for now: no coaching or warning, the first Next goes on, and the coach's
+        # readings are still kept; no baseline, so no slip checks.
+        self.assertFalse(session.FIT_COACH)
+        s, states, _ = self.run_round("reading", next_after=0.6, speed=20, fit_coach=False)
+        sj, fit = self.files(s)
+        self.assertEqual(s.state, "done")
+        self.assertEqual((sj["fit"], fit["fitment"]["result"]), ("off", "off"))
+        self.assertTrue(fit["readings"])
+        self.assertIsNone(fit["baseline"])
+        self.assertFalse(fit["slips"])
+        self.assertFalse(any(st["note"] for st in states))
+        self.assertFalse(any(st["fit_words"] or st["fit_card"] for st in states))
+        self.assertTrue(all(st["fit_coach"] is False for st in states))
+
+    def test_skip_redo_stop(self):
+        def steer(s, states):
+            def wait_for(cond):
+                end = time.monotonic() + 30
+                while time.monotonic() < end and not cond():
+                    time.sleep(0.01)
+            # In the jumps take: redo the step while it runs, then skip the rest of the take.
+            # (Its ready first: the step's name comes one emit before the dots take's "running" goes.)
+            wait_for(lambda: s._status["step"] == "jumps" and s._status["state"] == "ready")
+            wait_for(lambda: s._status["step"] == "jumps" and s._status["state"] == "running")
+            s.redo()
+            wait_for(lambda: s._status["step"] == "jumps" and s._status["state"] == "ready")
+            wait_for(lambda: s._status["step"] == "jumps" and s._status["state"] == "running")
+            s.skip()
+            wait_for(lambda: s._status["step"] == "circle-slow" and s._status["state"] == "running")
+            s.stop(wait=0)
+        s, states, profile = self.run_round("jumps", steer=steer, speed=10, profile={"eyes_ipd": 0.063})
+        self.assertEqual(s.state, "stopped")
+        sj, _ = self.files(s)
+        self.assertEqual(sj["takes"], ["01-dots", "02-jumps", "03-pursuit"])
+        with open(os.path.join(s.session_dir, "takes", "02-jumps", "take.json")) as f:
+            self.assertEqual(json.load(f)["status"], "skipped")
+        with open(os.path.join(s.session_dir, "takes", "03-pursuit", "take.json")) as f:
+            self.assertEqual(json.load(f)["status"], "stopped")
+        redo = [e for e in self.prompts(s, "02-jumps") if e["event"] == "redo"]
+        self.assertEqual(len(redo), 1)
+        self.assertLess(redo[0]["from"], redo[0]["to"])
+        self.assertNotIn("bounties", profile, "a stopped round isn't counted")
+
+    # A round of two takes: a nudge with dots done while holding it (a chain), then one prompt.
+    TWO = [{"id": "nudge", "kind": "prompt", "group": "one", "dot": [0, 0], "seconds": 1, "text": "Push"},
+           {"id": "hold", "kind": "dots", "group": "one", "dots": [[0, 0], [5, 0]], "hold": 0.5, "ready": False,
+            "text": "Keep holding"},
+           {"id": "other", "kind": "prompt", "group": "two", "dot": [0, 0], "seconds": 1, "text": "Other"}]
+
+    def run_two(self, steer, **kw):
+        write_round(self.tmp, self.TWO)
+        return self.run_round("t", steer=steer, speed=10, next_after=0.6, rounds_dir=self.tmp, fit_coach=False,
+                              profile={"eyes_ipd": 0.063}, **kw)
+
+    @staticmethod
+    def at(s, step, state):
+        end = time.monotonic() + 30
+        while time.monotonic() < end and not (s._status["step"] == step and s._status["state"] == state):
+            time.sleep(0.005)
+        return s._status["step"] == step and s._status["state"] == state
+
+    def take_status(self, s, take):
+        with open(os.path.join(s.session_dir, "takes", take, "take.json")) as f:
+            return json.load(f)["status"]
+
+    def redos(self, s, take):
+        return [e["id"] for e in self.prompts(s, take) if e["event"] == "redo"]
+
+    def test_redo_in_the_countdown(self):
+        # R in the countdown starts this step again (it used to go back to the one before).
+        seen = {}
+        def steer(s, states):
+            self.assertTrue(self.at(s, "other", "countdown"))
+            s.redo()
+            seen["ready"] = self.at(s, "other", "ready")
+            seen["recording"] = s._recording
+        s, _, _ = self.run_two(steer)
+        self.assertEqual(s.state, "done")
+        self.assertTrue(seen["ready"])
+        self.assertFalse(seen["recording"], "the ready screen isn't recorded")
+        takes = self.files(s)[0]["takes"]
+        self.assertEqual(self.redos(s, takes[-1]), ["other"])
+        self.assertEqual(self.redos(s, takes[-2]), [])
+
+    def test_back_into_the_take_before(self):
+        # Back at a new take's first ready screen: the take before stays open until its Next,
+        # so its last step (with the nudge it follows on from) goes again in the same take.
+        seen = {}
+        def steer(s, states):
+            self.assertTrue(self.at(s, "other", "ready"))
+            seen["can_redo"] = s._status["can_redo"]
+            s.redo()
+            seen["back"] = self.at(s, "nudge", "ready")
+        s, _, _ = self.run_two(steer)
+        self.assertEqual(s.state, "done")
+        self.assertTrue(seen["can_redo"])
+        self.assertTrue(seen["back"], "back to the chain's head, not to hold")
+        takes = self.files(s)[0]["takes"]
+        self.assertEqual(len(takes), 3, takes)        # the opener's dots, one, two: no extra take
+        one = self.prompts(s, takes[1])
+        self.assertEqual(self.redos(s, takes[1]), ["nudge"])
+        self.assertEqual([e["id"] for e in one if e["event"] == "step"], ["nudge", "hold", "nudge", "hold"])
+        self.assertEqual(self.take_status(s, takes[1]), "complete")
+
+    def test_redo_a_step_that_follows_on(self):
+        # R while holding: the nudge goes again with it.
+        def steer(s, states):
+            self.assertTrue(self.at(s, "hold", "running"))
+            s.redo()
+            self.assertTrue(self.at(s, "nudge", "ready"))
+        s, _, _ = self.run_two(steer)
+        self.assertEqual(s.state, "done")
+        takes = self.files(s)[0]["takes"]
+        steps = [e["id"] for e in self.prompts(s, takes[1]) if e["event"] == "step"]
+        self.assertEqual(self.redos(s, takes[1]), ["nudge"])
+        self.assertEqual(steps, ["nudge", "hold", "nudge", "hold"])
+
+    def test_stop_or_skip_at_the_next_ready_screen(self):
+        # The take before is finished, not stopped or skipped, when the next take's ready screen
+        # is where it ends.
+        def stop(s, states):
+            self.assertTrue(self.at(s, "other", "ready"))
+            s.stop(wait=0)
+        s, _, _ = self.run_two(stop)
+        self.assertEqual(s.state, "stopped")
+        takes = self.files(s)[0]["takes"]
+        self.assertEqual(len(takes), 2)
+        self.assertEqual(self.take_status(s, takes[1]), "complete")
+
+        def skip(s, states):
+            self.assertTrue(self.at(s, "other", "ready"))
+            s.skip()
+        s, _, _ = self.run_two(skip)
+        self.assertEqual(s.state, "done")
+        takes = self.files(s)[0]["takes"]
+        self.assertEqual(len(takes), 2, "two was skipped before it started")
+        self.assertEqual(self.take_status(s, takes[1]), "complete")
+
+    def test_nothing_to_go_back_to(self):
+        # After a reseat Back is off: the headset sits differently now.
+        rnd = [{"id": "a", "kind": "prompt", "dot": [0, 0], "seconds": 1, "text": "A"},
+               {"id": "off", "kind": "reseat", "text": "off and on"},
+               {"id": "b", "kind": "prompt", "dot": [0, 0], "seconds": 1, "text": "B"}]
+        write_round(self.tmp, rnd)
+        seen = {}
+        def steer(s, states):
+            self.assertTrue(self.at(s, "b", "ready"))
+            seen["can_redo"] = s._status["can_redo"]
+        s, states, _ = self.run_round("t", steer=steer, speed=10, next_after=0.4, rounds_dir=self.tmp,
+                                      fit_coach=False, profile={"eyes_ipd": 0.063})
+        self.assertEqual(s.state, "done")
+        self.assertFalse(seen["can_redo"], "nothing goes back over a reseat")
+
+    def test_pause_stops_the_recording(self):
+        def steer(s, states):
+            end = time.monotonic() + 30
+            while time.monotonic() < end and not (s._status["step"] == "dots" and s._status["state"] == "running"):
+                time.sleep(0.01)
+            s.pause()
+            time.sleep(0.3)
+            self.assertFalse(s._recording)
+            s.resume()
+        s, _, _ = self.run_round("face", steer=steer, speed=10, profile={"eyes_ipd": 0.063})
+        self.assertEqual(s.state, "done")
+        events = [e["event"] for e in self.prompts(s, "01-dots")]
+        i = events.index("pause")
+        self.assertEqual(events[i - 1:i + 3], ["rec", "pause", "resume", "rec"])
+
+    def test_shift_check(self):
+        moved = {"on": False}
+
+        def steer(s, states):
+            # Once the fitment is done, the eyes appear 12 degrees up: the headset slipped down.
+            end = time.monotonic() + 30
+            while time.monotonic() < end and s._status["state"] in ("starting", "fitment"):
+                time.sleep(0.01)
+            ref = s.ref
+            summary = {str(cam): {"n": 90, "finds": 90, "found": 1.0,
+                                  "x": ref.eyes[cam]["centre"][0] + ref.eyes[cam]["jacobian"][0][1] * 12,
+                                  "y": ref.eyes[cam]["centre"][1] + ref.eyes[cam]["jacobian"][1][1] * 12,
+                                  "a": 28.0, "spacing": 15.0, "extra": 0.0} for cam in (0, 1)}
+            s._eyes.fake = lambda text: json.dumps(summary) if text == "fit" else "ok"
+            moved["on"] = True
+        s, states, _ = self.run_round("glasses", steer=steer, profile={"eyes_ipd": 0.063})
+        self.assertTrue(moved["on"])
+        _, fit = self.files(s)
+        self.assertGreaterEqual(len(fit["slips"]), 2, "the nine dots' middle, twice, and the grid's")
+        self.assertIn("up", fit["slips"][0]["hints"])
+        self.assertTrue(any(st["note"].startswith("The headset moved") for st in states))
+        self.assertTrue(any(e["event"] == "slip" for e in self.prompts(s, "01-dots")))
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_sounds.py

@@ -0,0 +1,53 @@
+#!/usr/bin/env python3
+"""sounds.py: the beeps' WAV files, and a player that never stops a round."""
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+import wave
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+import sounds  # noqa: E402
+
+
+class SoundsTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-sounds-")
+        self.env = mock.patch.dict(os.environ, {"XDG_RUNTIME_DIR": self.tmp})
+        self.env.start()
+
+    def tearDown(self):
+        self.env.stop()
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def test_wavs(self):
+        self.assertEqual(set(sounds.SOUNDS), {"blink", "close", "open", "go", "done"})
+        self.assertEqual(len({sounds.SOUNDS[n] for n in ("blink", "close", "go", "done")}), 4, "told apart by ear")
+        self.assertEqual(sounds.SOUNDS["close"], sounds.SOUNDS["open"], "open is the same beep again")
+        for name, (hz, secs) in sounds.SOUNDS.items():
+            path = sounds.wav_path(name)
+            self.assertTrue(path.startswith(self.tmp))
+            with wave.open(path) as w:
+                self.assertEqual((w.getnchannels(), w.getsampwidth(), w.getframerate()), (1, 2, sounds.RATE))
+                self.assertEqual(w.getnframes(), int(sounds.RATE * secs))
+                frames = w.readframes(w.getnframes())
+            first, last = int.from_bytes(frames[:2], "little", signed=True), int.from_bytes(frames[-2:], "little", signed=True)
+            self.assertEqual((first, last), (0, 0), "faded in and out: no click")
+
+    def test_player_off_and_missing(self):
+        logged = []
+        self.assertFalse(sounds.Player(on=False, log=logged.append).play("blink"))
+        self.assertEqual(logged, [])
+        with mock.patch.object(sounds.shutil, "which", return_value=None):
+            p = sounds.Player(log=logged.append)
+            self.assertFalse(p.play("close"))
+        self.assertEqual(logged, ["no pw-play: the sound cues are off"])
+        with self.assertRaises(ValueError):
+            p.play("honk")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_takes.py

@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+"""The eye recorder's store and encrypted export (gaze/rec/takes.py), on a made-up round of
+random frames (never real eye images): listing, deleted ranges across the two clocks, the export's
+chunks, logs and checksums, and unpacking it with a key. Needs zstd (Python 3.14's, as eyerec.py)
+and the cryptography library: the dev container has both.
+
+  python3 gaze/rec/tests/test_takes.py
+"""
+import json
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+import eyerec  # noqa: E402
+import takes  # noqa: E402
+from recorder import age  # noqa: E402
+
+try:
+    age._crypto()
+    HAVE = eyerec.compress is not None
+except RuntimeError:
+    HAVE = False
+
+RID = "20261014-120000-edges"
+TID = "01-dots"
+MONO0 = 1000 * 10**9          # the take starts at 1000 s MONOTONIC
+OFF = 5 * 10**9 + 123         # RAW is 5 s ahead
+FPS = 90.0
+PER_CHUNK = 10
+CHUNKS = 3
+
+
+def raw_s(mono_ns):
+    return (mono_ns + OFF) / 1e9
+
+
+def make_round(base, health=True):
+    """A round with one take: CHUNKS chunks of PER_CHUNK random frames per camera, frame k of
+    camera c at MONO0 + k / FPS (plus 1 ms for camera 1). Returns {(c, k): frame bytes}."""
+    rdir = os.path.join(base, "rounds", RID)
+    tdir = os.path.join(rdir, "takes", TID)
+    frames = {}
+    lines = [eyerec.INDEX_HEADER]
+    for c in eyerec.CAMERAS:
+        os.makedirs(os.path.join(tdir, "cam%d" % c))
+        for chunk in range(CHUNKS):
+            data = []
+            for i in range(PER_CHUNK):
+                k = chunk * PER_CHUNK + i
+                frames[(c, k)] = os.urandom(eyerec.FRAME)
+                data.append(frames[(c, k)])
+                lines.append("%d\t%d\t%d\t%d\t%.6f\n" % (c, chunk, i, k % 8, raw_s(MONO0 + int(k / FPS * 1e9)) + c * 0.001))
+            with open(os.path.join(tdir, "cam%d" % c, "%06d.zst" % chunk), "wb") as f:
+                f.write(eyerec.compress(b"".join(data)))
+    with open(os.path.join(tdir, "index.tsv"), "w") as f:
+        f.writelines(lines)
+    end = MONO0 + int(CHUNKS * PER_CHUNK / FPS * 1e9)
+    with open(os.path.join(tdir, "take.json"), "w") as f:
+        json.dump({"group": "dots", "started_ns": MONO0, "ended_ns": end, "status": "complete", "steps": [],
+                   "deleted": [], "clock": [[MONO0, OFF], [end, OFF]], "notes": ""}, f)
+    with open(os.path.join(tdir, "own.jsonl"), "w") as f:
+        for k in range(CHUNKS * PER_CHUNK):
+            f.write(json.dumps({"t": raw_s(MONO0 + int(k / FPS * 1e9)), "cam": 0, "x": 1.0}) + "\n")
+    with open(os.path.join(tdir, "prompts.jsonl"), "w") as f:
+        f.write(json.dumps({"t": MONO0, "event": "take"}) + "\n")
+    with open(os.path.join(tdir, "targets.jsonl"), "w") as f:
+        f.write(json.dumps({"t": raw_s(MONO0), "dot": "on", "yaw": 0, "pitch": 0}) + "\n")
+    with open(os.path.join(rdir, "session.json"), "w") as f:
+        json.dump({"schema": 1, "round": RID, "bounty": "edges", "status": "done", "takes": [TID],
+                   "started": "2026-10-14T12:00:00+0000", "checklist": {"glasses": "none"},
+                   "uploads": [{"pr": 1}]}, f)
+    with open(os.path.join(rdir, "fit.json"), "w") as f:
+        json.dump({"fitment": {"state": "ok"}}, f)
+    with open(os.path.join(rdir, "session.log"), "w") as f:
+        f.write("a log that names /home/someone\n")
+    if health:
+        with open(os.path.join(rdir, "health.json"), "w") as f:
+            json.dump({"conditions": ["test"]}, f)
+    with open(os.path.join(base, "profile.json"), "w") as f:
+        json.dump({"contributor": "c0ffee", "consent": {"version": "test-1"}, "health": {"x": 1},
+                   "optional": {"notes": "private"}}, f)
+    return frames
+
+
+def load(path, mode="r"):
+    with open(path, mode) as f:
+        return json.load(f) if path.endswith(".json") else f.read()
+
+
+def frame_mono(k):
+    return MONO0 + int(k / FPS * 1e9)
+
+
+@unittest.skipUnless(HAVE, "needs zstd and the cryptography library (the dev container)")
+class TakesTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-takes-")
+        self.base = os.path.join(self.tmp, "contrib")
+        self.frames = make_round(self.base)
+        self.store = takes.Store(self.base)
+        self.ident, self.rec = age.keygen()
+        self.key = os.path.join(self.tmp, "key.txt")
+        with open(self.key, "w") as f:
+            f.write("# test key\n" + self.ident + "\n")
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def test_ids(self):
+        for bad in ("../x", "20261014-120000-../x", "", "x"):
+            with self.assertRaises(ValueError):
+                self.store.round_dir(bad)
+        for bad in ("..", "01-a/b", ".x"):
+            with self.assertRaises(ValueError):
+                self.store.take_dir(RID, bad)
+        self.store.round_dir("20261014-120000-edges-2")
+
+    def test_clock(self):
+        clock = [[0, 100], [1000, 200]]
+        self.assertEqual(takes.raw_offset_ns(-5, clock), 100)
+        self.assertEqual(takes.raw_offset_ns(500, clock), 150)
+        self.assertEqual(takes.raw_offset_ns(5000, clock), 200)
+        self.assertEqual(takes.raw_offset_ns(5, []), 0)
+
+    def test_listing(self):
+        [r] = self.store.rounds()
+        self.assertEqual((r["id"], r["bounty"], r["takes"], r["exported"]), (RID, "edges", 1, False))
+        [t] = self.store.takes(RID)
+        self.assertEqual(t["frames"], [30, 30])
+        self.assertEqual(t["deleted_frames"], [0, 0])
+        self.assertAlmostEqual(t["duration_s"], 29 / FPS, places=2)
+
+    def test_ranges(self):
+        # index.tsv's times are whole microseconds: the ranges have a millisecond to spare
+        self.store.delete_range(RID, TID, frame_mono(12) - 10**6, frame_mono(25))
+        self.store.delete_range(RID, TID, frame_mono(20), frame_mono(29) + 10**6)
+        self.assertEqual(self.store.ranges(RID, TID), [[frame_mono(12) - 10**6, frame_mono(29) + 10**6]])
+        [t] = self.store.takes(RID)
+        self.assertEqual(t["deleted_frames"][0], 18)
+        self.store.restore_range(RID, TID, 0)
+        self.assertEqual(self.store.ranges(RID, TID), [])
+
+    def test_export_and_unpack(self):
+        # frames 15..29 deleted: chunk 0 whole, chunk 1 half (10..14 kept), chunk 2 gone
+        self.store.delete_range(RID, TID, frame_mono(15) - 1000, frame_mono(29) + 5 * 10**6)
+        path = self.store.export(RID, recipients=[self.rec])
+        self.assertFalse(os.path.exists(path + ".partial"))
+        m = load(os.path.join(path, "manifest.json"))
+        [e] = m["takes"]
+        self.assertEqual(e["frames_deleted"][0], 15)
+        self.assertEqual(e["frames"][0], 15)
+        self.assertEqual(e["chunks"], 4)
+        self.assertEqual(e["logs"]["own.jsonl"], {"rows": 15, "deleted": 15})
+        self.assertNotIn("uploads", m["session"])
+        self.assertNotIn("health", m["profile"])
+        self.assertNotIn("notes", m["profile"]["optional"])
+        self.assertEqual(m["encryption"]["recipients"], [self.rec])
+        files = sorted(os.path.relpath(os.path.join(r, n), path) for r, _, ns in os.walk(path) for n in ns)
+        self.assertIn("health.json.age", files)
+        self.assertNotIn("health.json", files)
+        self.assertNotIn("session.log", files)
+        self.assertFalse([f for f in files if f.endswith(".zst")], "every chunk is encrypted")
+        self.assertIn("takes/%s/cam0/000001.zst.age" % TID, files)
+        self.assertNotIn("takes/%s/cam0/000002.zst.age" % TID, files)
+        # chunk 0 untouched: its encrypted copy decrypts to the recorded file
+        src = load(os.path.join(self.base, "rounds", RID, "takes", TID, "cam0", "000000.zst"), "rb")
+        enc = load(os.path.join(path, "takes", TID, "cam0", "000000.zst.age"), "rb")
+        self.assertEqual(age.decrypt_bytes(enc, self.ident), src)
+        with open(os.path.join(path, "SHA256SUMS")) as f:
+            self.assertEqual(sorted(line.split("  ", 1)[1].strip() for line in f), [f for f in files if f != "SHA256SUMS"])
+
+        out = takes.unpack(path, self.key, os.path.join(self.tmp, "unpacked"))
+        self.assertEqual(load(os.path.join(out, "health.json")), {"conditions": ["test"]})
+        got = {}
+        tdir = os.path.join(out, "takes", TID)
+        for c, t, slot, data in eyerec.iter_frames(tdir):
+            got.setdefault(c, []).append(data)
+        for c in eyerec.CAMERAS:
+            self.assertEqual(got[c], [self.frames[(c, k)] for k in range(15)])
+        self.assertEqual(load(os.path.join(tdir, "take.json"))["export"]["deleted_raw_s"][0][0],
+                         raw_s(frame_mono(15) - 1000))
+
+        [r] = self.store.rounds()
+        self.assertTrue(r["exported"] and not r["export_stale"])
+
+    def test_default_recipients(self):
+        path = self.store.export(RID)
+        m = load(os.path.join(path, "manifest.json"))
+        self.assertEqual(m["encryption"]["recipients"], age.read_recipients(takes.RECIPIENTS))
+        with self.assertRaises(ValueError):   # the test key isn't one of them
+            takes.unpack(path, self.key, os.path.join(self.tmp, "no"))
+
+    def test_bad_chunks(self):
+        tdir = os.path.join(self.base, "rounds", RID, "takes", TID)
+        shutil.copyfile(os.path.join(tdir, "cam0", "000000.zst"), os.path.join(tdir, "cam0", "000007.zst"))
+        with open(os.path.join(tdir, "cam1", "000001.zst"), "r+b") as f:
+            f.truncate(100)
+        os.remove(os.path.join(tdir, "cam1", "000002.zst"))
+        path = self.store.export(RID, recipients=[self.rec])
+        [e] = load(os.path.join(path, "manifest.json"))["takes"]
+        self.assertEqual(e["left_out"], {"not_indexed": 1, "missing": 1, "bad": 1})
+        self.assertEqual(e["frames"], [30, 10])
+        with open(os.path.join(path, "takes", TID, "index.tsv")) as f:
+            self.assertEqual(sum(1 for line in f if line.startswith("1\t")), 10)
+
+    def test_cancel(self):
+        with self.assertRaises(takes.Cancelled):
+            self.store.export(RID, recipients=[self.rec], cancel=lambda: True)
+        self.assertFalse(os.path.exists(self.store.export_dir(RID) + ".partial"))
+        self.assertFalse(os.path.exists(self.store.export_dir(RID)))
+
+    def test_delete(self):
+        self.store.export(RID, recipients=[self.rec])
+        self.store.delete_take(RID, TID)
+        meta = load(os.path.join(self.base, "rounds", RID, "session.json"))
+        self.assertEqual((meta["takes"], meta["deleted_takes"]), ([], [TID]))
+        self.store.delete_round(RID)
+        self.assertEqual(self.store.rounds(), [])
+        self.assertFalse(os.path.exists(self.store.export_dir(RID)))
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/tests/test_validate.py

@@ -0,0 +1,150 @@
+#!/usr/bin/env python3
+"""The eye recorder's export check (gaze/rec/validate.py) on takes.py's export of a made-up
+round of random frames (test_takes.make_round): a good one, with and without a key, and broken
+ones. Needs zstd and the cryptography library: the dev container has both.
+
+  python3 gaze/rec/tests/test_validate.py
+"""
+import json
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+import uuid
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+sys.path.insert(0, os.path.join(HERE, ".."))
+import takes  # noqa: E402
+import validate  # noqa: E402
+from recorder import age  # noqa: E402
+from recorder.files import write_sums  # noqa: E402
+from test_takes import HAVE, RID, TID, make_round  # noqa: E402
+
+
+@unittest.skipUnless(HAVE, "needs zstd and the cryptography library (the dev container)")
+class ValidateTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-validate-")
+        base = os.path.join(self.tmp, "contrib")
+        make_round(base)
+        self.contributor = str(uuid.uuid4())
+        with open(os.path.join(base, "profile.json"), "w") as f:
+            json.dump({"contributor": self.contributor,
+                       "consent": {"version": "test-1", "adult": True, "region_ok": True, "accepted": "2026-10-14"},
+                       "bounties": {}}, f)
+        self.ident, self.rec = age.keygen()
+        self.keys = os.path.join(self.tmp, "recipients.txt")
+        with open(self.keys, "w") as f:
+            f.write("# test\n%s\n" % self.rec)
+        self.store = takes.Store(base)
+        self.path = self.store.export(RID, recipients=[self.rec])
+        self.take = os.path.join(self.path, "takes", TID)
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def check(self, identity=None, resum=False):
+        if resum:   # a change made on purpose, with checksums to match, so the other checks see it
+            os.remove(os.path.join(self.path, "SHA256SUMS"))
+            write_sums(self.path)
+        return validate.validate(self.path, identity=identity, recipients_path=self.keys)
+
+    def has(self, report, text):
+        self.assertTrue(any(text in e for e in report.errors), (text, report.errors))
+
+    def manifest(self, change):
+        p = os.path.join(self.path, "manifest.json")
+        with open(p) as f:
+            m = json.load(f)
+        change(m)
+        with open(p, "w") as f:
+            json.dump(m, f)
+
+    def test_good(self):
+        r = self.check()
+        self.assertEqual(r.errors, [])
+        self.assertEqual(r.summary["frames"], 60)
+        self.assertTrue(r.summary["health"])
+        r = self.check(identity=self.ident)
+        self.assertEqual(r.errors, [])
+
+    def test_shipped_keys(self):
+        r = validate.validate(self.path)   # recipients.txt's, not the test key
+        self.has(r, "other keys than the dataset's")
+
+    def test_checksum(self):
+        with open(os.path.join(self.take, "cam0", "000001.zst.age"), "r+b") as f:
+            f.seek(200)
+            b = f.read(1)
+            f.seek(200)
+            f.write(bytes([b[0] ^ 1]))
+        self.has(self.check(), "checksum mismatch")
+
+    def test_unencrypted(self):
+        shutil.copyfile(os.path.join(self.take, "index.tsv"), os.path.join(self.take, "cam0", "000009.zst"))
+        with open(os.path.join(self.path, "health.json"), "w") as f:
+            f.write("{}")
+        r = self.check(resum=True)
+        self.has(r, "cam0/000009.zst: not an allowed file (unencrypted")
+        self.has(r, "health.json: not an allowed file (unencrypted")
+
+    def test_cut_short(self):
+        # Without a key only some sizes show a cut (a last piece under a tag); the key finds any.
+        p = os.path.join(self.take, "cam1", "000000.zst.age")
+        with open(p, "rb") as f:
+            data = f.read()
+        body = data.index(b"\n", data.index(b"\n---") + 1) + 1 + 16
+        whole = (len(data) - body) // (age.CHUNK + age.TAG)
+        with open(p, "r+b") as f:
+            f.truncate(body + whole * (age.CHUNK + age.TAG) + 5)
+        self.has(self.check(resum=True), "cam1/000000.zst.age: a size no age file has")
+        with open(p, "r+b") as f:
+            f.truncate(len(data) - 65000)
+        self.assertEqual(self.check(resum=True).errors, [])
+        self.has(self.check(identity=self.ident, resum=True), "cam1/000000.zst.age: doesn't decrypt")
+
+    def test_wrong_content(self):
+        with open(os.path.join(self.take, "cam0", "000000.zst.age"), "wb") as f:
+            f.write(age.encrypt_bytes(os.urandom(1000), [self.rec]))
+        self.assertEqual(self.check(resum=True).errors, [], "without a key it can't tell")
+        self.has(self.check(identity=self.ident), "doesn't decompress")
+        other, _ = age.keygen()
+        self.has(self.check(identity=other), "doesn't decrypt")
+
+    def test_index(self):
+        p = os.path.join(self.take, "index.tsv")
+        with open(p) as f:
+            lines = f.readlines()
+        with open(p, "w") as f:
+            f.writelines(lines[:3] + lines[4:])   # camera 0, chunk 0, frame 2 gone
+        r = self.check(resum=True)
+        self.has(r, "frames aren't numbered")
+        self.has(r, "the manifest says [30, 30] frames, index.tsv has [29, 30]")
+        os.remove(os.path.join(self.take, "cam1", "000001.zst.age"))
+        self.has(self.check(resum=True), "cam1/000001.zst.age: in index.tsv but missing")
+
+    def test_manifest(self):
+        def change(m):
+            m["session"]["dry_run"] = True
+            m["session"]["replay"] = "/dev/shm/x"
+            m["profile"]["health"] = {"x": 1}
+            m["profile"]["consent"]["adult"] = False
+            m["contributor"] = m["profile"]["contributor"] = "c0ffee"
+        self.manifest(change)
+        r = self.check(resum=True)
+        for text in ("dry-run round", "from a replay", "unexpected keys: health", "18 or older", "random uuid4"):
+            self.has(r, text)
+
+    def test_two_recipients(self):
+        _, second = age.keygen()
+        with open(self.keys, "a") as f:
+            f.write(second + "\n")
+        self.has(self.check(), "other keys than the dataset's")
+        self.manifest(lambda m: m["encryption"].update(recipients=[self.rec, second]))
+        self.has(self.check(resum=True), "encrypted to 1 recipients, not 2")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/rec/validate.py

@@ -0,0 +1,467 @@
+#!/usr/bin/env python3
+"""Check an eye recorder export before it's uploaded, or when it's received (takes.py's
+exports/<round>/).
+
+The window runs it before an upload and refuses on errors; the maintainer runs it on each
+submission. The images are encrypted, so without a key it checks everything around them:
+  - SHA256SUMS: every file listed and matching, nothing missing from it;
+  - an allow-list of files: anything else (an unencrypted chunk, health.json, a symlink) is an error;
+  - every .age file is age v1 to as many X25519 recipients as the manifest names, and its size
+    is one a whole age file can have; the manifest's recipients are recipients.txt's;
+  - manifest.json: the schema, required keys and types, the consent, a random uuid4 contributor,
+    no dry run or replay, no health answers outside health.json.age;
+  - each take's index.tsv against its chunks (every chunk named is there, every one there is
+    named, frames numbered from 0, times going forward) and against the manifest's counts;
+  - every .jsonl line is a JSON object with a time "t", and take.json and fit.json parse;
+  - the total size.
+With --identity KEYFILE (one of the dataset's keys, the maintainer's side) it also decrypts each
+chunk in memory and checks it decompresses to the frames index.tsv gives it.
+
+Standard library only without --identity (zstd and the cryptography library with it). Python
+3.12 or later, Linux or Windows.
+
+usage: validate.py DIR [--json] [--identity KEYFILE]
+       exit status 0: no errors, 1: errors, 2: not an export
+"""
+import argparse
+import json
+import math
+import os
+import re
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.dirname(os.path.dirname(HERE)))
+from recorder import age  # noqa: E402
+from recorder.checks import (Cancelled, Report, check_sums, is_uuid4, listed, need,  # noqa: E402,F401
+                             read_json_file)
+
+EXPORT_SCHEMA = 1
+RECIPIENTS = os.path.join(HERE, "recipients.txt")
+TOP_FILES = {"manifest.json", "fit.json", "health.json.age", "SHA256SUMS"}
+TAKE_FILES = {"take.json", "index.tsv", "targets.jsonl", "ipd.jsonl", "prompts.jsonl", "own.jsonl",
+              "steamvr.jsonl", "poses.jsonl"}
+CAM_DIRS = ("cam0", "cam1")
+ROUND_RE = re.compile(r"^\d{8}-\d{6}-[a-z0-9][a-z0-9-]{0,39}$")
+TAKE_RE = re.compile(r"^\d{2,}-[a-z0-9][a-z0-9_-]{0,63}$")
+CHUNK_FILE_RE = re.compile(r"^(\d{6})\.zst\.age$")
+TAKE_STATUSES = ("complete", "stopped", "skipped")
+PROFILE_KEYS = {"schema", "contributor", "consent", "optional", "checklist", "bounties", "eyes_ipd"}
+INDEX_HEADER = "# camera\tchunk\tframe\tslot\tt_raw"
+FRAME = 512 * 400
+CHUNK_FRAMES_MAX = 1000   # eyerec.py writes 90 a chunk
+# One round is about 2.6 GB (2026-10-14 replay: 20 MB/s on disk); encryption adds next to nothing.
+WARN_BYTES = 6 * 1000 ** 3
+MAX_BYTES = 15 * 1000 ** 3
+
+
+# ---------------------------------------------------------------- the files
+
+def walk_files(root, report):
+    """The export's files as relative paths, and allow-list errors on the way."""
+    files = []
+    for dirpath, dirnames, filenames in os.walk(root):
+        rel_dir = os.path.relpath(dirpath, root).replace(os.sep, "/")
+        rel_dir = "" if rel_dir == "." else rel_dir
+        parts = rel_dir.split("/") if rel_dir else []
+        for d in list(dirnames):
+            rel = f"{rel_dir}/{d}" if rel_dir else d
+            ok = ((not parts and d == "takes") or (parts == ["takes"] and TAKE_RE.match(d))
+                  or (len(parts) == 2 and parts[0] == "takes" and d in CAM_DIRS))
+            if os.path.islink(os.path.join(dirpath, d)):
+                report.error(f"{rel}: a symlink, not allowed in an export")
+                dirnames.remove(d)
+            elif not ok:
+                report.error(f"{rel}/: not an allowed folder")
+                dirnames.remove(d)
+        for name in filenames:
+            rel = f"{rel_dir}/{name}" if rel_dir else name
+            full = os.path.join(dirpath, name)
+            if os.path.islink(full) or not os.path.isfile(full):
+                report.error(f"{rel}: a symlink or special file, not allowed in an export")
+                continue
+            ok = ((not parts and name in TOP_FILES) or (len(parts) == 2 and name in TAKE_FILES)
+                  or (len(parts) == 3 and CHUNK_FILE_RE.match(name)))
+            if not ok:
+                hint = " (unencrypted: export it again)" if name.endswith(".zst") or name == "health.json" else ""
+                report.error(f"{rel}: not an allowed file{hint}")
+            files.append(rel)
+    return sorted(files)
+
+
+age_shape = age.shape   # recorder/age.py: shared with the hand recorder's debug exports
+
+
+def check_jsonl(path, rel, report, cancelled):
+    """Every line a JSON object with a finite time "t" (RAW seconds, or MONOTONIC ns in prompts)."""
+    try:
+        with open(path, encoding="utf-8") as f:
+            for n, line in enumerate(f, 1):
+                if n % 50000 == 0 and cancelled():
+                    raise Cancelled()
+                if not line.strip():
+                    continue
+                try:
+                    obj = json.loads(line)
+                except ValueError as e:
+                    report.error(f"{rel} line {n}: not valid JSON: {e}")
+                    return
+                t = obj.get("t") if isinstance(obj, dict) else None
+                if isinstance(t, bool) or not isinstance(t, (int, float)) or not math.isfinite(t):
+                    report.error(f"{rel} line {n}: not an object with a time \"t\"")
+                    return
+    except UnicodeDecodeError:
+        report.error(f"{rel}: not UTF-8 text")
+    except OSError as e:
+        report.error(f"{rel}: can't read it: {e.strerror or e}")
+
+
+def read_index(path, rel, report):
+    """index.tsv as {(camera, chunk): [(frame, t)]}, or None when it doesn't parse."""
+    out = {}
+    with open(path, encoding="utf-8", errors="replace") as f:
+        for n, line in enumerate(f, 1):
+            line = line.rstrip("\n")
+            if n == 1:
+                if line != INDEX_HEADER:
+                    report.error(f"{rel}: the first line isn't the header")
+                    return None
+                continue
+            if not line:
+                continue
+            fields = line.split("\t")
+            try:
+                c, chunk, i, slot = (int(x) for x in fields[:4])
+                t = float(fields[4])
+                if len(fields) != 5 or not math.isfinite(t) or c not in (0, 1) or min(chunk, i, slot) < 0:
+                    raise ValueError
+            except (ValueError, IndexError):
+                report.error(f"{rel} line {n}: not camera, chunk, frame, slot, time")
+                return None
+            out.setdefault((c, chunk), []).append((i, t))
+    return out
+
+
+def check_take(root, tid, entry, sizes, recipients, identity, report, step, cancelled):
+    base = f"takes/{tid}/"
+    for name in sorted(TAKE_FILES - {"take.json", "index.tsv"}):
+        rel = base + name
+        if rel in sizes:
+            check_jsonl(os.path.join(root, rel), rel, report, cancelled)
+        elif name in ("targets.jsonl", "prompts.jsonl", "own.jsonl"):
+            report.warn(f"{rel} is missing")
+    if base + "take.json" in sizes:
+        tj = read_json_file(os.path.join(root, base + "take.json"), base + "take.json", report)
+        if tj is not None and not isinstance(tj, dict):
+            report.error(f"{base}take.json: not a JSON object")
+    else:
+        report.warn(f"{base}take.json is missing")
+    chunks = {}
+    for rel in sizes:
+        if rel.startswith(base) and rel.count("/") == 3:
+            m = CHUNK_FILE_RE.match(rel.rsplit("/", 1)[1])
+            if m:
+                chunks[(int(rel.split("/")[2][3:]), int(m.group(1)))] = rel
+    if base + "index.tsv" not in sizes:
+        if chunks:
+            report.error(f"{base}index.tsv is missing, but there are {len(chunks)} chunks")
+        return
+    index = read_index(os.path.join(root, base + "index.tsv"), base + "index.tsv", report)
+    if index is None:
+        return
+    frames = [0, 0]
+    unnamed = sorted(set(chunks) - set(index))
+    if unnamed:
+        report.error(f"{base}: chunks index.tsv doesn't name: {listed(chunks[k] for k in unnamed)}")
+    last = [-math.inf, -math.inf]
+    for key in sorted(index):
+        if cancelled():
+            raise Cancelled()
+        c, chunk = key
+        rows = index[key]
+        frames[c] += len(rows)
+        name = f"{base}cam{c}/{chunk:06d}.zst.age"
+        if key not in chunks:
+            report.error(f"{name}: in index.tsv but missing")
+            continue
+        if [i for i, _ in rows] != list(range(len(rows))) or len(rows) > CHUNK_FRAMES_MAX:
+            report.error(f"{base}index.tsv: chunk {c}/{chunk:06d}'s frames aren't numbered 0 to {len(rows) - 1}")
+        times = [t for _, t in rows]
+        if any(b < a for a, b in zip(times, times[1:])) or times[0] < last[c]:
+            report.error(f"{base}index.tsv: camera {c}'s times go back at chunk {chunk:06d}")
+        last[c] = times[-1]
+        problem = age_shape(os.path.join(root, name), recipients)
+        if problem:
+            report.error(f"{name}: {problem}")
+        elif identity:
+            problem = decrypted_problem(os.path.join(root, name), identity, len(rows))
+            if problem:
+                report.error(f"{name}: {problem}")
+        step(sizes[name])
+    if entry is not None and isinstance(entry.get("frames"), list) and entry["frames"] != frames:
+        report.error(f"take {tid}: the manifest says {entry['frames']} frames, index.tsv has {frames}")
+
+
+def decrypted_problem(path, identity, nframes):
+    sys.path.insert(0, HERE)
+    import eyerec   # zstd, as the recorder has it
+    try:
+        with open(path, "rb") as f:
+            data = eyerec.decompress(age.decrypt_bytes(f.read(), identity))
+    except ValueError as e:
+        return f"doesn't decrypt: {e}"
+    except Exception as e:
+        return f"doesn't decompress: {e}"
+    if len(data) != nframes * FRAME:
+        return f"{len(data)} bytes, but index.tsv gives it {nframes} frames"
+    return None
+
+
+# ---------------------------------------------------------------- the manifest
+
+def check_manifest(m, report, root, shipped):
+    """The manifest's keys and types. Returns ({take id: entry}, the recipients)."""
+    if not isinstance(m, dict):
+        report.error("manifest: not a JSON object")
+        return {}, []
+    if m.get("schema") != EXPORT_SCHEMA:
+        report.error(f"manifest: schema {m.get('schema')!r}, expected {EXPORT_SCHEMA}")
+    tool = need(m, "tool", str, "", report)
+    if tool is not None and not tool.startswith("ft-eyerec"):
+        report.warn(f"manifest: tool is {tool!r}")
+    need(m, "exported", str, "", report)
+    need(m, "bounty", str, "", report)
+    rid = need(m, "round_id", str, "", report)
+    if rid is not None and not ROUND_RE.match(rid):
+        report.error(f"manifest: round_id {rid!r} isn't a round id")
+    if rid and ROUND_RE.match(os.path.basename(root)) and os.path.basename(root) != rid:
+        report.warn(f"the folder is named {os.path.basename(root)}, the manifest's round is {rid}")
+    contributor = need(m, "contributor", str, "", report)
+    if contributor is not None and not is_uuid4(contributor):
+        report.error(f"manifest: the contributor id {contributor!r} isn't a random uuid4")
+    parent = os.path.basename(os.path.dirname(root))
+    if contributor and is_uuid4(parent) and parent != contributor:
+        report.error(f"the export sits under contributor {parent}, but its manifest says {contributor}")
+    consent = need(m, "consent_version", str, "", report)
+    if consent is not None and not consent.strip():
+        report.error("manifest: the consent version is empty")
+
+    recipients = []
+    enc = need(m, "encryption", dict, "", report)
+    if enc is not None:
+        if enc.get("format") != "age-encryption.org/v1":
+            report.error(f"manifest: encryption.format {enc.get('format')!r}")
+        r = enc.get("recipients")
+        if not isinstance(r, list) or not r or not all(isinstance(x, str) for x in r):
+            report.error("manifest: encryption.recipients isn't a list of keys")
+        else:
+            recipients = r
+            try:
+                for x in r:
+                    age.parse_recipient(x)
+            except ValueError as e:
+                report.error(f"manifest: encryption.recipients: {e}")
+            if shipped is not None and sorted(r) != sorted(shipped):
+                report.error("manifest: encrypted to other keys than the dataset's (recipients.txt)")
+
+    profile = need(m, "profile", dict, "", report)
+    if profile is not None:
+        if profile.get("contributor") != contributor:
+            report.error("manifest: profile.contributor differs from contributor")
+        pc = need(profile, "consent", dict, "profile.", report)
+        if pc is not None:
+            if pc.get("version") != consent:
+                report.error("manifest: profile.consent.version differs from consent_version")
+            if pc.get("adult") is not True:
+                report.error("manifest: the contributor didn't confirm being 18 or older")
+            if pc.get("region_ok") is not True:
+                report.error("manifest: the contributor didn't confirm not living in Illinois, Texas or Washington")
+            need(pc, "accepted", str, "profile.consent.", report)
+        optional = profile.get("optional")
+        if isinstance(optional, dict) and str(optional.get("notes") or "").strip():
+            report.warn("the profile's notes are included: check they hold nothing private")
+        extra = set(profile) - PROFILE_KEYS
+        if extra:
+            report.error(f"manifest: the profile has unexpected keys: {listed(sorted(extra))}")
+
+    session = need(m, "session", dict, "", report)
+    if session is not None:
+        for key, t in (("started", str), ("checklist", dict), ("takes", list), ("bounty", str)):
+            need(session, key, t, "session.", report)
+        if session.get("dry_run"):
+            report.error("this is a dry-run round: it has no real recordings")
+        if session.get("replay"):
+            report.error("this round was recorded from a replay, not the headset's cameras")
+        if session.get("status") not in ("done", None):
+            report.warn(f"the round's status is {session.get('status')!r}")
+        if "uploads" in session:
+            report.warn("manifest: session.uploads is included (earlier uploads' links)")
+        if "health" in session:
+            report.error("manifest: health answers outside health.json.age")
+        checklist = session.get("checklist")
+        if isinstance(checklist, dict) and str(checklist.get("notes") or "").strip():
+            report.warn("the checklist's notes are included: check they hold nothing private")
+    if "/home/" in json.dumps(m):
+        report.warn("manifest: it mentions a home folder path (a username may show)")
+
+    entries = {}
+    takes = need(m, "takes", list, "", report)
+    for i, t in enumerate(takes or []):
+        where = f"takes[{i}]."
+        if not isinstance(t, dict):
+            report.error(f"manifest: takes[{i}] isn't an object")
+            continue
+        tid = need(t, "id", str, where, report)
+        if tid is None:
+            continue
+        if not TAKE_RE.match(tid):
+            report.error(f"manifest: take id {tid!r} isn't one")
+            continue
+        if tid in entries:
+            report.error(f"manifest: take {tid} is listed twice")
+        entries[tid] = t
+        for key, typ in (("group", str), ("status", str), ("frames", list), ("frames_deleted", list),
+                         ("duration_s", (int, float)), ("chunks", int)):
+            need(t, key, typ, where, report)
+        if t.get("status") not in TAKE_STATUSES:
+            report.warn(f"manifest: take {tid} has status {t.get('status')!r}")
+        left = t.get("left_out")
+        if isinstance(left, dict) and any(left.values()):
+            report.warn(f"take {tid}: chunks the export left out: {left}")
+    return entries, recipients
+
+
+def summarize(m):
+    """What a pull request's description says about the export."""
+    if not isinstance(m, dict):
+        return {}
+    session = m.get("session") if isinstance(m.get("session"), dict) else {}
+    checklist = session.get("checklist") if isinstance(session.get("checklist"), dict) else {}
+    takes = [t for t in m.get("takes") or [] if isinstance(t, dict)]
+    seconds = sum(t.get("duration_s") or 0 for t in takes if isinstance(t.get("duration_s"), (int, float)))
+    frames = sum(sum(f for f in t.get("frames") or [] if isinstance(f, int)) for t in takes
+                 if isinstance(t.get("frames"), list))
+    enc = m.get("encryption") if isinstance(m.get("encryption"), dict) else {}
+    return {"round": m.get("round_id", ""), "bounty": m.get("bounty", ""), "contributor": m.get("contributor", ""),
+            "takes": len(takes), "frames": frames, "minutes": round(seconds / 60, 1),
+            "glasses": checklist.get("glasses", ""), "health": "health.json.age" in (enc.get("files") or []),
+            "consent_version": m.get("consent_version", ""), "tool": m.get("tool", "")}
+
+
+# ---------------------------------------------------------------- all of it
+
+def validate(root, progress=None, cancel=None, identity=None, recipients_path=RECIPIENTS):
+    """Check the export at root. progress(fraction 0..1, text) is called as it goes; cancel()
+    (or cancel.is_set()) returning true raises Cancelled. identity: an "AGE-SECRET-KEY-1..."
+    string, to check inside the chunks. recipients_path: the keys the export must be encrypted
+    to (None: don't compare). Returns a Report."""
+    root = os.path.abspath(root)
+    report = Report(root)
+    cancelled = getattr(cancel, "is_set", cancel) or (lambda: False)
+    tell = progress or (lambda fraction, text: None)
+    if not os.path.isdir(root):
+        report.error(f"{root}: no such folder")
+        return report
+    shipped = None
+    if recipients_path:
+        try:
+            shipped = age.read_recipients(recipients_path)
+        except (OSError, ValueError) as e:
+            report.error(f"the dataset's keys ({recipients_path}): {e}")
+    files = walk_files(root, report)
+    sizes = {rel: os.path.getsize(os.path.join(root, rel)) for rel in files}
+    report.bytes = sum(sizes.values())
+    total = max(1, 2 * report.bytes)   # checksums, then the chunks again
+    done = 0
+
+    def step(n, text):
+        nonlocal done
+        done += n
+        tell(min(done / total, 0.999), text)
+
+    tell(0.0, "Checking checksums")
+    check_sums(root, files, report, lambda n: step(n, "Checking checksums"), cancelled)
+
+    manifest = None
+    if "manifest.json" in files:
+        manifest = read_json_file(os.path.join(root, "manifest.json"), "manifest.json", report)
+    else:
+        report.error("manifest.json is missing")
+    entries, recipients = check_manifest(manifest, report, root, shipped) if manifest is not None else ({}, [])
+    report.summary = summarize(manifest)
+
+    if "fit.json" in files:
+        fit = read_json_file(os.path.join(root, "fit.json"), "fit.json", report)
+        if fit is not None and not isinstance(fit, dict):
+            report.error("fit.json: not a JSON object")
+    else:
+        report.warn("no fit.json: the round's fit readings are missing")
+    enc_files = ((manifest or {}).get("encryption") or {}).get("files") if isinstance(manifest, dict) else None
+    listed_health = isinstance(enc_files, list) and "health.json.age" in enc_files
+    if ("health.json.age" in files) != listed_health:
+        report.error("health.json.age and the manifest's encryption.files don't agree")
+    if "health.json.age" in files and recipients:
+        problem = age_shape(os.path.join(root, "health.json.age"), recipients)
+        if problem:
+            report.error(f"health.json.age: {problem}")
+
+    take_dirs = sorted({rel.split("/")[1] for rel in files if rel.startswith("takes/") and rel.count("/") >= 2})
+    for tid in take_dirs:
+        if tid not in entries:
+            report.error(f"takes/{tid}/ isn't in the manifest")
+    for tid, entry in entries.items():
+        if tid not in take_dirs:
+            report.error(f"take {tid} is in the manifest but has no folder")
+    for tid in take_dirs:
+        if cancelled():
+            raise Cancelled()
+        check_take(root, tid, entries.get(tid), sizes, recipients, identity, report,
+                   lambda n, tid=tid: step(n, f"Checking {tid}"), cancelled)
+
+    if report.bytes > MAX_BYTES:
+        report.error(f"the export is {report.bytes / 1000 ** 3:.1f} GB, over the {MAX_BYTES / 1000 ** 3:.0f} GB limit")
+    elif report.bytes > WARN_BYTES:
+        report.warn(f"the export is large: {report.bytes / 1000 ** 3:.1f} GB")
+    tell(1.0, "Checked")
+    return report
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Check an eye recorder export (an exports/<round>/ folder).")
+    ap.add_argument("dir")
+    ap.add_argument("--json", action="store_true", help="print the result as JSON")
+    ap.add_argument("--identity", help="an age identity file: also decrypt and check every chunk")
+    a = ap.parse_args()
+    if not os.path.isdir(a.dir):
+        print(f"{a.dir}: no such folder", file=sys.stderr)
+        return 2
+    if hasattr(os, "setpriority"):   # not on Windows
+        try:
+            os.setpriority(os.PRIO_PROCESS, 0, 19)   # stay out of VR's way
+        except OSError:
+            pass
+    identity = None
+    if a.identity:
+        with open(a.identity) as f:
+            identity = next((line.strip() for line in f if line.strip().startswith("AGE-SECRET-KEY-1")), None)
+        if not identity:
+            print(f"{a.identity}: no age identity in it", file=sys.stderr)
+            return 2
+    report = validate(a.dir, identity=identity)
+    if a.json:
+        print(json.dumps(report.as_dict(), indent=2))
+    else:
+        for e in report.errors:
+            print(f"error: {e}")
+        for w in report.warnings:
+            print(f"warning: {w}")
+        s = report.summary
+        if s:
+            print(f"{s.get('round')}: {s.get('takes')} takes, {s.get('frames')} frames, {s.get('minutes')} min, "
+                  f"{report.bytes / 1000 ** 2:.1f} MB, consent {s.get('consent_version') or '-'}")
+        print("OK" if report.ok else f"{len(report.errors)} errors")
+    return 0 if report.ok else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

gaze/run.sh

@@ -1,6 +1,7 @@
 #!/usr/bin/env bash
 # Install, start, stop, or inspect the gaze service (ft-gazed) on the Frame.
-# Usage: gaze/run.sh install|uninstall   # user service, starts with SteamVR
+# Usage: gaze/run.sh install|uninstall   # user service, starts with SteamVR; install also keeps
+#                                        # SteamVR's eye cameras at 90 fps (gaze/eye-rate.py)
 #        gaze/run.sh start|stop|restart|status|log [lines]
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
@@ -11,10 +12,13 @@ case ${1:-status} in
   install)
     [ "$FRAME_RELEASE" = 1 ] || "$root/gaze/build.sh"
     fill_template "$root/gaze/$unit" | on_frame "mkdir -p ~/.config/systemd/user && cat > ~/.config/systemd/user/$unit"
-    on_frame "chmod +x gaze/ft-gazed gaze/ft-gazectl"
+    on_frame "chmod +x gaze/ft-gazed gaze/ft-gazectl gaze/eye-rate.py"
+    on_frame "/usr/bin/python3 gaze/eye-rate.py on" || true
     "$frame" --host "set -e; systemctl --user daemon-reload; systemctl --user enable $unit
 $(start_with_steamvr $unit)" ;;
-  uninstall) "$frame" --host "systemctl --user disable --now $unit 2>/dev/null; rm -f ~/.config/systemd/user/$unit; systemctl --user daemon-reload; echo removed" ;;
+  uninstall)
+    "$frame" --host "systemctl --user disable --now $unit 2>/dev/null; rm -f ~/.config/systemd/user/$unit; systemctl --user daemon-reload; echo removed"
+    on_frame "/usr/bin/python3 gaze/eye-rate.py off" || true ;;
   start|stop|restart) "$frame" --host "systemctl --user $1 $unit; systemctl --user is-active $unit" ;;
   status) "$frame" --host "systemctl --user is-active $unit" || true; on_frame "gaze/ft-gazectl status" || true ;;
   log) "$frame" --host "journalctl --user -u $unit --no-pager -o cat -n ${2:-30}" ;;

gaze/test/blind-recalibration-test.py

@@ -0,0 +1,265 @@
+#!/usr/bin/env python3
+"""Offline test of our own eye tracker's calibration going blind (gaze/gazecheck.py, OWN_GONE):
+a full calibration while ours is calibrated but sends no gaze. Live 2026-10-18 a calibration
+with the left eye closed mapped only the right eye; the next one, with the right eye closed,
+took no dot, since every click waited for our tracker's gaze ("Waiting: the eye tracker isn't
+sending a gaze"). Now the dot stands in for the gaze while there's none, as in the first
+calibration (first-calibration-test.py), and ft-eyes checks the pupils it sees.
+
+Same harness as first-calibration-test.py: ft-gazed's Service with its sockets renamed and HOME
+in a temp folder, a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR
+sees both eyes; ours sends a gaze while OWN_FLAG exists), a fake ft-eyes control socket
+(calibrated), and no panel. Nothing reaches the live gaze service, the pointer helper, ft-eyes,
+or SteamVR, so it's safe next to them.
+
+  gaze/test/blind-recalibration-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-blind-recal-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import json  # noqa: E402
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import socket  # noqa: E402
+import subprocess  # noqa: E402
+import sys  # noqa: E402
+import threading  # noqa: E402
+import time  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_blind_recal_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
+gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
+gazed.TrackedEye = lambda: lambda now=None: None  # both eyes, whatever SteamVR's settings say
+DOTS = 4
+real_dots = gazecheck.check_dots
+gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS]  # a short calibration
+gazecheck.Fusion.MIN = DOTS   # ...whose dots still fit a fusion (SteamVR's eyes at each)
+gazecheck.LOOKED_AWAY = 1e9   # the fake SteamVR's eyes stay put whatever the dot (looked-away-test.py has that check)
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, ours sends a gaze (the right
+# eye's, at 1, 2) while OWN_FLAG exists, and {"ok":0} as with a stale file otherwise.
+FAKE = os.path.join(HOME, "ft-gaze")
+OWN_FLAG = os.path.join(HOME, "own-gaze")
+with open(FAKE, "w") as f:
+    f.write('''import json, os, select, sys, time
+flag = sys.argv[1]
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        if not os.read(0, 4096):
+            break
+    own = {"hy": 1.0, "hp": 2.0, "eyes": [None, [1.0, 2.0]]} if os.path.exists(flag) else {"ok": 0}
+    print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}, "left": {"hy": 0.0, "hp": 2.0}, "right": {"hy": 2.0, "hp": 2.0},
+          "own": own}}), flush=True)
+''')
+SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
+
+
+def own_gaze(on):
+    if on:
+        open(OWN_FLAG, "w").close()
+    elif os.path.exists(OWN_FLAG):
+        os.remove(OWN_FLAG)
+
+
+def start_helper(self):
+    """ft-gaze, straight from here instead of the dev container."""
+    self.proc = subprocess.Popen([sys.executable, FAKE, OWN_FLAG], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+
+
+def start_eyes(self):
+    """ft-eyes' process: a stand-in. Its control socket is the fake below."""
+    self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.eyes_proc.stderr.fileno(), False)
+    self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
+
+
+def start_panel(self):
+    self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.panel_proc.stderr.fileno(), False)
+    self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+
+
+gazed.Service.start_helper = start_helper
+gazed.Service.start_eyes = start_eyes
+gazecheck.Checks.start_panel = start_panel
+
+# The fake ft-eyes: calibrated (the right eye only, as after a calibration with the left closed).
+eyes_state = {"cal": {"made": "test", "dots": 27}, "points": []}
+eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+eyes.bind(gazed.EYES_SOCKET)
+eyes.settimeout(0.2)
+
+
+def eyes_answer():
+    while True:
+        try:
+            data, addr = eyes.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        w = data.decode().split()
+        if w[0] == "status":
+            reply = json.dumps({"calibration": eyes_state["cal"], "calibrating": False, "dots": 0, "mapped": ["right"],
+                                "eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
+        elif w[0] == "calib-start":
+            eyes_state["points"] = []
+            reply = "ok"
+        elif w[0] == "calib-point":
+            eyes_state["points"].append(tuple(map(float, w[1:5])))
+            reply = "ok 0 50 nan 1.00 (left eye only: the right eye was seen in only 0 frames)"
+        elif w[0] == "calib-fit":
+            eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
+            reply = f"ok {len(eyes_state['points'])} dots, fit median 1.00 deg, eyes right not used (0 dots, needs 12), left 1.00 ({len(eyes_state['points'])} dots)"
+        elif w[0] == "click":
+            continue
+        else:
+            reply = f"fail unknown command {w[0]}"
+        if addr:
+            eyes.sendto(reply.encode(), addr)
+
+
+threading.Thread(target=eyes_answer, daemon=True).start()
+
+helper_state = {"reply": "ok on worn", "heard": []}
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def helper_answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(helper_state["reply"].encode(), addr)
+        else:
+            helper_state["heard"].append(data.decode())
+
+
+threading.Thread(target=helper_answer, daemon=True).start()
+own_gaze(True)
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(3)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(65536).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+def check_state():
+    return svc.checks.check or {}
+
+
+def click_dot(i, settle=0.0):
+    """Wait for dot i, then click: is it taken (within 2 s)?"""
+    wait(lambda: check_state().get("i") == i and not check_state().get("done_at"), 3)
+    time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1 + settle)
+    before = check_state().get("captured", 0)
+    ask("calaccept")
+    return wait(lambda: check_state().get("captured", 0) > before, 2)
+
+
+check("ours runs, calibrated", ask("wake 60") == "ok" and wait(lambda: svc.checks.calibrated() is True, 6), True)
+check("SteamVR sees the eyes (the fake ft-gaze is sending)", wait(svc.checks.eyes_seen, 6), True)
+check("calibrate: a full calibration opens",
+      ask("calibrate").startswith("ok") and wait(lambda: check_state().get("kind") == "full", 6), True)
+check("not blind: ours is calibrated", check_state().get("blind"), False)
+
+check("dot 1, ours sends a gaze: taken", click_dot(0), True)
+check("...on our gaze, not blind", (check_state().get("run_blind"), check_state().get("went_blind")), (False, False))
+
+# Dot 2: the eye ours maps closes, and its gaze stops. The old way (OWN_GONE never) the click waits.
+own_gaze(False)
+gazecheck.OWN_GONE = 1e9
+check("dot 2, no gaze from ours, the old way: the click takes nothing", click_dot(1), False)
+check("...and the panel says it waits for a gaze",
+      wait(lambda: "isn't sending a gaze" in check_state().get("note", ""), 2), True)
+gazecheck.OWN_GONE = 0.5
+check("now: the dot stands in for the gaze, and the click waiting there takes it",
+      wait(lambda: check_state().get("captured") == 2, 3), True)
+check("...said once in the log", sum("our tracker sends no gaze" in m for m in logs), 1)
+t0, t1, yaw, pitch = eyes_state["points"][-1]
+dot = gazecheck.check_dots("full", True)[1]
+check("ft-eyes got dot 2's direction", (round(yaw, 3), round(pitch, 3)), (round(dot[0], 3), round(dot[1], 3)))
+
+own_gaze(True)
+check("dot 3, ours sends a gaze again: taken on it", click_dot(2) and not check_state().get("run_blind"), True)
+own_gaze(False)
+check("dot 4, gone again: blind once OWN_GONE passes, taken", click_dot(3, gazecheck.OWN_GONE), True)
+t0, t1, yaw, pitch = eyes_state["points"][-1]
+check("its window is the look up to the click (about CHECK_WINDOW)", abs((t1 - t0) - gazecheck.CHECK_WINDOW) < 0.15, True)
+check("still said once in the log", sum("our tracker sends no gaze" in m for m in logs), 1)
+check("all dots: ours fits its calibration", wait(lambda: not svc.checks.check and len(eyes_state["points"]) == DOTS, 5), True)
+check("the fit's reply in the log", any("left 1.00 (4 dots)" in m for m in logs), True)
+
+# A check of our gaze (five) has nothing to check without one: it waits for ours to send, and
+# never goes blind.
+check("five check without our gaze: it waits for ours to send", ask("fivecheck"), "ok waking the eye tracker first")
+check("...and doesn't open meanwhile", wait(lambda: svc.checks.check is not None, 3), False)
+own_gaze(True)
+check("ours sends: the five check opens", wait(lambda: check_state().get("kind") == "five", 6), True)
+own_gaze(False)
+check("its click without our gaze takes nothing", click_dot(0, gazecheck.OWN_GONE), False)
+check("...and it never went blind", (check_state().get("run_blind"), check_state().get("went_blind")), (False, False))
+ask("calquit")
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+svc.running = False
+time.sleep(0.7)
+shutil.rmtree(HOME, ignore_errors=True)
+os._exit(1 if failures else 0)

gaze/test/eye-rate-test.py

@@ -0,0 +1,138 @@
+#!/usr/bin/env python3
+"""Offline test of gaze/eye-rate.py (SteamVR's eye cameras at 90 fps): on, off, and the camera
+rate, with SteamVR off (its settings file) and on (vrcmd). HOME is a temporary folder and
+vrcmd and pgrep are stand-ins, so the live SteamVR settings are never read or changed.
+
+  gaze/test/eye-rate-test.py
+"""
+import contextlib
+import importlib.machinery
+import importlib.util
+import io
+import json
+import os
+import tempfile
+from pathlib import Path
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+tmp = Path(tempfile.mkdtemp(prefix="ft-eye-rate-test-"))
+os.environ["HOME"] = str(tmp)  # eye-rate.py's paths come from it
+
+loader = importlib.machinery.SourceFileLoader("eyerate", os.path.join(HERE, "..", "eye-rate.py"))
+er = importlib.util.module_from_spec(importlib.util.spec_from_loader("eyerate", loader))
+loader.exec_module(er)
+assert str(er.HOME) == str(tmp), "eye-rate.py must take HOME from the environment"
+
+failures = []
+
+
+def check(what, got, want):
+    if got != want:
+        failures.append(what)
+        print(f"FAIL {what}: got {got!r}, want {want!r}", flush=True)
+
+
+def main_with(*args):
+    """eye-rate.py's main() with these arguments: (exit code, what it printed)."""
+    er.sys.argv = ["eye-rate.py", *args]
+    out = io.StringIO()
+    with contextlib.redirect_stdout(out), contextlib.redirect_stderr(out):
+        code = er.main()
+    return code, out.getvalue()
+
+
+settings = er.STEAMVR_SETTINGS[0]
+
+
+def file_value():
+    return json.loads(settings.read_text()).get("steamvr", {}).get("gazeDepEyeToHeadEnable")
+
+
+# --- SteamVR off: the settings file ---
+er.steamvr_up = lambda: False
+settings.parent.mkdir(parents=True)
+settings.write_text(json.dumps({"steamvr": {"analogGain": 0.25}, "dashboard": {"x": 1}}, indent=3))
+check("off by default (not in the file)", er.get(), None)
+code, _ = main_with("on")
+check("on: exit 0", code, 0)
+check("on: written to the file", file_value(), True)
+check("on: SteamVR's other settings kept", json.loads(settings.read_text())["steamvr"]["analogGain"], 0.25)
+check("on: other sections kept", json.loads(settings.read_text())["dashboard"], {"x": 1})
+check("on: marked as Frametop's", er.MARK.exists(), True)
+code, out = main_with("on")
+check("on again: says it's on already", "already" in out, True)
+code, _ = main_with("off")
+check("off: written to the file", file_value(), False)
+check("off: mark gone", er.MARK.exists(), False)
+
+# Someone turned it on themselves: off leaves it.
+settings.write_text(json.dumps({"steamvr": {"gazeDepEyeToHeadEnable": True}}))
+code, out = main_with("on")
+check("on, set by the user: no mark", er.MARK.exists(), False)
+code, out = main_with("off")
+check("off, set by the user: left on", file_value(), True)
+check("off, set by the user: says so", "left as they are" in out, True)
+
+# No settings file yet (SteamVR never ran): on makes one.
+settings.unlink()
+code, _ = main_with("on")
+check("no file: on makes it", file_value(), True)
+main_with("off")
+
+# A broken file isn't overwritten.
+settings.write_text("{not json")
+code, _ = main_with("on")
+check("broken file: on fails", code, 1)
+check("broken file: left alone", settings.read_text(), "{not json")
+check("broken file: no mark", er.MARK.exists(), False)
+settings.unlink()
+
+# --- SteamVR on: vrcmd ---
+live = {"value": False}
+calls = []
+
+
+def fake_vrcmd(*args):
+    calls.append(args)
+    if args[0] == "--settings-bool":
+        return f"steamvr.gazeDepEyeToHeadEnable={'true' if live['value'] else 'false'}\n"
+    if args[0] == "--set-settings-bool":
+        live["value"] = args[2] == "1"  # as vrcmd does: "true" would read as 0
+        return f"steamvr.gazeDepEyeToHeadEnable={args[2]}\n"
+    return ""
+
+
+er.steamvr_up = lambda: True
+er.vrcmd = fake_vrcmd
+code, _ = main_with("on")
+check("SteamVR on: set through vrcmd", live["value"], True)
+check("SteamVR on: sets 1, never 'true'", [c for c in calls if c[0] == "--set-settings-bool"][-1][2], "1")
+check("SteamVR on: the file isn't touched", settings.exists(), False)
+check("SteamVR on: marked", er.MARK.exists(), True)
+code, _ = main_with("off")
+check("SteamVR on: off through vrcmd", live["value"], False)
+check("SteamVR on: mark gone", er.MARK.exists(), False)
+
+# vrcmd that doesn't take it: on fails, no mark.
+er.vrcmd = lambda *args: "" if args[0] == "--set-settings-bool" else "steamvr.gazeDepEyeToHeadEnable=false\n"
+code, _ = main_with("on")
+check("vrcmd didn't take it: on fails", code, 1)
+check("vrcmd didn't take it: no mark", er.MARK.exists(), False)
+
+# --- The camera rate, from eyetracking.txt ---
+check("no eyetracking.txt: unknown", er.camera_rate(), None)
+er.EYE_LOG.parent.mkdir(parents=True)
+head = "Sat Oct 10 2026 22:35:43.000000 [Info] - "
+er.EYE_LOG.write_text("\n".join([
+    head + "eyetracking 2.18.2 startup with PID=1, config=x",
+    head + "CStereoAdspCams: Set framerate 15.000000",
+    head + "Eye tracking shutting down",
+    head + "eyetracking 2.18.2 startup with PID=2, config=x",
+    head + "Eye tracking started"]) + "\n")
+check("no rate set since the last start: 90", er.camera_rate(), 90.0)
+with er.EYE_LOG.open("a") as f:
+    f.write(head + "CStereoAdspCams: Set framerate 15.000000\n")
+check("15 set since the last start", er.camera_rate(), 15.0)
+
+print("eye-rate-test:", "FAILED " + ", ".join(failures) if failures else "all passed")
+raise SystemExit(1 if failures else 0)

gaze/test/first-calibration-test.py

@@ -6,7 +6,9 @@ its dots. Until 2026-10-05 it couldn't: a fresh install that chose our tracker n
 Runs ft-gazed's Service with its sockets renamed and HOME in a temp folder (state and settings
 go there), a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR sees both
 eyes; "own" is {"ok":0}, as with an uncalibrated ft-eyes), a fake ft-eyes control socket, and no
-panel (a stand-in process). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
+panel (a stand-in process). The fake ft-eyes also answers late or not at all, as a slow one
+does: the service must keep running meanwhile (until 2026-10-09 it waited, up to 3 s a dot and
+10 s for the fit, and the helper's 3 s panel lease ran out). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
 or SteamVR, so it's safe next to them.
 
   gaze/test/first-calibration-test.py
@@ -43,9 +45,12 @@ gazecheck.SCREENS = f"\0{tag}_screens"
 gazecheck.PANEL = f"\0{tag}_panel"
 gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
 gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
-DOTS = 3
+gazed.TrackedEye = lambda: lambda now=None: None  # both eyes, whatever SteamVR's settings say
+DOTS = 4
 real_dots = gazecheck.check_dots
 gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS]  # a short calibration
+gazecheck.Fusion.MIN = DOTS   # ...whose dots still fit a fusion (SteamVR's eyes at each)
+gazecheck.LOOKED_AWAY = 1e9   # the fake SteamVR's eyes stay put whatever the dot (looked-away-test.py has that check)
 logs = []
 gazed.log = gazecheck.log = lambda msg: logs.append(msg)
 
@@ -58,7 +63,8 @@ while True:
         if not os.read(0, 4096):
             break
     print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
-          "open": [0.8, 0.8]}, "own": {"ok": 0}}}), flush=True)
+          "open": [0.8, 0.8]}, "left": {"hy": 0.0, "hp": 2.0}, "right": {"hy": 2.0, "hp": 2.0},
+          "own": {"ok": 0}}}), flush=True)
 ''')
 SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
 
@@ -92,8 +98,10 @@ gazed.Service.start_helper = start_helper
 gazed.Service.start_eyes = start_eyes
 gazecheck.Checks.start_panel = start_panel
 
-# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that.
-eyes_state = {"cal": None, "points": [], "fail": None}
+# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that;
+# "delay" holds calib-point's and calib-fit's answers that many seconds; "drop" leaves the next
+# calib-point unanswered.
+eyes_state = {"cal": None, "points": [], "fail": None, "delay": 0.0, "drop": False}
 eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
 eyes.bind(gazed.EYES_SOCKET)
 eyes.settimeout(0.2)
@@ -116,19 +124,31 @@ def eyes_answer():
             reply = "ok"
         elif w[0] == "calib-point":
             eyes_state["points"].append(tuple(map(float, w[1:5])))
+            if eyes_state["drop"]:
+                eyes_state["drop"] = False
+                continue
             reply, eyes_state["fail"] = eyes_state["fail"] or "ok 50 50 1.00 1.00", None
         elif w[0] == "calib-fit":
             eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
             reply = f"ok {len(eyes_state['points'])} dots"
         else:
             reply = f"fail unknown command {w[0]}"
-        if addr:
+        if addr and w[0] in ("calib-point", "calib-fit") and eyes_state["delay"]:
+            threading.Timer(eyes_state["delay"], send_late, (reply, addr)).start()
+        elif addr:
             eyes.sendto(reply.encode(), addr)
 
 
+def send_late(reply, addr):
+    try:
+        eyes.sendto(reply.encode(), addr)
+    except OSError:
+        pass  # the service gave up on it
+
+
 threading.Thread(target=eyes_answer, daemon=True).start()
 
-helper_state = {"reply": "ok off worn", "heard": []}
+helper_state = {"reply": "ok off worn", "heard": [], "calpanel": []}  # calpanel: when "calpanel 1" came
 helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
 helper.bind(gazed.POINTER)
 helper.settimeout(0.2)
@@ -146,6 +166,8 @@ def helper_answer():
             helper.sendto(helper_state["reply"].encode(), addr)
         else:
             helper_state["heard"].append(data.decode())
+            if data == b"calpanel 1":
+                helper_state["calpanel"].append(time.monotonic())
 
 
 threading.Thread(target=helper_answer, daemon=True).start()
@@ -225,14 +247,57 @@ ask("calaccept")
 check("ft-eyes refusing a dot: its reason reaches the panel's note",
       wait(lambda: "left eye in only 3 frames" in check_state().get("note", ""), 2), True)
 check("dot 2, second try: taken", take_dot(1), True)
-check("dot 3: taken", take_dot(2), True)
 
+# A slow ft-eyes (2.5 s): the service goes on meanwhile, and a second click is ignored.
+eyes_state["delay"] = 2.5
+wait(lambda: check_state().get("i") == 2 and not check_state().get("done_at"), 3)
+time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+before = len(eyes_state["points"])
+ask("calaccept")
+check("dot 3, ft-eyes slow: the service waits for it", wait(lambda: svc.checks.asking is not None, 2), True)
+t = time.monotonic()
+ask("status")
+check("the service still answers meanwhile", time.monotonic() - t < 0.5, True)
+ask("calaccept")
+check("dot 3: taken once ft-eyes answers", wait(lambda: check_state().get("captured") == 3, 4), True)
+check("the helper's panel lease was renewed while ft-eyes took its time",
+      sum(t < at < t + eyes_state["delay"] for at in helper_state["calpanel"]) >= 2, True)
+check("and the calibration is still open (a blocked service took that as the headset off)",
+      check_state().get("kind"), "full")
+check("the second click asked ft-eyes nothing", len(eyes_state["points"]), before + 1)
+eyes_state["delay"] = 0.0
+
+# No answer: the dot isn't taken, after calib-point's 3 s.
+eyes_state["drop"] = True
+wait(lambda: check_state().get("i") == 3 and not check_state().get("done_at"), 3)
+time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+ask("calaccept")
+check("dot 4, no answer from ft-eyes: not taken, and the panel says so",
+      wait(lambda: "didn't answer" in check_state().get("note", ""), 5), True)
+eyes_state["delay"] = 2.5  # the fit too
+check("dot 4, second try: taken", take_dot(3) or wait(lambda: check_state().get("captured") == 4, 4), True)
+
+check("while ours fits, the panel stays", wait(lambda: check_state().get("fitting") is True, 3), True)
+ask("calquit")
+check("and a right click doesn't close it", check_state().get("kind"), "full")
 check("all dots: ours fits its calibration (calib-fit)",
-      wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 4), True)
+      wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 5), True)
+gaps = [b - a for a, b in zip(helper_state["calpanel"], helper_state["calpanel"][1:])]
+check("the helper's panel lease (3 s) never ran out", bool(gaps) and max(gaps) < 3.0, True)
 check("the service sees it calibrated", wait(lambda: svc.checks.calibrated() is True, 4), True)
 check("and gaze mode stays on", "gaze off" in helper_state["heard"], False)
 check("and no second calibration opens", wait(lambda: svc.checks.check is not None, 3), False)
 check("one calibration in the log", sum(m.startswith("full check:") for m in logs), 1)
+fusion = gazecheck.Fusion.load(gazecheck.FUSION)
+check("SteamVR's eyes at the dots ours took: a fusion saved", (fusion.info.get("dots"), fusion.info.get("how"))
+      if fusion else None, (DOTS, "calibration"))
+dots = [d[:2] for d in real_dots("full", True)[:DOTS]]
+check("its points: SteamVR's eyes' mean, and the dot", [p[:2] for p in fusion.points] if fusion else None,
+      [[1.0, 2.0]] * DOTS)
+check("...the dots", [tuple(round(v, 3) for v in p[2:]) for p in fusion.points] if fusion else None,
+      [tuple(round(v, 3) for v in d) for d in dots])
+check("the service loaded it", wait(lambda: svc.fusion is not None and svc.fusion.ready(), 3), True)
+check("GAZE_FUSION unset: not in use", (svc.fusing, svc.status()["fusion"]["setting"]), (False, "off"))
 
 print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
 svc.running = False

gaze/test/fit-coach-check-test.py

@@ -0,0 +1,247 @@
+#!/usr/bin/env python3
+"""Offline test of the fit coach in the headset fit check (gaze/gazecheck.py, "fit"): with
+SteamVR's tracker in use, the check keeps ft-eyes running (the lease), asks it for the last
+second of pupil finds ("fit"), judges them only while the gaze is straight ahead, and puts the
+coach's hint first in the panel's hints.
+
+The harness is first-calibration-test.py's: ft-gazed's Service with its sockets renamed, HOME in
+a temp folder, a fake pointer helper, a fake ft-gaze (its gaze from a file the test writes), a
+fake ft-eyes control socket (its finds from the shipped reference, offset as the test says),
+and the panel's commands recorded instead of sent. Safe next to the live gaze service.
+
+  gaze/test/fit-coach-check-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-fit-coach-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import json  # noqa: E402
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import socket  # noqa: E402
+import subprocess  # noqa: E402
+import sys  # noqa: E402
+import threading  # noqa: E402
+import time  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import fitcoach  # noqa: E402
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_fit_coach_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
+gazed.read_settings = lambda: ("steam", "auto", "auto", 55.0)  # SteamVR's tracker in use
+gazed.own_installed = lambda: True
+gazed.TrackedEye = lambda: lambda now=None: None
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, its gaze from LOOK.
+LOOK = os.path.join(HOME, "look.json")
+FAKE = os.path.join(HOME, "ft-gaze")
+
+
+def look(yaw, pitch):
+    with open(LOOK + ".new", "w") as f:
+        json.dump([yaw, pitch], f)
+    os.replace(LOOK + ".new", LOOK)
+
+
+look(0.0, 0.0)
+with open(FAKE, "w") as f:
+    f.write(f'''import json, os, select, sys, time
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        if not os.read(0, 4096):
+            break
+    try:
+        hy, hp = json.load(open({LOOK!r}))
+    except (OSError, ValueError):
+        continue
+    print(json.dumps({{"t": time.monotonic(), "src": {{"mmap1": {{"hy": hy, "hp": hp, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}}}}}}), flush=True)
+''')
+SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
+started = {"eyes": 0}
+
+
+def start_helper(self):
+    self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+
+
+def start_eyes(self):
+    started["eyes"] += 1
+    self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.eyes_proc.stderr.fileno(), False)
+    self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
+
+
+def start_panel(self):
+    self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.panel_proc.stderr.fileno(), False)
+    self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+
+
+panel = []
+gazed.Service.start_helper = start_helper
+gazed.Service.start_eyes = start_eyes
+gazecheck.Checks.start_panel = start_panel
+gazecheck.Checks.to_panel = lambda self, command: panel.append(command)
+
+# The fake ft-eyes: "fit" answers with finds where the shipped reference puts eyes offset by
+# eyes_state["offset"] (yaw, pitch degrees, both eyes alike).
+REF = fitcoach.Reference.load()
+eyes_state = {"offset": (0.0, 0.0), "asked": 0}
+eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+eyes.bind(gazed.EYES_SOCKET)
+eyes.settimeout(0.2)
+
+
+def finds():
+    yaw, pitch = eyes_state["offset"]
+    out = {}
+    for cam in (0, 1):
+        (cx, cy), ((a, b), (c, d)) = REF.eyes[cam]["centre"], REF.eyes[cam]["jacobian"]
+        out[cam] = {"n": 90, "finds": 90, "found": 1.0, "x": cx + a * yaw + b * pitch, "y": cy + c * yaw + d * pitch,
+                    "a": 28.0, "spacing": 15.0, "extra": 0.0}
+    return out
+
+
+def eyes_answer():
+    while True:
+        try:
+            data, addr = eyes.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        w = data.decode().split()
+        if w[0] == "status":
+            reply = json.dumps({"calibration": None, "calibrating": False, "dots": 0,
+                                "eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
+        elif w[0] == "fit":
+            eyes_state["asked"] += 1
+            reply = json.dumps(finds())
+        else:
+            reply = f"fail unknown command {w[0]}"
+        if addr:
+            try:
+                eyes.sendto(reply.encode(), addr)
+            except OSError:
+                pass  # the service gave up on it (as ft-eyes' serve() does)
+
+
+threading.Thread(target=eyes_answer, daemon=True).start()
+
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def helper_answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(b"ok off worn", addr)
+
+
+threading.Thread(target=helper_answer, daemon=True).start()
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(3)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(65536).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+def coach_line():
+    """The first hint line the panel got last, if it's the coach's."""
+    for cmd in reversed(panel):
+        if cmd.startswith("hints "):
+            first = cmd[len("hints "):].split("|")[0]
+            return first if first.startswith("Headset position") else None
+    return None
+
+
+ask("wake 60")
+check("the tracker sends", wait(lambda: time.monotonic() - svc.checks.sample_at < 1, 6), True)
+check("SteamVR's tracker in use: ft-eyes isn't running", svc.eyes_proc is None, True)
+check("fitcheck is taken", ask("fitcheck").startswith("ok"), True)
+check("the fit check opens", wait(lambda: (svc.checks.check or {}).get("kind") == "fit", 6), True)
+check("the fit check starts ft-eyes for the coach", wait(lambda: svc.eyes_proc is not None, 3), True)
+check("and asks it for finds", wait(lambda: eyes_state["asked"] > 0, 3), True)
+check("looking ahead, well fitted: good",
+      wait(lambda: (coach_line() or "").startswith("Headset position: good."), 4), True)
+check("marked a first guess (the reference is from one face)", "first guess" in (coach_line() or ""), True)
+
+eyes_state["offset"] = (0.0, 12.0)   # both eyes appear to look up: the headset is low
+check("the headset low: push it up",
+      wait(lambda: "Push the headset up" in (coach_line() or ""), 4), True)
+check("it's the first hint line", (coach_line() or "").startswith("Headset position: Push"), True)
+
+look(25.0, 0.0)                      # looking away: the pupils move with the gaze, so no new reading
+check("looking away: the last centred reading, marked so",
+      wait(lambda: "when you last looked straight ahead" in (coach_line() or ""), 4), True)
+eyes_state["offset"] = (0.0, 0.0)
+time.sleep(1.5)
+check("and it isn't judged meanwhile", "Push the headset up" in (coach_line() or ""), True)
+look(0.0, 0.0)
+check("looking ahead again: judged again", wait(lambda: (coach_line() or "").startswith("Headset position: good."), 4), True)
+
+ask("calquit")
+check("closing logs the last reading", wait(lambda: any(m.startswith("fit coach, last reading: ok") for m in logs), 2), True)
+check("the check closed", wait(lambda: svc.checks.check is None, 2), True)
+check("ft-eyes stops once its lease runs out",
+      wait(lambda: svc.eyes_proc is None, gazecheck.COACH_LEASE + 4), True)
+
+print("all passed" if not failures else f"{len(failures)} failed: {failures}", flush=True)
+shutil.rmtree(HOME, ignore_errors=True)
+os._exit(1 if failures else 0)

gaze/test/fit-watch-test.py

@@ -0,0 +1,316 @@
+#!/usr/bin/env python3
+"""Offline test of the fit watch (gaze/gazecheck.py, GAZE_FIT_WATCH): the fit check keeping a
+good fit as yours, the put-on check in the quick check that opens when the headset goes on
+(holds the dot, says what to move, lets go once it's back or on a click), and the slip note
+with our tracker (ft-eyes' glint jump past the threshold for SLIP_HOLD; gone with the slip;
+not again within NOTE_COOLDOWN).
+
+fit-coach-check-test.py's harness: ft-gazed's Service with its sockets renamed, HOME in a temp
+folder, a fake pointer helper, a fake ft-gaze, a fake ft-eyes control socket (its finds from
+the shipped reference, offset as the test says, and its status with the glint jump), and the
+panel's commands recorded instead of sent. Safe next to the live gaze service.
+
+  gaze/test/fit-watch-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-fit-watch-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import json  # noqa: E402
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import socket  # noqa: E402
+import subprocess  # noqa: E402
+import sys  # noqa: E402
+import threading  # noqa: E402
+import time  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import fitcoach  # noqa: E402
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_fit_watch_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
+SETTINGS = {"tracker": "steam"}   # SteamVR's tracker in use first, then ours
+gazed.read_settings = lambda: (SETTINGS["tracker"], "auto", "auto", 55.0)
+gazed.own_installed = lambda: True
+gazed.TrackedEye = lambda: lambda now=None: None
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, its gaze from LOOK.
+LOOK = os.path.join(HOME, "look.json")
+FAKE = os.path.join(HOME, "ft-gaze")
+
+
+def look(yaw, pitch):
+    with open(LOOK + ".new", "w") as f:
+        json.dump([yaw, pitch], f)
+    os.replace(LOOK + ".new", LOOK)
+
+
+look(0.0, 0.0)
+with open(FAKE, "w") as f:
+    f.write(f'''import json, os, select, sys, time
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        if not os.read(0, 4096):
+            break
+    try:
+        hy, hp = json.load(open({LOOK!r}))
+    except (OSError, ValueError):
+        continue
+    print(json.dumps({{"t": time.monotonic(), "src": {{"mmap1": {{"hy": hy, "hp": hp, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}}}}}}), flush=True)
+''')
+SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
+started = {"eyes": 0}
+
+
+def start_helper(self):
+    self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+
+
+def start_eyes(self):
+    started["eyes"] += 1
+    self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.eyes_proc.stderr.fileno(), False)
+    self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
+
+
+def start_panel(self):
+    self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.panel_proc.stderr.fileno(), False)
+    self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+
+
+panel = []
+gazed.Service.start_helper = start_helper
+gazed.Service.start_eyes = start_eyes
+gazecheck.Checks.start_panel = start_panel
+gazecheck.Checks.to_panel = lambda self, command: panel.append(command)
+
+# The fake ft-eyes: "fit" answers with finds where the shipped reference puts eyes offset by
+# eyes_state["offset"] (yaw, pitch degrees, both eyes alike).
+REF = fitcoach.Reference.load()
+eyes_state = {"offset": (0.0, 0.0), "jump": (0.0, 0.0), "asked": 0}
+eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+eyes.bind(gazed.EYES_SOCKET)
+eyes.settimeout(0.2)
+
+
+def finds():
+    yaw, pitch = eyes_state["offset"]
+    out = {}
+    for cam in (0, 1):
+        (cx, cy), ((a, b), (c, d)) = REF.eyes[cam]["centre"], REF.eyes[cam]["jacobian"]
+        out[cam] = {"n": 90, "finds": 90, "found": 1.0, "x": cx + a * yaw + b * pitch, "y": cy + c * yaw + d * pitch,
+                    "a": 28.0, "spacing": 15.0, "extra": 0.0}
+    return out
+
+
+def eyes_answer():
+    while True:
+        try:
+            data, addr = eyes.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        w = data.decode().split()
+        if w[0] == "status":
+            yaw, pitch = eyes_state["jump"]   # the slip, as both pupils' move in degrees
+            jump = {}
+            for cam, name in ((0, "right"), (1, "left")):
+                (a, b), (c, d) = REF.eyes[cam]["jacobian"]
+                jump[name] = {"reseat": False, "jump": bool(yaw or pitch),
+                              "jump_px": [a * yaw + b * pitch, c * yaw + d * pitch]}
+            reply = json.dumps({"calibration": {"dots": 27}, "calibrating": False, "dots": 27, "eyes": jump})
+        elif w[0] == "fit":
+            eyes_state["asked"] += 1
+            reply = json.dumps(finds())
+        else:
+            reply = f"fail unknown command {w[0]}"
+        if addr:
+            try:
+                eyes.sendto(reply.encode(), addr)
+            except OSError:
+                pass  # the service gave up on it (as ft-eyes' serve() does)
+
+
+threading.Thread(target=eyes_answer, daemon=True).start()
+
+HELPER = {"gaze": False}
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def helper_answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(b"ok on worn" if HELPER["gaze"] else b"ok off worn", addr)
+
+
+threading.Thread(target=helper_answer, daemon=True).start()
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(3)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(65536).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+def since(n):
+    """The panel's commands from n on."""
+    return panel[n:]
+
+
+def quick(label):
+    """Opens the quick check as the headset going on does; returns where its panel commands start."""
+    n = len(panel)
+    check(label, svc.checks.start("quick", gazecheck.PUTON_REASON), "ok")
+    return n
+
+
+def puton():
+    return ((svc.checks.check or {}).get("puton") or {}).get("state")
+
+
+def closed_with_capture(seconds=8):
+    return wait(lambda: svc.checks.check is None and any(m.startswith("quick check done") for m in logs[mark:]), seconds)
+
+
+ask("wake 60")
+check("the tracker sends", wait(lambda: time.monotonic() - svc.checks.sample_at < 1, 6), True)
+check("the eyes count as seen", wait(lambda: svc.checks.eyes_seen(), 3), True)
+
+# The fit watch off (the default): the quick check is as it was.
+mark = len(logs)
+quick("fit watch off: a quick check opens")
+check("  without a put-on check", puton(), None)
+check("  and takes its dot", closed_with_capture(), True)
+
+# A good fit in the fit check is kept as yours.
+check("no saved fit yet", os.path.exists(fitcoach.BASELINE), False)
+check("fitcheck is taken", ask("fitcheck").startswith("ok"), True)
+check("the fit check opens", wait(lambda: (svc.checks.check or {}).get("kind") == "fit", 6), True)
+check("it reads a good fit", wait(lambda: ((svc.checks.check or {}).get("coach") or {}).get("good") is not None, 6), True)
+ask("calquit")
+check("closing it saves the fit as yours", wait(lambda: os.path.exists(fitcoach.BASELINE), 2), True)
+check("with both eyes", sorted(json.load(open(fitcoach.BASELINE))["eyes"]), ["0", "1"])
+
+gazed.CONF.parent.mkdir(parents=True, exist_ok=True)
+with open(gazed.CONF, "w") as f:
+    f.write("GAZE_FIT_WATCH=1\n")
+
+# Put on as saved: the dot is held for the reading, then taken.
+mark = len(logs)
+quick("put on as saved: the quick check opens")
+check("  reading the fit first", puton(), "reading")
+check("  sits as saved", wait(lambda: any("sits as your saved fit" in m for m in logs[mark:]), 5), True)
+check("  then the dot is taken", closed_with_capture(), True)
+
+# Put on low: it says to push it up, holds the dot past the quick check's timeout, lets go once back.
+eyes_state["offset"] = (0.0, 12.0)
+mark = len(logs)
+n = quick("put on low: the quick check opens")
+check("  the headset moved", wait(lambda: puton() == "moved", 5), True)
+check("  the panel widens", "show five" in since(n), True)
+check("  and says to push it up", wait(lambda: any(c.startswith("text Push the headset up") for c in since(n)), 2), True)
+time.sleep(gazecheck.QUICK_TIMEOUT + 1)
+check("  the dot waits past the quick check's timeout", (svc.checks.check or {}).get("captured"), 0)
+eyes_state["offset"] = (0.0, 0.0)
+check("  pushed back: let go", wait(lambda: any("is back as your saved fit" in m for m in logs[mark:]), 5), True)
+check("  saying so", "note Good: it's back where it was" in since(n), True)
+check("  then the dot is taken", closed_with_capture(), True)
+
+# Put on low, then a click: taken as it sits.
+eyes_state["offset"] = (0.0, 12.0)
+mark = len(logs)
+quick("put on low again")
+check("  the headset moved", wait(lambda: puton() == "moved", 5), True)
+ask("calaccept")
+check("  a click lets go", wait(lambda: any("a click" in m for m in logs[mark:]), 3), True)
+check("  and takes the dot", closed_with_capture(), True)
+eyes_state["offset"] = (0.0, 0.0)
+
+# The slip note: our tracker, gaze mode on.
+SETTINGS["tracker"] = "own"
+HELPER["gaze"] = True
+with open(gazed.CONF, "w") as f:
+    f.write("GAZE_FIT_WATCH=1\n# our tracker now\n")
+check("our tracker in use", wait(lambda: svc.kind == "own" and svc.eyes_proc is not None, 6), True)
+check("gaze mode on", wait(lambda: svc.checks.gaze_on is True, 4), True)
+check("ft-eyes' status comes", wait(lambda: time.monotonic() - svc.own_at < 2, 4), True)
+n = len(panel)
+time.sleep(2)
+check("no slip: no note", "show note" in since(n), False)
+eyes_state["jump"] = (0.0, -12.0)    # the pupils dropped: the headset rode up
+start = time.monotonic()
+check("a slip: the note", wait(lambda: "show note" in since(n), gazecheck.SLIP_HOLD + 4), True)
+check("  not before SLIP_HOLD", time.monotonic() - start >= gazecheck.SLIP_HOLD, True)
+check("  saying so", "title Your headset slipped" in since(n), True)
+check("  and which way", any(c.startswith("text Let the headset down") for c in since(n)), True)
+check("  no pointer hiding for it", any(c == "calpanel 1" for c in since(n)), False)
+n = len(panel)
+eyes_state["jump"] = (0.0, 0.0)
+check("the slip gone: the note goes", wait(lambda: "hide" in since(n), 4), True)
+n = len(panel)
+eyes_state["jump"] = (0.0, -12.0)
+time.sleep(gazecheck.SLIP_HOLD + 3)
+check("again within NOTE_COOLDOWN: no note", "show note" in since(n), False)
+eyes_state["jump"] = (0.0, 0.0)
+
+print("all passed" if not failures else f"{len(failures)} failed: {failures}", flush=True)
+shutil.rmtree(HOME, ignore_errors=True)
+os._exit(1 if failures else 0)

gaze/test/fitcoach-test.py

@@ -0,0 +1,238 @@
+#!/usr/bin/env python3
+"""The fit coach (gaze/fitcoach.py) on made-up readings: the reference fit, which way each hint
+points with both cameras' orientations, the state's hysteresis, lost eyes, and a baseline.
+
+  python3 gaze/test/fitcoach-test.py
+"""
+import math
+import os
+import random
+import sys
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.dirname(HERE))
+import fitcoach  # noqa: E402
+
+# About the maintainer's headset's (gaze/fit-reference.json): camera 1's image is turned 180
+# degrees from camera 0's, so its signs are the other way.
+J = {0: [[1.27, -0.42], [-0.48, -1.34]], 1: [[-1.34, -0.53], [-0.48, 1.48]]}
+CENTRE = {0: [341.0, 221.4], 1: [334.7, 173.5]}
+
+
+def ref(follow=None, faces=1):
+    return fitcoach.Reference({c: {"centre": CENTRE[c], "jacobian": J[c]} for c in (0, 1)}, ipd=0.065,
+                              faces=faces, cameras_follow_lenses=follow)
+
+
+def pupil(cam, yaw, pitch, centre=None):
+    """Where camera cam sees the pupil when the eye appears to look (yaw, pitch) degrees."""
+    cx, cy = centre or CENTRE[cam]
+    (a, b), (c, d) = J[cam]
+    return cx + a * yaw + b * pitch, cy + c * yaw + d * pitch
+
+
+def summary(right=(0, 0), left=(0, 0), found=(1.0, 1.0), spacing=(14.0, 16.0), extra=(0.0, 0.0)):
+    out = {}
+    for cam, (yaw, pitch) in ((0, right), (1, left)):
+        x, y = pupil(cam, yaw, pitch)
+        out[cam] = {"n": 90, "finds": int(90 * found[cam]), "found": found[cam], "x": x, "y": y, "a": 28.0,
+                    "spacing": spacing[cam], "extra": extra[cam]}
+    return out
+
+
+def keys(reading):
+    return [h["key"] for h in reading["hints"]]
+
+
+class ReferenceTest(unittest.TestCase):
+    def test_fit_recovers_both_orientations(self):
+        rng = random.Random(1)
+        rows = []
+        for made in ("a", "b"):
+            for _ in range(27):
+                yaw, pitch = rng.uniform(-20, 20), rng.uniform(-15, 15)
+                rows.append({"made": made, "truth": [yaw, pitch],
+                             "eye": {str(c): {"pupil": [v + rng.gauss(0, 0.5) for v in pupil(c, yaw, pitch)]} for c in (0, 1)}})
+        # A third calibration that fits badly is left out.
+        for _ in range(27):
+            yaw, pitch = rng.uniform(-20, 20), rng.uniform(-15, 15)
+            rows.append({"made": "bad", "truth": [yaw, pitch],
+                         "eye": {str(c): {"pupil": [v + rng.gauss(0, 15) for v in pupil(c, yaw, pitch)]} for c in (0, 1)}})
+        r = fitcoach.Reference.from_dots(rows)
+        for c in (0, 1):
+            self.assertEqual(r.eyes[c]["fits"], 2)
+            for i in range(2):
+                self.assertAlmostEqual(r.eyes[c]["centre"][i], CENTRE[c][i], delta=0.5)
+                for j in range(2):
+                    self.assertAlmostEqual(r.eyes[c]["jacobian"][i][j], J[c][i][j], delta=0.05)
+        # degrees() undoes the Jacobian.
+        x, y = pupil(1, 7.0, -3.0)
+        yaw, pitch = r.degrees(1, x - r.eyes[1]["centre"][0], y - r.eyes[1]["centre"][1])
+        self.assertAlmostEqual(yaw, 7.0, delta=0.2)
+        self.assertAlmostEqual(pitch, -3.0, delta=0.2)
+
+    def test_json_round_trip(self):
+        r = ref(follow=True, faces=4)
+        back = fitcoach.Reference.from_json(r.to_json())
+        self.assertEqual(back.eyes[1]["jacobian"], J[1])
+        self.assertTrue(back.cameras_follow_lenses)
+        self.assertFalse(back.provisional)
+        self.assertTrue(ref().provisional)
+
+    def test_the_shipped_reference_loads(self):
+        r = fitcoach.Reference.load()
+        self.assertIsNotNone(r, fitcoach.REFERENCE)
+        self.assertEqual(sorted(r.eyes), [0, 1])
+        self.assertTrue(r.provisional)
+
+
+class AssessTest(unittest.TestCase):
+    def test_centred_is_ok(self):
+        r = fitcoach.assess(summary(), ref())
+        self.assertEqual(r["state"], "ok")
+        self.assertEqual(r["hints"], [])
+        self.assertTrue(r["provisional"])
+
+    def test_eyes_high_means_push_up(self):
+        r = fitcoach.assess(summary(right=(0, 12), left=(0, 12)), ref())
+        self.assertEqual(r["state"], "adjust")
+        self.assertEqual(keys(r), ["up"])
+        r = fitcoach.assess(summary(right=(0, -12), left=(0, -12)), ref())
+        self.assertEqual(keys(r), ["down"])
+
+    def test_eyes_left_means_slide_left(self):
+        r = fitcoach.assess(summary(right=(11, 0), left=(11, 0)), ref())
+        self.assertEqual(keys(r), ["left"])
+        self.assertAlmostEqual(r["terms"]["left"], 11, delta=0.1)
+        r = fitcoach.assess(summary(right=(-11, 0), left=(-11, 0)), ref())
+        self.assertEqual(keys(r), ["right"])
+
+    def test_ipd_only_advised_once_the_cameras_are_known_to_follow_the_lenses(self):
+        apart = summary(right=(-10, 0), left=(10, 0))   # each eye outward of its lens
+        r = fitcoach.assess(apart, ref(follow=None))
+        self.assertAlmostEqual(r["terms"]["ipd"], 10, delta=0.1)
+        self.assertNotIn("ipd-wider", keys(r))
+        self.assertFalse(r["ipd_advice"])
+        self.assertEqual(r["state"], "ok", "the common terms are centred")
+        r = fitcoach.assess(apart, ref(follow=True))
+        self.assertEqual(keys(r), ["ipd-wider"])
+        self.assertEqual(r["state"], "adjust")
+        r = fitcoach.assess(summary(right=(10, 0), left=(-10, 0)), ref(follow=True))
+        self.assertEqual(keys(r), ["ipd-narrower"])
+
+    def test_tilt(self):
+        r = fitcoach.assess(summary(right=(0, -5), left=(0, 5)), ref())
+        self.assertEqual(keys(r), ["tilt-left-up"])
+        r = fitcoach.assess(summary(right=(0, 5), left=(0, -5)), ref())
+        self.assertEqual(keys(r), ["tilt-right-up"])
+
+    def test_hysteresis_between_ok_and_adjust(self):
+        mid = summary(right=(0, 6.5), left=(0, 6.5))
+        self.assertEqual(fitcoach.assess(mid, ref(), previous={"state": "ok"})["state"], "ok")
+        self.assertEqual(fitcoach.assess(mid, ref(), previous={"state": "adjust"})["state"], "adjust")
+        self.assertEqual(fitcoach.assess(mid, ref())["state"], "adjust")
+        self.assertEqual(fitcoach.assess(mid, ref(), previous={"state": "lost"})["state"], "adjust")
+        # Adjust without a term past ADJUST_DEG: the biggest one's hint, so there's something to do.
+        self.assertEqual(keys(fitcoach.assess(mid, ref())), ["up"])
+        self.assertEqual(keys(fitcoach.assess(mid, ref(), previous={"state": "ok"})), [])
+
+    def test_a_lost_eye_comes_first(self):
+        r = fitcoach.assess(summary(right=(0, 12), left=(0, 12), found=(1.0, 0.3)), ref())
+        self.assertEqual(r["state"], "lost")
+        self.assertEqual(keys(r)[0], "lost")
+        self.assertIn("left eye", r["hints"][0]["text"])
+
+    def test_waiting_without_frames(self):
+        r = fitcoach.assess({0: {"n": 0}, 1: {"n": 0}}, ref())
+        self.assertEqual(r["state"], "waiting")
+        self.assertEqual(fitcoach.assess({}, ref())["state"], "waiting")
+
+    def test_reflections(self):
+        r = fitcoach.assess(summary(extra=(0.0, 0.8)), ref())
+        self.assertEqual(keys(r), ["reflections"])
+        self.assertIn("left", r["hints"][0]["text"])
+
+
+class BaselineTest(unittest.TestCase):
+    def test_against_the_baseline(self):
+        good = summary(right=(4, -3), left=(4, -3))     # this face's good fit isn't the reference's
+        base = fitcoach.make_baseline(good, ipd=0.065)
+        r = fitcoach.assess(good, ref(), ipd=0.065, baseline=base)
+        self.assertEqual((r["state"], r["against"]), ("ok", "baseline"))
+        self.assertFalse(fitcoach.moved(r))
+        # Back on, 7 degrees lower on the face: moved, and which way to put it back.
+        slipped = summary(right=(4, 4), left=(4, 4))
+        r = fitcoach.assess(slipped, ref(), ipd=0.065, baseline=base)
+        self.assertTrue(fitcoach.moved(r))
+        self.assertAlmostEqual(r["terms"]["up"], 7, delta=0.1)
+
+    def test_eye_relief_and_ipd_change(self):
+        base = fitcoach.make_baseline(summary(), ipd=0.065)
+        near = summary(spacing=(14.0 * 1.12, 16.0 * 1.12))
+        r = fitcoach.assess(near, ref(), ipd=0.0665, baseline=base)
+        self.assertEqual(keys(r), ["ipd-changed", "nearer"])
+        self.assertIn("65.0 mm, now 66.5 mm", r["hints"][0]["text"])
+        self.assertTrue(fitcoach.moved(r))
+        far = summary(spacing=(14.0 * 0.9, 16.0 * 0.9))
+        self.assertEqual(keys(fitcoach.assess(far, ref(), ipd=0.065, baseline=base)), ["further"])
+
+
+class ObservationsTest(unittest.TestCase):
+    def test_window_and_medians(self):
+        obs = fitcoach.Observations(window=1.0)
+        # A pair 14 px apart and four stray ones: MANY_GLINTS.
+        glints = [(300.0, 200.0), (301.0, 214.0), (250.0, 120.0), (260.0, 120.0), (270.0, 120.0), (280.0, 120.0)]
+        for i in range(60):
+            t = i / 30
+            obs.add(0, t, {"x": 300 + (i % 3), "y": 210.0, "a": 25.0, "glints": glints})
+            obs.add(1, t, None if i % 2 else {"x": 320.0, "y": 170.0, "a": 30.0, "glints": []})
+        s = obs.summary()
+        self.assertEqual(s[0]["n"], 31)                     # the last second only
+        self.assertEqual(s[0]["x"], 301)
+        self.assertAlmostEqual(s[0]["spacing"], math.hypot(1, 14), places=5)
+        self.assertEqual(s[0]["extra"], 1.0)
+        self.assertAlmostEqual(s[1]["found"], 15 / 31, places=3)
+        self.assertIsNone(s[1]["spacing"])
+
+    def test_glint_spacing_matches_eyes_pupil(self):
+        try:
+            sys.path.insert(0, os.path.join(os.path.dirname(HERE), "tracker"))
+            import eyes_pupil
+        except ImportError:
+            self.skipTest("no numpy or OpenCV here")
+        rng = random.Random(3)
+        for _ in range(200):
+            p = {"x": 300.0, "y": 200.0, "glints": [(rng.uniform(270, 330), rng.uniform(170, 230)) for _ in range(rng.randint(0, 5))]}
+            pair = eyes_pupil.glint_pair(p)
+            want = None if pair is None else math.hypot(pair[1][0] - pair[0][0], pair[1][1] - pair[0][1])
+            got = fitcoach.glint_spacing(p)
+            if want is None:
+                self.assertIsNone(got)
+            else:
+                self.assertAlmostEqual(got, want, places=6)
+
+
+class SlipTest(unittest.TestCase):
+    """The slip notice's reading: ft-eyes' glint jumps (pupil px) as position terms and a hint."""
+
+    def jumps(self, yaw, pitch):
+        # Both pupils moved as if the eyes looked (yaw, pitch): each camera's own direction.
+        return {c: (pupil(c, yaw, pitch)[0] - CENTRE[c][0], pupil(c, yaw, pitch)[1] - CENTRE[c][1]) for c in (0, 1)}
+
+    def test_terms(self):
+        t = fitcoach.slip_terms(self.jumps(2.0, 9.0), ref())
+        self.assertAlmostEqual(t["up"], 9.0, places=6)
+        self.assertAlmostEqual(t["left"], 2.0, places=6)
+        self.assertEqual(fitcoach.slip_terms({}, ref()), {})
+        self.assertEqual(fitcoach.slip_terms({0: (3.0, 4.0)}, None), {})
+
+    def test_hint(self):
+        self.assertEqual(fitcoach.position_hint(fitcoach.slip_terms(self.jumps(0, 9.0), ref()))["key"], "up")
+        self.assertEqual(fitcoach.position_hint(fitcoach.slip_terms(self.jumps(0, -9.0), ref()))["key"], "down")
+        self.assertEqual(fitcoach.position_hint(fitcoach.slip_terms(self.jumps(-7.0, 6.0), ref()))["key"], "right")
+        self.assertIsNone(fitcoach.position_hint(fitcoach.slip_terms(self.jumps(3.0, -4.0), ref())))
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/test/fusion-test.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Offline test of the fusion (gazecal.Fusion, ft-gazed's GAZE_FUSION): SteamVR's affine from
+calibration dots, the clicks' offset per wear, which tracker carries the gaze when, saving and
+loading fusion.json, the setting, and ft-gazed's live gaze with it on. ft-gazed's Service is
+built without its sockets, and HOME is a temp folder, so nothing reaches the live gaze service.
+
+  gaze/test/fusion-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-fusion-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import math  # noqa: E402
+import shutil  # noqa: E402
+import sys  # noqa: E402
+from collections import deque  # noqa: E402
+from pathlib import Path  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+from gazecal import EyeWeights, Fusion, GazeLock  # noqa: E402
+
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(HERE, "..", "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+
+failures = []
+
+
+def check(what, got, want):
+    if got != want:
+        failures.append(what)
+        print(f"FAIL {what}: got {got!r}, want {want!r}", flush=True)
+    else:
+        print(f"ok   {what}", flush=True)
+
+
+def near(a, b, tol=1e-6):
+    return a is not None and b is not None and all(abs(x - y) < tol for x, y in zip(a, b))
+
+
+# --- gazecal.Fusion ---
+f = Fusion()
+check("not fitted: not ready, no SteamVR gaze", (f.ready(), f.steam_gaze((0, 0), (0, 0))), (False, None))
+# SteamVR's eyes' mean reads 1 degree high everywhere: the affine's offset takes it out
+f.fit([(y, p + 1, y, p) for y in (-20, -10, 0, 10, 20) for p in (-10, 0, 10)], {"how": "test"})
+check("fitted on 15 dots: ready", (f.ready(), f.info["dots"], f.info["how"]), (True, 15, "test"))
+check("an offset: taken out", near(f.steam_gaze((14.0, 6.0), (16.0, 6.0)), (15.0, 5.0)), True)
+check("...leaving nothing on its dots", max(f.misses()) < 1e-6, True)
+# ...and 10 % wide too: the gain is taken out as well without the ridge (with it, a little short:
+# on 2026-10-17's runs it fitted other sittings better)
+wide = Fusion()
+wide.RIDGE = 0.0
+wide.fit([(1.1 * y, 1.1 * p + 1, y, p) for y in (-20, -10, 0, 10, 20) for p in (-10, 0, 10)])
+check("a gain, no ridge: taken out", near(wide.steam_gaze((1.1 * 15 - 1, 1.1 * 5 + 1), (1.1 * 15 + 1, 1.1 * 5 + 1)),
+                                         (15.0, 5.0), 1e-6), True)
+wide.RIDGE = Fusion.RIDGE
+wide.fit(wide.points)
+check("with the ridge: most of it", 0.0 < max(wide.misses()) < 0.6, True)
+check("one SteamVR eye: no gaze from it", f.steam_gaze((1.0, 2.0), None), None)
+check("the mean of the eyes", Fusion.mean((0.0, 2.0), (2.0, 4.0)), (1.0, 3.0))
+
+# Who carries the gaze
+ours, steam = (2.0, 0.0), (0.0, 2.0)
+check("both our eyes and SteamVR's: half and half", f.combine(ours, 2, steam), (1.0, 1.0))
+check("one of our eyes: SteamVR's", f.combine(ours, 1, steam), steam)
+check("none of ours: SteamVR's", f.combine(None, 0, steam), steam)
+check("no SteamVR gaze: ours", f.combine(ours, 1, None), ours)
+check("a share of 1: ours where both are found", f.combine(ours, 2, steam, share=1.0), ours)
+
+# The clicks' offset: per wear, a median, stale clicks left out
+g = Fusion()
+g.fit([(y, p, y, p - 1) for y in (-10, 0, 10) for p in (-10, 0, 10)])   # 1 degree low... corrected
+g.wear = 100.0
+check("no clicks: no offset", g.offset(), (0.0, 0.0))
+check("a click 0.5 off: its miss", round(g.click((0.0, 0.0), (0.0, 0.0), 0.5, -1.0), 3), 0.5)
+check("...and it's the offset", near(g.offset(), (0.5, 0.0)), True)
+check("a click 20 degrees off (a look elsewhere): not taken", (round(g.click((0, 0), (0, 0), 20.0, -1.0), 1),
+                                                              near(g.offset(), (0.5, 0.0))), (20.0, True))
+g.wear = 200.0
+check("the headset on again: the old wear's clicks don't count", g.offset(), (0.0, 0.0))
+check("steam_gaze without the clicks", near(g.steam_gaze((3.0, 3.0), (3.0, 3.0), clicks=False), (3.0, 2.0)), True)
+
+# Saving and loading
+path = Path(HOME) / "fusion.json"
+f.save(path)
+h = Fusion.load(path)
+check("saved and loaded: the same correction", near(h.steam_gaze((5.0, 5.0), (7.0, 5.0)), f.steam_gaze((5.0, 5.0), (7.0, 5.0))),
+      True)
+check("...and its info", (h.info["how"], h.info["dots"], len(h.points)), ("test", 15, 15))
+check("a missing file: None", Fusion.load(Path(HOME) / "none.json"), None)
+path.write_text("{ broken")
+check("a broken file: None", Fusion.load(path), None)
+
+# --- GAZE_FUSION ---
+conf = Path(HOME) / ".config" / "frametop.conf"
+conf.parent.mkdir(parents=True, exist_ok=True)
+check("no frametop.conf: off", gazed.read_fusion(), False)
+for text, want in (("GAZE_FUSION=on\n", True), ("GAZE_FUSION = ON  # yes\n", True), ("GAZE_FUSION=off\n", False),
+                   ("GAZE_FUSION=yes\n", False), ("# GAZE_FUSION=on\n", False), ("GAZE_TRACKER=own\n", False)):
+    conf.write_text(text)
+    check(f"frametop.conf {text.strip()!r}", gazed.read_fusion(), want)
+
+
+# --- ft-gazed's live gaze: the parts of Service the samples go through, no sockets ---
+def service(fusion_on=True, one=None):
+    svc = gazed.Service.__new__(gazed.Service)
+    svc.override, svc.tracker, svc.source, svc.one_eye = None, "own", "mmap1", one
+    svc.counts = dict.fromkeys(("samples", "sent", "blinks", "one_eye", "one_eye_used", "lost_left", "lost_right",
+                                "looking_down", "dropped", "fused", "steam_only"), 0)
+    svc.opens, svc.lost = (deque(maxlen=90), deque(maxlen=90)), [False, False]
+    svc.fix, svc.last_sample, svc.low_until = GazeLock(radius=1.0), 0.0, [0.0, 0.0]
+    svc.bridge = gazed.EyeBridge()
+    svc.weights = {"own": EyeWeights()}
+    svc.fusion_on = fusion_on
+    svc.fusion = Fusion()
+    svc.fusion.fit([(y, p + 1, y, p) for y in (-10, 0, 10) for p in (-10, 0, 10)])   # SteamVR 1 degree high
+    svc.checks = type("Checks", (), {"active": False, "pending": None})()
+    svc.sent = []
+    svc.send = lambda t, hy, hp, rhy, rhp, eyes: svc.sent.append((round(hy, 3), round(hp, 3), eyes))
+    return svc
+
+
+def feed(svc, n=30, own_eyes=((4.0, 2.0), (4.0, 2.0)), steam=True, opens=(0.8, 0.8), unc=(0.001, 0.001)):
+    for i in range(n):
+        src = {"mmap1": {"unc": list(unc), "open": list(opens)},
+               "own": {"hy": 4.0, "hp": 2.0, "eyes": [list(e) if e else None for e in own_eyes]}}
+        if steam:
+            src["left"], src["right"] = {"hy": 1.0, "hp": 3.0}, {"hy": 3.0, "hp": 3.0}   # mean (2, 3): corrected (2, 2)
+        svc.on_eyes_sample({"t": i / 90, "src": src}, True)
+
+
+svc = service()
+check("fusing with our tracker, on, and fitted", svc.fusing, True)
+check("ft-gaze prints SteamVR's eyes too", svc.wanted_sources(), "own,mmap1,left,right")
+feed(svc)
+check("both our eyes: half ours (4, 2), half SteamVR's corrected (2, 2)", svc.sent[-1][:2], (3.0, 2.0))
+check("...counted as fused", (svc.counts["fused"], svc.counts["steam_only"]), (30, 0))
+check("...and the eyes sent with it are ours (for lessons)", svc.sent[-1][2], [(4.0, 2.0), (4.0, 2.0)])
+svc = service()
+feed(svc, own_eyes=((4.0, 2.0), None))
+check("one of our eyes: SteamVR's", svc.sent[-1][:2], (2.0, 2.0))
+check("...counted", (svc.counts["steam_only"], svc.counts["one_eye"]), (30, 30))
+svc = service()
+feed(svc, own_eyes=(None, None))
+check("neither of ours: SteamVR's carries it", (len(svc.sent), svc.sent[-1][:2] if svc.sent else None), (30, (2.0, 2.0)))
+svc = service()
+feed(svc, steam=False)
+check("SteamVR's eyes missing: ours", svc.sent[-1][:2], (4.0, 2.0))
+svc = service()
+feed(svc, n=60, unc=(0.02, 0.001))
+check("SteamVR lost its left eye: ours", svc.sent[-1][:2], (4.0, 2.0))
+svc = service()
+feed(svc, n=60, opens=(0.0, 0.0))
+check("a blink (both lids down): nothing", (svc.sent, svc.counts["blinks"]), ([], 60))
+svc = service(one=1)
+feed(svc)
+check("SteamVR tracking one eye: no fusion", (svc.sent[-1][:2], svc.counts["fused"]), ((4.0, 2.0), 0))
+svc = service(fusion_on=False)
+check("GAZE_FUSION off: not fusing, SteamVR's eyes not asked for", (svc.fusing, svc.wanted_sources()),
+      (False, "own,mmap1"))
+feed(svc)
+check("...ours alone", svc.sent[-1][:2], (4.0, 2.0))
+svc = service()
+svc.fusion = Fusion()
+check("no fitted fusion: not fusing", svc.fusing, False)
+svc = service()
+svc.tracker = "steam"
+svc.models = {name: gazed.Correction() for name in gazed.SOURCES}
+check("SteamVR's tracker: no fusion", svc.fusing, False)
+
+shutil.rmtree(HOME, ignore_errors=True)
+print("FAILED: " + ", ".join(failures) if failures else "all passed")
+sys.exit(1 if failures else 0)

gaze/test/gaze-lock-test.py

@@ -0,0 +1,180 @@
+#!/usr/bin/env python3
+"""Offline test of the fixation lock on the gaze's direction in the room (gazecal.GazeLock) and
+of ft-gazed's blink settle for our tracker (BLINK_SETTLE). Made-up samples only: ft-gazed's
+Service is built without its sockets, so nothing reaches the live gaze service.
+
+  gaze/test/gaze-lock-test.py
+"""
+import importlib.machinery
+import importlib.util
+import math
+import os
+import random
+import sys
+from collections import deque
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+from gazecal import EyeWeights, Fixation, GazeLock, angles, direction, rotate  # noqa: E402
+
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(HERE, "..", "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+
+failures = []
+
+
+def check(what, ok, detail=""):
+    if not ok:
+        failures.append(what)
+        print(f"FAIL {what} {detail}", flush=True)
+
+
+def head(yaw, pitch=0.0, roll=0.0):
+    """A head pose's quaternion [w, x, y, z]: yaw about +Y (left), then pitch about +X (up), then
+    roll about -Z, as a headset's pose matrix would give."""
+    def q(axis, deg):
+        h = math.radians(deg) / 2
+        return [math.cos(h)] + [math.sin(h) * a for a in axis]
+
+    def mul(a, b):
+        w1, x1, y1, z1 = a
+        w2, x2, y2, z2 = b
+        return [w1 * w2 - x1 * x2 - y1 * y2 - z1 * z2, w1 * x2 + x1 * w2 + y1 * z2 - z1 * y2,
+                w1 * y2 - x1 * z2 + y1 * w2 + z1 * x2, w1 * z2 + x1 * y2 - y1 * x2 + z1 * w2]
+    return mul(mul(q((0, 1, 0), yaw), q((1, 0, 0), pitch)), q((0, 0, -1), roll))
+
+
+def room(q, yaw, pitch):
+    """Where a head-relative gaze points in the room, as (yaw, pitch)."""
+    return angles(rotate(q, direction(yaw, pitch)))
+
+
+def head_rel(q, ryaw, rpitch):
+    return angles(rotate(q, direction(ryaw, rpitch), inverse=True))
+
+
+def off(a, b):
+    return math.degrees(math.acos(max(-1.0, min(1.0, sum(x * y for x, y in zip(direction(*a), direction(*b)))))))
+
+
+# --- The conventions: ft-gaze's Direction and Angles, and the quaternion as a pose rotation
+check("straight ahead is -Z", direction(0, 0) == (-0.0, 0.0, -1.0), direction(0, 0))
+check("yaw turns left (-X), pitch looks up", direction(90, 0)[0] < -0.99 and direction(0, 90)[1] > 0.99)
+rng = random.Random(1)
+worst = 0.0
+for _ in range(500):
+    q = head(rng.uniform(-180, 180), rng.uniform(-60, 60), rng.uniform(-30, 30))
+    y, p = rng.uniform(-50, 50), rng.uniform(-40, 40)
+    back = head_rel(q, *room(q, y, p))
+    worst = max(worst, abs(back[0] - y), abs(back[1] - p))
+check("into the room and back", worst < 1e-9, worst)
+check("a head turned 30 deg left, eyes ahead: the room's 30 left", max(map(abs, (room(head(30), 0, 0)[0] - 30,
+                                                                              room(head(30), 0, 0)[1]))) < 1e-9)
+check("a head pitched 10 up, eyes 5 up: the room's 15 up", abs(room(head(0, 10), 0, 5)[1] - 15) < 1e-9)
+
+# --- A look at a spot in the room while the head turns (the eyes turn the other way)
+def turning(speed, lock, poses=True, noise=0.05, seconds=2.0):
+    """The pointer's place in the room (drawn with the newest head, as the helper does) while the
+    eyes hold room yaw 10, pitch 5 and the head turns at `speed` deg/s: the worst miss, and the
+    jumps over 0.5 deg between samples."""
+    rng = random.Random(2)
+    worst, jumps, last = 0.0, 0, None
+    for i in range(int(90 * seconds)):
+        t = i / 90
+        q = head(-20 + speed * t, 3.0, 2.0)
+        hy, hp = head_rel(q, 10.0 + rng.gauss(0, noise), 5.0 + rng.gauss(0, noise))
+        if isinstance(lock, GazeLock):
+            out = lock(hy, hp, t, q if poses else None, q if poses else None)
+        else:
+            out = lock(hy, hp, t, 1.0)
+        at = room(q, *out)
+        if i >= 3:
+            worst = max(worst, off(at, (10.0, 5.0)))
+            jumps += last is not None and off(at, last) > 0.5
+        last = at
+    return worst, jumps
+
+
+w, j = turning(20, GazeLock())
+check("room lock, head turning 20 deg/s: the pointer stays on the spot", w < 0.3 and j == 0, (w, j))
+w, j = turning(60, GazeLock())
+check("room lock, head turning 60 deg/s: still on it", w < 0.3 and j == 0, (w, j))
+w, j = turning(20, Fixation(1.0))
+check("the head-relative lock rides along and snaps back (why the room lock)", w > 0.8 and j >= 10, (w, j))
+w, j = turning(20, GazeLock(), poses=False)
+check("no poses (an older ft-gaze): the head-relative lock", w > 0.8 and j >= 10, (w, j))
+
+# A look elsewhere moves it at once (two samples), and a still head is as before
+lock = GazeLock()
+q = head(15, -5, 1)
+for i in range(30):
+    out = lock(2.0, 1.0, i / 90, q, q)
+check("still: the look", off(out, (2.0, 1.0)) < 1e-6, out)
+lock(20.0, 1.0, 30 / 90, q, q)
+out = lock(20.0, 1.0, 31 / 90, q, q)
+check("a saccade: there after two samples", off(out, (20.0, 1.0)) < 1e-6, out)
+one = lock(2.0, 1.0, 32 / 90, q, q)
+check("one stray sample is held off", off(one, (20.0, 1.0)) < 1e-6, one)
+fix, lock2 = Fixation(1.0), GazeLock()
+same = 0.0
+for i in range(200):
+    y, p = 3 + 0.3 * math.sin(i / 7), -2 + 0.3 * math.cos(i / 5)
+    a, b = fix(y, p, i / 90, 1.0), lock2(y, p, i / 90, q, q)
+    same = max(same, off(a, b))
+check("a still head: as the head-relative lock", same < 0.02, same)
+lock2(3, -2, 3.0, q, q)
+lock2(3, -2, 3.1, None, None)
+check("switching to no poses starts over", lock2.count == 0 and lock2.head.count == 1)
+# Newest pose differs from the sample's: the room direction is kept, re-expressed in the newest head
+lock3 = GazeLock()
+then, now = head(0), head(4)
+out = lock3(10.0, 0.0, 0.0, then, now)
+check("drawn with the newest head", abs(out[0] - 6.0) < 1e-9 and abs(out[1]) < 1e-9, out)
+
+
+# --- ft-gazed's own path: an eye stays out for BLINK_SETTLE after its lids came down
+def service():
+    svc = gazed.Service.__new__(gazed.Service)
+    svc.override, svc.tracker, svc.source, svc.one_eye = None, "own", "mmap1", None
+    svc.counts = dict.fromkeys(("samples", "sent", "blinks", "one_eye", "one_eye_used", "lost_left", "lost_right",
+                                "looking_down", "dropped"), 0)
+    svc.opens, svc.lost = (deque(maxlen=90), deque(maxlen=90)), [False, False]
+    svc.fix, svc.last_sample, svc.low_until = GazeLock(radius=1.0), 0.0, [0.0, 0.0]
+    svc.bridge = gazed.EyeBridge()
+    svc.weights = {"own": EyeWeights()}
+    svc.sent = []
+    svc.send = lambda t, hy, hp, rhy, rhp, eyes: svc.sent.append((t, hy, hp))
+    return svc
+
+
+def blink(settle):
+    """The right eye's lids down for 3 samples, and its pupil found wrong (30, -20) for 3 more
+    after they're up: the gaze sent from 60 on."""
+    saved = gazed.BLINK_SETTLE
+    gazed.BLINK_SETTLE = settle
+    try:
+        svc = service()
+        q = head(0)
+        for i in range(120):
+            closing = 60 <= i < 63
+            right = [30.0, -20.0] if 60 <= i < 66 else [6.0, 2.0]
+            svc.on_eyes_sample({"t": i / 90, "head": {"q": q}, "src": {
+                "mmap1": {"unc": [0.001, 0.001], "open": [0.8, 0.0 if closing else 0.8]},
+                "own": {"hy": 5.0, "hp": 2.0, "hq": q, "eyes": [[4.0, 2.0], right]}}}, True)
+    finally:
+        gazed.BLINK_SETTLE = saved
+    return svc, [s for s in svc.sent if s[0] >= 60 / 90]
+
+
+svc, sent = blink(gazed.BLINK_SETTLE)
+worst = max(off(s[1:], (5.0, 2.0)) for s in sent)
+check("settle: the bad finds after the lids came up don't reach the pointer", worst < 1.1, worst)
+check("settle: then both eyes again", off(sent[-1][1:], (5.0, 2.0)) < 0.2, sent[-1])
+check("settle: counted as one-eye samples", svc.counts["one_eye"] >= 6, svc.counts)
+_, sent = blink(0.0)
+worst = max(off(s[1:], (5.0, 2.0)) for s in sent)
+check("without it, they do (the test's control)", worst > 5, worst)
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed")
+sys.exit(1 if failures else 0)

gaze/test/idle-test.py

@@ -34,6 +34,7 @@ gazecheck.SCREENS = f"\0{tag}_screens"
 gazecheck.PANEL_PROG = gazed.REPO / "nonexistent-panel"  # "isn't built": no panel
 gazed.IDLE_AFTER, gazed.WAKE_SETTLE = 1.0, 3.0
 gazed.read_settings = lambda: ("steam", "steam", "auto", 55.0)
+gazed.TrackedEye = lambda: lambda now=None: None  # both eyes, whatever SteamVR's settings say
 logs = []
 gazed.log = gazecheck.log = lambda msg: logs.append(msg)
 

gaze/test/looked-away-test.py

@@ -0,0 +1,148 @@
+#!/usr/bin/env python3
+"""Offline test of the full calibration's looked-away check (gaze/gazecheck.py, Checks.looked_away):
+a dot for our tracker where SteamVR's two eyes were more than LOOKED_AWAY from it, past their
+usual offset in that calibration, isn't taken, and ft-eyes isn't asked. Live 2026-10-18: the
+palm menu opened the Steam menu mid-calibration, and three dots were taken 17-35 degrees from
+where the user looked (both trackers agreed). A SteamVR that is off by itself (its eyes in one
+place whatever the dot) mustn't block the calibration: past AWAY_GIVE_UP dots in a row, the
+check stops.
+
+Drives Checks.capture with synthetic looks and a fake service: nothing reaches the live gaze
+service, the pointer helper, ft-eyes, or SteamVR. HOME is a temp folder (checks.jsonl goes there).
+
+  gaze/test/looked-away-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-looked-away-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE follows HOME
+
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import sys  # noqa: E402
+import types  # noqa: E402
+
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
+import gazecheck  # noqa: E402
+
+tag = f"ft_gaze_looked_away_test_{os.getpid()}"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.EYES = f"\0{tag}_eyes"
+logs = []
+gazecheck.log = lambda msg: logs.append(msg)
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(f"{'ok  ' if ok else 'FAIL'}  {label}" + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+class Weights:
+    def add(self, miss):
+        pass
+
+
+svc = types.SimpleNamespace(one_eye=None, weights={"own": Weights(), "steam": Weights()})
+checks = gazecheck.Checks(svc, selectors.DefaultSelector())
+asked = []
+checks.ask_eyes = lambda command, timeout, done: asked.append(command)
+DOTS = gazecheck.check_dots("full", True)
+
+
+def new_check():
+    checks.check = {"kind": "full", "own": True, "dots": DOTS, "i": 0, "tries": 0, "skipped": 0, "captured": 0,
+                    "points": {}, "reasons": {}, "fit_reasons": set(), "note": "", "blind": False, "fusion": [],
+                    "away_dots": [], "away_off": False, "steam_offsets": []}
+
+
+def look(at, steam_off=(0.0, 0.0), n=30):
+    """A look at `at` (yaw, pitch) by SteamVR's eyes (each this far from it, plus steam_off), our
+    tracker's at the dot: the capture window as on_sample keeps it."""
+    y, p = at[0] + steam_off[0], at[1] + steam_off[1]
+    src = {"mmap1": {"unc": [0.001, 0.001], "open": [0.8, 0.8]}, "left": {"hy": y - 1.0, "hp": p},
+           "right": {"hy": y + 1.0, "hp": p}, "own": {"ok": 1, "eyes": [list(at), list(at)]}}
+    return [(0.0, {"t": 100.0 + k / 90, "src": src}, y, p) for k in range(n)]
+
+
+def capture(i, at, steam_off=(0.0, 0.0)):
+    """Dot i shown, a click while the eyes are at `at`: was ft-eyes asked (the check let it go on)?"""
+    c = checks.check
+    c["i"], c["done_at"] = i, None
+    before = len(asked)
+    checks.capture(look(at, steam_off))
+    return len(asked) > before
+
+
+def answer_ok():
+    """ft-eyes takes the dot it was asked about (point_done's own path, with "ok")."""
+    c = checks.check
+    yaw, pitch, rnd = c["dots"][c["i"]]
+    rec = {"true": [yaw, pitch], "steam": [(yaw, pitch), (yaw, pitch)], "own": True}
+    checks.point_done(rec, [0.0, 0.0], "ok")
+
+
+# Looking at the dots: each taken, as before.
+new_check()
+for i in range(3):
+    check(f"dot {i + 1}: looking at it, ft-eyes is asked", capture(i, DOTS[i][:2]), True)
+    answer_ok()
+check("no notes for those", checks.check["note"], "")
+
+# Looking 25 degrees off (the Steam menu, mid-calibration): not taken, and the panel says why.
+dot = DOTS[3][:2]
+check("dot 4: looked 25 degrees away: ft-eyes isn't asked", capture(3, dot, (25.0, 0.0)), False)
+check("...the panel says why", "looked away" in checks.check["note"], True)
+check("...and the dot stays for another try", (checks.check["i"], checks.check["tries"]), (3, 1))
+check("...the reason is counted", checks.check["reasons"], {"you looked away from the dot (by SteamVR's eyes)": 1})
+check("dot 4, second try, looking at it: taken", capture(3, dot), True)
+answer_ok()
+
+# 14 degrees off: under LOOKED_AWAY, taken (SteamVR's own error at the corners reached 9 live).
+check("dot 5: 14 degrees off: taken", capture(4, DOTS[4][:2], (0.0, 14.0)), True)
+answer_ok()
+
+# A SteamVR that is off by itself, 20 degrees at every dot: the usual offset counts once it has
+# AWAY_OFFSET_MIN dots, so every dot is taken.
+new_check()
+for i in range(gazecheck.AWAY_OFFSET_MIN):
+    capture(i, DOTS[i][:2], (3.0, 0.0))   # a small offset first: these are taken
+    answer_ok()
+check("its usual offset in this calibration is learned", len(checks.check["steam_offsets"]), gazecheck.AWAY_OFFSET_MIN)
+check("then 18.5 degrees the same way: 15.5 past the usual 3, refused",
+      capture(3, DOTS[3][:2], (18.5, 0.0)), False)
+check("and 14 degrees the same way: taken (11 past the usual 3)", capture(3, DOTS[3][:2], (14.0, 0.0)), True)
+answer_ok()
+
+# A SteamVR off by 20 degrees from the start: the first dots are refused, then the check stops
+# (AWAY_GIVE_UP dots in a row) and the rest are taken.
+new_check()
+logs.clear()
+check("SteamVR 20 degrees off, dot 1: refused", capture(0, DOTS[0][:2], (20.0, 0.0)), False)
+check("dot 1, second try: refused, and skipped", (capture(0, DOTS[0][:2], (20.0, 0.0)), checks.check["skipped"]),
+      (False, 1))
+check("dot 2: refused", capture(1, DOTS[1][:2], (20.0, 0.0)), False)
+check("dot 2 again: refused, and skipped", (capture(1, DOTS[1][:2], (20.0, 0.0)), checks.check["skipped"]), (False, 2))
+check("dot 3: a third dot in a row: SteamVR is off, taken", capture(2, DOTS[2][:2], (20.0, 0.0)), True)
+check("...and the log says the check is off", any("looked-away check is off" in m for m in logs), True)
+answer_ok()
+check("dot 4: taken (the check stays off)", capture(3, DOTS[3][:2], (20.0, 0.0)), True)
+
+# Our tracker alone without SteamVR's eyes (one eye tracked, or SteamVR lost them): no check.
+new_check()
+svc.one_eye = 0
+check("one eye tracked by SteamVR: no check, 30 degrees off is taken", capture(0, DOTS[0][:2], (30.0, 0.0)), True)
+svc.one_eye = None
+
+# The reason's words, for the note and a failed calibration's status.
+check("reject_reason for it", gazecheck.reject_reason("looked away (25 deg by SteamVR)", None),
+      ("looked away", "you looked away from the dot (by SteamVR's eyes)", False))
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+shutil.rmtree(HOME, ignore_errors=True)
+sys.exit(1 if failures else 0)

gaze/test/mmap-layout-test.cpp

@@ -19,7 +19,7 @@ int failures = 0;
     } while (0)
 
 // The file as the eye server writes it, one sample at a time, with every field from the
-// timestamp on moved by `shift` (0 stable, 5 the 0.4.x beta).
+// timestamp on moved by `shift` (0 on SteamOS 0.3, 5 on 0.4).
 struct File {
     std::vector<uint8_t> bytes = std::vector<uint8_t>(324122);  // eye-server.mmap's size
     EyeFile eyes;
@@ -162,6 +162,40 @@ void NoMapping() {
     CHECK(f.eyes.Counter() == f.n);
 }
 
+// QuatJson (the head poses printed for the gaze service's room lock): for poses turned every
+// way, including past 90 degrees where each of its four branches is taken, the quaternion turns
+// vectors as the pose matrix does.
+void Quaternions() {
+    auto axis = [](Vec3 a, double deg) {  // a rotation matrix about unit axis a
+        vr::HmdMatrix34_t m{};
+        const Vec3 cols[3] = {RotateAbout({1, 0, 0}, a, deg * M_PI / 180), RotateAbout({0, 1, 0}, a, deg * M_PI / 180),
+                              RotateAbout({0, 0, 1}, a, deg * M_PI / 180)};
+        for (int c = 0; c < 3; ++c) m.m[0][c] = cols[c].x, m.m[1][c] = cols[c].y, m.m[2][c] = cols[c].z;
+        return m;
+    };
+    double worst = 0;
+    int n = 0;
+    for (int ax = 0; ax < 7; ++ax) {
+        const Vec3 a = Normalize(Vec3{double(ax & 1), double((ax >> 1) & 1), double((ax >> 2) & 1)} +
+                                 Vec3{0.1 * ax, -0.07 * ax, 0.05});
+        for (double deg = -179; deg <= 180; deg += 23) {
+            const vr::HmdMatrix34_t m = axis(a, deg);
+            double q[4];
+            CHECK(std::sscanf(QuatJson(m).c_str(), "[%lf,%lf,%lf,%lf]", &q[0], &q[1], &q[2], &q[3]) == 4);
+            const Vec3 u{q[1], q[2], q[3]};
+            for (const Vec3 v : {Vec3{0, 0, -1}, Vec3{1, 0, 0}, Vec3{0.3, 0.8, -0.5}}) {
+                const Vec3 c = Cross(u, v);
+                const Vec3 byQ = v + c * (2 * q[0]) + Cross(u, c) * 2;
+                worst = std::max(worst, Length(byQ - Rotate(m, v)));
+            }
+            ++n;
+        }
+    }
+    CHECK(n > 100);
+    CHECK(worst < 1e-5);
+    if (worst >= 1e-5) std::printf("quaternion off by %g\n", worst);
+}
+
 }  // namespace
 
 int main() {
@@ -171,6 +205,7 @@ int main() {
     TornWrite();
     ReadsTheLayoutFound();
     NoMapping();
+    Quaternions();
     if (failures) {
         std::printf("%d failed\n", failures);
         return 1;

gaze/test/one-eye-calibration-test.py

@@ -0,0 +1,254 @@
+#!/usr/bin/env python3
+"""Offline test of our tracker with one eye lost (2026-10-18): a calibration dot is taken with
+the eye that was seen when the other wasn't (ft-eyes calib-point), each eye is fitted on its
+own dots and an eye seen at too few is left out (calib-fit, eyes_model.Calibration.fit), and
+ft-gazed eases the step when an eye drops out (gazecal.EyeBridge). A second person's eyes each
+lost the pupil looking 20 degrees outward, so until now their outer dots failed, and the rest
+fitted both eyes from the middle only.
+
+Pupil positions are written into ft-eyes' eye histories as its frames would leave them: no
+cameras, SteamVR or live gaze service. Needs the tracker's Python (numpy, OpenCV).
+
+  scripts/in-box /opt/frametop/venv/bin/python gaze/test/one-eye-calibration-test.py
+"""
+import importlib.machinery
+import importlib.util
+import json
+import math
+import os
+import shutil
+import sys
+import tempfile
+from pathlib import Path
+
+HOME = tempfile.mkdtemp(prefix="ft-eyes-one-eye-test-")
+os.environ.update(HOME=HOME, FT_EYES_CAMS="/nonexistent", FT_EYES_STATE=os.path.join(HOME, "eyes"),
+                  FT_EYES_MODEL="classic")
+GAZE = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(GAZE))
+sys.path.insert(0, str(GAZE / "tracker"))
+
+import gazecheck  # noqa: E402
+from gazecal import EyeBridge, EyeWeights, GazeLock  # noqa: E402
+
+loader = importlib.machinery.SourceFileLoader("fteyes", str(GAZE / "tracker" / "ft-eyes"))
+fe = importlib.util.module_from_spec(importlib.util.spec_from_loader(loader.name, loader))
+loader.exec_module(fe)
+loader = importlib.machinery.SourceFileLoader("ftgazed", str(GAZE / "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader(loader.name, loader))
+loader.exec_module(gazed)
+
+failures = []
+
+
+def check(what, got, want):
+    ok = got == want
+    print(f"{'ok  ' if ok else 'FAIL'}  {what}" + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(what)
+
+
+DOTS = [d[:2] for d in gazecheck.check_dots("full", True)]   # 27 dots, out to +-20
+CENTRE = {0: (300.0, 200.0), 1: (330.0, 205.0)}               # right, left: pupil looking ahead
+
+
+def pupil(eye, yaw, pitch):
+    """Where an eye's pupil sits for a gaze: 4 px a degree, yaw +left moves it to lower x."""
+    x0, y0 = CENTRE[eye]
+    return x0 - 4.0 * yaw + 0.02 * yaw * pitch, y0 - 4.0 * pitch
+
+
+class Clock:
+    t = 100.0
+
+
+def look(tracker, yaw, pitch, seen=(0, 1), jitter=()):
+    """Half a second at a dot: 45 frames per eye that's seen (jitter: those that jump about),
+    then calib-point over that window."""
+    t0 = Clock.t
+    for i in range(45):
+        t = t0 + i / 90
+        for e in tracker.eyes:
+            if e.eye not in seen:
+                continue
+            x, y = pupil(e.eye, yaw, pitch)
+            if e.eye in jitter:
+                x += 0.5 * (i - 22)   # drifting: half its frames over 5.5 px from the middle
+            e.history.append((t, x, y, math.nan, math.nan))
+    Clock.t = t0 + 1.0
+    return tracker.command(f"calib-point {t0:.6f} {t0 + 0.5:.6f} {yaw:.4f} {pitch:.4f}")
+
+
+def outward_lost(yaw):
+    """The second person: the right eye lost looking 14+ degrees right (yaw -), the left 14+ left."""
+    return tuple(e for e in (0, 1) if not (e == 0 and yaw <= -14 or e == 1 and yaw >= 14))
+
+
+# --- calib-point: one eye is enough for a dot ---
+tracker = fe.Tracker()
+check("calib-start", tracker.command("calib-start"), "ok")
+r = look(tracker, 0.0, 0.0)
+check("both eyes seen: taken", r.split()[:3], ["ok", "45", "45"])
+check("...with no note", "only" in r, False)
+r = look(tracker, -20.0, 0.0, seen=(1,))
+check("right eye lost: taken with the left", r.startswith("ok 0 45"), True)
+check("...and says so", r.endswith("(left eye only: the right eye was seen in only 0 frames)"), True)
+check("...the dot holds the left eye alone", sorted(tracker.dots[-1]["eye"]), [1])
+r = look(tracker, 10.0, 0.0, jitter=(1,))
+check("left eye jumping about: taken with the right", "(right eye only: the left eye moved" in r, True)
+check("...the dot holds the right eye alone", sorted(tracker.dots[-1]["eye"]), [0])
+n = len(tracker.dots)
+r = look(tracker, 0.0, 10.0, seen=())
+check("neither eye seen: not taken", r, "fail neither eye was seen enough (right 0 frames, left 0 frames)")
+r = look(tracker, 0.0, 10.0, jitter=(0, 1))
+check("both moved: not taken, as before", r, "fail the right eye moved (5.5 px)")
+r = look(tracker, 0.0, 10.0, seen=(1,), jitter=(1,))
+check("one lost and the other moved: the move is the reason", r, "fail the left eye moved (5.5 px)")
+check("...none of those kept", len(tracker.dots), n)
+
+# The panel's words for them (gazecheck.reject_reason).
+check("reject_reason: neither eye", gazecheck.reject_reason(
+    "fail neither eye was seen enough (right 3 frames, left 0 frames)", None),
+    ("eyes not seen", "our tracker saw neither eye well enough", True))
+check("reject_reason: one eye moved, as before", gazecheck.reject_reason("fail the left eye moved (12.0 px)", None),
+      ("left eye moved", "your left eye moved while you looked", False))
+
+# --- calib-fit: a full calibration where each eye is lost looking outward ---
+tracker = fe.Tracker()
+tracker.command("calib-start")
+for yaw, pitch in DOTS:
+    look(tracker, yaw, pitch, seen=outward_lost(yaw))
+seen = {c: sum(1 for k in tracker.dots if c in k["eye"]) for c in (0, 1)}
+check("all 27 dots taken", len(tracker.dots), 27)
+check("each eye at the dots it was seen at", seen, {0: 22, 1: 22})
+r = tracker.command("calib-fit")
+check("fitted", r.startswith("ok 27 dots, fit median 0.0"), True)
+check("...each eye's dots in the reply", r.endswith("(22 dots)") and "(22 dots), left 0.0" in r, True)
+check("both eyes mapped", tracker.status()["mapped"], ["right", "left"])
+check("...the dots per eye saved with it", tracker.cal.info["eye_dots"], {"right": 22, "left": 22})
+right, left = tracker.eyes
+check("each eye calibrated", (right.cal is not None, left.cal is not None), (True, True))
+g = tracker.cal.gaze(1, *pupil(1, -20.0, 0.0))
+check("the left eye at 20 right (only it saw that dot): on it", (round(g[0], 1), round(g[1], 1)), (-20.0, 0.0))
+rows = [json.loads(x) for x in open(fe.CALIBRATION.with_name("calibration-dots.jsonl"))]
+check("calibration-dots.jsonl keeps the one-eye dots", sorted(len(x["eye"]) for x in rows)[:4], [1, 1, 1, 1])
+check("...and its spreads come back from them", sorted(fe.spread_from_dots(tracker.cal)), [0, 1])
+loaded = fe.eyes_model.Calibration.load(fe.CALIBRATION)
+check("the saved calibration maps both", loaded.eyes(), [0, 1])
+
+# --- calib-fit with one eye seen at too few dots: the other alone ---
+tracker = fe.Tracker()
+tracker.command("calib-start")
+for i, (yaw, pitch) in enumerate(DOTS):
+    look(tracker, yaw, pitch, seen=(1,) if i % 3 else (0, 1))   # the right eye at 9 of 27
+r = tracker.command("calib-fit")
+check("right eye at 9 dots: fitted with the left alone", r.endswith("(27 dots)")
+      and "right not used (9 dots, needs 12)" in r, True)
+check("...the left mapped only", tracker.status()["mapped"], ["left"])
+right, left = tracker.eyes
+check("...the right eye gives no gaze", (right.cal, left.cal is not None), (None, True))
+check("...a click teaches the left only", tracker.command(f"click {Clock.t:.6f} 0 0").split()[1:3], ["nan", "nan"])
+fresh = fe.Tracker()
+check("...and a restart says so", (fresh.cal.eyes(), fresh.eyes[0].cal), ([1], None))
+
+# --- calib-fit with too few dots for either eye ---
+tracker = fe.Tracker()
+tracker.command("calib-start")
+for i, (yaw, pitch) in enumerate(DOTS[:14]):
+    look(tracker, yaw, pitch, seen=(0,) if i % 2 else (1,))
+r = tracker.command("calib-fit")
+check("7 dots each: no calibration", r, "fail neither eye was seen at enough dots (right 7, left 7; need 12)")
+check("...the last one stays", fe.eyes_model.Calibration.load(fe.CALIBRATION).eyes(), [1])
+
+# --- gazecal.EyeBridge: no jump when an eye drops out ---
+b = EyeBridge()
+for i in range(90):   # both eyes, a degree apart: their mean is the gaze
+    out = b(i / 90, [(4.0, 2.0), (6.0, 2.0)], (5.0, 2.0))
+check("both eyes: the combined gaze", out, (5.0, 2.0))
+t = 1.0
+out = b(t, [None, (6.0, 2.0)], (6.0, 2.0))
+check("the left drops out: the gaze stays put", (abs(out[0] - 5.0) < 0.05, out[1]), (True, 2.0))
+out = b(t + 0.3, [None, (6.0, 2.0)], (6.0, 2.0))
+check("...eases toward the right eye", 5.5 < out[0] < 5.8, True)
+out = b(t + 2.0, [None, (6.0, 2.0)], (6.0, 2.0))
+check("...and is the right eye's after 2 s", round(out[0], 2), 6.0)
+out = b(t + 2.1, [(4.0, 2.0), (6.0, 2.0)], (5.0, 2.0))
+check("both back: the combined gaze", out, (5.0, 2.0))
+check("no eyes learned yet: as it comes", EyeBridge()(0.0, [None, (6.0, 2.0)], (6.0, 2.0)), (6.0, 2.0))
+
+
+# --- ft-gazed's own path uses it ---
+def service():
+    svc = gazed.Service.__new__(gazed.Service)
+    svc.override, svc.tracker, svc.source, svc.one_eye = None, "own", "mmap1", None
+    svc.models = {name: gazed.Correction() for name in gazed.SOURCES}
+    svc.counts = dict.fromkeys(("samples", "sent", "blinks", "one_eye", "one_eye_used", "lost_left", "lost_right",
+                                "looking_down", "dropped"), 0)
+    from collections import deque
+    svc.opens, svc.vergence = (deque(maxlen=90), deque(maxlen=90)), deque(maxlen=90)
+    svc.lost, svc.bad_at, svc.fallback = [False, False], [0.0, 0.0], gazed.EyeFallback()
+    svc.fix, svc.last_sample, svc.low_until = GazeLock(radius=1.0), 0.0, [0.0, 0.0]
+    svc.bridge = EyeBridge()
+    svc.fix = lambda gy, gp, t, then, q: (gy, gp)   # what goes into the lock
+    svc.weights, svc.fusion_on, svc.fusion = {"own": EyeWeights()}, False, None
+    svc.sent = []
+    svc.send = lambda t, hy, hp, rhy, rhp, eyes: svc.sent.append((round(hy, 2), round(hp, 2)))
+    return svc
+
+
+svc = service()
+for i in range(90):
+    svc.on_eyes_sample({"t": i / 90, "src": {"mmap1": {"unc": [0.001, 0.001], "open": [0.8, 0.8]},
+                                             "own": {"hy": 5.0, "hp": 2.0, "eyes": [[4.0, 2.0], [6.0, 2.0]]}}}, True)
+svc.on_eyes_sample({"t": 1.0, "src": {"mmap1": {"unc": [0.001, 0.001], "open": [0.8, 0.8]},
+                                      "own": {"hy": 6.0, "hp": 2.0, "eyes": [None, [6.0, 2.0]]}}}, True)
+check("ft-gazed, our tracker: both eyes, then the left lost: no jump",
+      (svc.sent[-2], abs(svc.sent[-1][0] - 5.0) < 0.05), ((5.0, 2.0), True))
+check("...counted as one eye", svc.counts["one_eye"], 1)
+
+
+# --- a new calibration starts the eye weights over (ft-gazed's note_own_calibration) ---
+logs = []
+gazed.log = lambda msg: logs.append(msg)
+svc = service()
+svc.bias, svc.own_made, svc.dirty = "auto", None, False
+svc.own = {"calibration": {"made": "2026-10-18 20:58:26", "dots": 27}}
+svc.note_own_calibration()
+check("the first calibration it hears of: adopted, nothing dropped", (svc.own_made, logs), ("2026-10-18 20:58:26", []))
+for _ in range(6):
+    svc.weights["own"].add([2.0, 0.5])   # the left eye's mapping was the worse one
+before = svc.weights["own"].weights()
+check("misses on that calibration lean toward the right eye", before[1] > 0.9, True)
+svc.note_own_calibration()
+check("the same calibration again: kept", svc.weights["own"].weights(), before)
+svc.own = {"calibration": {"made": "2026-10-18 21:11:33", "dots": 26}}
+svc.note_own_calibration()
+check("a new calibration: the misses start over", svc.weights["own"].misses, [[], []])
+check("...the eyes count alike until each has MIN again", tuple(svc.weights["own"].weights()), (0.5, 0.5))
+check("...said in the log", sum("eye weights start over" in m for m in logs), 1)
+check("...and saved with the stamp", (svc.own_made, svc.dirty), ("2026-10-18 21:11:33", True))
+svc.own = {}
+svc.note_own_calibration()
+check("a status without a calibration changes nothing", svc.own_made, "2026-10-18 21:11:33")
+
+# The stamp lives with the misses, so a restart keeps them and a calibration made meanwhile doesn't.
+svc.lives = {name: gazed.PointerLessons() for name in gazed.SOURCES}
+svc.weights["own"].add([1.0, 1.0])
+gazed.STATE.mkdir(parents=True, exist_ok=True)
+svc.save_lessons()
+again = service()
+again.bias, again.lives = "auto", {name: gazed.PointerLessons() for name in gazed.SOURCES}
+again.source = "mmap1"
+again.load_lessons()
+check("after a restart: the stamp and the misses", (again.own_made, again.weights["own"].misses),
+      ("2026-10-18 21:11:33", [[1.0], [1.0]]))
+again.own = {"calibration": {"made": "2026-10-18 21:11:33"}}
+again.note_own_calibration()
+check("...the same calibration: kept", again.weights["own"].misses, [[1.0], [1.0]])
+again.own = {"calibration": {"made": "2026-10-18 21:30:00"}}
+again.note_own_calibration()
+check("...one made while it was down: started over", again.weights["own"].misses, [[], []])
+
+shutil.rmtree(HOME, ignore_errors=True)
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+sys.exit(1 if failures else 0)

gaze/test/one-eye-test.py

@@ -0,0 +1,176 @@
+#!/usr/bin/env python3
+"""Offline test of Track Dominant Eye Only (SteamOS 0.4): SteamVR's tracker ignores one eye,
+and the gaze code then goes by the other (gazecal.tracked_eye, steady_samples' `eye`,
+fitcheck's `ignore`, ft-gazed's live gaze). Reads only the temporary settings files it writes;
+ft-gazed's Service is built without its sockets, so nothing reaches the live gaze service.
+
+  gaze/test/one-eye-test.py
+"""
+import importlib.machinery
+import importlib.util
+import json
+import os
+import sys
+import tempfile
+from collections import deque
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+from fitcheck import FitCheck  # noqa: E402
+from gazecal import (  # noqa: E402
+    Correction,
+    EyeFallback,
+    EyeWeights,
+    GazeLock,
+    TrackedEye,
+    steady_samples,
+    tracked_eye,
+)
+
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(HERE, "..", "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+
+failures = []
+
+
+def check(what, got, want):
+    if got != want:
+        failures.append(what)
+        print(f"FAIL {what}: got {got!r}, want {want!r}", flush=True)
+
+
+tmp = tempfile.mkdtemp(prefix="ft-one-eye-test-")
+first, second = os.path.join(tmp, "a.vrsettings"), os.path.join(tmp, "b.vrsettings")
+paths = (first, second)
+
+
+def write(path, steamvr):
+    with open(path, "w") as f:
+        f.write(steamvr if isinstance(steamvr, str) else json.dumps({"steamvr": steamvr}, indent=3))
+
+
+# --- Reading SteamVR's settings ---
+check("no settings file: both eyes", tracked_eye(paths), None)
+write(second, {"eyeTrackingDominantEyeOnly": True})
+check("only the second file: it counts (right, SteamVR's default eye)", tracked_eye(paths), 1)
+write(first, {"supersampleScale": 1.0})
+check("the first file counts, without the setting: both eyes", tracked_eye(paths), None)
+write(first, {"eyeTrackingDominantEyeOnly": True, "dominantEye": 0})
+check("dominant eye left", tracked_eye(paths), 0)
+write(first, {"eyeTrackingDominantEyeOnly": False, "dominantEye": 0})
+check("setting off", tracked_eye(paths), None)
+write(first, "{ not json")
+check("a broken file: both eyes", tracked_eye(paths), None)
+
+eye = TrackedEye(paths)
+write(first, {"eyeTrackingDominantEyeOnly": True, "dominantEye": 1})
+check("TrackedEye reads it", eye(now=100.0), 1)
+write(first, {"eyeTrackingDominantEyeOnly": True, "dominantEye": 0, "pad": "x" * 10})
+check("TrackedEye waits CHECK seconds", eye(now=101.0), 1)
+check("then sees the change", eye(now=102.5), 0)
+
+# --- One look at a dot: the left eye lost all along (what SteamVR's tracker may report for
+# the eye it ignores), the right seen, and the vergence jumping with the lost eye ---
+look = [{"t": i / 90, "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.02, 0.001], "open": [0.0, 0.8],
+                                        "lr": 2.8 if i % 2 else 9.0}}} for i in range(40)]
+why = {}
+check("both eyes judged: nothing kept", len(steady_samples(look, why=why)), 0)
+check("both eyes judged: why", why, {"lost_left": 40})
+why = {}
+check("right eye only: all kept", len(steady_samples(look, why=why, eye=1)), 40)
+check("right eye only: nothing dropped", why, {})
+why = {}
+check("left eye only: nothing kept", len(steady_samples(look, why=why, eye=0)), 0)
+blink = [dict(s, src={"mmap1": dict(s["src"]["mmap1"], open=[0.0, 0.05])}) if 10 <= i < 15 else s
+         for i, s in enumerate(look)]
+why = {}
+check("right eye only: its blinks still drop", len(steady_samples(blink, why=why, eye=1)), 35)
+check("right eye only: as blinks", why, {"blink": 5})
+
+# --- The headset fit check ---
+fit = FitCheck(ignore=0)
+for i in range(400):
+    fit.feed({"src": {"mmap1": {"hy": 0.0, "hp": 0.0, "unc": [0.02, 0.001], "open": [0.0, 0.8]}}}, i / 90)
+check("fit: the ignored eye's card", fit.status(0)[0], "not tracked")
+check("fit: the tracked eye's card", fit.status(1)[0], "tracking")
+hints = fit.hints()
+check("fit: says which eye counts", hints[0].startswith("SteamVR tracks only your right eye"), True)
+check("fit: no losses blamed on the ignored eye", any("Left eye: lost" in h for h in hints), False)
+check("fit: the tracked eye is fine", hints[-1], "Your right eye is tracked everywhere you've looked so far.")
+both = FitCheck()
+for i in range(400):
+    both.feed({"src": {"mmap1": {"hy": 0.0, "hp": 0.0, "unc": [0.02, 0.001], "open": [0.0, 0.8]}}}, i / 90)
+check("fit, both eyes judged: the left eye is lost", both.status(0)[0], "LOST")
+
+# --- ft-gazed's live gaze: the parts of Service the samples go through, no sockets ---
+def service(one, source="mmap1"):
+    svc = gazed.Service.__new__(gazed.Service)
+    svc.override, svc.tracker, svc.source, svc.one_eye = None, "steam", source, one
+    svc.models = {name: Correction() for name in gazed.SOURCES}
+    svc.counts = dict.fromkeys(("samples", "sent", "blinks", "one_eye", "one_eye_used", "lost_left", "lost_right",
+                                "looking_down", "dropped"), 0)
+    svc.opens, svc.vergence = (deque(maxlen=90), deque(maxlen=90)), deque(maxlen=90)
+    svc.lost, svc.bad_at, svc.fallback = [False, False], [0.0, 0.0], EyeFallback()
+    svc.fix, svc.last_sample, svc.low_until = GazeLock(radius=1.0), 0.0, [0.0, 0.0]
+    svc.bridge = gazed.EyeBridge()
+    svc.correction = lambda name, hy, hp: (0.0, 0.0)
+    svc.sent = []
+    svc.send = lambda t, hy, hp, rhy, rhp, eyes: svc.sent.append((hy, hp))
+    return svc
+
+
+def feed(svc, n=90):
+    for i in range(n):
+        svc.on_source_sample({"t": i / 90, "src": {
+            "mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.02, 0.001], "open": [0.0, 0.8]},
+            "mmap2": {"hy": 3.0, "hp": 2.0, "eyes": [[9.0, 9.0], [5.0, 2.0]]}}})
+
+
+svc = service(None, "mmap2")
+feed(svc)
+check("mmap2, both eyes judged: a lost eye and no fallback yet drop the gaze", svc.sent, [])
+svc = service(1, "mmap2")
+feed(svc)
+check("mmap2, right eye only: its own reading goes out", (len(svc.sent), svc.sent[-1] if svc.sent else None), (90, (5.0, 2.0)))
+check("mmap2, right eye only: no blinks counted", svc.counts["blinks"], 0)
+svc = service(0, "mmap1")
+feed(svc)
+check("left eye only, and it's lost: nothing goes out", (svc.sent, svc.counts["blinks"]), ([], 90))
+svc = service(1)
+check("no calibration: the source, as a whole", svc.kind, "source")
+svc.models["right"].samples = 9
+check("right eye only and calibrated: each eye's own", svc.kind, "eyes")
+svc.one_eye = None
+check("both eyes judged: the left needs a calibration too", svc.kind, "source")
+
+
+# Our own tracker sees both eyes whatever SteamVR tracks; only the blinks come from SteamVR.
+def feed_own(svc, n=90, right_open=0.8):
+    for i in range(n):
+        svc.on_eyes_sample({"t": i / 90, "src": {
+            "mmap1": {"unc": [0.02, 0.001], "open": [0.0, right_open]},
+            "own": {"hy": 5.0, "hp": 2.0, "eyes": [[4.0, 2.0], [6.0, 2.0]]}}}, True)
+
+
+svc = service(1)
+svc.tracker, svc.weights = "own", {"own": EyeWeights()}
+check("own tracker, right eye only: our tracker", svc.kind, "own")
+feed_own(svc)
+check("own tracker, right eye only: SteamVR's closed left doesn't drop our left",
+      (len(svc.sent), svc.counts["one_eye"], svc.counts["blinks"]), (90, 0, 0))
+svc = service(1)
+svc.tracker, svc.weights = "own", {"own": EyeWeights()}
+feed_own(svc, right_open=0.0)
+check("own tracker, right eye only: its blink drops both", (svc.sent, svc.counts["blinks"]), ([], 90))
+svc = service(None)
+svc.tracker, svc.weights = "own", {"own": EyeWeights()}
+feed_own(svc)
+check("own tracker, both eyes judged: SteamVR's closed left still drops it", svc.counts["one_eye"], 90)
+
+for p in paths:
+    if os.path.exists(p):
+        os.remove(p)
+os.rmdir(tmp)
+print("FAILED: " + ", ".join(failures) if failures else "all passed")
+sys.exit(1 if failures else 0)

gaze/test/size-guard-test.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""Offline test of ft-eyes' size guard (eyes_pupil.SizeGuard, EYES_SIZE_GUARD): synthetic frames
+fed to its eyes as the live loop does. A find two thirds of the pupil's usual size in one eye,
+while the other's is as usual, counts as not found: no gaze from it, the search stays on the
+last pupil, the finds log marks it "guard". With the guard off, it's taken. Needs the
+tracker's Python (numpy, OpenCV); no cameras or SteamVR.
+
+  scripts/in-box /opt/frametop/venv/bin/python gaze/test/size-guard-test.py
+"""
+import importlib.machinery
+import importlib.util
+import json
+import os
+import sys
+import tempfile
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+HOME = tempfile.mkdtemp(prefix="ft-eyes-size-guard-test-")
+TRACKER = Path(__file__).resolve().parents[1] / "tracker"
+
+
+def load(rule):
+    os.environ.update(HOME=HOME, FT_EYES_CAMS="/nonexistent", FT_EYES_STATE=os.path.join(HOME, "eyes"),
+                      FT_EYES_MODEL="classic", FT_EYES_SIZE_GUARD=rule)
+    loader = importlib.machinery.SourceFileLoader(f"fteyes_{rule}", str(TRACKER / "ft-eyes"))
+    mod = importlib.util.module_from_spec(importlib.util.spec_from_loader(loader.name, loader))
+    loader.exec_module(mod)
+    return mod
+
+
+def frame(a, b):
+    img = np.full((400, 512), 100, np.uint8)
+    cv2.ellipse(img, (256, 200), (a, b), 0, 0, 360, 0, -1)
+    return img
+
+
+failed = 0
+
+
+def check(ok, what):
+    global failed
+    print(("ok   " if ok else "FAIL ") + what)
+    failed += not ok
+
+
+for rule, refused in (("both", True), ("off", False)):
+    fe = load(rule)
+    tracker = fe.Tracker()
+    log = os.path.join(HOME, f"finds-{rule}.jsonl")
+    tracker.command(f"finds start {log}")
+    right, left = tracker.eyes
+    t = 100.0
+    for i in range(40):           # both pupils as usual
+        right.feed(t, frame(30, 24))
+        left.feed(t + 0.001, frame(30, 24))
+        t += 1 / 90
+    last = right.last
+    left.feed(t, frame(30, 24))
+    right.feed(t + 0.001, frame(20, 15))   # 0.67 of usual, the left eye as usual
+    tracker.command("finds stop")
+    rows = [json.loads(x) for x in open(log)]
+    check(tracker.status()["size_guard"] == rule, f"{rule}: status says the rule")
+    check((right.refused == 1) == refused, f"{rule}: the small find refused: {right.refused}")
+    check((right.last is last) == refused, f"{rule}: the search stays on the last pupil")
+    check(bool(rows[-1].get("guard")) == refused, f"{rule}: the finds log marks it: {rows[-1]}")
+    check(abs(rows[-1]["a"] - 20) < 1.5, f"{rule}: ...with its size")
+    check(left.refused == 0 and sum(1 for r in rows if r.get("guard")) == (1 if refused else 0),
+          f"{rule}: nothing else refused")
+    counts = tracker.status()["size_guard_counts"]
+    got = {e: (c["kept"] >= 30, c["refused"], c["restarts"]) for e, c in counts.items()}
+    want = {"right": (True, 1, 0), "left": (True, 0, 0)} if refused else \
+        {e: (False, 0, 0) for e in ("right", "left")}   # the rule off: nothing judged
+    check(got == want, f"{rule}: status counts the guard's finds: {counts}")
+    zero = {e: {"kept": 0, "refused": 0, "restarts": 0} for e in (0, 1)}
+    _, line = fe.guard_line(tracker.guard, zero)
+    check(("right refused 1 of " in line and "left refused 0 of " in line and "restarts" not in line) if refused
+          else line == "", f"{rule}: its minute's log line: {line!r}")
+    if refused:
+        # The right pupil really smaller (one eye's light) for over GUARD_RESET: its usual size
+        # starts over, once, and that's counted.
+        was = {e: tracker.guard.counts(e) for e in (0, 1)}
+        t += 0.01
+        for i in range(int(fe.eyes_pupil.GUARD_RESET * 90) + 30):
+            left.feed(t, frame(30, 24))
+            right.feed(t + 0.001, frame(20, 15))
+            t += 1 / 90
+        check(tracker.guard.counts(1)["restarts"] == 0 and tracker.guard.counts(0)["restarts"] == 1,
+              f"{rule}: a pupil that stayed smaller: one restart: {tracker.status()['size_guard_counts']}")
+        _, line = fe.guard_line(tracker.guard, was)
+        check(", 1 restarts" in line, f"{rule}: ...in the log line: {line!r}")
+
+print(f"{'FAILED' if failed else 'ok'}: size guard")
+sys.exit(1 if failed else 0)

gaze/test/test_eyes_pupil.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+"""Offline pupil detection: shared preprocessing preserves both closing attempts.
+
+Synthetic images exercise normal pupils, glints, the small-kernel retry, lost eyes,
+and windowed searches. No camera, calibration, or SteamVR connection is needed.
+"""
+import sys
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+import cv2
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tracker"))
+import eyes_pupil as pupil
+
+
+def frame(retry=False):
+    img = np.full((400, 512), 100, np.uint8)
+    centre = (44, 200) if retry else (256, 200)
+    if retry:
+        img[:, :11] = 0  # large closing joins the pupil to this edge region
+    cv2.ellipse(img, centre, (30, 24), 0, 0, 360, 0, -1)
+    return img
+
+
+def separate_attempts(img, ox, oy):
+    """The old preprocessing order: each attempt gets a newly calculated mask."""
+    for close in (pupil.CLOSE, pupil.CLOSE_TIGHT):
+        dark = (cv2.GaussianBlur(img, (5, 5), 0) < pupil.DARK).astype(np.uint8)
+        p = pupil._find(img, dark, ox, oy, close)
+        if p is not None:
+            return p
+    return None
+
+
+class PupilTest(unittest.TestCase):
+    def test_first_attempt(self):
+        img = frame()
+        with patch.object(pupil.cv2, "GaussianBlur", wraps=cv2.GaussianBlur) as blur, \
+                patch.object(pupil, "_find", wraps=pupil._find) as find:
+            p = pupil.find_pupil(img)
+        self.assertEqual((blur.call_count, find.call_count), (1, 1))
+        self.assertAlmostEqual(p["x"], 256, delta=0.1)
+        self.assertAlmostEqual(p["y"], 200, delta=0.1)
+
+    def test_retry_preserves_mask_and_kernels(self):
+        img = frame(retry=True)
+        dark = (cv2.GaussianBlur(img, (5, 5), 0) < pupil.DARK).astype(np.uint8)
+        self.assertIsNone(pupil._find(img, dark, 0, 0))
+        expected = pupil._find(img, dark, 0, 0, pupil.CLOSE_TIGHT)
+        self.assertIsNotNone(expected)
+        kernels = {k: v.copy() for k, v in pupil._KERNELS.items()}
+        with patch.object(pupil.cv2, "GaussianBlur", wraps=cv2.GaussianBlur) as blur, \
+                patch.object(pupil, "_find", wraps=pupil._find) as find:
+            self.assertEqual(pupil._find_either(img, 0, 0), expected)
+        self.assertEqual((blur.call_count, find.call_count), (1, 2))
+        first, second = (call.args[1] for call in find.call_args_list)
+        self.assertIs(first, second)
+        np.testing.assert_array_equal(first, dark)
+        for k, v in kernels.items():
+            np.testing.assert_array_equal(pupil._KERNELS[k], v)
+
+    def test_exact_results_for_full_and_cropped_images(self):
+        rng = np.random.default_rng(512400)
+        images = [np.zeros((400, 512), np.uint8), np.full((400, 512), 255, np.uint8),
+                  frame(), frame(retry=True)]
+        glints = frame()
+        for y in (194, 207):
+            cv2.circle(glints, (256, y), 2, 255, -1)
+        images.append(glints)
+        for _ in range(12):
+            img = rng.integers(35, 180, (400, 512), np.uint8)
+            cv2.ellipse(img, (int(rng.integers(30, 480)), int(rng.integers(25, 375))),
+                        (int(rng.integers(10, 45)), int(rng.integers(8, 25))),
+                        int(rng.integers(0, 180)), 0, 360, 0, -1)
+            images.append(img)
+        for i, img in enumerate(images):
+            for x, y in ((0, 0), (11, 15), (200, 120)):
+                roi = img if (x, y) == (0, 0) else img[y:y + 180, x:x + 200]
+                before = roi.copy()
+                with self.subTest(image=i, offset=(x, y)):
+                    self.assertEqual(pupil._find_either(roi, x, y), separate_attempts(roi, x, y))
+                    np.testing.assert_array_equal(roi, before)
+
+    def test_window_hint_and_full_frame_fallback(self):
+        img = frame()
+        expected = pupil.find_pupil(img)
+        self.assertEqual(pupil.find_pupil(img, expected), expected)
+        self.assertEqual(pupil.find_pupil(img, {"x": 50, "y": 50, "a": 5}), expected)
+        self.assertEqual(pupil.find_pupil(np.zeros_like(img), expected), None)
+
+    def test_a_much_smaller_spot_isnt_the_pupil(self):
+        """The right camera's dark spot of 2026-10-17: darker and smaller than the pupil, so it
+        wins on score unless the pupil's recent size rules it out."""
+        img = np.full((400, 512), 100, np.uint8)
+        cv2.ellipse(img, (256, 200), (30, 24), 0, 0, 360, 20, -1)
+        cv2.ellipse(img, (215, 250), (10, 7), 0, 0, 360, 0, -1)
+        self.assertAlmostEqual(pupil.find_pupil(img)["x"], 215, delta=0.5)
+        p = pupil.find_pupil(img, smallest=15)
+        self.assertAlmostEqual(p["x"], 256, delta=0.5)
+        self.assertAlmostEqual(pupil.find_pupil(img, p, 15)["x"], 256, delta=0.5)
+        self.assertIsNone(pupil.find_pupil(img, p, 40))
+
+    def test_pupil_size(self):
+        size = pupil.PupilSize()
+        for i in range(pupil.SIZE_FINDS - 1):
+            size.add(i / 90, {"a": 20.0})
+        self.assertEqual(size.smallest(9 / 90), 0.0)  # too few finds yet
+        size.add(9 / 90, {"a": 24.0})
+        size.add(10 / 90, None)  # lost frames don't count
+        self.assertAlmostEqual(size.smallest(10 / 90), pupil.SMALLEST * 20.0)
+        self.assertEqual(size.smallest(10 / 90 + pupil.SIZE_WINDOW + 0.2), 0.0)  # all of them too old
+
+
+def warm(guard, t=0.0, a=(20.0, 20.0)):
+    """Both eyes' usual sizes: GUARD_FINDS finds each, 90 Hz, from t. Returns the next t."""
+    for i in range(pupil.GUARD_FINDS):
+        for e in (0, 1):
+            assert guard.keep(e, t + i / 90, a[e])
+    return t + pupil.GUARD_FINDS / 90
+
+
+class SizeGuardTest(unittest.TestCase):
+    def test_a_misfind_in_one_eye(self):
+        g = pupil.SizeGuard("both")
+        t = warm(g)
+        self.assertTrue(g.keep(1, t, 19.0))
+        self.assertFalse(g.keep(0, t + 0.001, 13.0))   # 0.65 of usual, the other eye at 0.95
+        self.assertTrue(g.keep(0, t + 1 / 90, 19.5))
+        self.assertEqual(g.refused, {0: 1, 1: 0})
+
+    def test_light_shrinks_both(self):
+        """Both pupils down to under half their usual size: kept with "both", refused with "long"."""
+        for rule, kept in (("both", True), ("long", False)):
+            g = pupil.SizeGuard(rule)
+            t = warm(g)
+            ok = []
+            for i in range(20):   # narrowing over 20 frames, both eyes alike
+                a = 20.0 * (1 - 0.03 * i)
+                ok += [g.keep(0, t, a), g.keep(1, t + 0.001, a)]
+                t += 1 / 90
+            with self.subTest(rule=rule):
+                self.assertEqual(all(ok), kept)
+
+    def test_other_eye_not_seen(self):
+        g = pupil.SizeGuard("both")
+        t = warm(g) + 1.0   # the other eye's last find is old
+        self.assertTrue(g.keep(0, t, 13.0))    # 0.65: over GUARD_ALONE
+        self.assertFalse(g.keep(0, t + 0.01, 11.0))
+
+    def test_refusals_going_on_start_over(self):
+        g = pupil.SizeGuard("long")
+        t = warm(g)
+        ok = [g.keep(0, t + i / 90, 10.0) for i in range(int(pupil.GUARD_RESET * 90) + 5)]
+        self.assertFalse(ok[0])
+        self.assertTrue(ok[-1])        # taken again after GUARD_RESET s
+        self.assertTrue(g.keep(0, t + 2.0, 10.0))
+
+    def test_off_and_unknown(self):
+        self.assertTrue(pupil.SizeGuard("off").keep(0, 0.0, 1.0))
+        self.assertEqual(pupil.SizeGuard("nonsense").rule, "both")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/tracker/atomic-update.conf

@@ -0,0 +1,4 @@
+# Installed to /etc/atomic-update.conf.d/frametop-eyegrab.conf by gaze/tracker/install.sh.
+# A SteamOS update deletes every /etc file its keep list (/usr/lib/rauc/atomic-update-keep.conf)
+# doesn't name. That list keeps the unit, but not the program it runs.
+/etc/frametop/ft-eyegrab

gaze/tracker/build.sh

@@ -13,6 +13,7 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
 "$root/scripts/sync.sh" >/dev/null
 exec "$root/scripts/frame.sh" -C gaze/tracker 'set -e; mkdir -p build
 gcc -std=gnu11 -O2 -Wall -Wextra -pthread -o build/ft-eyegrab ft-eyegrab.c
+strip --strip-unneeded build/ft-eyegrab
 max=$(objdump -T build/ft-eyegrab | grep -oE "GLIBC_[0-9.]+" | sort -uV | tail -1)
 echo "built build/ft-eyegrab, newest glibc symbol: $max"
 [ "$(printf "%s\n" "$max" GLIBC_2.39 | sort -V | tail -1)" = GLIBC_2.39 ] || { echo "needs newer glibc than the host has" >&2; exit 1; }
@@ -21,6 +22,12 @@ if [ "${FRAME_IN_BOX:-0}" = 1 ] && [ -x /opt/frametop/venv/bin/python ]; then
   python3 -m venv --without-pip build/venv
   /opt/frametop/venv/bin/python -c "import sysconfig; print(sysconfig.get_path(\"purelib\"))" \
     >"$(build/venv/bin/python -c "import sysconfig; print(sysconfig.get_path(\"purelib\"))")/frametop-image.pth"
+  # The eye net (eyes_net.py) needs ncnn, which the image does not have yet: add it here. It needs
+  # the network; without it ft-eyes uses the pupil finder.
+  if ! build/venv/bin/python -c "import ncnn" 2>/dev/null; then
+    { build/venv/bin/python -m ensurepip >/dev/null && build/venv/bin/python -m pip install -q --disable-pip-version-check \
+        --no-deps "$(grep -E "^ncnn==" requirements.txt)"; } || echo "build/venv: no ncnn (the eye net needs it)" >&2
+  fi
 elif ! cmp -s requirements.txt build/venv/requirements.done; then
   rm -rf build/venv
   python3 -m venv build/venv

gaze/tracker/eyes_model.py

@@ -75,13 +75,26 @@ def pair_mid(pair):
     return ((pair[0][0] + pair[1][0]) / 2, (pair[0][1] + pair[1][1]) / 2)
 
 
+def eye_of(click, eye):
+    """A click's features for one eye, or None: `eye` is a list by eye (the lab tools) or a dict
+    of the eyes seen (ft-eyes' calibration dots, which may hold one)."""
+    e = click["eye"]
+    return e.get(eye) if isinstance(e, dict) else e[eye]
+
+
 class Calibration:
     """The three fits per eye. Build from clicks (ft-eyes-score's features) or load from JSON.
 
     `spread` is each eye's RMS miss (degrees) on the clicks its pupil fit was made from.
     `combine` weights the eyes by its inverse square: on practice2 (one headset position,
     leave-one-out) that gave 0.75 median against 0.84 for the plain average, since one eye
-    is usually much better than the other (left 0.73, right 1.32 there)."""
+    is usually much better than the other (left 0.73, right 1.32 there).
+
+    Each eye is fitted on the clicks it was seen in, so a click may hold one eye only, and an
+    eye seen in under MIN_CLICKS is left out (the calibration then maps the other alone). A
+    second person's eyes (2026-10-17 rounds) each lost the pupil looking 20 degrees outward:
+    with only the clicks that saw both, each eye's fit had none out there and guessed (frame
+    errors 8.1 degrees median, 84 p90, held-out takes); each on its own clicks, 2.3 and 5.0."""
 
     def __init__(self, fits=None, info=None, spread=None):
         self.fits = fits or {}   # (name, eye) -> Quad
@@ -93,21 +106,26 @@ class Calibration:
         truth = lambda cs: np.array([k["truth"] for k in cs])  # noqa: E731
         fits, spread = {}, {}
         for c in (0, 1):
-            cs = [k for k in clicks if k["eye"][c] is not None]
+            cs = [k for k in clicks if eye_of(k, c) is not None]
             if len(cs) < MIN_CLICKS:
                 continue
-            fits["pupil", c] = Quad([k["eye"][c]["pupil"] for k in cs], truth(cs))
-            miss = fits["pupil", c]([k["eye"][c]["pupil"] for k in cs]) - truth(cs)
+            pupils = [eye_of(k, c)["pupil"] for k in cs]
+            fits["pupil", c] = Quad(pupils, truth(cs))
+            miss = fits["pupil", c](pupils) - truth(cs)
             spread[c] = float(np.sqrt(np.mean(np.sum(miss ** 2, axis=1))))
-            fits["where", c] = Quad(truth(cs), [k["eye"][c]["pupil"] for k in cs])
-            gs = [k for k in cs if k["eye"][c]["mid"] is not None]
+            fits["where", c] = Quad(truth(cs), pupils)
+            gs = [k for k in cs if eye_of(k, c)["mid"] is not None]
             if len(gs) >= MIN_CLICKS:
-                fits["glint", c] = Quad([k["eye"][c]["pupil"] - k["eye"][c]["mid"] for k in gs], truth(gs))
+                fits["glint", c] = Quad([eye_of(k, c)["pupil"] - eye_of(k, c)["mid"] for k in gs], truth(gs))
         return cls(fits, info, spread)
 
     def has(self, name, eye):
         return (name, eye) in self.fits
 
+    def eyes(self):
+        """The eyes it maps (0 right, 1 left)."""
+        return [c for c in (0, 1) if self.has("pupil", c)]
+
     def slip(self, eye, rows):
         """Median slip in pixels from pair sightings `rows` (t, px, py, mx, my), or None."""
         if not self.has("glint", eye) or len(rows) < SLIP_MIN:

gaze/tracker/eyes_net.py

@@ -0,0 +1,56 @@
+"""The eye net: one eye camera's frame in, that eye's raw gaze out (frame-eyes' train/gaze).
+
+A model folder holds model.json and the ncnn files it names:
+  {"name": "eyes-v0", "variant": "s100", "input": [100, 128], "gaze_scale": 30, "out": 6,
+   "param": "model.ncnn.param", "bin": "model.ncnn.bin", "in": "in0", "outname": "out0", ...}
+The net's output holds both cameras' heads (OUT numbers each, camera 0's first): gaze yaw and
+pitch / gaze_scale, pupil x / 512 and y / 400, eye open (logit), pupil visible (logit).
+
+The gaze is raw: each person's own offset is left to the calibration (eyes_model.Calibration,
+with this gaze where the pupil centre would be). So a calibration belongs to one model: ft-eyes
+keeps each model's in its own folder.
+
+ncnn fp16 on one thread: 1.4 ms a frame for s100 on an A720 core (frame-eyes notes/e0-runtime.md).
+"""
+import json
+import math
+import os
+
+import numpy as np
+
+W, H = 512, 400
+
+
+class Net:
+    def __init__(self, folder):
+        import cv2
+        import ncnn
+        self.cv2 = cv2
+        self.folder = folder
+        with open(os.path.join(folder, "model.json")) as f:
+            self.info = json.load(f)
+        self.name = str(self.info["name"])
+        self.in_h, self.in_w = self.info["input"]
+        self.scale = float(self.info.get("gaze_scale", 30.0))
+        self.out = int(self.info.get("out", 6))
+        self.in_name, self.out_name = self.info.get("in", "in0"), self.info.get("outname", "out0")
+        net = ncnn.Net()
+        net.opt.num_threads = 1
+        net.opt.use_vulkan_compute = False
+        net.opt.use_fp16_storage = net.opt.use_fp16_arithmetic = net.opt.use_fp16_packed = True
+        for key, load in (("param", net.load_param), ("bin", net.load_model)):
+            if load(os.path.join(folder, self.info[key])) != 0:
+                raise RuntimeError(f"{folder}: can't load {self.info[key]}")
+        self.ncnn, self.net = ncnn, net
+
+    def __call__(self, img, cam):
+        """img: one camera's 400x512 uint8 frame -> dict(gaze=(yaw, pitch), x, y, open, visible),
+        open and visible as probabilities."""
+        x = self.cv2.resize(img, (self.in_w, self.in_h), interpolation=self.cv2.INTER_AREA)
+        x = x.astype(np.float32)[None] * (1.0 / 255.0)
+        ex = self.net.create_extractor()
+        ex.input(self.in_name, self.ncnn.Mat(x))
+        o = np.array(ex.extract(self.out_name)[1]).reshape(-1)[cam * self.out:(cam + 1) * self.out]
+        sig = lambda v: 1.0 / (1.0 + math.exp(-float(v)))  # noqa: E731
+        return {"gaze": (float(o[0]) * self.scale, float(o[1]) * self.scale), "x": float(o[2]) * W,
+                "y": float(o[3]) * H, "open": sig(o[4]), "visible": sig(o[5])}

gaze/tracker/eyes_pupil.py

@@ -5,7 +5,14 @@ background are just as dark, but they touch the image edge, so any dark region t
 the edge is dropped. Closing the glints' holes can join the pupil to that background (the
 left camera's, when you look more than about 20 degrees left), so when nothing is found
 the search runs again with a smaller closing.
+
+A dark spot well inside the frame can pass all that too: on 2026-10-17 the right camera took
+one about 9 px across for the pupil (23 px) about once a second, for 3-7 frames at a time, and
+the gaze jumped 60 degrees. A pupil doesn't shrink to half its size in a frame, so a find that
+much smaller than the last half second's (PupilSize) is skipped.
 """
+from collections import deque
+
 import cv2
 import numpy as np
 
@@ -22,42 +29,44 @@ MAX_ASPECT = 3.0   # long / short axis; the steep camera sees a squashed pupil
 GLINT = 200        # glints are near-saturated spots on or by the pupil
 CLOSE = 7          # px: closes the glints' holes in the pupil (the right eye's need this much)
 CLOSE_TIGHT = 3    # px: the retry, keeps a pupil near the dark background apart from it
+_KERNELS = {size: np.ones((size, size), np.uint8) for size in {3, CLOSE, CLOSE_TIGHT}}
 
 
 NEAR = 70          # the windowed search: this many pixels, or 3 pupil radii, around a hint
 
 
-def find_pupil(frame, near=None):
+def find_pupil(frame, near=None, smallest=0.0):
     """Return dict(x, y, a, b, angle, area, fill, glints) or None.
 
     `near` (a previous result) searches a window around it first (0.4 ms, not 1.4-2.1);
-    if the pupil isn't wholly inside the window it falls back to the whole frame."""
+    if the pupil isn't wholly inside the window it falls back to the whole frame. Blobs whose
+    long semi-axis is under `smallest` px don't count (PupilSize.smallest)."""
     if near is not None:
         r = int(max(NEAR, 3 * near['a']))
         x0, y0 = max(int(near['x']) - r, 0), max(int(near['y']) - r, 0)
-        p = _find_either(frame[y0:int(near['y']) + r, x0:int(near['x']) + r], x0, y0)
+        p = _find_either(frame[y0:int(near['y']) + r, x0:int(near['x']) + r], x0, y0, smallest)
         if p is not None:
             p['glints'] = find_glints(frame, p)
             return p
-    p = _find_either(frame, 0, 0)
+    p = _find_either(frame, 0, 0, smallest)
     if p is not None:
         p['glints'] = find_glints(frame, p)
     return p
 
 
-def _find_either(img, ox, oy):
-    p = _find(img, ox, oy)
-    return p if p is not None else _find(img, ox, oy, CLOSE_TIGHT)
+def _find_either(img, ox, oy, smallest=0.0):
+    # Both closing sizes use the same input mask; morphologyEx returns a new one.
+    dark = (cv2.GaussianBlur(img, (5, 5), 0) < DARK).astype(np.uint8)
+    p = _find(img, dark, ox, oy, smallest=smallest)
+    return p if p is not None else _find(img, dark, ox, oy, CLOSE_TIGHT, smallest)
 
 
-def _find(img, ox, oy, close=CLOSE):
-    """The pupil in `img` (a window at ox, oy of the frame), with frame coordinates. A dark
+def _find(img, dark, ox, oy, close=CLOSE, smallest=0.0):
+    """The pupil in `img` (a window at ox, oy), using its prepared `dark` mask. A dark
     region touching the window's edge doesn't count: it's background, rim, or cut off."""
-    f = cv2.GaussianBlur(img, (5, 5), 0)
-    dark = (f < DARK).astype(np.uint8)
     # Glints punch bright holes in the pupil; close them so the blob stays whole.
-    dark = cv2.morphologyEx(dark, cv2.MORPH_CLOSE, np.ones((close, close), np.uint8))
-    dark = cv2.morphologyEx(dark, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8))
+    dark = cv2.morphologyEx(dark, cv2.MORPH_CLOSE, _KERNELS[close])
+    dark = cv2.morphologyEx(dark, cv2.MORPH_OPEN, _KERNELS[3])
     n, lab, stats, _ = cv2.connectedComponentsWithStats(dark, connectivity=8)
     h, w = img.shape
     best = None
@@ -74,7 +83,7 @@ def _find(img, ox, oy, close=CLOSE):
             continue
         (ex, ey), (d1, d2), ang = cv2.fitEllipse(c)
         a, b = max(d1, d2) / 2, min(d1, d2) / 2
-        if b < 3 or a / b > MAX_ASPECT:
+        if b < 3 or a / b > MAX_ASPECT or a < smallest:
             continue
         fill = area / (np.pi * a * b)
         if fill < MIN_FILL or fill > 1.25:
@@ -90,6 +99,116 @@ def _find(img, ox, oy, close=CLOSE):
     return best[1] if best else None
 
 
+SMALLEST = 0.5     # a find under this share of the pupil's recent size is something else
+SIZE_WINDOW = 0.5  # s of finds the recent size comes from
+SIZE_FINDS = 10    # and at least this many of them
+
+
+class PupilSize:
+    """One eye's recent pupil size, the median long semi-axis of its finds over the last
+    SIZE_WINDOW s. A pupil narrows in bright light over a second or so, and turning the eye
+    changes how big the camera sees it by a few percent per frame; in the practice captures
+    (2026-09-29) under 0.4 % of finds were below half this median, many of them strays as well.
+    Only smaller finds are refused: if a wrong spot gets in (after a loss, with no recent size),
+    the real pupil isn't refused when it's back, and the median follows it."""
+
+    def __init__(self):
+        self.finds = deque()   # (t, a)
+
+    def smallest(self, t):
+        """The smallest long semi-axis a find at time t may have, in px (0: no limit yet)."""
+        while self.finds and self.finds[0][0] < t - SIZE_WINDOW:
+            self.finds.popleft()
+        if len(self.finds) < SIZE_FINDS:
+            return 0.0
+        return SMALLEST * float(np.median([a for _, a in self.finds]))
+
+    def add(self, t, p):
+        if p is not None:
+            self.finds.append((t, p['a']))
+
+
+GUARD_RULES = ("both", "long", "off")
+GUARD_LONG = 10.0   # s of kept finds an eye's usual size comes from
+GUARD_FINDS = 20    # and at least this many of them
+GUARD_BOTH = 0.75   # "both": a find under this share of the other eye's ratio to its usual size
+GUARD_ALONE = 0.6   # without the other eye just now (or "long"): under this share of its usual size
+GUARD_FRESH = 0.05  # s: the other eye's last find counts this long
+GUARD_RESET = 1.0   # s of refusals in a row: the pupil really did change, its usual size starts over
+
+
+class SizeGuard:
+    """Both eyes' usual pupil sizes, for the misfinds PupilSize lets through (2026-10-17, two
+    Gaze Bench runs): a dark spot two thirds of the pupil's size, held for a few frames, so the
+    last half second's median follows it in. Here each eye's usual size is the median long
+    semi-axis of its kept finds over the last GUARD_LONG s, and a find is judged by its ratio to
+    that. The pupils narrow and widen together in light (the consensual reflex), so with
+    "both" a find is refused when its ratio is under GUARD_BOTH of the other eye's latest
+    (capped at 1): light shrinks both, a misfind one. On those runs (8884 finds) good finds sat
+    at 0.84 of the usual size or more 95 % of the time, the right eye's misfinds at a median of
+    0.77 and a quarter of them under 0.52. "long" is the ratio alone, under GUARD_ALONE, which
+    is also the rule while the other eye isn't seen. A refused find counts as not found. If
+    refusals go on for GUARD_RESET s, the eye's usual size starts over, so a pupil that did
+    change (light with one eye lost) is taken again. Each eye's kept and refused finds and those
+    restarts are counted (counts), so a session shows whether it refused real pupils: many
+    restarts, or refusals while both eyes looked fine, say it did."""
+
+    def __init__(self, rule="both"):
+        self.rule = rule if rule in GUARD_RULES else "both"
+        self.finds = {0: deque(), 1: deque()}   # (t, a) of kept finds
+        self.usual = {0: None, 1: None}         # the median, recomputed every few finds
+        self.since = {0: 0, 1: 0}
+        self.last = {0: None, 1: None}          # (t, ratio) of each eye's last kept find
+        self.refusing = {0: None, 1: None}      # when the current run of refusals began
+        self.refused = {0: 0, 1: 0}
+        self.kept = {0: 0, 1: 0}
+        self.restarts = {0: 0, 1: 0}            # runs of refusals that went on GUARD_RESET s
+
+    def ratio(self, e, t, a):
+        h = self.finds[e]
+        while h and h[0][0] < t - GUARD_LONG:
+            h.popleft()
+            self.usual[e] = None
+        if len(h) < GUARD_FINDS:
+            return None
+        if self.usual[e] is None or self.since[e] >= 10:
+            self.usual[e], self.since[e] = float(np.median([x for _, x in h])), 0
+        return a / self.usual[e]
+
+    def keep(self, e, t, a):
+        """Is eye e's find at time t, long semi-axis a px, its pupil? (It learns from the kept ones.)"""
+        if self.rule == "off":
+            return True
+        r = self.ratio(e, t, a)
+        ok = True
+        if r is not None:
+            o = self.last[1 - e]
+            if self.rule == "both" and o is not None and t - o[0] <= GUARD_FRESH:
+                ok = r >= GUARD_BOTH * min(1.0, o[1])
+            else:
+                ok = r >= GUARD_ALONE
+        if not ok:
+            if self.refusing[e] is None:
+                self.refusing[e] = t
+            elif t - self.refusing[e] > GUARD_RESET:
+                self.finds[e].clear()
+                self.usual[e], ok, r = None, True, None
+                self.restarts[e] += 1
+        if not ok:
+            self.refused[e] += 1
+            return False
+        self.refusing[e] = None
+        self.kept[e] += 1
+        self.finds[e].append((t, a))
+        self.since[e] += 1
+        self.last[e] = (t, 1.0 if r is None else r)
+        return True
+
+    def counts(self, e):
+        """Eye e's finds judged so far: {"kept", "refused", "restarts"} (none with the rule off)."""
+        return {"kept": self.kept[e], "refused": self.refused[e], "restarts": self.restarts[e]}
+
+
 GLINT_RING = 140   # a glint sits on dark iris or pupil: its surroundings are below this
 GLINT_PAIR = (5, 45)  # the two LEDs' reflections: this far apart, in pixels, mostly vertical