Recorder tests: the eye dataset has three recipients since key 3 (dody)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.dockerignore

@@ -19,3 +19,6 @@ frametop-report-*.txt
 # the fine-tuned hand models: the image fetches its own pinned copy (hands/models.sh)
 hands/models/student/
 hands/models/.student-*
+# model versions downloaded from Input Settings (models/catalog.py): each install fetches its own
+hands/models/versions/
+gaze/tracker/models/

.gitignore

@@ -11,3 +11,5 @@ frametop-report-*.txt
 *.pgm
 .venv/
 .frame-job.d/
+# eye net versions picked in Input Settings (models/catalog.py downloads them; CC BY-NC 4.0)
+gaze/tracker/models/

README.md

@@ -37,7 +37,7 @@ You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or th
 
    The third and fourth choices, stable release and experimental release, download Frametop already built (`Frametop.zip`, about 1.4 GB, from the [releases](https://github.com/Frametop/frametop/releases)) and install it without compiling anything. `-s -- --release` picks the stable release without the question, and `-s -- --release --experimental` the experimental one.
 
-   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks along the way whether to install each optional part: voice dictation (yes by default; it downloads about 320 MB of speech models), hand tracking (experimental, yes by default), in a release the [Hand Recorder](https://frametop.github.io/frametop/hand-recorder) (yes by default; it records your hands for Frametop's open hand dataset), gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The stable version asks only about gaze mode, the eye tracker, and the Bluetooth fixes. Hand tracking, the Hand Recorder, the eye tracker, and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
+   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks along the way whether to install each optional part: voice dictation (yes by default; it downloads about 320 MB of speech models), hand tracking (experimental, yes by default), in a release the [Hand Recorder](https://frametop.github.io/frametop/hand-recorder) (yes by default; it records your hands for Frametop's open hand dataset), gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), in a release the Eye Recorder (yes by default, with our eye tracker; it records your eyes for Frametop's eye tracking dataset), and the Bluetooth fixes, then whether to restart SteamVR. The stable version asks only about gaze mode, the eye tracker, and the Bluetooth fixes. Hand tracking, the Hand Recorder, the eye tracker, and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
 
 After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings. SteamOS's own single-screen desktop is still there, as Native Desktop in the same list.
 

gaze/README.md

@@ -60,7 +60,7 @@ The tracker stops when the headset is off your head. SteamVR also calibrates gaz
 `gaze/tracker/` is an eye tracker of our own, because SteamVR's is about 1.5 degrees off after the best correction the gaze service can learn, and what's left is mostly look-to-look noise that no correction on top of its output can remove. Ours processes the eye cameras itself: 0.59 degrees (median) in its best live session against 0.83 for SteamVR's with the probe's correction, and after the headset was taken off and put back without recalibrating, 0.58 once your first clicks had taught it where the headset sat (`tracker/findings.md` has the measurements).
 
 - `ft-eyegrab` (C, root, the system service `frametop-eyegrab.service`) copies the eye-camera frames (512x400, 90 fps per eye, or 15 without `gaze/eye-rate.py on`) out of the DMA-BUFs SteamVR's `eyetracking` process holds into `/dev/shm/frametop-eyes-cams`, owned by you. It maps them read-only, and it only copies while someone touches `/dev/shm/frametop-eyes-want` (ft-eyes and the recorder do, every second). Otherwise it holds none of the tracker's buffers. Its unit keeps only the capabilities that needs (`CAP_SYS_PTRACE`, `CAP_DAC_READ_SEARCH`, `CAP_CHOWN`). `gaze/tracker/install.sh` builds it and installs it to `/etc/frametop` with sudo, which it asks for (`uninstall`, `status`, and `log` too). It also adds it to SteamOS's update keep list (`/etc/atomic-update.conf.d/frametop-eyegrab.conf`): an update deletes `/etc` files that list doesn't name, and without the grabber gaze mode falls back to SteamVR's tracker and its calibration. `scripts/doctor.sh` reports a grabber an update deleted.
-- `ft-eyes` (Python with numpy and OpenCV, in the dev container: `gaze/tracker/build.sh` puts the pinned `requirements.txt` in `gaze/tracker/build/venv`) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (`eyes_pupil.py`), and maps them to a gaze with a quadratic fit per eye (`eyes_model.py`). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. It publishes the gaze in `/dev/shm/frametop-eyes-gaze` (ft-gaze's source `own`) and takes calibration dots and clicks on `@ft_eyes`. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in `~/.local/state/frametop/gaze/eyes/`.
+- `ft-eyes` (Python with numpy and OpenCV, in the dev container: `gaze/tracker/build.sh` puts the pinned `requirements.txt` in `gaze/tracker/build/venv`) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (`eyes_pupil.py`), and maps them to a gaze with a quadratic fit per eye (`eyes_model.py`). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. It publishes the gaze in `/dev/shm/frametop-eyes-gaze` (ft-gaze's source `own`) and takes calibration dots and clicks on `@ft_eyes`. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in `~/.local/state/frametop/gaze/eyes/`. Instead of the pupil finder it can run an eye net (`eyes_net.py`): `EYES_MODEL` is `classic` (the default), `latest`, or a version from `gaze/tracker/models.json`, picked with its notes and benchmarks on Input Settings' Gaze page, which downloads it into `gaze/tracker/models/NAME` (`models/README.md`). Each net keeps its own calibration in `eyes/net-NAME/`.
 - `lab/` has the tools for improving it on recordings. `ft-eyes-record NAME` (or `ft-eyes-session`, with SteamVR's gaze alongside) records the cameras. `ft-eyes-score` fits and scores on recordings against the probe's practice clicks. `ft-eyes-e2e` runs the whole live path on two recordings (calibrate on one, click through the other). `ft-eyes-replay` plays a recording into a scratch share. Heavy ones are meant for a PC: if you have `frame-job` (a personal tool, not in this repo), `gaze/tracker/.frame-job` sends them there. `lab/py` runs them with that Python (in the dev container on the Frame; on a PC, the same venv from `requirements.txt`, which frame-job's setup makes).
 
 Ground rules, for anyone changing it:

gaze/ft-gazed

@@ -65,7 +65,10 @@ relearn the offset.
 Our own tracker (gaze/tracker/ft-eyes) runs here too, in the dev container, while
 GAZE_TRACKER=own and the gaze is in use (below), or the gaze probe asks for it ("eyes SECONDS",
 a lease the probe renews). It reads the eye-camera frames the root service frametop-eyegrab
-copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs.
+copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs. Which eye net it runs
+(EYES_MODEL: models/catalog.py) is read at its start, so ft-eyes restarts when that changes; starting
+it also runs `models/ft-models auto eyes` in the background (the catalog from GitHub, and the net
+EYES_MODEL needs if it isn't downloaded), and ft-eyes moves to a net that brings.
 
 Idle: ft-gaze and our own tracker run only while the gaze is in use: gaze mode on with someone
 wearing the headset (the pointer helper says both: "gaze ? headset" -> "ok on|off worn|away"), a
@@ -135,6 +138,9 @@ from gazecal import (DEFAULT_MODEL, EYE_FOUND, EYE_LOST, MODELS, STATE, Correcti
                      EyeWeights, Fixation, LiveCorrection, SteamEyeLog, TrackedEye)
 from gazecheck import Checks  # noqa: E402
 
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "models"))
+import catalog  # noqa: E402  (which eye net EYES_MODEL means: models/catalog.py)
+
 REPO = Path(__file__).resolve().parents[1]
 HELPER = REPO / "gaze" / "build" / "ft-gaze"
 IN_BOX = REPO / "scripts" / "in-box"  # runs a program in the container it was built in
@@ -160,6 +166,7 @@ LOOK = 0.3        # seconds of samples before that moment make the look (the pro
 RETRY = 3.0       # seconds before starting ft-gaze again
 EYES_RETRY = 10.0  # seconds before starting ft-eyes again after it stopped on its own
 EYES_LEASE_MAX = 120.0
+MODELS_CLI = REPO / "models" / "ft-models"  # "auto eyes": the eye catalog, and the net EYES_MODEL needs
 SETTLE = 0.3      # seconds after an eye is found again before the fallback learns from it
 IDLE_AFTER = 30.0  # seconds after the gaze was last in use before ft-gaze and our tracker stop
 WAKE_SETTLE = 15.0  # seconds after waking that the tracker may take to start sending
@@ -257,6 +264,8 @@ class Service:
         self.eyes_proc = None  # ft-eyes, while it runs
         self.eyes_until = 0.0  # the probe's lease (monotonic time)
         self.eyes_restart_at = 0.0
+        self.eyes_model = self.eyes_model_now()  # what EYES_MODEL means now: ft-eyes restarts when it changes
+        self.models_proc = None  # ft-models auto eyes, while it runs
         self.awake = False      # ft-gaze (and our tracker) run: the gaze is in use (see the top)
         self.idle_at = 0.0      # idle from then, unless it's in use again first
         self.woke_at = 0.0
@@ -293,8 +302,39 @@ class Service:
 
     # --- Settings, calibration and lessons ---
 
+    @staticmethod
+    def eyes_model_now():
+        """(EYES_MODEL, the folder it resolves to): ft-eyes reads it at start."""
+        try:
+            r = catalog.resolve("eyes")
+            return r["setting"], r["folder"]
+        except Exception as e:  # noqa: BLE001 (a broken catalog: ft-eyes falls back to the pupil finder)
+            return f"error {e}", None
+
+    def check_eyes_model(self):
+        now = self.eyes_model_now()
+        if now != self.eyes_model:
+            log(f"eye model {now[0]}" + (f" ({now[1]})" if now[1] else "") + (": restarting ft-eyes" if self.eyes_proc else ""))
+            self.eyes_model = now
+            if self.eyes_proc:
+                self.stop_eyes()
+                self.eyes_restart_at = 0.0
+
+    def models_auto(self):
+        """The eye catalog from GitHub (at most every 15 minutes) and the net EYES_MODEL needs, if it's
+        missing, in the background: ft-eyes starts with what's here, and moves to the new one after."""
+        if self.models_proc and self.models_proc.poll() is None:
+            return
+        try:
+            self.models_proc = subprocess.Popen([sys.executable, str(MODELS_CLI), "auto", "eyes"], stdin=subprocess.DEVNULL,
+                                                stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                                                start_new_session=True)
+        except OSError as e:
+            log(f"couldn't check for eye models: {e}")
+
     def load_settings(self):
         self.conf_mtime = mtime(CONF)
+        self.check_eyes_model()
         tracker, self.tracker_setting, bias, self.nudge_max = read_settings()
         if (tracker, bias) != (self.tracker, self.bias):
             auto = ""
@@ -558,6 +598,8 @@ class Service:
             log(f"ft-eyes isn't built: run {REPO}/gaze/tracker/build.sh")
             self.eyes_restart_at = time.monotonic() + 30
             return
+        self.models_auto()
+        self.eyes_model = self.eyes_model_now()
         env = dict(os.environ)
         env["XDG_RUNTIME_DIR"] = f"/run/user/{os.getuid()}"
         self.eyes_proc = subprocess.Popen([str(IN_BOX), str(EYES_PYTHON), str(EYES_PROG), "-v", "--watch-stdin"],
@@ -920,6 +962,9 @@ class Service:
         if mtime(CONF) != self.conf_mtime or (self.tracker_setting == "auto"
                                               and own_installed() != (self.tracker == "own")):
             self.load_settings()
+        if self.models_proc and self.models_proc.poll() is not None:
+            self.models_proc = None
+            self.check_eyes_model()   # a net it fetched: ft-eyes moves to it
         self.update_awake()
         want = self.eyes_wanted()
         if want and not self.eyes_proc and time.monotonic() >= self.eyes_restart_at:

gaze/rec/CONSENT.md

@@ -1,6 +1,6 @@
-# Recording your eyes for the Frametop eye dataset (DRAFT, not reviewed yet)
+# Recording your eyes for the Frametop eye dataset
 
-Version: 2026-10-14-draft
+Version: 2026-10-16
 
 Frametop's eye tracking needs a better model than it has: one that keeps working when you look far to the side, blink, squint, wear glasses or makeup, or when the headset moves on your face. To train it, we're collecting recordings of many people's eyes into a dataset that only the maintainers can read. This page explains what the eye recorder records, where it goes, who can see it, and what you agree to if you take part. Please read all of it.
 
@@ -55,7 +55,7 @@ The recorder doesn't record sound, the headset's outward cameras, your name, you
 ## Where it goes, and who can read it
 
 - The dataset is hosted on Hugging Face (<https://huggingface.co/datasets/DeeJanuz/frametop-eyes>), whose servers may be outside your country, for example in the USA. It's gated: people must accept its terms to download it, and what they get has the eye images and health answers encrypted.
-- Only the maintainers hold the keys that decrypt them. Today that's DeeJanuz, with one key on their own computer and a backup copy stored offline. The keys are never uploaded. If a key ever leaked, uploads made to it could be read: the maintainers would say so on the dataset's page and stop using it.
+- Only the maintainers hold the keys that decrypt them. Today that's DeeJanuz, with a key on each of two of their own computers (one of them trains the models) and a backup copy stored offline. The keys are never uploaded. If a key ever leaked, uploads made to it could be read: the maintainers would say so on the dataset's page and stop using it.
 - Decrypted images are kept only on the maintainers' own computers, for training and checking models, and aren't shared with anyone else or uploaded to other services.
 - Recordings stay in the dataset until they're withdrawn or the dataset is taken down.
 

gaze/rec/DESIGN.md

@@ -1,6 +1,6 @@
 # Eye recorder: design (2026-10-14)
 
-The eye recorder guides a person through recording their eyes with the headset's two eye cameras, at known gaze targets and through the cases where eye tracking breaks. It also fits the headset: it shows whether each eye sits where the cameras see it best, helps set the IPD, and catches the headset shifting on the face, with instructions to put it back. Recordings stay on the headset until the person uploads them. Only the maintainers can read what's uploaded (below, "Privacy").
+The eye recorder guides a person through recording their eyes with the headset's two eye cameras, at known gaze targets and through the cases where eye tracking breaks. It also fits the headset: it shows whether each eye sits where the cameras see it best, shows the IPD setting, and catches the headset shifting on the face, with instructions to put it back. Recordings stay on the headset until the person uploads them. Only the maintainers can read what's uploaded (below, "Privacy").
 
 It's the hand recorder's sibling (`hands/rec/DESIGN.md`): the same session runner pattern, headset button, review, export and Hugging Face upload, with eye cameras instead of hand cameras. Like the hand recorder it ships in Frametop and, later, as a standalone app for people without Frametop (Frametop/frametop-eye-recorder, Frametop as a submodule).
 
@@ -69,7 +69,7 @@ What it can tell, and the instruction it gives:
 | A glint missing, or extra bright spots | Glasses reflections, or the lens fogged or smudged | "Tilt your glasses" / "Wipe the lenses" |
 | An eye found in under half the frames, looking straight ahead | Lashes on the lens, a droopy lid, or the eye outside the camera's view | The per-eye card says which |
 
-The IPD row depends on the first headset test. If the cameras move with the lenses, a pupil's sideways offset in its image directly says how far that eye is from its lens axis, and the coach turns the IPD wheel's direction into "wider" or "narrower" until both offsets are about zero, at any IPD. If the cameras are fixed, the offset depends on the IPD setting too, and the coach needs the reference at each IPD step: the fitment step's IPD sweep (below, "Bounty board") builds that.
+The IPD row depends on the first headset test. If the cameras move with the lenses, a pupil's sideways offset in its image directly says how far that eye is from its lens axis, and the coach turns the IPD wheel's direction into "wider" or "narrower" until both offsets are about zero, at any IPD. If the cameras are fixed, the offset depends on the IPD setting too, and the coach needs the reference at each IPD step, built from rounds recorded at different settings (there's no IPD sweep: the setting can't be changed with the headset on).
 
 **Put-on and shift check.** After a good fit, the coach keeps that fit as the person's baseline (`~/.local/state/frametop/gaze/fit-baseline.json`: per eye, the pupil looking straight ahead and the glint pair, with the IPD setting). Then:
 - When the headset goes back on (frames resume after ft-eyes' 3 s gap), it shows the fit card for a few seconds while the person looks ahead. If the pupils sit more than a threshold (start with 1.5 mm equivalent, tuned from data) from the baseline, it says which way to move the headset, live, until it's back. If the IPD setting changed, it says so first.
@@ -79,10 +79,12 @@ In the recorder the fit coach runs as each round's fitment step, and the shift w
 ## Bounty board
 
 People pick what to record from a board of bounties. Each bounty is one round of about 5 minutes, in step mode: the headset's right button starts each step (press = next or pause, two presses = redo, hold = stop), as in the hand recorder. Every round starts the same way:
-1. **Fitment** (not recorded as a take, but its readings go into fit.json): the fit coach until the headset sits well (below, "Fit coach"). Skippable after a warning, and the round records that it was skipped. On a contributor's first round, and whenever the IPD setting has changed since their last, it ends with an IPD sweep: turn the wheel slowly from narrow to wide and back while looking at a centre dot, with the IPD logged as it changes (the panel looks 30 times a second, and on SteamVR's IPD event). Its own take, `NN-ipd`, before the nine dots.
+1. **Fitment** (not recorded as a take, but its readings go into fit.json): the fit coach until the headset sits well (below, "Fit coach"). Skippable after a warning, and the round records that it was skipped. (The coach is off for now, 2026-10-16: then it only asks for a comfortable fit.) It shows the IPD setting, read from SteamVR when the round starts. The setting can't be changed with the headset on (2026-10-16), so there's no IPD step: a wrong setting means stopping, setting it, and starting again. Each take's ipd.jsonl still logs it.
 2. **Nine dots** (recorded): a 3x3 grid to 20 degrees on the mid background. It's the round's own reference, so each round can be labelled alone, even after a re-seat.
 3. **The bounty's steps.**
 
+How to build a new one, and the rules a round script must follow (beeps for anything done with the eyes shut, cue words as labels, timing, reach, comfort): `rounds/README.md`.
+
 Every round uploads on its own. The board marks the bounties you've done, by your contributor id in `profile.json`, with how many times. You can do one again.
 
 The launch bounties:
@@ -90,7 +92,7 @@ The launch bounties:
 | Bounty | What | Edge case it covers |
 |---|---|---|
 | View edges | Dots at the panel's edges and corners: 30 degrees to the sides, 23 up and down, about 35 in the corners (the rec panel is 64 degrees across; every round's dots are checked against it) | Eyes turned far: the lid, the glints lost, the pupil foreshortened |
-| Blinks and lids | Blink on cue, close both for 2 s, wink each eye, squint, eyes wide, half-closed, looking down with lids lowered | Closed and partly closed eyes; one eye missing |
+| Blinks and lids | Blink on a beep, close both eyes on a beep and open them on the next (sounds.py: the eyes are shut, so words can't cue them), wink each eye, squint, eyes wide, half-closed, looking down with lids lowered | Closed and partly closed eyes; one eye missing |
 | Headset slips | Nudge the headset up, down, left, right, tilt it; then take it off and put it back on; dots after each | Headset shifting on the face: the put-on check's data |
 | Glasses | A 5x5 grid and the edges with glasses or inserts on (offered when the checklist says so) | Lens reflections, the glasses' frame in view |
 | Jumps and pursuit | A dot jumping between random spots (0.5 to 1.5 s each), then moving on circles, lines and a figure eight at two speeds | Saccades, overshoot, smooth pursuit, the tracker's lag |
@@ -137,7 +139,7 @@ ft-gazepanel gains `show rec`: a large head-locked panel (FULL_DEG, 64 degrees,
     session.json                the bounty, checklist, versions, takes, eye camera rate, IPD at start
     fit.json                    the fit coach's readings: the fitment step, and any slip warnings
     health.json                 the opt-in health answers (encrypted in the export)
-    takes/<NN>-<step group>/    numbered in order: NN-ipd (when there's a sweep), NN-dots (the nine dots), then the bounty's
+    takes/<NN>-<step group>/    numbered in order: NN-dots (the nine dots), then the bounty's
       cam0/000000.zst ...       right eye: one zstd chunk per second, frames back to back (512x400 u8)
       cam1/000000.zst ...       left eye
       index.tsv                 per frame: camera, chunk, offset in it, slot, CLOCK_MONOTONIC_RAW time
@@ -161,7 +163,7 @@ Uploads: `gaze/rec/hub.py` sends nothing to the real dataset until `CONSENT.md`
 
 ## Build order
 
-1. **Headset test 1:** do the eye cameras move with the IPD wheel? (Look at one dot, turn the wheel end to end, watch ft-eyes' pupil x.) Decides the IPD guidance.
+1. **Headset test 1:** do the eye cameras move with the lenses when the IPD changes? The setting can't be changed with the headset on, so compare nine-dot rounds recorded at two settings (ft-eyes' pupil x at the centre dot). Decides the IPD guidance.
 2. Pull the shared recorder code out of `hands/rec/` (hand recorder tests pass).
 3. `eyerec.py` (frame writer) and its replay test against a capture.
 4. `fitcoach.py` with readings from a capture, and the fit card on the existing fit panel.

gaze/rec/UPLOAD.md

@@ -1,4 +1,3 @@
-DRAFT: this text waits for its review, with CONSENT.md.
 ### About uploading
 
 - **What's sent:** the export in `@EXPORT_PATH@` (@EXPORT_SIZE@), into `contributions/@CONTRIBUTOR@/@ROUND@` in [@DATASET@](https://huggingface.co/datasets/@DATASET@). Upload first checks that every file is complete and matches its checksum, and that the eye images and health answers are encrypted.

gaze/rec/eyerec.py

@@ -165,6 +165,8 @@ class Writer:
             got = False
             for c in CAMERAS:
                 n = share.count(c)
+                if n <= seen[c]:   # nothing new (and never step back: that would copy frames twice)
+                    continue
                 if n - seen[c] > share.slots:   # fell behind: those are gone
                     with self._lock:
                         if self._on:

gaze/rec/ft-eyerec

@@ -1,6 +1,9 @@
 #!/bin/bash
-# Launch the Frametop Eye Recorder from a Plasma session on the Frame host.
-# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there).
+# Launch the Frametop Eye Recorder from a Plasma session on the Frame host (its app menu entry,
+# gaze/rec/install.sh).
+# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there; a
+# release's image has the recorders' cryptography and huggingface_hub in /opt/frametop/venv, which
+# distrobox leaves off the front of PATH, so it goes first here).
 # podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
 # the container's cgroup; the Frametop session runs on a private bus from
 # dbus-run-session). The session's Wayland socket and bus go to the app itself.
@@ -14,8 +17,8 @@ export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
 # From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
 # host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
 cd "$HOME" || exit 1
-"$here/../../scripts/container-up.sh"
-exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+exec "$here/../../scripts/in-box" env PATH="/opt/frametop/venv/bin:/opt/frametop/bin:$PATH:/usr/sbin:/sbin" \
+  WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
   XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
   QT_QPA_PLATFORM="wayland;xcb" \
   python3 "$here/ft_eyerec.py" "$@"

gaze/rec/ft_eyerec.py

@@ -416,11 +416,11 @@ class Backend(RecorderBackend):
         if not mod or not b:
             return ""
         try:
-            plan = mod.build_plan(mod.load_round(mod.round_path(b["script"])), 0, ipd_sweep=False)
+            plan = mod.build_plan(mod.load_round(mod.round_path(b["script"])), 0)
         except (OSError, ValueError) as e:
             return f"Couldn't read the round: {e}"
-        return (f"{len(plan)} steps, about {mod.plan_seconds(plan) / 60:.0f} min recorded, after the headset's fit "
-                "check (and an IPD sweep on your first round, or when the IPD setting changed)")
+        return (f"{len(plan)} steps, about {mod.plan_seconds(plan) / 60:.0f} min recorded, after putting the "
+                "headset on")
 
     # --- the checklist and health answers
     @Property("QVariantMap", notify=profileRelay)

gaze/rec/hub.py

@@ -8,7 +8,8 @@ nobody types a token anywhere. An upload:
   2. stops if this export (same SHA256SUMS) was uploaded before, unless --again;
   3. stops unless CONSENT.md and UPLOAD.md are here and neither is a draft. FT_EYEREC_ALLOW_UPLOAD=1
      opens it for a rehearsal, and only to another dataset than the real one (FT_EYEREC_DATASET):
-     the real one takes nothing before the texts are reviewed;
+     the real one takes nothing before the texts are reviewed; and it stops for an export made under
+     a draft consent text (agree to the current one, then export again);
   4. checks the login and access to the dataset, opens the pull request first, uploads the folder
      to contributions/<contributor>/<round> in it, and records it under "uploads" in session.json.
 --dry-run does all of it except the network calls and the records, and says what it would upload.
@@ -155,6 +156,9 @@ def upload(store, rnd, dry_run=False, again=False, progress=None, log=None):
     if not dry_run and not upload_allowed(repo):
         raise HubError("closed", "Eye contributions aren't open yet: the consent and upload texts are waiting for a "
                                  f"legal review. (A rehearsal: {ALLOW_ENV}=1 with {DATASET_ENV} set to a test dataset.)")
+    if repo == HF_DATASET and str(summary.get("consent_version") or "").endswith("draft"):
+        raise HubError("draft-consent", "This export was made when you'd agreed to a draft of the consent text. "
+                                        "Agree to the current one on the Welcome page, then export the round again.")
     path_in_repo = f"contributions/{summary.get('contributor', '')}/{rnd}"
     message, description = describe(summary, report, sha, report.bytes)
     files = []

gaze/rec/install.sh

@@ -5,6 +5,10 @@
 # runs) and add "Frametop Eye Recorder" to the app menu (for Frametop's desktop, as the hand
 # recorder's). It doesn't change Frametop's gaze setup; the window offers SteamVR's 90 fps eye
 # cameras (gaze/eye-rate.py) when they're slower.
+# A release (FRAME_RELEASE) brings the gaze tools built in its image (pack/Containerfile), so there it
+# only adds the menu entry; its install.sh offers this.
+# The recorder reads the eye cameras through our eye tracker's frame grabber
+# (frametop-eyegrab.service, gaze/tracker/install.sh): without it, the window says so.
 # Usage: gaze/rec/install.sh            install or update
 #        gaze/rec/install.sh uninstall  remove the menu entry. Recordings stay where they are.
 set -euo pipefail
@@ -14,17 +18,23 @@ entry='~/.local/share/applications/frametop-eyerec.desktop'
 
 case ${1:-install} in
   install)
-    echo "== 1/3 dev container packages"
-    "$root/setup/dev-container.sh"
-    echo "== 2/3 the gaze tools: ft-gaze, ft-gazepanel and ft-eyes"
-    "$root/gaze/build.sh"
-    "$root/gaze/tracker/build.sh"
-    echo "== 3/3 the menu entry"
+    if [ "$FRAME_RELEASE" = 1 ]; then
+      echo "== the menu entry"
+    else
+      echo "== 1/3 dev container packages"
+      "$root/setup/dev-container.sh"
+      echo "== 2/3 the gaze tools: ft-gaze, ft-gazepanel and ft-eyes"
+      "$root/gaze/build.sh"
+      "$root/gaze/tracker/build.sh"
+      echo "== 3/3 the menu entry"
+    fi
     fill_template "$root/gaze/rec/ft-eyerec.desktop" |
       on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
     echo
     echo "Installed. On Frametop's desktop, open \"Frametop Eye Recorder\" from the app menu."
     echo "Recordings go to ~/.local/share/frametop/eyes/contrib."
+    on_frame 'systemctl is-enabled -q frametop-eyegrab.service 2>/dev/null' ||
+      echo "It needs our eye tracker's frame grabber, which isn't installed: gaze/tracker/install.sh"
     ;;
   uninstall)
     on_frame "rm -f $entry"

gaze/rec/main.qml

@@ -298,7 +298,7 @@ Kirigami.ApplicationWindow {
                     padding: Kirigami.Units.largeSpacing
                     wrapMode: Text.Wrap
                     text: "Each bounty is a short round of about 5 minutes for one case where eye tracking struggles. "
-                          + "Pick one; the most wanted come first. Every round starts with fitting the headset and "
+                          + "Pick one; the most wanted come first. Every round starts with putting the headset on and "
                           + "nine dots. You can do a bounty again." + (backend.boardUpdated ? " (Board of "
                           + backend.boardUpdated + ".)" : "")
                 }
@@ -603,10 +603,10 @@ Kirigami.ApplicationWindow {
                 Controls.Label {
                     Layout.maximumWidth: Kirigami.Units.gridUnit * 26
                     wrapMode: Text.Wrap
-                    text: "Press Start, then put the headset on. A panel in the headset first helps you fit it: "
-                          + "look at the dot and move the headset until the panel says good (this window shows it "
-                          + "too). On your first round, and whenever the IPD setting changed, you then turn the IPD "
-                          + "wheel slowly from end to end. Then nine dots, then the bounty's steps. Look at each dot "
+                    text: "Press Start, then put the headset on so it's comfortable and the picture is sharp, "
+                          + "and look at the dot on the panel in the headset. It shows the headset's IPD setting: "
+                          + "it can't be changed with the headset on, so set it before you start. Then nine dots, "
+                          + "then the bounty's steps. Look at each dot "
                           + "until its ring fills, and keep your head still unless a step says otherwise.\n\n"
                           + "Each step waits until you're ready: press the button on the right side of the headset, "
                           + "or Space or Next in this window. A 3-2-1 countdown follows, then the step. Nothing is "
@@ -735,13 +735,6 @@ Kirigami.ApplicationWindow {
                         font: Kirigami.Theme.smallFont
                         text: "A first guess: the coach has been measured on one face so far."
                     }
-                    Controls.Label {
-                        visible: sessionView.st.ipd !== undefined && sessionView.st.ipd !== null
-                        Layout.fillWidth: true
-                        wrapMode: Text.Wrap
-                        opacity: 0.7
-                        text: "The headset's IPD setting: " + ((sessionView.st.ipd || 0) * 1000).toFixed(1) + " mm"
-                    }
                 }
             }
 
@@ -805,7 +798,8 @@ Kirigami.ApplicationWindow {
                     FitCard {
                         Layout.alignment: Qt.AlignTop
                         Layout.preferredWidth: Kirigami.Units.gridUnit * 18
-                        visible: sessionView.state === "fitment" || (sessionView.state === "paused" && sessionView.card !== null)
+                        visible: (sessionView.state === "fitment" && sessionView.st.fit_coach !== false)
+                                 || (sessionView.state === "paused" && sessionView.card !== null)
                         card: sessionView.card
                     }
 
@@ -821,6 +815,16 @@ Kirigami.ApplicationWindow {
                             text: sessionView.state === "fitment" ? "Headset position: " + (sessionView.st.fit_words || "")
                                                                   : (sessionView.st.prompt || "")
                         }
+                        // The IPD setting (SteamOS's, through SteamVR), shown while the headset goes on
+                        Controls.Label {
+                            Layout.fillWidth: true
+                            visible: sessionView.state === "fitment"
+                            wrapMode: Text.Wrap
+                            text: sessionView.st.ipd === undefined || sessionView.st.ipd === null
+                                  ? "Couldn't read the IPD setting from SteamVR."
+                                  : "Your IPD setting: " + (sessionView.st.ipd * 1000).toFixed(1) + " mm. If that's not "
+                                    + "right for you, stop (Esc), take the headset off, set it, and start again."
+                        }
                         // The countdown's 3, 2, 1, then a cue's word, big
                         Controls.Label {
                             visible: !!sessionView.st.big && sessionView.state !== "paused"
@@ -849,7 +853,9 @@ Kirigami.ApplicationWindow {
                             visible: sessionView.waiting && sessionView.state !== "paused"
                             opacity: 0.7
                             text: sessionView.state === "fitment"
-                                  ? "Press Next once the fit says good. Pressing it before warns first; a second press starts anyway."
+                                  ? (sessionView.st.fit_coach === false
+                                     ? "Put the headset on comfortably, with a sharp picture, look at the dot, then press Next."
+                                     : "Press Next once the fit says good. Pressing it before warns first; a second press starts anyway.")
                                   : (sessionView.st.ready_text || "Ready? Press Space or click Next")
                                     + ". A 3-2-1 countdown starts the recording."
                             wrapMode: Text.Wrap

gaze/rec/recipients.txt

@@ -1,4 +1,6 @@
 # The eye dataset's recipients (gaze/rec/DESIGN.md "Files"): exports encrypt each chunk and health.json
-# to all of them (recorder/age.py). Key 1 is kept on the maintainer's training machine, key 2 offline.
+# to all of them (recorder/age.py). Key 1 is kept on the 7i, key 2 offline,
+# key 3 on dody (the GPU training machine, which decrypts pull requests itself).
 age1a0tc98pj2s7gdfgrv4gam6ah0qv69ymf3t2cc25hcwepmats69hst35z2e
 age1r0zlq7kcyg4dz4y3vkl9j36jw3zc6ly6reddh9ag5kfy6aw0fy6s5hepat
+age1e5vwgc03s9mk3dj5j39lcgxg6qyvkml77qh7g8svygar49r2kd7svldt70

gaze/rec/rounds/README.md

@@ -0,0 +1,106 @@
+# Building an eye bounty
+
+A bounty is two things:
+
+- **A board entry** in `gaze/rec/bounties.json`: id, title, blurb, minutes, priority, the script it runs,
+  and what it needs from the checklist (`recorder/bounties.py` has the format).
+- **A round script**, `gaze/rec/rounds/<script>.json`: the steps (`session.py`'s docstring has the
+  format of every step kind).
+
+Scripts ship with the recorder. The dataset's copy of the board (`DeeJanuz/frametop-eyes`,
+`bounties.json`) can only describe, reorder, reprioritise or retire bounties, so a new bounty reaches
+people only with a Frametop release that carries its script.
+
+`load_round` enforces the rules marked **(checked)**: a script that breaks one doesn't load, and the
+tests fail.
+
+## What the round is for
+
+Every recorded second becomes training data, labelled by where the dot was. So:
+
+- **Know what each step labels.** In most steps the eyes are on a dot, and the dot's position is the
+  label. Where they aren't (eyes shut, reading, looking around freely), the cue words or the step's
+  kind say so, and training leaves those frames out or treats them as their own case.
+- **One edge case per bounty.** Say in the board's blurb and the DESIGN.md table which case it covers
+  and why the trackers fail there.
+- **Rounds open the same way.** The recorder adds putting the headset on and the nine dots before every
+  round. Don't repeat them. The group `dots` is theirs **(checked)**.
+
+## Cues the person can't see get a beep
+
+The panel can't cue someone whose eyes are shut. A cue that asks for that plays a beep
+(`gaze/rec/sounds.py`) as its third field: `["Close your eyes", 2.5, "close"]`.
+
+- **Blink:** a cue whose word asks for a blink plays `blink` **(checked)**.
+- **Both eyes shut:** a cue asking to close or shut both eyes plays `close` **(checked)**, and the next
+  cue plays `open` **(checked)**, so the person hears when to open them. A step never ends with the eyes
+  shut.
+- **Say it in the step's text**, which is read before Next: "Close your eyes when you hear a beep, and
+  then open your eyes when you hear the beep again." The words still show on the panel too.
+- **One eye or half-closed** (winks, squints, half-closed lids) keep words alone: the open eye reads
+  them. They may beep too.
+- **No voice.** Beeps only; generated speech sounds odd in the headset. A new kind of cue sound goes
+  into `sounds.SOUNDS` and the tests.
+
+## Cue words are labels
+
+Training reads the cue words in `prompts.jsonl` (`frame-eyes/train/data/ingest.py`, `shut_cams`):
+
+- A word with **blink, close, shut, wink or yawn** marks the eyes as shut while it shows. **Left** or
+  **right** limits that to one eye, and **half** means the lids are partly open, so the frames keep
+  their labels.
+- Use those words only for that. "Open", "Look", "Relax" and "Normal" are the words for going back.
+- Keep words short (40 characters at most): they show big on the panel.
+
+## Timing
+
+- **Dots:** a hold of 1.2 s or more after a settle of about 0.6 s (the defaults). The eyes take that
+  long to land on a dot and stay.
+- **Shut eyes:** 2 s or more closed, then 2 s or more open before anything else, so the eyes settle back
+  on the dot.
+- **Blinks on a beep:** uneven gaps, 1.2 to 2 s, so the beep can't be predicted.
+- **Length:** about 4 to 6 minutes recorded. Check with `python3 gaze/rec/session.py <script> --plan`, and
+  put that, rounded, in the board's `minutes`.
+
+## Where things can go
+
+- **Dots and paths stay on the panel** **(checked)**: about 30 degrees to the sides, 23 up and down, less
+  in the corners (`off_panel`).
+- **Distance:** 0.25 to 10 m **(checked)**. 1.5 m unless distance is what the bounty is about.
+- **Head still,** unless the bounty is about moving it: say "Keep your head still" in the text of dot steps.
+
+## Takes
+
+- Steps of the same `group` share one take, and a group's steps must follow each other **(checked)**.
+  Group steps that a reviewer would delete together.
+- A `reseat` step isn't recorded, and a round with one gets no shift check.
+- Every step waits for Next unless it has `"ready": false`. Only chain steps that follow on naturally.
+
+## Comfort, safety and privacy
+
+- Comfortable eye turns only: "as far as is comfortable", never "as far as you can".
+- No flashing: brightness changes are steps of a second or more (`light` levels), never a flicker.
+- Nothing that asks for sound, the outward cameras, or anything about the person beyond the checklist.
+  Talking is fine (it moves the face), but the microphone is never recorded.
+- Text to read is neutral and made up: no names, places or questions about the person.
+
+## Writing the text
+
+- Plain words, one instruction per step, the action first: "Look at each dot until its ring fills."
+- `|` starts a new line on the panel, `||` leaves a blank line.
+- Say what to do, not why. The why goes in the round's `intro` or the board's blurb.
+
+## Before it ships
+
+1. `python3 gaze/rec/session.py <script> --plan`: the steps and their length.
+2. The tests, in the dev container: `cd gaze/rec && for t in tests/test_*.py; do python3 $t; done`.
+   `test_shipped_rounds_load` loads every script; update its count.
+3. A dry run: `python3 gaze/rec/session.py <script> --dry-run --speed 20 --next-after 0.1`.
+4. A replay end to end: `frame-eyes/tools/eyerec-dryrun <bounty>` (record, export, validate), and the
+   export through `eyes_hub.py check` and `ingest.py` on dody. Delete those rounds afterwards: their
+   labels are wrong.
+5. A real round in the headset, with a note in `HEADSET-TESTS.md`: is every cue clear, and is the
+   length right?
+6. Add the bounty to `gaze/rec/bounties.json` and a row to DESIGN.md's bounty table. After a release
+   carries the script, put it on the dataset's board too (`frame-eyes/dataset-repo/bounties.json`,
+   uploaded to `DeeJanuz/frametop-eyes`).

gaze/rec/rounds/blinks.json

@@ -2,12 +2,12 @@
  "schema": 1,
  "id": "blinks",
  "title": "Blinks and lids",
- "intro": "Blinks, closed and half-closed eyes. Keep looking at the dot whenever your eyes are open.",
+ "intro": "Blinks, closed and half-closed eyes, some of them on a beep. Keep looking at the dot whenever your eyes are open.",
  "steps": [
   {
    "id": "blink-cue",
    "kind": "prompt",
-   "text": "Look at the dot and blink each time the word says so.",
+   "text": "Look at the dot and blink once on each beep.",
    "dot": [
     0,
     0
@@ -15,42 +15,50 @@
    "cues": [
     [
      "Blink",
-     1.5
+     1.6,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.3,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     2.0,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.5,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.8,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.2,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.7,
+     "blink"
     ],
     [
      "Blink",
-     1.5
+     1.5,
+     "blink"
     ]
    ]
   },
   {
    "id": "closed",
    "kind": "prompt",
-   "text": "Look at the dot, then close both eyes when it says so.",
+   "text": "Close your eyes when you hear a beep, and then open your eyes when you hear the beep again.",
    "dot": [
     0,
     0
@@ -61,20 +69,24 @@
      2
     ],
     [
-     "Close both eyes",
-     2.5
+     "Close your eyes",
+     2.5,
+     "close"
     ],
     [
-     "Open",
-     2
+     "Open your eyes",
+     2,
+     "open"
     ],
     [
-     "Close both eyes",
-     2.5
+     "Close your eyes",
+     2.5,
+     "close"
     ],
     [
-     "Open",
-     2
+     "Open your eyes",
+     2,
+     "open"
     ]
    ]
   },

gaze/rec/session.py

@@ -8,11 +8,13 @@ Then, in step mode:
 
   1. Fitment, not a take: the fit coach (gaze/fitcoach.py) on a centre dot until the headset
      sits well and the person presses Next; Next before that warns, a second press goes on and
-     the round records it was skipped. Its readings go into fit.json.
-  2. On the contributor's first round, or when SteamVR's IPD setting changed since their last,
-     an IPD sweep: one take, the wheel turned end to end while looking at a centre dot.
-  3. Nine dots, a take: a 3x3 grid to 20 degrees on the mid background, the round's reference.
-  4. The bounty's steps, from its round script (rounds/<script>.json, below).
+     the round records it was skipped. Its readings go into fit.json. The coach is off for now
+     (FIT_COACH): then this only asks for a comfortable fit, Next goes on, and its readings are
+     kept without being shown. Either way it shows the IPD setting, read from SteamVR when the
+     round starts: it can't be changed with the headset on, so there's no IPD step, and a wrong
+     setting means stopping and setting it first.
+  2. Nine dots, a take: a 3x3 grid to 20 degrees on the mid background, the round's reference.
+  3. The bounty's steps, from its round script (rounds/<script>.json, below).
 
 Each step that's "ready" shows first and waits for Next; then a 3-2-1 countdown, recorded, then
 the step. Steps of the same group share a take (takes/NN-group): the eye frames (eyerec.py),
@@ -34,7 +36,9 @@ Round scripts, rounds/<id>.json: {"schema": 1, "id", "title", "intro", "shift_ch
   path     "path": "circle|line|eight <numbers>" (the panel's path command), "seconds"
   jumps    "count", "range": [yaw, pitch] (a box), "each": [shortest, longest] s
   prompt   "dot": [yaw, pitch] or null (none), and "seconds" or "cues": [[word, s]...] (shown
-           big in turn), "cue_distance": {word: metres}
+           big in turn; [word, s, sound] also plays a beep as it starts, for steps done with the
+           eyes shut: sound blink, close or open, sounds.py; rounds/README.md has the rules,
+           and load_round enforces them), "cue_distance": {word: metres}
   light    "dot", "levels": [[background, s]...]
   world    "point": [x, y, z] (metres from the head when it starts, -z ahead), "seconds"
   reseat   not recorded: the text, Next, then a fit reading
@@ -76,6 +80,7 @@ sys.path.insert(0, GAZE)
 sys.path.insert(0, HERE)
 import eyerec  # noqa: E402  (gaze/rec/eyerec.py: the frame writer)
 import fitcoach  # noqa: E402  (gaze/fitcoach.py)
+import sounds  # noqa: E402  (gaze/rec/sounds.py: the beeps)
 from recorder import bounties  # noqa: E402
 from recorder.button import BUTTON_DEVICE, ButtonReader, find_button, mouse_connected, read_input_devices  # noqa: E402
 from recorder.common import (clean_text, clock_sample, in_container, mono_ns, os_version,  # noqa: E402
@@ -110,7 +115,6 @@ FIT_ASK_S = 0.5           # the fit coach's reading, this often during the fitme
 FIT_GOOD_S = 2.0          # OK this long: the fit is good
 READING_WAIT_S = 6.0      # a reseat's fit reading waits this long for both eyes
 CENTRE_DEG = 1.0          # a dot this near the middle gets the shift check after its hold
-SPLIT_IPD_M = 0.0005      # the IPD setting changed: an IPD sweep
 VERSION = "ft-eyerec 0.1"
 READY_TEXT = "Ready? Press Space or click Next"
 READY_BUTTON = "Ready? Press the button on the right side of the headset"
@@ -124,11 +128,20 @@ FIT_TITLE = "Headset fit"
 FIT_TEXT = "Look at the dot. Adjust the headset until this says good."
 FIT_ADJUST = "Adjust the headset until it says good"
 FIT_SKIP_NOTE = "The fit isn't good yet: recordings may be less useful. Press again to start anyway."
-IPD_STEP = {"id": "ipd", "kind": "prompt", "group": "ipd", "dot": [0, 0], "seconds": 25,
-            "text": "Keep looking at the dot and turn the IPD wheel slowly: all the way narrow, then all the "
-                    "way wide, then back to where the picture is sharpest."}
+# The fit coach is off until it's trained on more people's fits (2026-10-16: its hints jumped
+# around and "good" was out of reach). Off, the fitment only asks for a comfortable fit and a look
+# at the dot, Next goes straight on, and the coach's readings still go into fit.json, silently:
+# they're what it will be trained on. No baseline then, so no slip notes either.
+# FT_EYEREC_FIT_COACH=1 (or session.py --fit-coach) turns it back on.
+FIT_COACH = os.environ.get("FT_EYEREC_FIT_COACH", "") == "1"
+FIT_OFF_TITLE = "Put the headset on"
+FIT_OFF_TEXT = "Put the headset on so it's comfortable and the picture is sharp. Then look at the dot."
+# The IPD setting is the headset's (SteamOS, read through SteamVR), shown at the fitment. It can't be
+# changed with the headset on, so a wrong one means stopping the round.
+IPD_TEXT = "Your IPD setting: {mm:.1f} mm. If that's not right for you, stop, take the headset off and set it."
+IPD_UNKNOWN = "Couldn't read the IPD setting from SteamVR."
 NINE_DOTS_TEXT = "Look at each dot until its ring fills. Keep your head still."
-DONE_SCREEN = {"title": "All done", "text": "Thank you! You can take off the headset.|Review the round in the Eye "
+DONE_SCREEN = {"title": "All done", "text": "Thank you! This recording session will close. Continue in the application.|Review the round in the Eye "
                                             "recorder window.", "seconds": 5}
 STOPPED_SCREEN = {"title": "Stopped", "text": "What was recorded so far is saved. Review it in the Eye recorder window.",
                   "seconds": 4}
@@ -203,6 +216,32 @@ def _dot(v, what):
     return d
 
 
+def cue_needs(word):
+    """The sound a cue must have because the person can't see the panel then (rounds/README.md):
+    "blink" for a blink, "close" for shutting both eyes, else None. Half-closed eyes and one eye
+    (a wink: the other one reads the panel) don't need one."""
+    w = word.lower()
+    if "half" in w or "left" in w or "right" in w:
+        return None
+    if "blink" in w:
+        return "blink"
+    if "close" in w or "shut" in w:
+        return "close"
+    return None
+
+
+def _check_sounds(cues, sid):
+    """Eyes-shut cues beep, and the cue after a "close" plays "open": with the eyes shut, the
+    words can't tell anyone when to open them."""
+    for k, (word, _secs, *sound) in enumerate(cues):
+        need = cue_needs(word)
+        if need and sound != [need]:
+            raise ValueError('%s: cue "%s" needs the sound "%s" (the eyes are shut then; rounds/README.md)'
+                             % (sid, word, need))
+        if sound == ["close"] and (k + 1 == len(cues) or cues[k + 1][2:] != ["open"]):
+            raise ValueError('%s: after "%s" comes a cue with the sound "open" (rounds/README.md)' % (sid, word))
+
+
 def _step(raw, i):
     """One step, checked and filled in."""
     if not isinstance(raw, dict):
@@ -263,12 +302,16 @@ def _step(raw, i):
         st["dot"] = None if "dot" in raw and raw["dot"] is None else _dot(raw.get("dot", [0, 0]), sid + " dot")
         cues = raw.get("cues")
         if cues is not None:
+            want = "%s: cues want [[word, seconds]...] or [word, seconds, sound] (sound: %s)" % (
+                sid, ", ".join(sounds.SOUNDS))
             if not isinstance(cues, list) or not cues:
-                raise ValueError("%s: cues want [[word, seconds]...]" % sid)
-            st["cues"] = [[clean_text(c[0])[:40], _num(c[1], 0.2, 60, sid + " cue")] for c in cues
-                          if isinstance(c, list) and len(c) == 2 and isinstance(c[0], str)]
+                raise ValueError(want)
+            st["cues"] = [[clean_text(c[0])[:40], _num(c[1], 0.2, 60, sid + " cue")] + c[2:] for c in cues
+                          if isinstance(c, list) and len(c) in (2, 3) and isinstance(c[0], str)
+                          and (len(c) == 2 or c[2] in sounds.SOUNDS)]
             if len(st["cues"]) != len(cues):
-                raise ValueError("%s: cues want [[word, seconds]...]" % sid)
+                raise ValueError(want)
+            _check_sounds(st["cues"], sid)
             cd = raw.get("cue_distance") or {}
             st["cue_distance"] = {str(k): _num(v, 0.25, 10, sid + " cue distance") for k, v in cd.items()}
         else:
@@ -311,8 +354,8 @@ def load_round(path):
             raise ValueError("%s: group %s comes back after another" % (path, s["group"]))
         groups.add(s["group"])
         last = s["group"]
-    if groups & {"dots", "ipd"}:
-        raise ValueError("%s: the groups dots and ipd are the opener's" % path)
+    if "dots" in groups:
+        raise ValueError("%s: the group dots is the opener's" % path)
     rid = raw.get("id")
     return {"id": rid if isinstance(rid, str) else os.path.basename(path)[:-5], "title": str(raw.get("title", rid)),
             "intro": str(raw.get("intro", "")), "shift_check": raw.get("shift_check", True) is not False and
@@ -346,13 +389,10 @@ def nine_dots(rng):
             "shuffle": False}
 
 
-def build_plan(rnd, seed, ipd_sweep):
+def build_plan(rnd, seed):
     """The round's steps in order, the opener's first, shuffles and jumps made from seed."""
     rng = random.Random(seed)
-    steps = []
-    if ipd_sweep:
-        steps.append(_step(IPD_STEP, 0))
-    steps.append(nine_dots(rng))
+    steps = [nine_dots(rng)]
     for s in rnd["steps"]:
         s = json.loads(json.dumps(s))
         if s["kind"] == "dots" and s["shuffle"]:
@@ -561,7 +601,7 @@ class Session:
     def __init__(self, base_dir, script, *, bounty=None, profile=None, checklist=None, health=None, dry_run=False,
                  speed=1.0, on_status=None, next_after=None, cams=None, panel_bin=None, panel_no_vr=False,
                  steamvr_gaze=True, button=True, button_device=None, seed=None, rounds_dir=None,
-                 profile_path=None, eyes_python=None):
+                 profile_path=None, eyes_python=None, fit_coach=None):
         self.base_dir = os.path.abspath(os.path.expanduser(base_dir or BASE_DIR))
         self.script = script
         self.round = load_round(round_path(script, rounds_dir))
@@ -575,6 +615,7 @@ class Session:
         self.steamvr_gaze = steamvr_gaze and not dry_run
         self.button, self.button_device = button, button_device
         self.eyes_python = eyes_python or EYES_PYTHON
+        self.fit_coach = FIT_COACH if fit_coach is None else bool(fit_coach)
         self.seed = seed
         self.print = print
         self.quiet_units = () if dry_run or cams else QUIET_UNITS
@@ -589,10 +630,11 @@ class Session:
         self._status = {"state": "starting", "bounty": self.bounty["id"], "title": "", "step": "", "step_index": 0,
                         "step_count": 0, "prompt": "", "note": "", "take": None, "error": "", "waiting": False,
                         "countdown": 0, "big": "", "can_redo": False, "fit": None, "fit_words": "", "fit_card": None,
-                        "ipd": None, "ready_text": READY_TEXT, "button": False, "seconds_left": 0.0}
+                        "ipd": None, "ready_text": READY_TEXT, "button": False, "seconds_left": 0.0,
+                        "fit_coach": self.fit_coach}
         self._last_emit = 0.0
         self._log_file = None
-        self._panel = self._eyes = None
+        self._panel = self._eyes = self._sounds = None
         self._procs = {}            # name: Popen
         self._gaze_thread = None
         self._gaze_lock = threading.Lock()
@@ -739,6 +781,7 @@ class Session:
     def _setup(self):
         self._panel = Socket(PANEL_SOCKET, self.dry_run, fake_panel, self._log, self.print)
         self._eyes = Socket(EYES_SOCKET, self.dry_run, fake_eyes(self.ref), self._log)
+        self._sounds = sounds.Player(on=not self.dry_run, log=self._log)
         if not self.dry_run:
             for path, build in ((self.panel_bin, "gaze/build.sh"), (FT_EYES, "gaze/tracker/build.sh")):
                 if not os.path.exists(path):
@@ -755,9 +798,8 @@ class Session:
         r = self._panel.cmd("ipd", reply=True)
         self._ipd = float(r.split()[1]) if r and r.startswith("ok ") else None
         self._status["ipd"] = self._ipd
-        last = self.profile.get("eyes_ipd")
-        sweep = self._ipd is not None and (not isinstance(last, (int, float)) or abs(self._ipd - last) >= SPLIT_IPD_M)
-        self.plan = build_plan(self.round, self.seed, sweep)
+        self._log("IPD setting: %s" % ("%.1f mm" % (self._ipd * 1000) if self._ipd is not None else "unknown"))
+        self.plan = build_plan(self.round, self.seed)
         self._fit["reference"] = {"source": self.ref.source, "faces": self.ref.faces, "made": self.ref.made,
                                   "provisional": self.ref.provisional} if self.ref else None
         self._session_json = {
@@ -765,7 +807,7 @@ class Session:
             "title": self.bounty.get("title") or self.round["title"], "board_updated": self.bounty.get("updated"),
             "checklist": self.checklist, "recorder": VERSION, "os": os_version(), "steamvr": steamvr_version(),
             "started": time.strftime("%Y-%m-%dT%H:%M:%S%z"), "ended": None, "status": "running", "seed": self.seed,
-            "dry_run": self.dry_run, "replay": self.cams or None, "ipd_start": self._ipd, "ipd_sweep": sweep,
+            "dry_run": self.dry_run, "replay": self.cams or None, "ipd_start": self._ipd,
             "fit": None, "eye_fps": None, "takes": [],
             "plan": [{"id": s["id"], "kind": s["kind"], "group": s["group"]} for s in self.plan]}
         if self.health:
@@ -1029,6 +1071,9 @@ class Session:
         self._panel.set("title", "title " + clean_text(title))
         self._panel.set("text", "text " + clean_text(text))
 
+    def _ipd_text(self):
+        return IPD_UNKNOWN if self._ipd is None else IPD_TEXT.format(mm=self._ipd * 1000)
+
     # --- the fit coach
     def _reading(self, previous=None, baseline=None):
         """The fit coach's reading of ft-eyes' last second, and the finds it's from (or None, None)."""
@@ -1039,10 +1084,13 @@ class Session:
 
     def _fitment(self):
         """The fit coach on a centre dot until the fit is good and Next is pressed; Next before
-        that warns, and a second press goes on (fit.json records it was skipped)."""
+        that warns, and a second press goes on (fit.json records it was skipped). With the coach
+        off (FIT_COACH), _fitment_quiet instead."""
+        if not self.fit_coach:
+            return self._fitment_quiet()
         title = "%s: %s" % (FIT_TITLE, self._session_json["title"])
         intro = self.round["intro"]
-        self._show(title, FIT_TEXT + ("||" + intro if intro else ""))
+        self._show(title, FIT_TEXT + "||" + self._ipd_text() + ("||" + intro if intro else ""))
         self._panel.cmd("dot 0 0 look")
         f = {"reading": None, "summary": None, "since": 0.0, "good": False, "ask": 0.0, "warned": False}
         started = mono_ns()
@@ -1069,7 +1117,7 @@ class Session:
                     self._panel.set("note", "note ")
             self._panel.cmd("dot 0 0 %s" % ("done" if good else "look"))
             self._panel.set("action", "action " + (self._status["ready_text"] if good or f["warned"] else FIT_ADJUST))
-            self._show(title, "Headset position: " + words + ("||" + intro if intro else ""))
+            self._show(title, "Headset position: " + words + "||" + self._ipd_text() + ("||" + intro if intro else ""))
             self._emit(force=False, fit=r["state"] if r else None, fit_words=words, fit_card=fit_card(r))
 
         self._redo_ok = False
@@ -1098,6 +1146,37 @@ class Session:
         self._panel.cmd("dot 0 0 off")
         self._emit(note="", fit=None, fit_words="", fit_card=None)
 
+    def _fitment_quiet(self):
+        """The fitment with the coach off: a comfortable fit and the centre dot, Next goes on.
+        The coach's readings are kept in fit.json, but nothing of them is shown."""
+        intro = self.round["intro"]
+        self._show("%s: %s" % (FIT_OFF_TITLE, self._session_json["title"]),
+                   FIT_OFF_TEXT + "||" + self._ipd_text() + ("||" + intro if intro else ""))
+        self._panel.cmd("dot 0 0 look")
+        self._panel.set("action", "action " + self._status["ready_text"])
+        f = {"reading": None, "ask": 0.0}
+        started = mono_ns()
+
+        def tick(dt):
+            f["ask"] -= dt
+            if f["ask"] > 0:
+                return
+            f["ask"] = FIT_ASK_S
+            r, _ = self._reading(f["reading"])
+            if r is not None:
+                f["reading"] = r
+                self._fit["readings"].append({"t": mono_ns(), "phase": "fitment", **compact(r)})
+
+        self._redo_ok = False
+        self._await_next(tick)
+        self._fit["fitment"] = {"from": started, "to": mono_ns(), "result": "off", "ipd": self._ipd,
+                                "last": compact(f["reading"]) if f["reading"] else None}
+        self._session_json["fit"] = "off"
+        self._save_session()
+        self._save_fit()
+        self._log("fitment: the coach is off%s" % ((", it read " + coach_words(f["reading"])) if f["reading"] else ""))
+        self._panel.cmd("dot 0 0 off")
+
     def _shift_check(self, step):
         """After a centre dot's hold: has the headset moved since the fitment?"""
         if not self.round["shift_check"] or not self._fit["baseline"]:
@@ -1134,7 +1213,8 @@ class Session:
         if r:
             self._fit["readings"].append({"t": mono_ns(), "phase": "reseat", "step": step["id"], **compact(r)})
             words = coach_words(r)
-            self._panel.set("note", "note Headset position: " + words if r["state"] != "ok" else "note ")
+            if self.fit_coach:
+                self._panel.set("note", "note Headset position: " + words if r["state"] != "ok" else "note ")
             self._log("    reseat: %s" % words)
         self._save_fit()
 
@@ -1295,7 +1375,7 @@ class Session:
                    countdown=0, big="", seconds_left=round(step_seconds(st), 1))
 
     def _title(self, st):
-        return {"ipd": "IPD sweep", "dots": "Nine dots"}.get(st["group"], self.round["title"])
+        return {"dots": "Nine dots"}.get(st["group"], self.round["title"])
 
     def _first_dot(self, st):
         """Where the eyes should be when the step starts: shown during its countdown."""
@@ -1390,12 +1470,17 @@ class Session:
             self._wait(st["seconds"])
             return
         total, gone = sum(c[1] for c in st["cues"]), 0.0
-        for word, secs in st["cues"]:
+        for word, secs, *sound in st["cues"]:
             self._panel.set("big", "big " + word)
             dist = st["cue_distance"].get(word)
             if dist:
                 self._panel.cmd("distance %.3f" % dist)
-            self._event("cue", id=st["id"], word=word, **({"distance": dist} if dist else {}))
+            self._event("cue", id=st["id"], word=word, **({"distance": dist} if dist else {}),
+                        **({"sound": sound[0]} if sound else {}))
+            if sound:
+                self._sounds.play(sound[0])
+                if self.dry_run:
+                    self.print("sound: " + sound[0])
             self._emit(big=word)
             self._wait(secs, bar=False)
             gone += secs
@@ -1468,6 +1553,7 @@ def main():
     ap.add_argument("--no-headset-button", action="store_true", help="don't read the headset's button")
     ap.add_argument("--button-device", metavar="PATH", help="test: read the button from this device or FIFO")
     ap.add_argument("--seed", type=int, help="the shuffles and jumps (default: from the round's id)")
+    ap.add_argument("--fit-coach", action="store_true", help="the fit coach on (off by default for now: FIT_COACH)")
     ap.add_argument("--glasses", choices=("none", "glasses", "inserts", "contacts"), help="the checklist's answer")
     a = ap.parse_args()
 
@@ -1504,8 +1590,8 @@ def main():
         print("round %s: %s" % (script, e))
         return 2
     if a.plan:
-        plan = build_plan(rnd, a.seed or 0, ipd_sweep=False)
-        print("%s: %d steps, about %.1f min recorded (plus the fitment, and the IPD sweep on a first round)"
+        plan = build_plan(rnd, a.seed or 0)
+        print("%s: %d steps, about %.1f min recorded (plus the fitment)"
               % (rnd["title"], len(plan), plan_seconds(plan) / 60))
         for s in plan:
             extra = len(s["dots"]) if "dots" in s else ""
@@ -1536,7 +1622,8 @@ def main():
     s = Session(base, script, bounty=entry, profile=profile, checklist=checklist, dry_run=a.dry_run, speed=a.speed,
                 on_status=on_status, next_after=a.next_after, cams=a.cams, panel_bin=a.panel, panel_no_vr=a.panel_no_vr,
                 steamvr_gaze=not a.no_steamvr_gaze, button=not a.no_headset_button and (not a.dry_run or a.button_device),
-                button_device=a.button_device, seed=a.seed, profile_path=profile_path, eyes_python=a.eyes_python)
+                button_device=a.button_device, seed=a.seed, profile_path=profile_path, eyes_python=a.eyes_python,
+                fit_coach=True if a.fit_coach else None)
 
     def on_signal(*_):
         print("stopping", flush=True)

gaze/rec/sounds.py

@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+"""The eye recorder's sound cues: plain beeps for steps done with the eyes shut, where the panel's
+words can't be seen. A round script names one as a cue's third field (session.py: "cues":
+[[word, seconds, sound]...]); it plays as the cue starts, and prompts.jsonl's "cue" line says which.
+
+  blink   a short high beep: blink once
+  close   a beep: close your eyes
+  open    the same beep again: open them
+
+No voice, no sound files: the beeps are made here (a sine with a short fade, so it doesn't click)
+into the run folder, and played with PipeWire's pw-play on the default output (the headset's
+speakers). A beep starts some tens of ms after its cue line (pw-play's start and its 20 ms
+latency); a person's reaction to it takes several times that.
+
+  sounds.py              play each one, a second apart
+  sounds.py NAME...      play those
+"""
+import math
+import os
+import shutil
+import struct
+import subprocess
+import sys
+import time
+import wave
+
+RATE = 48000
+VOLUME = 0.6        # pw-play's stream volume; the beep itself is at half scale
+# name: (pitch Hz, seconds)
+SOUNDS = {
+    "blink": (1000.0, 0.09),
+    "close": (660.0, 0.18),
+    "open": (660.0, 0.18),
+}
+FADE_S = 0.005
+
+
+def _dir():
+    d = os.path.join(os.environ.get("XDG_RUNTIME_DIR") or "/run/user/%d" % os.getuid(), "frametop", "eyerec-sounds")
+    os.makedirs(d, exist_ok=True)
+    return d
+
+
+def wav_path(name):
+    """The sound's WAV file, made the first time (and again when its pitch or length changed)."""
+    hz, secs = SOUNDS[name]
+    path = os.path.join(_dir(), "%s-%d-%d.wav" % (name, hz, secs * 1000))
+    if not os.path.exists(path):
+        n, fade = int(RATE * secs), int(RATE * FADE_S)
+        frames = bytearray()
+        for i in range(n):
+            env = min(1.0, i / fade, (n - 1 - i) / fade)
+            frames += struct.pack("<h", int(0.5 * 32767 * env * math.sin(2 * math.pi * hz * i / RATE)))
+        with wave.open(path + ".tmp", "wb") as w:
+            w.setnchannels(1)
+            w.setsampwidth(2)
+            w.setframerate(RATE)
+            w.writeframes(bytes(frames))
+        os.replace(path + ".tmp", path)
+    return path
+
+
+class Player:
+    """Plays sounds without waiting for them. Off (dry runs, tests): it only says what it would play.
+    Without pw-play, or when a sound fails, it logs that once and the round goes on silent."""
+
+    def __init__(self, on=True, log=None):
+        self.log = log or (lambda text: None)
+        self.pw_play = shutil.which("pw-play") if on else None
+        self.procs = []
+        self.warned = False
+        if on and not self.pw_play:
+            self._warn("no pw-play: the sound cues are off")
+        if self.pw_play:
+            for name in SOUNDS:     # made now, not while a step runs
+                wav_path(name)
+
+    def _warn(self, text):
+        if not self.warned:
+            self.warned = True
+            self.log(text)
+
+    def play(self, name):
+        if name not in SOUNDS:
+            raise ValueError("no sound %r" % name)
+        if not self.pw_play:
+            return False
+        self.procs = [p for p in self.procs if p.poll() is None]
+        env = dict(os.environ, XDG_RUNTIME_DIR=os.environ.get("XDG_RUNTIME_DIR") or "/run/user/%d" % os.getuid())
+        try:
+            self.procs.append(subprocess.Popen(
+                [self.pw_play, "--latency", "20ms", "--volume", str(VOLUME), "--media-role", "Notification",
+                 wav_path(name)], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                env=env))
+            return True
+        except OSError as e:
+            self._warn("can't play the sound cues: %s" % e)
+            return False
+
+    def close(self):
+        for p in self.procs:
+            try:
+                p.wait(timeout=1)
+            except subprocess.TimeoutExpired:
+                p.kill()
+        self.procs = []
+
+
+def main(argv):
+    names = argv or list(SOUNDS)
+    bad = [n for n in names if n not in SOUNDS]
+    if bad:
+        sys.exit("no sound %s; there are: %s" % (", ".join(bad), ", ".join(SOUNDS)))
+    p = Player(log=lambda text: print(text, file=sys.stderr))
+    for i, name in enumerate(names):
+        if i:
+            time.sleep(1.0)
+        print(name)
+        p.play(name)
+    p.close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

gaze/rec/tests/test_hub.py

@@ -63,16 +63,35 @@ class HubTest(unittest.TestCase):
         self.assertEqual(hub.rhub.uploads(hub.meta_path(self.store, RID)), [])
 
     def test_gate(self):
-        self.assertFalse(hub.texts_ready(), "the texts are drafts")
-        with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: ""}):
-            self.assertFalse(hub.upload_allowed(hub.dataset_id()), "never the real dataset before the review")
-            with self.assertRaises(hub.HubError) as cm:
-                hub.upload(self.store, RID)
-            self.assertEqual(cm.exception.kind, "closed")
-        with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: "a/test"}):
+        self.assertTrue(hub.texts_ready(), "the texts are reviewed (2026-10-16)")
+        with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "", hub.DATASET_ENV: ""}):
             self.assertTrue(hub.upload_allowed(hub.dataset_id()))
-        with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "", hub.DATASET_ENV: "a/test"}):
-            self.assertFalse(hub.upload_allowed(hub.dataset_id()))
+        # While a text is a draft: never the real dataset, a rehearsal only with ALLOW_ENV.
+        with mock.patch.object(hub, "texts_ready", return_value=False):
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: ""}):
+                self.assertFalse(hub.upload_allowed(hub.dataset_id()), "never the real dataset before the review")
+                with self.assertRaises(hub.HubError) as cm:
+                    hub.upload(self.store, RID)
+                self.assertEqual(cm.exception.kind, "closed")
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "1", hub.DATASET_ENV: "a/test"}):
+                self.assertTrue(hub.upload_allowed(hub.dataset_id()))
+            with mock.patch.dict(os.environ, {hub.ALLOW_ENV: "", hub.DATASET_ENV: "a/test"}):
+                self.assertFalse(hub.upload_allowed(hub.dataset_id()))
+
+    def test_draft_consent_blocks(self):
+        # An export made under a draft consent text never goes to the real dataset.
+        prof = os.path.join(self.store.base, "profile.json")
+        with open(prof) as f:
+            p = json.load(f)
+        p["consent"]["version"] = "2026-10-14-draft"
+        with open(prof, "w") as f:
+            json.dump(p, f)
+        shutil.rmtree(self.store.export_dir(RID))
+        self.store.export(RID, low_priority=False)
+        with mock.patch.dict(os.environ, {hub.DATASET_ENV: ""}):
+            with self.assertRaises(hub.HubError) as cm:
+                hub.upload(self.store, RID, dry_run=True)
+        self.assertEqual(cm.exception.kind, "draft-consent")
 
     def test_invalid_blocks(self):
         with open(os.path.join(self.store.export_dir(RID), "health.json"), "w") as f:

gaze/rec/tests/test_session.py

@@ -69,6 +69,21 @@ class RoundTest(unittest.TestCase):
         self.bad([{"id": "a", "kind": "dots", "group": "g", "dots": [[0, 0]]}, {"id": "b", "kind": "dots", "dots": [[0, 0]]},
                   {"id": "c", "kind": "dots", "group": "g", "dots": [[0, 0]]}], "comes back")
         self.bad([{"id": "a", "kind": "dots", "group": "dots", "dots": [[0, 0]]}], "opener")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1, "honk"]]}], "sound: blink")
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1, "blink", 2]]}], "cues want")
+        # The eyes-shut rules (rounds/README.md)
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Blink", 1]]}], 'needs the sound "blink"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Close both eyes", 2], ["Open", 2]]}], 'needs the sound "close"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Shut your eyes", 2, "close"]]}], 'comes a cue with the sound "open"')
+        self.bad([{"id": "a", "kind": "prompt", "cues": [["Close your eyes", 2, "close"], ["Open", 2]]}], '"open"')
+        for word in ("Close your left eye", "Half closed", "Squint", "Open"):
+            session.load_round(write_round(self.tmp, [{"id": "a", "kind": "prompt", "cues": [[word, 1]]}]))
+
+    def test_sound_cues(self):
+        r = session.load_round(write_round(self.tmp, [{"id": "a", "kind": "prompt",
+                                                       "cues": [["Look", 1], ["Blink", 1.5, "blink"]]}]))
+        self.assertEqual(r["steps"][0]["cues"], [["Look", 1.0], ["Blink", 1.5, "blink"]])
+        self.assertEqual(session.step_seconds(r["steps"][0]), 2.5)
 
     def test_grid_and_defaults(self):
         r = session.load_round(write_round(self.tmp, [{"id": "a", "kind": "dots", "grid": [3, 2, 10, 5], "ready": False},
@@ -81,18 +96,17 @@ class RoundTest(unittest.TestCase):
 
     def test_plan(self):
         rnd = session.load_round(os.path.join(session.ROUNDS_DIR, "jumps.json"))
-        a = session.build_plan(rnd, 7, ipd_sweep=True)
-        self.assertEqual([s["group"] for s in a[:3]], ["ipd", "dots", "jumps"])
-        nine = a[1]["dots"]
+        a = session.build_plan(rnd, 7)
+        self.assertEqual([s["group"] for s in a[:2]], ["dots", "jumps"])
+        nine = a[0]["dots"]
         self.assertEqual((nine[0], nine[-1], len(nine)), ([0.0, 0.0], [0.0, 0.0], 10))
         self.assertEqual(sorted(map(tuple, nine[1:-1])), sorted(map(tuple, (d for d in session.grid(3, 3, 20, 20) if d != [0, 0]))))
-        jumps = a[2]
+        jumps = a[1]
         self.assertEqual(len(jumps["dots"]), 40)
         self.assertTrue(all(0.5 <= t <= 1.5 for t in jumps["times"]))
         self.assertTrue(all(not session.off_panel(*d) for d in jumps["dots"]))
-        self.assertEqual(a, session.build_plan(rnd, 7, ipd_sweep=True), "the same seed, the same round")
-        self.assertNotEqual(a[1]["dots"], session.build_plan(rnd, 8, ipd_sweep=True)[1]["dots"])
-        self.assertEqual(session.build_plan(rnd, 7, ipd_sweep=False)[0]["group"], "dots")
+        self.assertEqual(a, session.build_plan(rnd, 7), "the same seed, the same round")
+        self.assertNotEqual(a[0]["dots"], session.build_plan(rnd, 8)[0]["dots"])
         self.assertGreater(session.plan_seconds(a), 60)
 
 
@@ -171,6 +185,7 @@ class DryRunTest(unittest.TestCase):
         with open(path, "w") as f:
             json.dump(profile or {}, f)
         states = []
+        kw.setdefault("fit_coach", True)   # the coach's own tests; test_fitment_coach_off has it off
         s = session.Session(self.tmp, script, profile=profile, dry_run=True, speed=kw.pop("speed", 40),
                             next_after=kw.pop("next_after", 0.25), profile_path=path, button=False,
                             on_status=lambda st: states.append(dict(st)), **kw)
@@ -198,8 +213,11 @@ class DryRunTest(unittest.TestCase):
         s, states, profile = self.run_round("depth")
         self.assertEqual(s.state, "done")
         sj, fit = self.files(s)
-        self.assertEqual(sj["takes"], ["01-ipd", "02-dots", "03-near", "04-mid", "05-far", "06-near-far"])
-        self.assertEqual((sj["status"], sj["fit"], sj["ipd_sweep"], sj["ipd_start"]), ("done", "good", True, 0.063))
+        # No IPD step: the setting is read and shown at the fitment.
+        self.assertEqual(sj["takes"], ["01-dots", "02-near", "03-mid", "04-far", "05-near-far"])
+        self.assertEqual((sj["status"], sj["fit"], sj["ipd_start"]), ("done", "good", 0.063))
+        self.assertNotIn("ipd_sweep", sj)
+        self.assertIn(session.IPD_TEXT.format(mm=63.0), "\n".join(self.out))
         self.assertEqual(fit["fitment"]["result"], "good")
         self.assertIsNotNone(fit["baseline"])
         self.assertEqual(profile["bounties"]["depth"]["rounds"], 1)
@@ -207,16 +225,29 @@ class DryRunTest(unittest.TestCase):
         for take in sj["takes"]:
             with open(os.path.join(s.session_dir, "takes", take, "take.json")) as f:
                 self.assertEqual(json.load(f)["status"], "complete", take)
-        events = [e["event"] for e in self.prompts(s, "03-near")]
+        events = [e["event"] for e in self.prompts(s, "02-near")]
         self.assertEqual(events[:4], ["take", "ready", "rec", "step"])
         self.assertEqual(events[-3:], ["wait", "rec", "end"])
         self.assertIn("ft_eyerecpanel: distance 0.500", self.out)
         self.assertIn("ft_eyerecpanel: distance 5.000", self.out)
-        # The IPD is the same next time: no sweep.
         s2, _, profile = self.run_round("depth", profile=profile)
         self.assertEqual(self.files(s2)[0]["takes"][0], "01-dots")
         self.assertEqual(profile["bounties"]["depth"]["rounds"], 2)
 
+    def test_blink_beeps(self):
+        # The blinks round's beeps: each in prompts.jsonl's cue line, played as the cue starts.
+        s, _, _ = self.run_round("blinks", profile={"eyes_ipd": 0.063})
+        self.assertEqual(s.state, "done")
+        played = [line[len("sound: "):] for line in self.out if line.startswith("sound: ")]
+        self.assertEqual(played, ["blink"] * 8 + ["close", "open", "close", "open"])
+        sj, _ = self.files(s)
+        [take] = [t for t in sj["takes"] if t.endswith("blink-cue")]
+        cues = [e for e in self.prompts(s, take) if e["event"] == "cue"]
+        self.assertEqual([(e["word"], e.get("sound")) for e in cues], [("Blink", "blink")] * 8)
+        [take] = [t for t in sj["takes"] if t.endswith("closed")]
+        cues = [e.get("sound") for e in self.prompts(s, take) if e["event"] == "cue"]
+        self.assertEqual(cues, [None, "close", "open", "close", "open"])
+
     def test_fitment_warns_then_skips(self):
         # Next before the fit has been good for FIT_GOOD_S: a warning, then the round goes on.
         s, states, _ = self.run_round("reading", next_after=0.01, speed=20)
@@ -225,6 +256,21 @@ class DryRunTest(unittest.TestCase):
         self.assertTrue(any(st["note"] == session.FIT_SKIP_NOTE for st in states))
         self.assertIsNone(fit["baseline"])
 
+    def test_fitment_coach_off(self):
+        # The default for now: no coaching or warning, the first Next goes on, and the coach's
+        # readings are still kept; no baseline, so no slip checks.
+        self.assertFalse(session.FIT_COACH)
+        s, states, _ = self.run_round("reading", next_after=0.6, speed=20, fit_coach=False)
+        sj, fit = self.files(s)
+        self.assertEqual(s.state, "done")
+        self.assertEqual((sj["fit"], fit["fitment"]["result"]), ("off", "off"))
+        self.assertTrue(fit["readings"])
+        self.assertIsNone(fit["baseline"])
+        self.assertFalse(fit["slips"])
+        self.assertFalse(any(st["note"] for st in states))
+        self.assertFalse(any(st["fit_words"] or st["fit_card"] for st in states))
+        self.assertTrue(all(st["fit_coach"] is False for st in states))
+
     def test_skip_redo_stop(self):
         def steer(s, states):
             def wait_for(cond):

gaze/rec/tests/test_sounds.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+"""sounds.py: the beeps' WAV files, and a player that never stops a round."""
+import os
+import shutil
+import sys
+import tempfile
+import unittest
+import wave
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, ".."))
+import sounds  # noqa: E402
+
+
+class SoundsTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="eyerec-sounds-")
+        self.env = mock.patch.dict(os.environ, {"XDG_RUNTIME_DIR": self.tmp})
+        self.env.start()
+
+    def tearDown(self):
+        self.env.stop()
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def test_wavs(self):
+        self.assertEqual(set(sounds.SOUNDS), {"blink", "close", "open"})
+        self.assertEqual(sounds.SOUNDS["close"], sounds.SOUNDS["open"], "open is the same beep again")
+        for name, (hz, secs) in sounds.SOUNDS.items():
+            path = sounds.wav_path(name)
+            self.assertTrue(path.startswith(self.tmp))
+            with wave.open(path) as w:
+                self.assertEqual((w.getnchannels(), w.getsampwidth(), w.getframerate()), (1, 2, sounds.RATE))
+                self.assertEqual(w.getnframes(), int(sounds.RATE * secs))
+                frames = w.readframes(w.getnframes())
+            first, last = int.from_bytes(frames[:2], "little", signed=True), int.from_bytes(frames[-2:], "little", signed=True)
+            self.assertEqual((first, last), (0, 0), "faded in and out: no click")
+
+    def test_player_off_and_missing(self):
+        logged = []
+        self.assertFalse(sounds.Player(on=False, log=logged.append).play("blink"))
+        self.assertEqual(logged, [])
+        with mock.patch.object(sounds.shutil, "which", return_value=None):
+            p = sounds.Player(log=logged.append)
+            self.assertFalse(p.play("close"))
+        self.assertEqual(logged, ["no pw-play: the sound cues are off"])
+        with self.assertRaises(ValueError):
+            p.play("honk")
+
+
+if __name__ == "__main__":
+    unittest.main()

gaze/tracker/build.sh

@@ -21,6 +21,12 @@ if [ "${FRAME_IN_BOX:-0}" = 1 ] && [ -x /opt/frametop/venv/bin/python ]; then
   python3 -m venv --without-pip build/venv
   /opt/frametop/venv/bin/python -c "import sysconfig; print(sysconfig.get_path(\"purelib\"))" \
     >"$(build/venv/bin/python -c "import sysconfig; print(sysconfig.get_path(\"purelib\"))")/frametop-image.pth"
+  # The eye net (eyes_net.py) needs ncnn, which the image does not have yet: add it here. It needs
+  # the network; without it ft-eyes uses the pupil finder.
+  if ! build/venv/bin/python -c "import ncnn" 2>/dev/null; then
+    { build/venv/bin/python -m ensurepip >/dev/null && build/venv/bin/python -m pip install -q --disable-pip-version-check \
+        --no-deps "$(grep -E "^ncnn==" requirements.txt)"; } || echo "build/venv: no ncnn (the eye net needs it)" >&2
+  fi
 elif ! cmp -s requirements.txt build/venv/requirements.done; then
   rm -rf build/venv
   python3 -m venv build/venv

gaze/tracker/eyes_net.py

@@ -0,0 +1,56 @@
+"""The eye net: one eye camera's frame in, that eye's raw gaze out (frame-eyes' train/gaze).
+
+A model folder holds model.json and the ncnn files it names:
+  {"name": "eyes-v0", "variant": "s100", "input": [100, 128], "gaze_scale": 30, "out": 6,
+   "param": "model.ncnn.param", "bin": "model.ncnn.bin", "in": "in0", "outname": "out0", ...}
+The net's output holds both cameras' heads (OUT numbers each, camera 0's first): gaze yaw and
+pitch / gaze_scale, pupil x / 512 and y / 400, eye open (logit), pupil visible (logit).
+
+The gaze is raw: each person's own offset is left to the calibration (eyes_model.Calibration,
+with this gaze where the pupil centre would be). So a calibration belongs to one model: ft-eyes
+keeps each model's in its own folder.
+
+ncnn fp16 on one thread: 1.4 ms a frame for s100 on an A720 core (frame-eyes notes/e0-runtime.md).
+"""
+import json
+import math
+import os
+
+import numpy as np
+
+W, H = 512, 400
+
+
+class Net:
+    def __init__(self, folder):
+        import cv2
+        import ncnn
+        self.cv2 = cv2
+        self.folder = folder
+        with open(os.path.join(folder, "model.json")) as f:
+            self.info = json.load(f)
+        self.name = str(self.info["name"])
+        self.in_h, self.in_w = self.info["input"]
+        self.scale = float(self.info.get("gaze_scale", 30.0))
+        self.out = int(self.info.get("out", 6))
+        self.in_name, self.out_name = self.info.get("in", "in0"), self.info.get("outname", "out0")
+        net = ncnn.Net()
+        net.opt.num_threads = 1
+        net.opt.use_vulkan_compute = False
+        net.opt.use_fp16_storage = net.opt.use_fp16_arithmetic = net.opt.use_fp16_packed = True
+        for key, load in (("param", net.load_param), ("bin", net.load_model)):
+            if load(os.path.join(folder, self.info[key])) != 0:
+                raise RuntimeError(f"{folder}: can't load {self.info[key]}")
+        self.ncnn, self.net = ncnn, net
+
+    def __call__(self, img, cam):
+        """img: one camera's 400x512 uint8 frame -> dict(gaze=(yaw, pitch), x, y, open, visible),
+        open and visible as probabilities."""
+        x = self.cv2.resize(img, (self.in_w, self.in_h), interpolation=self.cv2.INTER_AREA)
+        x = x.astype(np.float32)[None] * (1.0 / 255.0)
+        ex = self.net.create_extractor()
+        ex.input(self.in_name, self.ncnn.Mat(x))
+        o = np.array(ex.extract(self.out_name)[1]).reshape(-1)[cam * self.out:(cam + 1) * self.out]
+        sig = lambda v: 1.0 / (1.0 + math.exp(-float(v)))  # noqa: E731
+        return {"gaze": (float(o[0]) * self.scale, float(o[1]) * self.scale), "x": float(o[2]) * W,
+                "y": float(o[3]) * H, "open": sig(o[4]), "visible": sig(o[5])}

gaze/tracker/ft-eyes

@@ -38,10 +38,23 @@ Control socket: abstract datagram "@ft_eyes"; each command gets one reply line.
                                append every frame looked at to PATH as JSON lines (the eye
                                recorder's own.jsonl): {"t", "cam", "x", "y", "a", "b", "angle",
                                "glints": [[x, y], ...], "gaze": [yaw, pitch] once calibrated},
-                               or {"t", "cam", "lost": true}. "finds stop" replies "ok ROWS"
+                               or {"t", "cam", "lost": true}; with the net also "net": {"gaze",
+                               "x", "y", "open", "visible"}. "finds stop" replies "ok ROWS"
+
+The eye net (eyes_net.py) instead of the pupil finder: EYES_MODEL in ~/.config/frametop.conf (or
+FT_EYES_MODEL), picked on Input Settings' Gaze page: latest, a version's name (gaze/tracker/models.json,
+downloaded into gaze/tracker/models/NAME by models/catalog.py), or a model folder from frame-eyes'
+training (anything with a "/"); classic or unset is the pupil finder. Each eye's raw gaze from the net
+then stands where the pupil centre would: the calibration, the shifts clicks teach and the dots'
+spread are in its degrees, and there are no glints (no slip estimate or jump notice). Each model
+has its own state folder, net-NAME/ (calibration.json, state.json, clicks.jsonl), so switching
+asks for a calibration once per model and the classic one is kept. Read at start: the gaze
+service restarts ft-eyes when EYES_MODEL changes. If the model can't load, ft-eyes says
+so and uses the pupil finder. While the fit coach asks ("fit") or finds are logged, the pupil
+finder runs too, for them.
 
 Environment, for replays (lab/ft-eyes-e2e): FT_EYES_CAMS, FT_EYES_GAZE, FT_EYES_STATE (the
-state folder), FT_EYES_SOCKET (the socket's name). With FT_EYES_CAMS set, the want file isn't
+state folder), FT_EYES_SOCKET (the socket's name), FT_EYES_MODEL. With FT_EYES_CAMS set, the want file isn't
 touched.
 
 /dev/shm/frametop-eyes-gaze, 128 bytes, little-endian (mirrored in ft-gaze.cpp):
@@ -81,11 +94,40 @@ import eyes_model  # noqa: E402
 import eyes_pupil  # noqa: E402
 import fitcoach  # noqa: E402
 
+def load_net():
+    """The eye net EYES_MODEL picks (FT_EYES_MODEL wins), or None (the pupil finder). models/catalog.py
+    resolves it: classic (or unset: the catalog's default), latest (the newest version, once downloaded),
+    a downloaded version's name, or a folder of one's own (anything with a "/")."""
+    try:
+        sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "models"))
+        import catalog
+        r = catalog.resolve("eyes")
+    except Exception as e:  # noqa: BLE001 (a broken catalog mustn't stop the tracker)
+        print(f"ft-eyes: the model catalog didn't load ({type(e).__name__}: {e}); using the pupil finder",
+              file=sys.stderr, flush=True)
+        return None
+    if not r["folder"]:
+        if r["setting"] not in ("classic", "steamvr"):
+            print(f"ft-eyes: EYES_MODEL={r['setting']}: {r['why']}; using the pupil finder", file=sys.stderr, flush=True)
+        return None
+    try:
+        import eyes_net
+        return eyes_net.Net(r["folder"])
+    except Exception as e:  # noqa: BLE001 (any failure: say so and track with the pupil finder)
+        print(f"ft-eyes: the eye net in {r['folder']} didn't load ({type(e).__name__}: {e}); using the pupil finder",
+              file=sys.stderr, flush=True)
+        return None
+
+
+NET = load_net()
+
 CAMS = os.environ.get("FT_EYES_CAMS", "/dev/shm/frametop-eyes-cams")  # overrides for replays
 OUT = os.environ.get("FT_EYES_GAZE", "/dev/shm/frametop-eyes-gaze")
 WANT = None if "FT_EYES_CAMS" in os.environ else Path("/dev/shm/frametop-eyes-want")
 OUT_SIZE = 128
-CALIBRATION = Path(os.environ.get("FT_EYES_STATE", Path.home() / ".local/state/frametop/gaze/eyes")) / "calibration.json"
+STATE_BASE = Path(os.environ.get("FT_EYES_STATE", Path.home() / ".local/state/frametop/gaze/eyes"))
+FEATURE = "net:" + NET.name if NET else "pupil"   # what the calibration maps: each model has its own
+CALIBRATION = (STATE_BASE / ("net-" + NET.name) if NET else STATE_BASE) / "calibration.json"
 W, H = 512, 400
 FRESH = 0.03          # an eye's reading counts toward the output for this long (s)
 LOST_EVERY = 3        # while an eye is lost, search the whole frame only every 3rd frame
@@ -98,7 +140,8 @@ HISTORY = 12.0        # seconds of pupil positions kept per eye, for dots and cl
 GAP = 3.0             # s without frames: the headset was off, and may sit differently now
 CLICK_BEFORE = 0.3    # a click's frames: the 300 ms before it (like the probe's fixation)
 CALIB_MIN = 15        # frames an eye needs in a calibration dot's window
-CALIB_SPREAD = 4.0    # px: more than this and the eye moved during the dot
+CALIB_SPREAD = 2.5 if NET else 4.0   # px (the net: degrees): more than this and the eye moved during the dot
+CLASSIC_FOR = 3.0     # s: with the net, the pupil finder runs this long after the fit coach asks
 # Waiting for frames. They come only as a counter in shared memory, so there's nothing to block
 # on: sleep until a camera's next frame is due, then look every POLL. ft-eyegrab passes each one
 # on within a ms or two of when its camera starts the one after, so they arrive 11.1 ms apart,
@@ -187,6 +230,7 @@ class Eye:
         self.obs = None         # the fit coach's window (fitcoach.Observations), shared by both eyes
         self.fed = None         # (frame time, CLOCK_MONOTONIC_RAW then) of the newest frame fed
         self.finds = None       # the finds log (Tracker.finds), shared by both eyes
+        self.coach = [0.0]      # with the net: until when the fit coach wants pupils (monotonic), shared
 
     def use(self, cal, shift=None):
         self.cal = cal if cal is not None and cal.has("pupil", self.eye) else None
@@ -199,6 +243,8 @@ class Eye:
         if self.last_t is not None and t - self.last_t > GAP:
             self.shift.reseat()
         self.last_t = t
+        if NET:
+            return self.feed_net(t, img)
         if self.last is None:
             self.lost += 1
             if self.lost % LOST_EVERY:
@@ -227,6 +273,31 @@ class Eye:
             self.finds.write(self, t, p)
         self.work += time.perf_counter() - t0
 
+    def feed_net(self, t, img):
+        """The net's raw gaze is the feature: history, calibration and shifts as for a pupil."""
+        t0 = time.perf_counter()
+        r = NET(img, self.eye)
+        classic = self.finds is not None or time.monotonic() < self.coach[0]
+        p = None
+        if classic:
+            p = eyes_pupil.find_pupil(img, self.last)
+            self.last = p
+            if self.obs is not None:
+                self.obs.add(self.eye, t, p)
+                self.fed = (t, time.clock_gettime(time.CLOCK_MONOTONIC_RAW))
+        if r["visible"] >= 0.5:
+            self.found += 1
+            fx, fy = r["gaze"]
+            self.history.append((t, fx, fy, math.nan, math.nan))
+            while self.history and self.history[0][0] < t - HISTORY:
+                self.history.popleft()
+            if self.cal:
+                g = self.cal.gaze(self.eye, fx, fy, self.shift.value)
+                self.gaze = (t, float(g[0]), float(g[1]), r["x"], r["y"])
+        if self.finds is not None:
+            self.finds.write(self, t, p, r)
+        self.work += time.perf_counter() - t0
+
     def window(self, t0, t1):
         """Median pupil and glint midpoint over [t0, t1], the frame count, and the spread."""
         rows = np.array([r for r in self.history if t0 <= r[0] <= t1]).reshape(-1, 5)
@@ -246,13 +317,18 @@ class Finds:
         self.f = open(path, "a", buffering=1 << 16)
         self.rows = 0
 
-    def write(self, eye, t, p):
+    def write(self, eye, t, p, net=None):
         if p is None:
             row = {"t": round(t, 6), "cam": eye.eye, "lost": True}
         else:
             row = {"t": round(t, 6), "cam": eye.eye, "x": round(float(p["x"]), 2), "y": round(float(p["y"]), 2),
                    "a": round(float(p["a"]), 2), "b": round(float(p["b"]), 2), "angle": round(float(p["angle"]), 1),
                    "glints": [[round(float(x), 1), round(float(y), 1)] for x, y in p.get("glints") or []]}
+            if eye.gaze and eye.gaze[0] == t and not NET:
+                row["gaze"] = [round(eye.gaze[1], 3), round(eye.gaze[2], 3)]
+        if net is not None:
+            row["net"] = {"gaze": [round(v, 3) for v in net["gaze"]], "x": round(net["x"], 1), "y": round(net["y"], 1),
+                          "open": round(net["open"], 3), "visible": round(net["visible"], 3)}
             if eye.gaze and eye.gaze[0] == t:
                 row["gaze"] = [round(eye.gaze[1], 3), round(eye.gaze[2], 3)]
         self.f.write(json.dumps(row) + "\n")
@@ -270,10 +346,16 @@ class Tracker:
         self.dots = []  # calibration dots so far, in ft-eyes-score's click form
         self.calibrating = False
         self.fit = fitcoach.Observations()
+        coach = [0.0]
         for e in self.eyes:
-            e.obs = self.fit
+            e.obs, e.coach = self.fit, coach
         if CALIBRATION.exists():
             self.cal = eyes_model.Calibration.load(CALIBRATION)
+            if self.cal.info.get("feature", "pupil") != FEATURE:
+                print(f"ft-eyes: {CALIBRATION} is for {self.cal.info.get('feature', 'pupil')}, not {FEATURE}:"
+                      " calibrate again", file=sys.stderr, flush=True)
+                self.cal = None
+        if self.cal:
             if not self.cal.spread:
                 self.cal.spread = spread_from_dots(self.cal)
         shifts = {}
@@ -319,6 +401,7 @@ class Tracker:
                 return "ok"
             return "fail finds start PATH | finds stop"
         if w[0] == "fit":
+            self.eyes[0].coach[0] = time.monotonic() + CLASSIC_FOR
             # In frame time, which a replay's frames keep from the recording: the newest frame's,
             # plus how long ago it came (frames stopping, the headset off, empty the window).
             fed = [e.fed for e in self.eyes if e.fed]
@@ -345,7 +428,8 @@ class Tracker:
             if len(self.dots) < eyes_model.MIN_CLICKS:
                 return f"fail only {len(self.dots)} dots (need {eyes_model.MIN_CLICKS})"
             cal = eyes_model.Calibration.fit(self.dots, {"made": time.strftime("%Y-%m-%d %H:%M:%S"),
-                                                       "dots": len(self.dots), "from": "probe calibration"})
+                                                       "dots": len(self.dots), "from": "probe calibration",
+                                                       "feature": FEATURE})
             if not all(cal.has(n, c) for n in ("pupil", "where") for c in (0, 1)):
                 return "fail not enough dots with both eyes"
             errs = [float(np.hypot(*(np.mean([cal.gaze(c, *k["eye"][c]["pupil"]) for c in (0, 1)], axis=0)
@@ -389,6 +473,7 @@ class Tracker:
 
     def status(self):
         return {"calibration": self.cal.info if self.cal else None, "calibrating": self.calibrating,
+                "feature": FEATURE, "model": NET.folder if NET else None,
                 "dots": len(self.dots),
                 "eyes": {EYES[e.eye]: {"shift": e.shift.value.tolist(), "clicks": len(e.shift.meas),
                                        "jump": bool(np.any(e.shift.jump)), "jump_px": e.shift.jump.tolist(),
@@ -493,7 +578,8 @@ def main():
     sock.bind(SOCKET)
     sock.setblocking(False)
     cal = tracker.cal
-    print("ft-eyes: " + (f"calibration from {cal.info.get('made')} ({cal.info.get('from', cal.info.get('capture'))})"
+    print("ft-eyes: " + (f"the eye net {NET.name} ({NET.folder}); " if NET else "")
+          + (f"calibration from {cal.info.get('made')} ({cal.info.get('from', cal.info.get('capture'))})"
                            if cal else "not calibrated: run the probe's calibration with the Own tracker")
           + f"; waiting for {CAMS}", file=sys.stderr, flush=True)
     cams = wait_for_cams(sock, tracker, want)

gaze/tracker/lab/ft-eyes-replay

@@ -2,10 +2,12 @@
 """ft-eyes-replay: play a recording into the shared-frame layout, as ft-eyegrab --share
 would, at the recorded pace, to test ft-eyes without the headset.
 
-Usage: lab/py lab/ft-eyes-replay NAME [PATH] [--from S] [--for S]
+Usage: lab/py lab/ft-eyes-replay NAME [PATH] [--from S] [--for S] [--now]
 NAME is a recording (a bare name is in eyes_lab.CAPTURES). PATH defaults to
 /dev/shm/frametop-eyes-cams-replay; run ft-eyes with FT_EYES_CAMS=PATH (and FT_EYES_GAZE=...
-so it doesn't overwrite the live output).
+so it doesn't overwrite the live output). Frames keep the recording's times, unless --now: then
+they're stamped with today's CLOCK_MONOTONIC_RAW as they're written, as ft-eyegrab's are (for
+tools that line them up with something live, such as the eye recorder's dots).
 """
 import mmap
 import os
@@ -42,6 +44,7 @@ def main(argv):
     count = [0, 0]
     t0 = idx[0, 3] + start
     wall0 = time.monotonic()
+    shift = time.clock_gettime(time.CLOCK_MONOTONIC_RAW) - t0 if "--now" in argv else 0.0
     try:
         for i in range(len(frames)):
             t = idx[i, 3]
@@ -58,7 +61,7 @@ def main(argv):
             seq = struct.unpack_from("<Q", mm, off)[0]
             struct.pack_into("<Q", mm, off, seq + 1)
             mm[off + 64:off + 64 + W * H] = frames[i].tobytes()
-            struct.pack_into("<dQII", mm, off + 8, t, n, cam, int(idx[i, 1]))
+            struct.pack_into("<dQII", mm, off + 8, t + shift, n, cam, int(idx[i, 1]))
             struct.pack_into("<Q", mm, off, seq + 2)
             count[cam] = n + 1
             struct.pack_into("<Q", mm, 24 + 8 * cam, n + 1)

gaze/tracker/models.json

@@ -0,0 +1,59 @@
+{
+ "catalog": 1,
+ "kind": "eyes",
+ "latest": "eyes-v0",
+ "default": "classic",
+ "baseline": "steamvr",
+ "metrics": [
+  {"key": "fresh", "label": "Right after calibrating, median", "unit": "°", "better": "lower",
+   "note": "One person, a later recording after taking the headset off and on again, 124 click targets."},
+  {"key": "fresh_p90", "label": "Right after calibrating, 90% of looks within", "unit": "°", "better": "lower"},
+  {"key": "reseat_clicks", "label": "After a re-seat, old calibration + click corrections, median", "unit": "°", "better": "lower",
+   "note": "The headset was taken off and put back on; the calibration is the one from before, and the pointer's clicks correct it."},
+  {"key": "reseat_clicks_p90", "label": "After a re-seat, with click corrections, 90% within", "unit": "°", "better": "lower"},
+  {"key": "reseat", "label": "After a re-seat, old calibration alone, median", "unit": "°", "better": "lower"}
+ ],
+ "models": [
+  {
+   "name": "classic",
+   "title": "Frametop classic (pupil and glints)",
+   "kind": "classic",
+   "status": "builtin",
+   "summary": "Our hand-written tracker: finds each pupil and its glints. Best once your clicks have corrected it.",
+   "notes": "No learned model: it finds the pupil and the two glints in each eye camera's frame and maps them with your calibration. The glints show when the headset slips, so the pointer's clicks correct it quickly after a re-seat. Right after a calibration it's less precise than SteamVR's.",
+   "cost": "CPU, in Frametop's container, while the gaze is in use. Not yet measured side by side with the nets.",
+   "bench": {"fresh": 2.83, "fresh_p90": 4.71, "reseat_clicks": 1.25, "reseat_clicks_p90": 3.35, "reseat": 14.03}
+  },
+  {
+   "name": "steamvr",
+   "title": "SteamVR (Valve's tracker)",
+   "kind": "valve",
+   "status": "builtin",
+   "summary": "Valve's own eye tracking, as SteamVR reports it, with Frametop's calibration on top.",
+   "notes": "The most precise right after calibrating, and it needs nothing installed. It drifts when the headset moves, and our click corrections help it less than the classic tracker.",
+   "cost": "None for Frametop: Valve's tracker runs on the headset's own processors.",
+   "bench": {"fresh": 1.18, "fresh_p90": 2.32, "reseat_clicks": 3.11, "reseat_clicks_p90": 5.43, "reseat": 3.73}
+  },
+  {
+   "name": "eyes-v0",
+   "title": "Frametop eye net v0",
+   "kind": "custom",
+   "status": "testing",
+   "date": "2026-10-16",
+   "summary": "Experimental: our first deep-learning eye net. Better than classic right after calibrating, worse after the headset moves.",
+   "notes": "A small network that reads each eye camera's frame and gives a raw gaze, which your calibration fits (each net keeps its own calibration, so calibrate once after switching).\n\nTrained on one person in one headset seating, so it doesn't carry over well when the headset sits differently: after a re-seat, the classic tracker with click corrections is more than twice as precise. Try it to help test the pipeline; later versions trained on more people should do better.",
+   "cost": "CPU, while the gaze is in use: 1.4 ms per pair of eye frames in a benchmark (12% of one big core at 90 frames a second), about 1.2 ms per frame inside ft-eyes.",
+   "bench": {"fresh": 1.61, "fresh_p90": 3.23, "reseat_clicks": 2.87, "reseat_clicks_p90": 6.41, "reseat": 4.0},
+   "format": 1,
+   "rev": "5f4341cc53d698f622368c22bf4bae1addbe20a0",
+   "files": {
+    "NOTICE": "2894eeb7d9269890ea54a5546b3fd7ee1c7e4cba0d0c1c64f8207dab68de7acf",
+    "model.json": "9d7e252137348148597b1a951066053868d4504229520459bb31e95795cd2101",
+    "model.ncnn.bin": "230cac3489333257ad99ad1ce5ce6fa76fe60a80f4fa92fb8bc78c7f50a9a44a",
+    "model.ncnn.param": "ec82393354ff522bbd17be57d96a68ebdd1337915f48dfa86ac989f4756f984b",
+    "LICENSE": "41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0",
+    "README.md": "2625af9abfe4bd350c268f39c0f7ee315cd40f83ada2f93253a0abf68c2f78c2"
+   }
+  }
+ ]
+}

gaze/tracker/requirements.txt

@@ -3,3 +3,6 @@
 # VTK, GDAL, and over a gigabyte of map data; these wheels are about 165 MB.
 numpy==2.5.3
 opencv-python-headless==5.0.0.93
+# The eye net (eyes_net.py, EYES_MODEL): ncnn's own wheel. Not in the image yet: build.sh adds
+# it to build/venv there from this line.
+ncnn==1.0.20260526

hands/.gitignore

@@ -13,3 +13,5 @@ taps-out/
 tools/taps-*.sh
 # recorded demo clips (hands/rec/demo/camgifs.py) show the room around the recorder: never shipped
 rec/demos-recorded/
+# hand model versions picked in Input Settings (models/catalog.py downloads them; CC BY-NC 4.0)
+models/versions/

hands/README.md

@@ -50,7 +50,8 @@ Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides
 - `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
 - `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
 - `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
-- `HANDS_MODELS` (unset: `hands/models/student`, the fine-tuned models, when `hands/models.sh` fetched them, else `hands/models/ncnn`, the stock MediaPipe models): a folder holding `palm.ncnn.*` and `hand.ncnn.*`, as `--models`. Use it to try other models without passing options to every launcher. The fine-tuned folders lack the `*-int8` files, so `--int8` without `--models` uses the stock ones.
+- `HANDS_MODEL=latest`: which hand models to run, picked on Input Settings' Hands page with notes and benchmarks: `latest` (`hands/models/student`, kept current by `models.sh`), `stock` (`hands/models/ncnn`, Google's MediaPipe models), or a version from `hands/models.json`, downloaded into `hands/models/versions/NAME` (see `models/README.md`). A version that isn't downloaded falls back to latest.
+- `HANDS_MODELS` (unset: what `HANDS_MODEL` picks; by default `hands/models/student`, the fine-tuned models, when `hands/models.sh` fetched them, else `hands/models/ncnn`, the stock MediaPipe models): a folder holding `palm.ncnn.*` and `hand.ncnn.*`, as `--models`. Use it to try other models without passing options to every launcher. The fine-tuned folders lack the `*-int8` files, so `--int8` without `--models` uses the stock ones.
 - `HANDS_MODELS_UPDATE=1`: when hand tracking starts, fetch the fine-tuned models GitHub's branch pins if they changed, without a new release (`models.sh` below). `0` keeps the ones it came with.
 - `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
 - `HANDS_SKELETON=1`: fit a skeleton with your bone lengths and publish it (see "Skeleton" below); `0` publishes the filtered landmarks as before. As `--skeleton`.

hands/frametop-hands.service

@@ -14,6 +14,8 @@ PartOf=steamvr.service
 # every 15 minutes, never more than a minute, offline or failing it starts with the ones it has).
 ExecStartPre=-/usr/bin/pkill -x ft-hands
 ExecStartPre=-/usr/bin/timeout 60 @REPO@/hands/models.sh auto
+# A version picked by name (HANDS_MODEL, Input Settings' Hands page) that isn't downloaded yet.
+ExecStartPre=-/usr/bin/timeout 60 /usr/bin/python3 @REPO@/models/ft-models auto hands
 ExecStart=@REPO@/scripts/in-box @REPO@/hands/build/ft-hands --status 60
 ExecStopPost=-/usr/bin/pkill -x ft-hands
 Restart=always

hands/models.json

@@ -0,0 +1,80 @@
+{
+ "catalog": 1,
+ "kind": "hands",
+ "latest": "v16b",
+ "default": "latest",
+ "baseline": "stock",
+ "metrics": [
+  {"key": "replay_bright", "label": "Hands tracked, bright room (90 s replay)", "unit": "", "better": "higher",
+   "note": "The maintainer's recordings replayed through ft-hands with its default settings."},
+  {"key": "replay_dim", "label": "Hands tracked, dim room (120 s replay)", "unit": "", "better": "higher"},
+  {"key": "replay_lit", "label": "Hands tracked, lit room (120 s replay)", "unit": "", "better": "higher"},
+  {"key": "found", "label": "Hand found in ft-hands' crops (held-out people)", "unit": "%", "better": "higher",
+   "note": "Contributors the model never trained on (Frametop Hands #4, #7, #8)."},
+  {"key": "landmark_px", "label": "Finger point error, median (held-out people)", "unit": "px", "better": "lower"},
+  {"key": "within5", "label": "Points within 5% of hand size (held-out people)", "unit": "%", "better": "higher"},
+  {"key": "shape_mm", "label": "3D hand shape error, median (held-out people)", "unit": "mm", "better": "lower"},
+  {"key": "side_ok", "label": "Left or right hand told correctly (held-out people)", "unit": "%", "better": "higher"},
+  {"key": "pinch_closed", "label": "Thumb-index pinch read closed (held-out people)", "unit": "%", "better": "higher",
+   "note": "Of the held-out views where the thumb and index tips touch, the share the model reads as touching."}
+ ],
+ "models": [
+  {
+   "name": "v16b",
+   "title": "Frametop v16b",
+   "kind": "custom",
+   "status": "recommended",
+   "date": "2026-10-16",
+   "summary": "v13b-p1 plus the pinch and tap bounty rounds: pinches and taps read closed more often.",
+   "notes": "v13b-p1's landmark model trained further with Frametop Hands contributions #12 (index taps), #13 (pinches) and #14. The palm detector is v13b-p1's, unchanged. Same networks as stock, so the same CPU cost.\n\nWhat to expect against v13b-p1: a thumb touching the index finger is read as touching more often (80% of held-out touches against 71%), so pinches and taps register more reliably. Finger points and hand shape are as close as before, and the replays track the same hands.\n\nKnown weak spots: a thumb touching the middle finger is still often read as a near miss, and empty views are taken for a hand slightly more often than with v13b-p1 (0.37% against 0.22% of empty crops).",
+   "cost": "Same as stock: two networks, fp16, on the cores hand tracking already uses.",
+   "bench": {"replay_bright": 960, "replay_dim": 1225, "replay_lit": 1434, "found": 98.7, "landmark_px": 3.8,
+             "within5": 94.7, "shape_mm": 8.1, "side_ok": 96.6, "pinch_closed": 80.0},
+   "format": 1,
+   "rev": "d96e56599b0af18fc05af2a0a4489985914b754a",
+   "files": {
+    "NOTICE": "0d31ad8d108fc01e9fc4c5612eb5a9b16d90f9a971e6677cafe4969c2388fc06",
+    "hand.ncnn.bin": "c602ad2344bd1676a628fc8ab4fca2f53574b9f12bc97c705f47762d23f5f941",
+    "hand.ncnn.param": "54e08f284ff6f121ff00f4a9c65afc38c0dbb22a32f4af44fc4e0fde6c4a1bbd",
+    "palm.ncnn.bin": "9865f5fa3c6bc885cb0b50f201c4ac1bd83b2c1ad64af4e122efa3fc56d61909",
+    "palm.ncnn.param": "c61df2dde5b221a55c35c9693e530845a95ce2732949a9b3d3c583e3b2bae92e",
+    "LICENSE": "41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0",
+    "README.md": "322fc0b146d301865592958ba4dd512bf249836816b578f0919a2f671f3f014c"
+   }
+  },
+  {
+   "name": "v13b-p1",
+   "title": "Frametop v13b-p1",
+   "kind": "custom",
+   "status": "older",
+   "date": "2026-10-14",
+   "summary": "Retrained on the Steam Frame's own cameras: about twice as many hands found in bright rooms as stock, and steadier fingers.",
+   "notes": "MediaPipe's palm detector and hand landmark model, retrained on the Frametop Hands dataset (contributions #3 to #7), the maintainer's recordings and flagged tracking failures. Same networks as stock, so the same CPU cost.\n\nWhat to expect against stock: hands are found far more often, most of all in bright rooms (twice as many tracked in the bright replay, 1.3 to 1.8 times in dim and lit ones), the finger points sit about a quarter closer, and the 3D hand shape is about 40% closer.\n\nKnown weak spot: a thumb touching the middle finger is often read as a near miss, so middle-finger pinches (right click) can need a firmer touch.",
+   "cost": "Same as stock: two networks, fp16, on the cores hand tracking already uses.",
+   "bench": {"replay_bright": 960, "replay_dim": 1224, "replay_lit": 1433, "found": 98.7, "landmark_px": 3.8,
+             "within5": 94.6, "shape_mm": 8.3, "side_ok": 96.5, "pinch_closed": 71.3},
+   "format": 1,
+   "rev": "41afd78fbd4527456fc4cbd840ffc9f39fdc939a",
+   "files": {
+    "NOTICE": "0e6b4fa8ea37665cb699abc36fcf1ee0ffaa63a78dfee788be6613dea6607f84",
+    "hand.ncnn.bin": "d6a6f00d845fb9ee490276d84b1e978da884361c372614546ed2b1b4d2f5c449",
+    "hand.ncnn.param": "54e08f284ff6f121ff00f4a9c65afc38c0dbb22a32f4af44fc4e0fde6c4a1bbd",
+    "palm.ncnn.bin": "9865f5fa3c6bc885cb0b50f201c4ac1bd83b2c1ad64af4e122efa3fc56d61909",
+    "palm.ncnn.param": "c61df2dde5b221a55c35c9693e530845a95ce2732949a9b3d3c583e3b2bae92e",
+    "LICENSE": "41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0",
+    "README.md": "29c6c5460e47412a9a37b22d6a7c8a9ed266a872bbcce93d3867f9e56cfe0ca1"
+   }
+  },
+  {
+   "name": "stock",
+   "title": "MediaPipe (stock, off the shelf)",
+   "kind": "stock",
+   "status": "builtin",
+   "summary": "Google's original palm and hand models, unchanged. Comes with Frametop.",
+   "notes": "Trained on colour photos from ordinary lenses. The Frame's tracking cameras are infrared, monochrome and fisheye, so these lose hands often, most of all in bright rooms. Useful to compare, or to go back to if a retrained version tracks worse for you.",
+   "cost": "Two networks, fp16, on the cores hand tracking already uses.",
+   "bench": {"replay_bright": 442, "replay_dim": 912, "replay_lit": 782, "found": 94.8, "landmark_px": 5.2,
+             "within5": 88.5, "shape_mm": 14.3, "side_ok": 93.8, "pinch_closed": 7.8}
+  }
+ ]
+}

hands/models.pin

@@ -5,12 +5,12 @@
 # needs to run them: raise it when a model needs a newer ft-hands, and older installs keep theirs.
 # Made by frame-hands' model-repo/pin.py.
 format 1
-rev 41afd78fbd4527456fc4cbd840ffc9f39fdc939a
-model v13b-p1
-file NOTICE 0e6b4fa8ea37665cb699abc36fcf1ee0ffaa63a78dfee788be6613dea6607f84
-file hand.ncnn.bin d6a6f00d845fb9ee490276d84b1e978da884361c372614546ed2b1b4d2f5c449
+rev d96e56599b0af18fc05af2a0a4489985914b754a
+model v16b
+file NOTICE 0d31ad8d108fc01e9fc4c5612eb5a9b16d90f9a971e6677cafe4969c2388fc06
+file hand.ncnn.bin c602ad2344bd1676a628fc8ab4fca2f53574b9f12bc97c705f47762d23f5f941
 file hand.ncnn.param 54e08f284ff6f121ff00f4a9c65afc38c0dbb22a32f4af44fc4e0fde6c4a1bbd
 file palm.ncnn.bin 9865f5fa3c6bc885cb0b50f201c4ac1bd83b2c1ad64af4e122efa3fc56d61909
 file palm.ncnn.param c61df2dde5b221a55c35c9693e530845a95ce2732949a9b3d3c583e3b2bae92e
 file LICENSE 41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0
-file README.md 29c6c5460e47412a9a37b22d6a7c8a9ed266a872bbcce93d3867f9e56cfe0ca1
+file README.md 322fc0b146d301865592958ba4dd512bf249836816b578f0919a2f671f3f014c

hands/rec/bounties/card-grabs.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/bounties/hand-gestures.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/bounties/index-taps.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/bounties/middle-taps.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/bounties/pinches.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/bounties/pokes.json

@@ -12,7 +12,7 @@
  "done": {
   "title": "All done",
   "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."
  },
  "stopped": {
   "title": "Stopped",

hands/rec/ft-handrec

@@ -1,6 +1,8 @@
 #!/bin/bash
 # Launch the Frametop Hand Recorder from a Plasma session on the Frame host.
-# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there).
+# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there; a
+# release's image has the recorders' cryptography and huggingface_hub in /opt/frametop/venv, which
+# distrobox leaves off the front of PATH, so it goes first here).
 # podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
 # the container's cgroup; the Frametop session runs on a private bus from
 # dbus-run-session). The session's Wayland socket and bus go to the app itself.
@@ -14,7 +16,8 @@ export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
 # From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
 # host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
 cd "$HOME" || exit 1
-exec "$here/../../scripts/in-box" env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+exec "$here/../../scripts/in-box" env PATH="/opt/frametop/venv/bin:/opt/frametop/bin:$PATH:/usr/sbin:/sbin" \
+  WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
   XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
   QT_QPA_PLATFORM="wayland;xcb" \
   python3 "$here/ft_handrec.py" "$@"

hands/rec/script-v2.json

@@ -4,7 +4,7 @@
  "intro_s": 4,
  "between_s": 3,
  "welcome": {"title": "Hand recorder", "seconds": 8, "text": "Follow the steps on this panel. Each one plays a short clip of what to do: copy it. In the clips the palm is light and the back of the hand is dark.|The button on the right side of the headset goes on when a step waits for you, and pauses while recording."},
- "done": {"title": "All done", "seconds": 6, "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."},
+ "done": {"title": "All done", "seconds": 6, "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."},
  "stopped": {"title": "Stopped", "seconds": 4, "text": "What was recorded so far is saved. Review it in the Hand recorder window."},
  "objects": {"pencil": "pencil or pen", "phone": "phone", "cup": "cup", "keyboard": "keyboard", "mouse": "mouse", "gamepad": "gamepad", "small": "small thing"},
  "sections": [

hands/rec/script.json

@@ -5,7 +5,7 @@
  "welcome": {"title": "Hand recorder", "seconds": 7,
   "text": "Follow the steps on this panel. Each one shows what to do with your hands, and where. A bar shows the time left.|The button on the right side of the headset goes on when a step waits for you, and pauses while recording."},
  "done": {"title": "All done", "seconds": 6,
-  "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."},
+  "text": "Thank you! This recording session will close. Continue in the application.|Review your recordings in the Hand recorder window."},
  "stopped": {"title": "Stopped", "seconds": 4,
   "text": "What was recorded so far is saved. Review it in the Hand recorder window."},
  "cue_names": {"flat": "Flat", "flat-back": "Backs", "spread": "Spread", "open": "Open", "fist": "Fist",

hands/track/main.cpp

@@ -45,8 +45,9 @@
 // HANDS_CAMERAS, HANDS_BRIGHT, HANDS_BRIGHT_ON, HANDS_BRIGHT_OFF, HANDS_COLOR_LEFT (which
 // colour camera is passthrough_left: color_video0 or color_video3), HANDS_COLOR_CROP
 // (subtract or none: tools/check_color.py tells both), HANDS_MISREAD_GUARD (0 or 1: the
-// tracker's guards for fine-tuned landmark models, Tracker::set_misread_guard), HANDS_MODELS (as
-// --models: a folder with palm.ncnn.* and hand.ncnn.*, e.g. fine-tuned ones), HANDS_REACQUIRE (seconds
+// tracker's guards for fine-tuned landmark models, Tracker::set_misread_guard), HANDS_MODEL (latest,
+// stock, or a version's name: models/catalog.py), HANDS_MODELS (as --models: a folder with palm.ncnn.*
+// and hand.ncnn.*, used instead of HANDS_MODEL), HANDS_REACQUIRE (seconds
 // the palm search aims at a lost hand, 0 off: Tracker::set_reacquire), HANDS_PAIR_SIDES (0 or 1: two
 // hands are one left and one right, Tracker::set_pair_sides), HANDS_PINCH_DIP (0 or 1: pinches also
 // begin on a sharp dip of the thumb-index distance, PinchParams::dip), HANDS_PINCH_FIST_CURL (1.2; 0 off:
@@ -69,6 +70,7 @@
 #include <unistd.h>
 
 #include <algorithm>
+#include <cctype>
 #include <cmath>
 #include <ctime>
 #include <memory>
@@ -456,10 +458,29 @@ int main(int argc, char **argv) {
         std::fflush(stdout);
         return true;
     };
-    if (models.empty())
-        models = model_root + (!int8 && access((model_root + "student/hand.ncnn.param").c_str(), R_OK) == 0 &&
-                                       access((model_root + "student/palm.ncnn.param").c_str(), R_OK) == 0
-                                   ? "student" : "ncnn");
+    // HANDS_MODEL (Input Settings' Hands page, models/catalog.py): latest (models/student, which
+    // hands/models.sh keeps current), stock (models/ncnn), or a downloaded version (models/versions/NAME).
+    // None of the fine-tuned ones has 8-bit files, so --int8 always takes the stock ones.
+    auto has_models = [](const std::string &dir) {
+        return access((dir + "/hand.ncnn.param").c_str(), R_OK) == 0 && access((dir + "/palm.ncnn.param").c_str(), R_OK) == 0;
+    };
+    if (models.empty()) {
+        const std::string pick = setting("HANDS_MODEL");
+        const bool name_ok = !pick.empty() && pick.size() <= 64 && std::isalnum((unsigned char)pick[0]) &&
+                             pick.find_first_not_of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._-") ==
+                                 std::string::npos;
+        if (pick == "stock" || pick == "off" || pick == "none") {
+            models = model_root + "ncnn";
+        } else if (name_ok && pick != "latest") {
+            if (!int8 && has_models(model_root + "versions/" + pick)) models = model_root + "versions/" + pick;
+            else if (!int8)
+                std::fprintf(stderr, "HANDS_MODEL=%s isn't downloaded (Frametop Input Settings > Hands, or "
+                             "models/ft-models fetch hands %s): using the latest\n", pick.c_str(), pick.c_str());
+        } else if (!pick.empty() && pick != "latest") {
+            std::fprintf(stderr, "HANDS_MODEL=%s isn't a model name: using the latest\n", pick.c_str());
+        }
+        if (models.empty()) models = model_root + (!int8 && has_models(model_root + "student") ? "student" : "ncnn");
+    }
     if (!load_calibration(calib, err) || !ring.open(ring_path.c_str(), err) || !nets.load(models, int8, err) ||
         (publish && (!pub.open(err) || (gestures_on && !gestures.open(pinch, grip, err)))) ||
         (!record.empty() && !start_recording(record, err))) {

input-settings/ft_input_settings.py

@@ -23,14 +23,17 @@ to the input relay over its control socket (@frametop_relay):
     go to ~/.config/frametop-voice.json (voice/voice_settings.py); ft-voiced reads them again
     within 2 s, and answers "status" on its control socket.
   - Gaze: the pointer's gaze mode (@ft_pointer_helper "gaze") and the gaze service
-    (gaze/ft-gazed, @ft_gazed: status, forget, reload), with its eye tracker (SteamVR's or
-    our own) and eye bias (GAZE_TRACKER, GAZE_EYE in frametop.conf).
+    (gaze/ft-gazed, @ft_gazed: status, forget, reload), with its eye tracker and eye bias
+    (GAZE_TRACKER, GAZE_EYE in frametop.conf). The tracker is picked from a catalog (models/catalog.py,
+    gaze/tracker/models.json): SteamVR's, our classic one, or one of our eye nets (EYES_MODEL: latest,
+    which follows new versions, or a version), each with notes and benchmarks, downloaded on use.
   - Hands: when hand tracking runs (HANDS_MODE: with the Frametop desktop, always with SteamVR,
     or off; hands/handsmode.py on the host starts and stops its services), the hand cutouts'
     margin (HANDS_MATTE_MARGIN, sent live to ft-screens: "cutouts margin DEG"), and three hand
     inputs, each on its own (handsmode.METHODS): clicks at the gaze (POINTER_HANDS,
     POINTER_GAZE_PINCH), the hand ray with gaze off (POINTER_HAND_RAY), and direct touch on
-    Frametop's panels (HANDS_NEAR, sent live to ft-screens: "handinput on|off").
+    Frametop's panels (HANDS_NEAR, sent live to ft-screens: "handinput on|off"), and the hand
+    models (HANDS_MODEL, from hands/models.json: latest, a version, or stock MediaPipe), the same way.
   - Bluetooth: paired devices, and re-applying the Bluetooth LE fixes after pairing.
   - A warning on every page when SteamVR won't load the ft_pointer driver (blocked after a
     crash, disabled, or SteamVR in safe mode) or hasn't loaded it (@ft_pointer doesn't answer
@@ -71,6 +74,12 @@ GAZE_PROBE = os.path.join(REPO, "gaze", "probe", "ft-gazeprobe")
 sys.path.insert(0, os.path.join(REPO, "hands"))
 import handsmode  # noqa: E402
 HANDSMODE = os.path.join(REPO, "hands", "handsmode.py")
+# The hand and eye model catalogs: versions, notes, benchmarks, downloads (standard library only).
+sys.path.insert(0, os.path.join(REPO, "models"))
+import catalog as model_catalog  # noqa: E402
+# How the pickers tag each entry: by status, and builtins by what they are.
+MODEL_TAGS = {"recommended": "Recommended", "testing": "Testing", "older": "Older"}
+MODEL_KIND_TAGS = {"stock": "Off the shelf", "valve": "Valve", "classic": "Built in", "custom": "Frametop"}
 SCREENS = "\0ft_screens"
 HANDS_MODES = {"frametop": "With Frametop: while the Frametop desktop runs",
                "always": "Always: whenever SteamVR runs, in games and the SteamVR dashboard too",
@@ -454,6 +463,7 @@ class Backend(QObject):
     captured = Signal(int, str)  # code, name
     capturedController = Signal(str, str)  # button, label
     shortcutCaptureChanged = Signal()
+    modelsChanged = Signal()
     message = Signal(str, bool)  # text, is error
 
     def __init__(self):
@@ -485,6 +495,8 @@ class Backend(QObject):
         self._hands = {}  # handsmode.py's status, {} until it answers
         self._hands_lock = threading.Lock()  # one handsmode.py at a time
         self._hands_refreshing = False
+        self._model_busy = {"hands": "", "eyes": ""}  # what a model download is doing, per kind
+        self._model_views = {}
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
         self.sock.bind("")  # autobind an abstract address the relay can reply to
         self.sock.setblocking(False)
@@ -1594,6 +1606,166 @@ class Backend(QObject):
         self._send(f"cutouts margin {value}", SCREENS)
         self.handsChanged.emit()
 
+    # --- models (hands and eyes: models/catalog.py) ---
+    def _model_view(self, kind):
+        """The picker's view of a kind: the catalog's entries as options, a "latest" one first (it
+        follows new versions), which one the settings pick ("choice") and which version runs ("running")."""
+        try:
+            v = model_catalog.view(kind)
+        except (OSError, ValueError, model_catalog.CatalogError) as e:
+            return {"error": f"The {kind} model list didn't load: {e}", "options": [], "metrics": []}
+        conf = read_conf()
+        by = {m["name"]: m for m in v["models"]}
+        choice = v["setting"]
+        note = ""
+        if kind == "eyes" and self.gazeTracker == "steam":
+            choice = "steamvr"
+        elif "/" in choice:
+            note = f"EYES_MODEL is a folder of your own ({choice}); pick one here to replace it."
+            choice = ""
+        if kind == "hands" and (os.environ.get("FT_HANDS_MODELS") or conf.get("HANDS_MODELS")):
+            note = ("HANDS_MODELS in frametop.conf names a folder of its own, and ft-hands uses that instead "
+                    "of what's picked here. Remove it to use this choice.")
+        running = v["latest"] if choice == "latest" else choice
+        if kind == "hands" and choice == "latest":
+            # hands/models.sh keeps latest in hands/models/student, on its own pin.
+            here = ""
+            try:
+                with open(os.path.join(REPO, "hands", "models", "student", "PIN")) as f:
+                    here = next((ln.split()[1] for ln in f if ln.startswith("model ")), "")
+            except (OSError, IndexError):
+                pass
+            running = here or "stock"
+            if not here:
+                note = note or (f"This install has no fine-tuned models yet, so stock runs: {v['latest']} comes "
+                                "when hand tracking next starts online (hands/models.sh).")
+            elif here != v["latest"]:
+                note = note or f"This install still has {here}: {v['latest']} comes when hand tracking next starts."
+        elif kind == "eyes" and choice == "latest":
+            running = v["resolved"]["name"] or "classic"
+            if not v["resolved"]["folder"]:
+                note = note or f"{v['latest']} isn't downloaded yet: the classic tracker runs until it is."
+        opts = []
+        latest = by.get(v["latest"])
+        if latest:
+            opts.append(dict(latest, name="latest", target=latest["name"], follows=True,
+                             title=f"Latest (follows updates): {latest['title']}",
+                             tag=MODEL_TAGS.get(latest["status"], ""),
+                             installed=True if kind == "hands" else latest["installed"]))
+        for m in v["models"]:
+            tag = MODEL_TAGS.get(m["status"]) or MODEL_KIND_TAGS.get(m["kind"], "")
+            opts.append(dict(m, target=m["name"], follows=False, tag=tag))
+        for o in opts:
+            o["bench"] = {k: (None if val is None else float(val)) for k, val in o["bench"].items()}
+        checked = time.strftime("%H:%M", time.localtime(v["checked"])) if v["checked"] else ""
+        return {"options": opts, "metrics": v["metrics"], "byName": by, "latest": v["latest"],
+                "baseline": v["baseline"], "choice": choice, "running": running, "note": note,
+                "source": v["source"], "checked": checked, "error": ""}
+
+    def _models_update(self):
+        self._model_views = {k: self._model_view(k) for k in ("hands", "eyes")}
+        self.modelsChanged.emit()
+
+    @Property("QVariantMap", notify=modelsChanged)
+    def handModels(self):
+        if "hands" not in self._model_views:   # (no signal from inside a getter: a binding loop)
+            self._model_views["hands"] = self._model_view("hands")
+        return self._model_views["hands"]
+
+    @Property("QVariantMap", notify=modelsChanged)
+    def eyeModels(self):
+        if "eyes" not in self._model_views:   # (no signal from inside a getter: a binding loop)
+            self._model_views["eyes"] = self._model_view("eyes")
+        return self._model_views["eyes"]
+
+    @Property("QVariantMap", notify=modelsChanged)
+    def modelBusy(self):
+        return self._model_busy
+
+    def _model_job(self, kind, work):
+        """Run a download (or a catalog check) off the UI thread, one per kind at a time."""
+        if self._model_busy[kind]:
+            return
+
+        def job():
+            try:
+                work()
+            except (OSError, ValueError, model_catalog.CatalogError) as e:
+                self.message.emit(str(e), True)
+            finally:
+                self._model_busy[kind] = ""
+                self._models_update()
+        self._model_busy[kind] = "working…"
+        self.modelsChanged.emit()
+        threading.Thread(target=job, daemon=True).start()
+
+    def _busy(self, kind, text):
+        self._model_busy[kind] = text
+        self.modelsChanged.emit()
+
+    @Slot()
+    def reloadModels(self):
+        """Read the catalogs and the downloads again (the pages call it when they open)."""
+        self._models_update()
+
+    @Slot(str)
+    def refreshModels(self, kind):
+        """The kind's catalog from GitHub (this install's branch): new versions and notes."""
+        if kind not in model_catalog.SPECS:
+            return
+
+        def work():
+            self._busy(kind, "checking for new versions…")
+            ok, msg = model_catalog.refresh(kind)
+            self.message.emit(("Hand" if kind == "hands" else "Eye") + " models: " + msg, not ok)
+        self._model_job(kind, work)
+
+    @Slot(str, str)
+    def useModel(self, kind, name):
+        """Pick a model: download it first if it needs that, then write the setting and apply it.
+        hands: HANDS_MODEL (latest, stock or a version), and hand tracking restarts to load it.
+        eyes: steamvr is GAZE_TRACKER=steam; anything else is our tracker (GAZE_TRACKER=own) with
+        EYES_MODEL (classic, latest or a version), which the gaze service applies (it restarts ft-eyes)."""
+        if kind not in model_catalog.SPECS or not (name == "latest" or model_catalog.NAME.fullmatch(name)):
+            return
+        view = self._model_views.get(kind) or self._model_view(kind)
+        target = view["latest"] if name == "latest" else name
+        e = view["byName"].get(target)
+        if e is None:
+            return
+
+        def work():
+            if not e["builtin"] and not (kind == "hands" and name == "latest"):
+                if not e["usable"]:
+                    raise model_catalog.CatalogError(f"{target} needs a newer Frametop")
+                if not e["installed"]:
+                    model_catalog.fetch(kind, target, progress=lambda t: self._busy(kind, t + "…"))
+            if kind == "hands":
+                write_conf_value("HANDS_MODEL", name)
+                r = host("systemctl", "--user", "try-restart", "frametop-hands.service")
+                running = r.returncode == 0 and host("systemctl", "--user", "is-active", "--quiet",
+                                                     "frametop-hands.service").returncode == 0
+                self.message.emit(f"Hand model: {e['title']}" + (", hand tracking restarted" if running
+                                                                  else " (used when hand tracking starts)"), False)
+                self.handsChanged.emit()
+            elif name == "steamvr":
+                self._set_gaze("GAZE_TRACKER", "steam", "Eye tracker: SteamVR's")
+            else:
+                write_conf_value("EYES_MODEL", name)
+                self._set_gaze("GAZE_TRACKER", "own", f"Eye tracker: {e['title']}"
+                               + ("" if e["builtin"] else " (calibrate it once: each net keeps its own calibration)"))
+        self._model_job(kind, work)
+
+    @Slot(str, str)
+    def removeModel(self, kind, name):
+        """Delete a downloaded version (not the one in use)."""
+        view = self._model_views.get(kind) or self._model_view(kind)
+        if kind not in model_catalog.SPECS or name in (view["running"], view["choice"]):
+            return
+        if model_catalog.remove(kind, name):
+            self.message.emit(f"Removed the download of {name}", False)
+        self._models_update()
+
     # --- bluetooth ---
     @Property("QVariantList", notify=bluetoothChanged)
     def bluetooth(self):

input-settings/main.qml

@@ -54,6 +54,212 @@ Kirigami.ApplicationWindow {
         })[backend.driverBlock] || ""
     }
 
+    // A model picker (Hands and Gaze pages): the catalog's versions (models/catalog.py) with their notes
+    // and benchmarks, set against the one running (viewing that one: against the baseline, stock or
+    // SteamVR). Choosing a version only shows it; "Use" downloads it if needed and switches to it.
+    // The catalog's texts come from GitHub, so they're shown as plain text.
+    component ModelPicker: ColumnLayout {
+        id: picker
+        property string kind
+        property var view: ({})
+        readonly property string busy: (backend.modelBusy || {})[kind] || ""
+        property string viewed: ""
+        readonly property var options: view.options || []
+        readonly property var opt: options.find(o => o.name === (viewed || view.choice)) || options[0] || ({})
+        readonly property string running: view.running || ""
+        readonly property string otherName: opt.target !== running ? running
+                                            : opt.target !== view.baseline ? (view.baseline || "") : ""
+        readonly property var other: otherName ? ((view.byName || {})[otherName] || null) : null
+        readonly property bool inUse: opt.name === view.choice
+        spacing: Kirigami.Units.smallSpacing
+        Layout.maximumWidth: Kirigami.Units.gridUnit * 30
+
+        function fmt(v, unit) {
+            if (v === undefined || v === null) return "–"
+            const n = Math.abs(v) >= 100 ? Math.round(v) : Math.round(v * 100) / 100
+            return n + (unit === "" || unit === "%" || unit === "°" ? unit : " " + unit)
+        }
+        function better(m, v, o) {
+            if (v === undefined || v === null || o === undefined || o === null || v === o) return 0
+            return (m.better === "lower" ? v < o : v > o) ? 1 : -1
+        }
+
+        Kirigami.InlineMessage {
+            Layout.fillWidth: true
+            visible: !!picker.view.error
+            type: Kirigami.MessageType.Error
+            text: picker.view.error || ""
+        }
+        Repeater {
+            model: picker.options
+            delegate: Controls.RadioButton {
+                required property var modelData
+                text: modelData.title + (modelData.tag ? "  ·  " + modelData.tag : "")
+                      + (modelData.name === picker.view.choice ? "  ·  in use" : "")
+                      + (!modelData.usable ? "  ·  needs a newer Frametop"
+                         : !modelData.installed ? "  ·  not downloaded" : "")
+                font.bold: modelData.name === picker.view.choice
+                checked: picker.opt.name === modelData.name
+                onToggled: if (checked) picker.viewed = modelData.name
+            }
+        }
+        Controls.Label {
+            visible: !!picker.view.note
+            text: picker.view.note || ""
+            textFormat: Text.PlainText
+            Layout.fillWidth: true
+            wrapMode: Text.WordWrap
+            color: Kirigami.Theme.neutralTextColor
+            font: Kirigami.Theme.smallFont
+        }
+
+        // The one being looked at: what it is, what to expect, what it costs, and its numbers.
+        Kirigami.Separator { Layout.fillWidth: true; visible: !!picker.opt.name }
+        Controls.Label {
+            text: (picker.opt.follows ? "Latest: " + picker.opt.target : picker.opt.title || "")
+                  + (picker.opt.date ? " (" + picker.opt.date + ")" : "")
+            textFormat: Text.PlainText
+            font.bold: true
+        }
+        Controls.Label {
+            visible: picker.opt.follows === true
+            text: "Moves to each new version we recommend, without asking. Pick a version below it to stay on that one."
+            Layout.fillWidth: true
+            wrapMode: Text.WordWrap
+            opacity: 0.7
+            font: Kirigami.Theme.smallFont
+        }
+        Controls.Label {
+            text: picker.opt.summary || ""
+            visible: text !== ""
+            textFormat: Text.PlainText
+            Layout.fillWidth: true
+            wrapMode: Text.WordWrap
+        }
+        Controls.Label {
+            text: picker.opt.notes || ""
+            visible: text !== ""
+            textFormat: Text.PlainText
+            Layout.fillWidth: true
+            wrapMode: Text.WordWrap
+            opacity: 0.85
+        }
+        Controls.Label {
+            text: picker.opt.cost ? "Cost: " + picker.opt.cost : ""
+            visible: text !== ""
+            textFormat: Text.PlainText
+            Layout.fillWidth: true
+            wrapMode: Text.WordWrap
+            opacity: 0.7
+            font: Kirigami.Theme.smallFont
+        }
+        GridLayout {
+            visible: (picker.view.metrics || []).some(m => picker.opt.bench && picker.opt.bench[m.key] !== undefined)
+            columns: 3
+            columnSpacing: Kirigami.Units.largeSpacing
+            rowSpacing: 2
+            Controls.Label { text: "Benchmark"; font.bold: true; Layout.fillWidth: true }
+            Controls.Label { text: picker.opt.target || ""; textFormat: Text.PlainText; font.bold: true }
+            Controls.Label {
+                text: picker.other ? picker.otherName + (picker.otherName === picker.running ? " (in use)" : "") : ""
+                textFormat: Text.PlainText
+                font.bold: true
+                opacity: 0.7
+            }
+            Repeater {
+                model: picker.view.metrics || []
+                delegate: Controls.Label {
+                    required property var modelData
+                    required property int index
+                    text: modelData.label
+                    textFormat: Text.PlainText
+                    Layout.row: index + 1
+                    Layout.column: 0
+                    Layout.fillWidth: true
+                    wrapMode: Text.WordWrap
+                    font: Kirigami.Theme.smallFont
+                    Controls.ToolTip.visible: !!modelData.note && mhover.hovered
+                    Controls.ToolTip.text: modelData.note || ""
+                    HoverHandler { id: mhover }
+                }
+            }
+            Repeater {
+                model: picker.view.metrics || []
+                delegate: Controls.Label {
+                    required property var modelData
+                    required property int index
+                    readonly property var v: (picker.opt.bench || {})[modelData.key]
+                    readonly property int b: picker.other ? picker.better(modelData, v, (picker.other.bench || {})[modelData.key]) : 0
+                    text: picker.fmt(v, modelData.unit)
+                    Layout.row: index + 1
+                    Layout.column: 1
+                    color: b > 0 ? Kirigami.Theme.positiveTextColor : b < 0 ? Kirigami.Theme.negativeTextColor
+                                                                       : Kirigami.Theme.textColor
+                }
+            }
+            Repeater {
+                model: picker.view.metrics || []
+                delegate: Controls.Label {
+                    required property var modelData
+                    required property int index
+                    text: picker.other ? picker.fmt((picker.other.bench || {})[modelData.key], modelData.unit) : ""
+                    Layout.row: index + 1
+                    Layout.column: 2
+                    opacity: 0.7
+                }
+            }
+        }
+        Controls.Label {
+            visible: (picker.view.metrics || []).some(m => picker.opt.bench && picker.opt.bench[m.key] !== undefined)
+            text: "Green: better than " + (picker.otherName || "the other") + ", red: worse. Hover a row for how it was measured."
+            textFormat: Text.PlainText
+            opacity: 0.7
+            font: Kirigami.Theme.smallFont
+        }
+        RowLayout {
+            Controls.Button {
+                text: picker.inUse ? "In use"
+                      : !picker.opt.installed ? "Download and use"
+                      : "Use this"
+                icon.name: picker.inUse ? "checkmark" : !picker.opt.installed ? "download" : "dialog-ok-apply"
+                enabled: !picker.inUse && picker.opt.usable !== false && picker.busy === "" && !!picker.opt.name
+                onClicked: backend.useModel(picker.kind, picker.opt.name)
+            }
+            Controls.Button {
+                text: "Remove download"
+                icon.name: "edit-delete"
+                visible: !picker.opt.builtin && !picker.opt.follows && picker.opt.installed === true
+                         && picker.opt.target !== picker.running && picker.opt.name !== picker.view.choice
+                enabled: picker.busy === ""
+                onClicked: backend.removeModel(picker.kind, picker.opt.name)
+            }
+            Controls.Button {
+                text: "Check for new versions"
+                icon.name: "view-refresh"
+                enabled: picker.busy === ""
+                onClicked: backend.refreshModels(picker.kind)
+            }
+            Controls.BusyIndicator {
+                visible: picker.busy !== ""
+                running: visible
+                Layout.preferredHeight: Kirigami.Units.iconSizes.medium
+                Layout.preferredWidth: Layout.preferredHeight
+            }
+            Controls.Label {
+                visible: picker.busy !== ""
+                text: picker.busy
+                opacity: 0.7
+                font: Kirigami.Theme.smallFont
+            }
+        }
+        Controls.Label {
+            text: "List from " + (picker.view.source || "this install")
+                  + (picker.view.checked ? ", checked " + picker.view.checked : "")
+            opacity: 0.6
+            font: Kirigami.Theme.smallFont
+        }
+    }
+
     function show(page) {
         pageStack.clear()
         pageStack.push(page)
@@ -1285,26 +1491,22 @@ Kirigami.ApplicationWindow {
                     opacity: 0.7
                     font: Kirigami.Theme.smallFont
                 }
-                RowLayout {
-                    Kirigami.FormData.label: "Eye tracker:"
-                    Controls.RadioButton {
-                        text: "Own tracker (recommended)"
-                        checked: backend.gazeTracker === "own"
-                        onToggled: if (checked) backend.setGazeTracker("own")
-                    }
-                    Controls.RadioButton {
-                        text: "SteamVR"
-                        checked: backend.gazeTracker === "steam"
-                        onToggled: if (checked) backend.setGazeTracker("steam")
-                    }
+                // Whose eye tracking: SteamVR's (GAZE_TRACKER=steam), or ours (own) with the classic
+                // pupil finder or an eye net (EYES_MODEL: latest or a version).
+                Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Eye tracker" }
+                ModelPicker {
+                    Kirigami.FormData.label: "Tracker:"
+                    kind: "eyes"
+                    view: backend.eyeModels
+                    Component.onCompleted: backend.reloadModels()
                 }
                 Controls.Label {
                     // Ours is the default once it's installed (GAZE_TRACKER=auto, gaze/ft-gazed).
                     visible: backend.gazeTracker === "steam" && !backend.gazeOwnInstalled
-                    text: "Our own tracker is more accurate. Install it with gaze/tracker/install.sh in the "
-                          + "Frametop folder, in a terminal (asks for sudo)"
+                    text: "Our own tracker (the classic one and the eye nets) needs its frame grabber. Install it "
+                          + "with gaze/tracker/install.sh in the Frametop folder, in a terminal (asks for sudo)"
                           + (backend.gazeTrackerAuto ? "; gaze then uses it, and you calibrate it once."
-                                                     : ", then pick Own tracker here and calibrate it once.")
+                                                     : ", then pick it here and calibrate it once.")
                     Layout.maximumWidth: Kirigami.Units.gridUnit * 30
                     wrapMode: Text.WordWrap
                     opacity: 0.7
@@ -1354,6 +1556,7 @@ Kirigami.ApplicationWindow {
                     opacity: 0.7
                     font: Kirigami.Theme.smallFont
                 }
+                Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Gaze pointer tuning" }
                 Repeater {
                     model: backend.gazeSettings
                     delegate: RowLayout {
@@ -1454,7 +1657,7 @@ Kirigami.ApplicationWindow {
                       + "a pinch clicks where you look; pinch, hold and move the hand to nudge the pointer first. Pinch nudge "
                       + "starts after: how far the hand moves before the pointer does. Pinch nudge speed: how far the pointer "
                       + "moves for each degree of hand movement after that. Eye tracker: SteamVR's, or our own "
-                      + "(gaze/tracker), which keeps its own calibration. Eye bias: the gaze "
+                      + "(gaze/tracker), the classic pupil finder or an eye net; each keeps its own calibration. Eye bias: the gaze "
                       + "combines both eyes, since their errors partly cancel; Left or Right counts that eye twice as "
                       + "much, and Auto weights each by how far off it was at your recent nudges."
             }
@@ -1532,6 +1735,15 @@ Kirigami.ApplicationWindow {
                     }
                 }
 
+                // Which hand models ft-hands runs (HANDS_MODEL): latest, a version, or stock MediaPipe.
+                Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Hand model" }
+                ModelPicker {
+                    Kirigami.FormData.label: "Model:"
+                    kind: "hands"
+                    view: backend.handModels
+                    Component.onCompleted: backend.reloadModels()
+                }
+
                 // Three ways the hands work the desktop, each on its own (handsmode.METHODS).
                 Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Hand input" }
                 Controls.Switch {

install.sh

@@ -8,15 +8,17 @@
 # distrobox the release brings, makes the release's container from its image, and builds nothing.
 #
 # Usage: ./install.sh [--yes] [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
-#                    [--no-bluetooth | --bluetooth]
-#   --yes           don't ask; installs voice dictation and gaze mode, and hand tracking, the Hand
-#                   Recorder and our eye tracker if sudo can run without a password prompt; skips
-#                   the Bluetooth fixes and the SteamVR restart
+#                    [--no-eye-recorder] [--no-bluetooth | --bluetooth]
+#   --yes           don't ask; installs voice dictation, gaze mode and the Eye Recorder, and hand
+#                   tracking, the Hand Recorder and our eye tracker if sudo can run without a
+#                   password prompt; skips the Bluetooth fixes and the SteamVR restart
 #   --no-voice      don't offer voice dictation (Frametop Voice: it downloads about 320 MB)
 #   --no-hands      don't offer hand tracking or the Hand Recorder
 #   --no-hand-recorder  don't offer the Hand Recorder (a release's; a clone builds it with
 #                   hands/rec/install.sh)
-#   --no-eye-tracker  don't offer our own eye tracker
+#   --no-eye-tracker  don't offer our own eye tracker (or the Eye Recorder, which needs it)
+#   --no-eye-recorder  don't offer the Eye Recorder (a release's; a clone builds it with
+#                   gaze/rec/install.sh)
 #   --no-bluetooth  don't offer the Bluetooth fixes
 #   --bluetooth     install the Bluetooth fixes without asking (with --yes: if sudo can run
 #                   without a password prompt)
@@ -25,17 +27,18 @@ set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
 . "$root/scripts/_env.sh"
 
-assume_yes=0 bluetooth=1 eye_tracker=1 hands=1 hand_recorder=1 voice=1
+assume_yes=0 bluetooth=1 eye_tracker=1 eye_recorder=1 hands=1 hand_recorder=1 voice=1
 for arg in "$@"; do
   case $arg in
     --yes) assume_yes=1 ;;
     --no-bluetooth) bluetooth=0 ;;
     --bluetooth) bluetooth=2 ;;
-    --no-eye-tracker) eye_tracker=0 ;;
+    --no-eye-tracker) eye_tracker=0 eye_recorder=0 ;;
+    --no-eye-recorder) eye_recorder=0 ;;
     --no-voice) voice=0 ;;
     --no-hands) hands=0 hand_recorder=0 ;;
     --no-hand-recorder) hand_recorder=0 ;;
-    -h|--help) sed -n '2,22p' "$0"; exit 0 ;;
+    -h|--help) sed -n '2,24p' "$0"; exit 0 ;;
     *) echo "unknown option: $arg" >&2; exit 2 ;;
   esac
 done
@@ -224,6 +227,27 @@ else
   echo "skipped: gaze mode uses SteamVR's eye tracker. Install ours later with: gaze/tracker/install.sh"
 fi
 
+# The Eye Recorder (gaze/rec), as the Hand Recorder: a release brings it built, so it's only the menu
+# entry, which needs our eye tracker's frame grabber (step 11). An entry from the release before
+# points into that release's folder: it's renewed without asking. A clone builds it with
+# gaze/rec/install.sh (a few minutes), so it isn't offered here.
+if [ "$FRAME_RELEASE" = 1 ]; then
+  recorder="the Eye Recorder (records your eyes for Frametop's eye tracking dataset)"
+  if [ "$eye_recorder" = 0 ]; then
+    echo "$recorder: skipped. Add it later with: gaze/rec/install.sh"
+  elif on_frame '[ -e ~/.local/share/applications/frametop-eyerec.desktop ]'; then
+    "$root/gaze/rec/install.sh" install >/dev/null && echo "$recorder: updated" ||
+      echo "$recorder: couldn't update it (above). Try again with: gaze/rec/install.sh"
+  elif ! on_frame 'systemctl is-enabled -q frametop-eyegrab.service 2>/dev/null'; then
+    echo "$recorder: skipped: it needs our eye tracker. Install both later with: gaze/tracker/install.sh, then gaze/rec/install.sh"
+  elif ask "Add the Eye Recorder to the app menu? It records your eyes for Frametop's eye tracking dataset, which trains our eye tracker; nothing leaves the headset until you press Upload, and uploads are encrypted." y; then
+    "$root/gaze/rec/install.sh" install ||
+      echo "$recorder: not installed (above). Try again later with: gaze/rec/install.sh"
+  else
+    echo "$recorder: skipped. Add it later with: gaze/rec/install.sh"
+  fi
+fi
+
 step "12/12 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
 if [ "$bluetooth" = 2 ] && [ "$assume_yes" = 1 ] && ! sudo_quiet; then
   echo "skipped: they need your password (sudo), and --yes doesn't ask. Install later with: setup/bluetooth/install.sh install"

justfile

@@ -20,7 +20,7 @@ test-python:
     set -uo pipefail
     python3 -c 'import PySide6, numpy, cv2, sherpa_onnx' || { echo "python3 can't import PySide6, numpy, cv2, and sherpa_onnx"; exit 1; }
     status=0
-    for t in input/test/*.py hands/tests/test_*.py hands/rec/tests/*.py recorder/tests/*.py gaze/test/*.py gaze/rec/tests/*.py pack/test/*.py voice/test/*.py spotwatch/test/*.py; do
+    for t in models/test/*.py input/test/*.py hands/tests/test_*.py hands/rec/tests/*.py recorder/tests/*.py gaze/test/*.py gaze/rec/tests/*.py pack/test/*.py voice/test/*.py spotwatch/test/*.py; do
         [ -f "$t" ] || continue
         if python3 "$t" >/dev/null; then echo "$t: ok"; else echo "$t: FAILED"; status=1; fi
     done

models/README.md

@@ -0,0 +1,50 @@
+# Model versions
+
+Users pick which hand and eye models Frametop runs on Frametop Input Settings' Hands and Gaze pages.
+Each page lists the versions in a catalog, with notes on what to expect and benchmark numbers set
+against the one in use. `catalog.py` reads the catalogs, downloads versions and says what a setting
+runs; `ft-models` is the same from a terminal.
+
+| Kind | Catalog | Setting (frametop.conf) | Downloads go to | Files from |
+|---|---|---|---|---|
+| hands | `hands/models.json` | `HANDS_MODEL`: `latest`, `stock`, a version | `hands/models/versions/NAME` | `DeeJanuz/frametop-hands-models` |
+| eyes | `gaze/tracker/models.json` | `EYES_MODEL`: `classic`, `latest`, a version; `GAZE_TRACKER=steam` for SteamVR's | `gaze/tracker/models/NAME` | `DeeJanuz/frametop-eyes-models` |
+
+- **Latest** follows the catalog's `latest`, so it moves when a new version ships. A version picked
+  by name stays put. For hands, latest is `hands/models/student`, which `hands/models.sh` keeps on
+  `hands/models.pin` (older installs only read that pin). The catalog's latest must be the same
+  files: `models/test/test_catalog.py` checks it.
+- **Where the list comes from.** Installs read their branch's catalog from GitHub (`main` for a
+  stable release, `experimental` for an experimental one or a clone on that branch), so a push
+  reaches them without a release: Input Settings' "Check for new versions", and the services when
+  they start (`ft-models auto`, at most every 15 minutes; `HANDS_MODELS_UPDATE=0` /
+  `EYES_MODELS_UPDATE=0` turn that off). Offline, or when GitHub's copy isn't valid, the copy in
+  the install is used.
+- **Trust.** A catalog only names files. They come from the kind's Hugging Face repo, which is
+  fixed in `catalog.py` and never taken from a catalog. Each download is pinned to a commit and must
+  match its sha256, or it isn't kept. Catalog checks are strict: one bad entry and the whole
+  catalog is ignored. The texts are shown as plain text.
+- **Who applies it.** ft-hands reads `HANDS_MODEL` at start, and Input Settings restarts hand
+  tracking after a change. ft-eyes reads `EYES_MODEL` at start, and the gaze service restarts it
+  when the setting changes. Each eye net keeps its own calibration (`net-NAME/`), so a switch asks
+  for one calibration.
+
+## Shipping a version
+
+1. Upload the version's folder to the kind's Hugging Face repo, next to the earlier ones (hands:
+   frame-hands' `model-repo/`, eyes: frame-eyes' `model-repo/`), with a NOTICE: how it was trained,
+   what it was tested on, the numbers.
+2. Print its files' hashes from that staging folder:
+   `models/ft-models hashes eyes ~/Desktop/Projects/frame-eyes/model-repo eyes-v1`
+3. Add an entry to the catalog: name, title, kind (`custom`), status (`testing` until it's the one
+   to recommend), date, summary, notes (what to expect against the current latest and the
+   baseline), cost, `bench` (the catalog's metrics, measured the same way as the other entries),
+   `format`, `rev` (the Hugging Face commit) and `files`. Set the old latest's status to `older`
+   when a new one is recommended.
+4. To make it latest, set `latest`. For hands, also re-pin `hands/models.pin` with frame-hands'
+   `model-repo/pin.py VERSION REV CHECKOUT`.
+5. Run `python3 models/test/test_catalog.py` and `models/ft-models fetch KIND NAME` (a real
+   download, checked), then push to experimental. Stable installs see it once it's on main.
+
+Raise `format` (and `SPECS[kind]["format"]` in `catalog.py`) when a version needs a newer runtime.
+Older installs then list the version as needing a newer Frametop instead of running it.

models/catalog.py

@@ -0,0 +1,465 @@
+"""The model catalogs: which hand and eye models there are, their notes and benchmarks, and the
+versions someone picked, downloaded and checked.
+
+Two catalogs, one per kind, in the repo and on GitHub:
+  hands  hands/models.json         ft-hands' palm detector + hand landmark model pairs
+  eyes   gaze/tracker/models.json  the eye tracker: SteamVR's, our classic pupil finder, our eye nets
+
+An install reads its branch's catalog from GitHub (main for a stable release, experimental for an
+experimental one or a clone on that branch: models.sh's rule), so a new version and its notes reach
+installs on a push, without a release. Offline, or when GitHub's copy isn't valid, the copy that came
+with the install is used. A catalog only names versions: the files come from the kind's own Hugging
+Face repo (HF_REPO here, never from the catalog), at the commit the entry pins, and every file must
+match the sha256 it lists.
+
+A catalog (JSON):
+  {"catalog": 1, "kind": "hands", "latest": "v13b-p1", "default": "latest", "baseline": "stock",
+   "metrics": [{"key", "label", "unit", "better": "higher"|"lower", "note"}],
+   "models": [{"name", "title", "kind", "status", "date", "summary", "notes", "cost", "bench": {key: n},
+               "format": 1, "rev": "<40 hex>", "files": {"name": "<sha256>"}}]}
+  kind    custom (ours, trained on Frametop data), stock (off the shelf, unchanged), valve (SteamVR's),
+          classic (our hand-written tracker, no model)
+  status  recommended, testing, older, builtin
+  latest  the custom version "Latest" follows: it changes as new versions ship
+  default what an unset setting uses ("latest", or a builtin's name)
+  format  what that version needs of the runtime (FORMATS here); newer ones show, but can't be used
+Builtin entries (no rev or files) ship with Frametop or come from SteamVR.
+
+Downloaded versions live in the kind's store (hands/models/versions/NAME, gaze/tracker/models/NAME),
+with PIN.json, the entry they were fetched by. hands' "latest" is hands/models/student, which
+hands/models.sh keeps on its own pin (models.pin); the catalog's latest must match it (the tests check).
+
+Standard library only: the input settings, the gaze service (on the host) and ft-eyes (in the box)
+import it. CLI: models/ft-models.
+"""
+import fcntl
+import hashlib
+import json
+import os
+import re
+import shutil
+import time
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parents[1]
+GITHUB = "Frametop/frametop"
+CACHE = Path(os.environ.get("XDG_CACHE_HOME") or Path.home() / ".cache") / "frametop" / "models"
+CATALOG_VERSION = 1           # the catalog layout this code reads
+MAX_CATALOG = 128 * 1024      # bytes
+MAX_FILE = 200 * 1024 * 1024  # bytes, per model file
+NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
+SHA = re.compile(r"[0-9a-f]{64}")
+REV = re.compile(r"[0-9a-f]{40}")
+KINDS_OF = ("custom", "stock", "valve", "classic")
+STATUSES = ("recommended", "testing", "older", "builtin")
+TEXT_MAX = {"title": 80, "date": 20, "summary": 400, "notes": 4000, "cost": 300}
+METRIC_TEXT_MAX = {"key": 40, "label": 120, "unit": 20, "note": 300}
+
+# Per kind: the catalog, the Hugging Face repo files come from, where versions go, the files a
+# version may have (and must), files at the repo's top, and the newest version format we can run.
+SPECS = {
+    "hands": {
+        "catalog": "hands/models.json",
+        "hf": "DeeJanuz/frametop-hands-models",
+        "store": "hands/models/versions",
+        "files": {"NOTICE", "hand.ncnn.bin", "hand.ncnn.param", "palm.ncnn.bin", "palm.ncnn.param"},
+        "required": {"hand.ncnn.bin", "hand.ncnn.param", "palm.ncnn.bin", "palm.ncnn.param"},
+        "top": {"LICENSE", "README.md"},
+        "format": 1,
+        "setting": "HANDS_MODEL",
+        "update": "HANDS_MODELS_UPDATE",
+    },
+    "eyes": {
+        "catalog": "gaze/tracker/models.json",
+        "hf": "DeeJanuz/frametop-eyes-models",
+        "store": "gaze/tracker/models",
+        "files": {"NOTICE", "model.json", "model.ncnn.bin", "model.ncnn.param"},
+        "required": {"model.json", "model.ncnn.bin", "model.ncnn.param"},
+        "top": {"LICENSE", "README.md"},
+        "format": 1,
+        "setting": "EYES_MODEL",
+        "update": "EYES_MODELS_UPDATE",
+    },
+}
+
+
+class CatalogError(ValueError):
+    pass
+
+
+def _check_text(d, limits, where):
+    for key, n in limits.items():
+        v = d.get(key, "")
+        if not isinstance(v, str) or len(v) > n:
+            raise CatalogError(f"{where}: {key} must be text of at most {n} characters")
+
+
+def validate(cat, kind):
+    """Check a catalog strictly; return it, or raise CatalogError. Anything unexpected fails the
+    whole catalog: GitHub's copy is then ignored for the one that came with the install."""
+    spec = SPECS[kind]
+    if not isinstance(cat, dict):
+        raise CatalogError("not an object")
+    if cat.get("catalog") != CATALOG_VERSION:
+        raise CatalogError(f"catalog version {cat.get('catalog')!r}, this Frametop reads {CATALOG_VERSION}")
+    if cat.get("kind") != kind:
+        raise CatalogError(f"a catalog of {cat.get('kind')!r}, not {kind}")
+    metrics = cat.get("metrics", [])
+    if not isinstance(metrics, list) or len(metrics) > 40:
+        raise CatalogError("metrics must be a list (at most 40)")
+    keys = set()
+    for m in metrics:
+        if not isinstance(m, dict):
+            raise CatalogError("a metric isn't an object")
+        _check_text(m, METRIC_TEXT_MAX, "metric")
+        if not m.get("key") or not m.get("label") or m.get("better", "higher") not in ("higher", "lower"):
+            raise CatalogError(f"metric {m.get('key')!r}: needs key, label, better higher|lower")
+        keys.add(m["key"])
+    models = cat.get("models")
+    if not isinstance(models, list) or not models or len(models) > 200:
+        raise CatalogError("models must be a non-empty list (at most 200)")
+    names = set()
+    for e in models:
+        if not isinstance(e, dict):
+            raise CatalogError("a model isn't an object")
+        name = e.get("name")
+        if not isinstance(name, str) or not NAME.fullmatch(name) or name in ("latest",):
+            raise CatalogError(f"bad model name {name!r}")
+        if name in names:
+            raise CatalogError(f"{name} listed twice")
+        names.add(name)
+        _check_text(e, TEXT_MAX, name)
+        if e.get("kind") not in KINDS_OF or e.get("status") not in STATUSES:
+            raise CatalogError(f"{name}: kind must be one of {KINDS_OF}, status one of {STATUSES}")
+        bench = e.get("bench", {})
+        if not isinstance(bench, dict) or any(k not in keys or not (v is None or isinstance(v, (int, float)))
+                                              or isinstance(v, bool) for k, v in bench.items()):
+            raise CatalogError(f"{name}: bench must map listed metrics to numbers")
+        if e["status"] == "builtin":
+            if "rev" in e or "files" in e:
+                raise CatalogError(f"{name}: a builtin has no rev or files")
+            continue
+        if not isinstance(e.get("format"), int) or e["format"] < 1:
+            raise CatalogError(f"{name}: format must be a positive number")
+        if not isinstance(e.get("rev"), str) or not REV.fullmatch(e["rev"]):
+            raise CatalogError(f"{name}: rev must be a full commit hash")
+        files = e.get("files")
+        if not isinstance(files, dict) or any(not isinstance(v, str) or not SHA.fullmatch(v) for v in files.values()):
+            raise CatalogError(f"{name}: files must map names to sha256s")
+        allowed = spec["files"] | spec["top"]
+        if set(files) - allowed or spec["required"] - set(files):
+            raise CatalogError(f"{name}: files must be among {sorted(allowed)} and include {sorted(spec['required'])}")
+    for key in ("latest", "baseline"):
+        v = cat.get(key)
+        if v is not None and v not in names:
+            raise CatalogError(f"{key} {v!r} isn't listed")
+    latest = cat.get("latest")
+    if latest is not None and next(e for e in models if e["name"] == latest)["status"] == "builtin":
+        raise CatalogError("latest must be a downloadable version")
+    default = cat.get("default", "latest")
+    if default != "latest" and default not in names:
+        raise CatalogError(f"default {default!r} isn't listed")
+    if default == "latest" and latest is None:
+        raise CatalogError("default is latest, but there's no latest")
+    return cat
+
+
+# --- where and which ---
+
+def store(kind, tree=REPO):
+    return Path(tree) / SPECS[kind]["store"]
+
+
+def channel(tree=REPO):
+    """The branch whose catalog an install follows: main or experimental, or None (any other branch:
+    only the copy in the checkout)."""
+    tree = Path(tree)
+    rel = tree / ".frametop-release"
+    if rel.exists():
+        ch = ""
+        for line in rel.read_text().splitlines():
+            if line.startswith("CHANNEL="):
+                ch = line.split("=", 1)[1].strip()
+        return "experimental" if ch == "experimental" else "main"
+    git = tree / ".git"
+    try:
+        if git.is_file():   # a worktree: "gitdir: PATH"
+            git = Path(git.read_text().split(":", 1)[1].strip())
+        head = (git / "HEAD").read_text().strip()
+    except (OSError, IndexError):
+        return None
+    branch = head.removeprefix("ref: refs/heads/") if head.startswith("ref: refs/heads/") else None
+    return branch if branch in ("main", "experimental") else None
+
+
+def read_conf(path=None):
+    path = Path(path) if path else Path.home() / ".config" / "frametop.conf"
+    conf = {}
+    try:
+        for line in path.read_text().splitlines():
+            line = line.split("#", 1)[0].strip()
+            if "=" in line:
+                k, v = line.split("=", 1)
+                conf[k.strip()] = v.strip().strip("\"'")
+    except OSError:
+        pass
+    return conf
+
+
+def bundled(kind, tree=REPO):
+    path = Path(tree) / SPECS[kind]["catalog"]
+    return validate(json.loads(path.read_text()), kind)
+
+
+def cached(kind):
+    """GitHub's copy from the last refresh, if it's still valid: (catalog, branch, fetched unix)."""
+    try:
+        d = json.loads((CACHE / f"{kind}.json").read_text())
+        return validate(d["catalog"], kind), d["branch"], float(d["fetched"])
+    except (OSError, ValueError, KeyError, TypeError):
+        return None
+
+
+def load(kind, tree=REPO):
+    """The catalog to show: GitHub's copy for this install's branch (from the last refresh), else the
+    one in the checkout. Returns (catalog, source), source "github BRANCH" or "this install"."""
+    ch = channel(tree)
+    c = cached(kind)
+    if c and ch and c[1] == ch:
+        return c[0], f"github {ch}"
+    return bundled(kind, tree), "this install"
+
+
+def refresh(kind, tree=REPO, timeout=15):
+    """Fetch the branch's catalog from GitHub into the cache. Returns (ok, message)."""
+    ch = channel(tree)
+    if not ch:
+        return False, "this checkout isn't on main or experimental: only its own catalog"
+    url = f"https://raw.githubusercontent.com/{GITHUB}/{ch}/{SPECS[kind]['catalog']}"
+    try:
+        with urllib.request.urlopen(url, timeout=timeout) as r:
+            data = r.read(MAX_CATALOG + 1)
+    except (urllib.error.URLError, OSError) as e:
+        return False, f"couldn't reach GitHub ({getattr(e, 'reason', e)})"
+    if len(data) > MAX_CATALOG:
+        return False, "GitHub's catalog is too big; keeping this one"
+    try:
+        cat = validate(json.loads(data), kind)
+    except (ValueError, CatalogError) as e:
+        return False, f"GitHub's catalog isn't valid for this Frametop ({e}); keeping this one"
+    CACHE.mkdir(parents=True, exist_ok=True)
+    tmp = CACHE / f".{kind}.json.tmp"
+    tmp.write_text(json.dumps({"branch": ch, "fetched": time.time(), "catalog": cat}))
+    os.replace(tmp, CACHE / f"{kind}.json")
+    return True, f"up to date with {ch}"
+
+
+def entry(cat, name):
+    return next((e for e in cat["models"] if e["name"] == name), None)
+
+
+def installed(kind, name, tree=REPO):
+    """A downloaded version's folder, if it has every file it needs and its pin, else None."""
+    if not NAME.fullmatch(name or ""):
+        return None
+    d = store(kind, tree) / name
+    if (d / "PIN.json").is_file() and all((d / f).is_file() for f in SPECS[kind]["required"]):
+        return d
+    return None
+
+
+def installed_names(kind, tree=REPO):
+    s = store(kind, tree)
+    try:
+        return sorted(p.name for p in s.iterdir() if not p.name.startswith(".") and installed(kind, p.name, tree))
+    except OSError:
+        return []
+
+
+def usable(kind, e):
+    """Can this Frametop run the entry: builtins, and versions whose format it knows."""
+    return e["status"] == "builtin" or e["format"] <= SPECS[kind]["format"]
+
+
+def setting(kind, conf=None):
+    """The kind's setting from frametop.conf (FT_ in the environment wins), or the catalog's default."""
+    key = SPECS[kind]["setting"]
+    v = os.environ.get("FT_" + key)
+    if v is None:
+        v = (conf if conf is not None else read_conf()).get(key, "")
+    return v.strip()
+
+
+# --- downloading ---
+
+def _sha(path):
+    h = hashlib.sha256()
+    with open(path, "rb") as f:
+        for block in iter(lambda: f.read(1 << 20), b""):
+            h.update(block)
+    return h.hexdigest()
+
+
+def verify(kind, name, tree=REPO):
+    """Check a downloaded version's files against its pin: [] or the names that don't match."""
+    d = installed(kind, name, tree)
+    if not d:
+        return ["(not downloaded)"]
+    pin = json.loads((d / "PIN.json").read_text())
+    return [f for f, sha in pin["files"].items() if not (d / f).is_file() or _sha(d / f) != sha]
+
+
+def fetch(kind, name, tree=REPO, cat=None, progress=None, timeout=30):
+    """Download a version into the store, every file checked, then swapped in whole. Returns the
+    folder. Raises CatalogError (or OSError) when it can't."""
+    spec = SPECS[kind]
+    cat = cat or load(kind, tree)[0]
+    e = entry(cat, name)
+    if e is None:
+        raise CatalogError(f"{name} isn't in the catalog")
+    if e["status"] == "builtin":
+        raise CatalogError(f"{name} comes with Frametop: nothing to download")
+    if not usable(kind, e):
+        raise CatalogError(f"{name} needs a newer Frametop")
+    s = store(kind, tree)
+    s.mkdir(parents=True, exist_ok=True)
+    with open(s / ".lock", "w") as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX)
+        d = s / name
+        if installed(kind, name, tree) and json.loads((d / "PIN.json").read_text()) == e and not verify(kind, name, tree):
+            return d
+        new = s / f".{name}.new"
+        new.mkdir(exist_ok=True)
+        for f, sha in sorted(e["files"].items()):
+            path = new / f
+            if path.is_file() and _sha(path) == sha:
+                continue
+            sub = f if f in spec["top"] else f"{name}/{f}"
+            url = f"https://huggingface.co/{spec['hf']}/resolve/{e['rev']}/{sub}"
+            if progress:
+                progress(f"downloading {f}")
+            part = new / (f + ".part")
+            try:
+                with urllib.request.urlopen(url, timeout=timeout) as r, open(part, "wb") as out:
+                    total = 0
+                    while block := r.read(1 << 20):
+                        total += len(block)
+                        if total > MAX_FILE:
+                            raise CatalogError(f"{f} is bigger than {MAX_FILE >> 20} MB")
+                        out.write(block)
+            except (urllib.error.URLError, OSError) as err:
+                part.unlink(missing_ok=True)
+                raise CatalogError(f"couldn't download {f} ({getattr(err, 'reason', err)})") from err
+            except CatalogError:
+                part.unlink(missing_ok=True)
+                raise
+            if _sha(part) != sha:
+                part.unlink()
+                raise CatalogError(f"{f}'s sha256 doesn't match the catalog; not kept")
+            os.replace(part, path)
+        (new / "PIN.json").write_text(json.dumps(e, indent=1))
+        old = s / f".{name}.old"
+        shutil.rmtree(old, ignore_errors=True)
+        if d.exists():
+            os.replace(d, old)
+        os.replace(new, d)
+        shutil.rmtree(old, ignore_errors=True)
+    return d
+
+
+def remove(kind, name, tree=REPO):
+    d = installed(kind, name, tree)
+    if d:
+        shutil.rmtree(d)
+    return bool(d)
+
+
+# --- what a setting means ---
+
+def resolve(kind, value=None, tree=REPO, cat=None):
+    """What the kind's setting runs now: {"setting", "name", "folder", "why"}. name is the version
+    (a catalog name, "latest"'s target), folder its downloaded folder or None (a builtin, or not
+    downloaded: then the runtime falls back and "why" says so). A value with a "/" is a folder of
+    one's own (for development), used as is."""
+    cat = cat or load(kind, tree)[0]
+    value = value if value is not None else setting(kind)
+    if not value:
+        value = cat.get("default", "latest")
+    if "/" in value:
+        return {"setting": value, "name": None, "folder": os.path.expanduser(value), "why": "a folder of your own"}
+    if value.lower() in ("off", "none"):
+        value = "classic" if kind == "eyes" else "stock"
+    if value == "latest":
+        name = cat.get("latest")
+        if kind == "hands":   # hands/models.sh keeps latest in hands/models/student
+            return {"setting": "latest", "name": name, "folder": None, "why": "latest (hands/models/student)"}
+        folder = installed(kind, name, tree) if name else None
+        if folder:
+            return {"setting": "latest", "name": name, "folder": str(folder), "why": "latest"}
+        # Not fetched yet: the newest custom one downloaded, while the auto update fetches it.
+        have = [e["name"] for e in cat["models"] if e["kind"] == "custom" and installed(kind, e["name"], tree)]
+        if have:
+            return {"setting": "latest", "name": have[0], "folder": str(installed(kind, have[0], tree)),
+                    "why": f"latest is {name}, not downloaded yet: using {have[0]}"}
+        return {"setting": "latest", "name": None, "folder": None, "why": f"latest is {name}, not downloaded yet"}
+    e = entry(cat, value)
+    if e is not None and e["status"] == "builtin":
+        return {"setting": value, "name": value, "folder": None, "why": e.get("title") or value}
+    folder = installed(kind, value, tree)
+    if folder:
+        return {"setting": value, "name": value, "folder": str(folder), "why": value}
+    return {"setting": value, "name": None, "folder": None, "why": f"{value} isn't downloaded"}
+
+
+def auto(kind, tree=REPO, every=15 * 60):
+    """What the services run when they start: refresh the catalog (at most every 15 minutes), then
+    fetch what the setting needs if it's missing. Not with KIND_MODELS_UPDATE=0. Returns a message."""
+    conf = read_conf()
+    upd = os.environ.get("FT_" + SPECS[kind]["update"], conf.get(SPECS[kind]["update"], "1"))
+    if upd.strip() == "0":
+        return f"{kind} models: not updating ({SPECS[kind]['update']}=0)"
+    c = cached(kind)
+    msg = ""
+    if not c or time.time() - c[2] > every:
+        msg = refresh(kind, tree)[1]
+    cat = load(kind, tree)[0]
+    value = setting(kind, conf) or cat.get("default", "latest")
+    want = cat.get("latest") if value == "latest" else value
+    if kind == "hands" and value == "latest":
+        return f"hands models: latest is hands/models.sh's ({msg})" if msg else "hands models: latest is hands/models.sh's"
+    e = entry(cat, want) if want and NAME.fullmatch(want) else None
+    if e is None or e["status"] == "builtin" or not usable(kind, e):
+        return f"{kind} models: nothing to fetch for {value!r}" + (f" ({msg})" if msg else "")
+    have = installed(kind, want, tree)
+    if have and json.loads((have / "PIN.json").read_text()) == e:
+        return f"{kind} models: {want} is here" + (f" ({msg})" if msg else "")
+    fetch(kind, want, tree, cat)
+    return f"{kind} models: fetched {want}"
+
+
+def view(kind, tree=REPO):
+    """Everything the input settings show for a kind: the catalog with each entry's state."""
+    cat, source = load(kind, tree)
+    c = cached(kind)
+    value = setting(kind) or cat.get("default", "latest")
+    have = set(installed_names(kind, tree))
+    models = []
+    for e in cat["models"]:
+        m = {k: e.get(k, "") for k in ("name", "title", "kind", "status", "date", "summary", "notes", "cost")}
+        m["bench"] = e.get("bench", {})
+        m["builtin"] = e["status"] == "builtin"
+        m["installed"] = m["builtin"] or e["name"] in have
+        m["usable"] = usable(kind, e)
+        m["size_mb"] = None
+        models.append(m)
+    listed = {e["name"] for e in cat["models"]}
+    for name in sorted(have - listed):   # downloaded once, no longer listed: still usable
+        models.append({"name": name, "title": name, "kind": "custom", "status": "older", "date": "",
+                       "summary": "No longer in the catalog; downloaded here.", "notes": "", "cost": "",
+                       "bench": {}, "builtin": False, "installed": True, "usable": True, "size_mb": None})
+    return {"kind": kind, "latest": cat.get("latest"), "default": cat.get("default", "latest"),
+            "baseline": cat.get("baseline"), "metrics": cat.get("metrics", []), "models": models,
+            "setting": value, "resolved": resolve(kind, value, tree, cat), "source": source,
+            "checked": c[2] if c else None}

models/ft-models

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""ft-models: the hand and eye model catalogs (models/catalog.py) from a terminal.
+
+  ft-models list hands|eyes [--json]    the versions, their notes, and what's in use
+  ft-models refresh hands|eyes          fetch this install's branch's catalog from GitHub
+  ft-models fetch hands|eyes NAME       download a version (every file checked against the catalog)
+  ft-models verify hands|eyes NAME      check a downloaded version's files again
+  ft-models remove hands|eyes NAME      delete a downloaded version
+  ft-models resolve hands|eyes [VALUE]  what the setting (or VALUE) runs now, as JSON
+  ft-models auto hands|eyes             what the services run at start: refresh at most every
+                                        15 minutes, fetch what the setting needs if it's missing
+  ft-models hashes hands|eyes STAGING VERSION
+                                        for a new catalog entry: the "files" of STAGING/VERSION and
+                                        STAGING's LICENSE and README.md, as uploaded (models/README.md)
+
+Pick a model in Frametop Input Settings (Hands and Gaze pages), or set HANDS_MODEL / EYES_MODEL in
+~/.config/frametop.conf: latest (follows new versions), a version's name, or a builtin (stock for
+hands; classic for eyes, and GAZE_TRACKER=steam for SteamVR's).
+"""
+import json
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+import catalog  # noqa: E402
+
+
+def main(argv):
+    if len(argv) < 2 or argv[0] not in ("list", "refresh", "fetch", "verify", "remove", "resolve", "auto", "hashes") \
+            or argv[1] not in catalog.SPECS:
+        print(__doc__.strip(), file=sys.stderr)
+        return 2
+    cmd, kind, rest = argv[0], argv[1], argv[2:]
+    try:
+        if cmd == "list":
+            v = catalog.view(kind)
+            if "--json" in rest:
+                print(json.dumps(v, indent=1))
+                return 0
+            print(f"{kind} models ({v['source']}); setting {v['setting']!r} -> {v['resolved']['why']}")
+            for m in v["models"]:
+                mark = "*" if m["name"] == v["resolved"]["name"] else " "
+                latest = " (latest)" if m["name"] == v["latest"] else ""
+                state = "builtin" if m["builtin"] else "downloaded" if m["installed"] else \
+                    "needs a newer Frametop" if not m["usable"] else "not downloaded"
+                print(f"{mark} {m['name']:<16} {m['status']:<12} {state:<15} {m['title']}{latest}")
+                if m["summary"]:
+                    print(f"  {'':<16} {m['summary']}")
+            return 0
+        if cmd == "refresh":
+            ok, msg = catalog.refresh(kind)
+            print(f"{kind} catalog: {msg}")
+            return 0 if ok else 1
+        if cmd == "resolve":
+            print(json.dumps(catalog.resolve(kind, rest[0] if rest else None)))
+            return 0
+        if cmd == "auto":
+            print(catalog.auto(kind))
+            return 0
+        if cmd == "hashes" and len(rest) == 2:
+            staging, version = Path(rest[0]), rest[1]
+            spec = catalog.SPECS[kind]
+            files = {p.name: catalog._sha(p) for p in sorted((staging / version).iterdir()) if p.is_file()}
+            files.update({n: catalog._sha(staging / n) for n in sorted(spec["top"]) if (staging / n).is_file()})
+            if set(files) - spec["files"] - spec["top"] or spec["required"] - set(files):
+                print(f"ft-models: a {kind} version has {sorted(spec['files'])} (needs {sorted(spec['required'])}),"
+                      f" plus {sorted(spec['top'])} at the top; found {sorted(files)}", file=sys.stderr)
+                return 1
+            print(json.dumps({"files": files}, indent=1))
+            return 0
+        if not rest:
+            print(f"ft-models {cmd} {kind} NAME", file=sys.stderr)
+            return 2
+        name = rest[0]
+        if cmd == "fetch":
+            d = catalog.fetch(kind, name, progress=lambda s: print(s, flush=True))
+            print(f"{kind} model {name} in {d}")
+        elif cmd == "verify":
+            bad = catalog.verify(kind, name)
+            print(f"{kind} model {name}: " + ("all files match" if not bad else "doesn't match: " + ", ".join(bad)))
+            return 1 if bad else 0
+        elif cmd == "remove":
+            print(f"{kind} model {name}: " + ("removed" if catalog.remove(kind, name) else "wasn't downloaded"))
+    except (catalog.CatalogError, OSError, ValueError) as e:
+        print(f"ft-models: {e}", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

models/test/test_catalog.py

@@ -0,0 +1,236 @@
+#!/usr/bin/env python3
+"""models/catalog.py: the catalogs in the repo are valid and agree with hands/models.pin, a bad catalog
+is refused whole, downloads keep only files that match, and settings resolve as the runtimes expect.
+No network: downloads go to a fake urlopen. Run: python3 models/test/test_catalog.py"""
+import copy
+import hashlib
+import io
+import json
+import os
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+HERE = Path(__file__).resolve().parent
+sys.path.insert(0, str(HERE.parent))
+import catalog  # noqa: E402
+
+REPO = HERE.parents[1]
+
+
+def sha(b):
+    return hashlib.sha256(b).hexdigest()
+
+
+FILES = {"model.json": b'{"name": "eyes-t1"}', "model.ncnn.param": b"param", "model.ncnn.bin": b"weights",
+         "NOTICE": b"notice", "LICENSE": b"licence", "README.md": b"card"}
+REV = "a" * 40
+
+
+def eye_catalog(**over):
+    cat = {"catalog": 1, "kind": "eyes", "latest": "eyes-t1", "default": "classic", "baseline": "steamvr",
+           "metrics": [{"key": "fresh", "label": "Fresh", "unit": "°", "better": "lower"}],
+           "models": [
+               {"name": "classic", "title": "Classic", "kind": "classic", "status": "builtin", "bench": {"fresh": 2.8}},
+               {"name": "steamvr", "title": "SteamVR", "kind": "valve", "status": "builtin"},
+               {"name": "eyes-t1", "title": "T1", "kind": "custom", "status": "testing", "format": 1, "rev": REV,
+                "files": {k: sha(v) for k, v in FILES.items()}, "bench": {"fresh": 1.6}},
+               {"name": "eyes-t0", "title": "T0", "kind": "custom", "status": "older", "format": 1, "rev": REV,
+                "files": {k: sha(v) for k, v in FILES.items()}},
+           ]}
+    cat.update(over)
+    return cat
+
+
+class FakeHF:
+    """urlopen for huggingface.co/DeeJanuz/frametop-eyes-models/resolve/REV/...: serves FILES."""
+
+    def __init__(self, files=FILES):
+        self.files, self.urls = dict(files), []
+
+    def __call__(self, url, timeout=None):
+        self.urls.append(url)
+        prefix = f"https://huggingface.co/{catalog.SPECS['eyes']['hf']}/resolve/{REV}/"
+        assert url.startswith(prefix), url
+        name = url[len(prefix):].split("/")[-1]
+        return io.BytesIO(self.files[name])
+
+
+class RepoCatalogs(unittest.TestCase):
+    def test_bundled_valid(self):
+        for kind in catalog.SPECS:
+            cat = catalog.bundled(kind)
+            self.assertEqual(cat["kind"], kind)
+
+    def test_hands_latest_matches_models_pin(self):
+        """hands/models.sh fetches latest by models.pin; the catalog's latest must be the same files."""
+        pin = {"files": {}}
+        for line in (REPO / "hands" / "models.pin").read_text().splitlines():
+            parts = line.split()
+            if not parts or parts[0].startswith("#"):
+                continue
+            if parts[0] == "file":
+                pin["files"][parts[1]] = parts[2]
+            else:
+                pin[parts[0]] = parts[1]
+        cat = catalog.bundled("hands")
+        e = catalog.entry(cat, cat["latest"])
+        self.assertEqual((e["name"], e["rev"], e["files"]), (pin["model"], pin["rev"], pin["files"]))
+
+    def test_texts_have_no_markup(self):
+        """Shown as plain text, but keep the catalogs free of markup anyway."""
+        for kind in catalog.SPECS:
+            for e in catalog.bundled(kind)["models"]:
+                for k in ("title", "summary", "notes", "cost"):
+                    self.assertNotIn("<", e.get(k, ""), (kind, e["name"], k))
+
+
+class Validate(unittest.TestCase):
+    def bad(self, cat):
+        with self.assertRaises(catalog.CatalogError):
+            catalog.validate(cat, "eyes")
+
+    def test_good(self):
+        catalog.validate(eye_catalog(), "eyes")
+
+    def test_refusals(self):
+        c = eye_catalog()
+        self.bad(dict(c, kind="hands"))
+        self.bad(dict(c, catalog=2))
+        self.bad(dict(c, latest="classic"))          # latest must be downloadable
+        self.bad(dict(c, latest="nope"))
+        self.bad(dict(c, default="nope"))
+        for change in (
+            lambda m: m.update(name="../x"),
+            lambda m: m.update(name="a/b"),
+            lambda m: m.update(rev="abc"),
+            lambda m: m["files"].update({"run.sh": "0" * 64}),           # not a model file
+            lambda m: m["files"].update({"model.json": "xyz"}),          # not a sha256
+            lambda m: m["files"].pop("model.ncnn.bin"),                  # a required file missing
+            lambda m: m.update(bench={"unknown": 1}),
+            lambda m: m.update(bench={"fresh": "1"}),
+            lambda m: m.update(status="best"),
+            lambda m: m.update(notes="x" * 5000),
+            lambda m: m.update(format=0),
+        ):
+            c2 = copy.deepcopy(c)
+            change(c2["models"][2])
+            self.bad(c2)
+        c2 = copy.deepcopy(c)
+        c2["models"][0]["rev"] = REV                 # a builtin has nothing to fetch
+        self.bad(c2)
+        c2 = copy.deepcopy(c)
+        c2["models"].append(copy.deepcopy(c2["models"][2]))
+        self.bad(c2)                                 # listed twice
+
+
+class Store(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.tree = Path(self.tmp.name)
+        self.cat = eye_catalog()
+        env = {"XDG_CACHE_HOME": str(self.tree / "cache"), "FT_EYES_MODEL": "", "FT_EYES_MODELS_UPDATE": ""}
+        self.env = mock.patch.dict(os.environ, env)
+        self.env.start()
+        self.cache = mock.patch.object(catalog, "CACHE", self.tree / "cache" / "frametop" / "models")
+        self.cache.start()
+
+    def tearDown(self):
+        self.cache.stop()
+        self.env.stop()
+        self.tmp.cleanup()
+
+    def fetch(self, name="eyes-t1", hf=None):
+        hf = hf or FakeHF()
+        with mock.patch.object(catalog.urllib.request, "urlopen", hf):
+            return catalog.fetch("eyes", name, tree=self.tree, cat=self.cat), hf
+
+    def test_fetch_and_verify(self):
+        d, hf = self.fetch()
+        self.assertEqual(d, self.tree / "gaze/tracker/models/eyes-t1")
+        self.assertEqual({p.name for p in d.iterdir()}, set(FILES) | {"PIN.json"})
+        self.assertEqual(catalog.verify("eyes", "eyes-t1", self.tree), [])
+        self.assertTrue(any(u.endswith(f"/{REV}/LICENSE") for u in hf.urls))     # top files from the top
+        self.assertTrue(any(u.endswith(f"/{REV}/eyes-t1/model.ncnn.bin") for u in hf.urls))
+        _, again = self.fetch()                     # already there and matching: nothing downloaded
+        self.assertEqual(again.urls, [])
+        (d / "model.ncnn.bin").write_bytes(b"changed")
+        self.assertEqual(catalog.verify("eyes", "eyes-t1", self.tree), ["model.ncnn.bin"])
+
+    def test_bad_file_not_kept(self):
+        with self.assertRaises(catalog.CatalogError):
+            self.fetch(hf=FakeHF(dict(FILES, **{"model.ncnn.bin": b"tampered"})))
+        self.assertIsNone(catalog.installed("eyes", "eyes-t1", self.tree))
+        new = self.tree / "gaze/tracker/models/.eyes-t1.new"
+        self.assertFalse(any(p.name.endswith(".part") or p.name == "model.ncnn.bin" for p in new.iterdir()))
+
+    def test_builtin_and_unknown_not_fetched(self):
+        for name in ("classic", "nope"):
+            with self.assertRaises(catalog.CatalogError):
+                self.fetch(name)
+
+    def test_newer_format_not_fetched(self):
+        self.cat["models"][2]["format"] = catalog.SPECS["eyes"]["format"] + 1
+        with self.assertRaises(catalog.CatalogError):
+            self.fetch()
+
+    def test_resolve(self):
+        r = lambda v: catalog.resolve("eyes", v, self.tree, self.cat)  # noqa: E731
+        self.assertEqual(r("")["name"], "classic")              # unset: the catalog's default
+        self.assertIsNone(r("classic")["folder"])
+        self.assertIsNone(r("latest")["folder"])                 # nothing downloaded: the pupil finder
+        self.assertIsNone(r("eyes-t1")["folder"])
+        self.assertEqual(r("/some/folder")["folder"], "/some/folder")
+        self.fetch("eyes-t0")
+        self.assertEqual(r("latest")["name"], "eyes-t0")         # latest missing: a downloaded custom one
+        self.fetch("eyes-t1")
+        self.assertEqual(r("latest")["folder"], str(self.tree / "gaze/tracker/models/eyes-t1"))
+        self.assertEqual(r("eyes-t0")["name"], "eyes-t0")
+
+    def test_hands_latest_is_student(self):
+        cat = catalog.bundled("hands")
+        self.assertIsNone(catalog.resolve("hands", "latest", self.tree, cat)["folder"])
+        self.assertIsNone(catalog.resolve("hands", "stock", self.tree, cat)["folder"])
+
+    def test_channel(self):
+        self.assertIsNone(catalog.channel(self.tree))
+        (self.tree / ".git").mkdir()
+        (self.tree / ".git" / "HEAD").write_text("ref: refs/heads/experimental\n")
+        self.assertEqual(catalog.channel(self.tree), "experimental")
+        (self.tree / ".git" / "HEAD").write_text("ref: refs/heads/my-branch\n")
+        self.assertIsNone(catalog.channel(self.tree))
+        (self.tree / ".frametop-release").write_text("VERSION=x\nCHANNEL=stable\n")
+        self.assertEqual(catalog.channel(self.tree), "main")
+
+    def test_auto_off(self):
+        with mock.patch.dict(os.environ, {"FT_EYES_MODELS_UPDATE": "0"}):
+            self.assertIn("not updating", catalog.auto("eyes", self.tree))
+
+    def test_auto_fetches_setting(self):
+        (self.tree / "gaze/tracker").mkdir(parents=True)
+        (self.tree / "gaze/tracker/models.json").write_text(json.dumps(self.cat))
+        with mock.patch.dict(os.environ, {"FT_EYES_MODEL": "latest"}), \
+                mock.patch.object(catalog, "refresh", lambda kind, tree: (False, "offline")), \
+                mock.patch.object(catalog.urllib.request, "urlopen", FakeHF()):
+            self.assertIn("fetched eyes-t1", catalog.auto("eyes", self.tree))
+            self.assertIn("is here", catalog.auto("eyes", self.tree))
+
+    def test_cached_catalog_only_for_its_branch(self):
+        (self.tree / "gaze/tracker").mkdir(parents=True)
+        (self.tree / "gaze/tracker/models.json").write_text(json.dumps(self.cat))
+        newer = eye_catalog(latest="eyes-t0")
+        catalog.CACHE.mkdir(parents=True)
+        (catalog.CACHE / "eyes.json").write_text(json.dumps({"branch": "experimental", "fetched": 0, "catalog": newer}))
+        self.assertEqual(catalog.load("eyes", self.tree)[0]["latest"], "eyes-t1")   # not on a branch: its own
+        (self.tree / ".frametop-release").write_text("CHANNEL=experimental\n")
+        cat, source = catalog.load("eyes", self.tree)
+        self.assertEqual((cat["latest"], source), ("eyes-t0", "github experimental"))
+        (catalog.CACHE / "eyes.json").write_text(json.dumps({"branch": "experimental", "fetched": 0,
+                                                              "catalog": dict(newer, latest="classic")}))
+        self.assertEqual(catalog.load("eyes", self.tree)[0]["latest"], "eyes-t1")   # invalid cache: ignored
+
+
+if __name__ == "__main__":
+    unittest.main()

pack/install-release.sh

@@ -20,6 +20,7 @@
 #
 # Usage: install-release.sh [--yes] [--any-steamos] [--unpack-only] [--dir DIR]
 #                           [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
+#                           [--no-eye-recorder]
 #                           [--no-bluetooth | --bluetooth]
 #   --unpack-only  stop after step 2 and the copy: don't run install.sh
 #   --dir DIR      where the releases go (default ~/.local/share/frametop/releases)
@@ -113,7 +114,7 @@ main() {
       --any-steamos) any=1 ;;
       --unpack-only) unpack_only=1 ;;
       --dir) base=${2:?--dir needs a folder}; shift ;;
-      --no-voice|--no-hands|--no-hand-recorder|--no-eye-tracker|--no-bluetooth|--bluetooth) pass+=("$1") ;;
+      --no-voice|--no-hands|--no-hand-recorder|--no-eye-tracker|--no-eye-recorder|--no-bluetooth|--bluetooth) pass+=("$1") ;;
       -h|--help) sed -n '2,27p' "$0"; return 0 ;;
       *) echo "unknown option: $1" >&2; return 2 ;;
     esac

recorder/tests/test_age.py

@@ -76,7 +76,7 @@ class AgeTest(unittest.TestCase):
 
     def test_files_and_recipients(self):
         rec = age.read_recipients(os.path.join(HERE, "..", "..", "gaze", "rec", "recipients.txt"))
-        self.assertEqual(len(rec), 2)
+        self.assertEqual(len(rec), 3)              # keys 1 (7i), 2 (offline), 3 (dody)
         with tempfile.TemporaryDirectory() as d:
             src, dst = os.path.join(d, "a"), os.path.join(d, "a.age")
             with open(src, "wb") as f:

scripts/report.sh

@@ -206,6 +206,7 @@ fi
 # looks wrong first. It wakes an idle gaze service for about 20 s, to see the tracker send.
 if has gaze; then
   python3 "$repo/scripts/gaze-report.py" </dev/null 2>&1 || echo "scripts/gaze-report.py failed"
+  python3 "$repo/models/ft-models" list eyes 2>&1 | head -n 12
 fi
 
 # Hand tracking: whether it runs, which models ft-hands loaded and whether they're the pinned ones,
@@ -220,6 +221,7 @@ if has hands; then
   [ -d "$student" ] && "$repo/hands/models.sh" check 2>&1 | tail -n 3
   echo "last checked for newer ones: $(date -r "$repo/hands/models/.student-checked" '+%F %T' 2>/dev/null || echo never)"
   grep -E '^\s*(FT_)?HANDS_MODELS(_UPDATE)?=' ~/.config/frametop.conf 2>/dev/null | sed 's/\s*#.*//; s/^/override (ft-hands loads these instead): /'
+  python3 "$repo/models/ft-models" list hands 2>&1 | head -n 12
   python3 "$repo/hands/camcheck.py" 2>&1 | head -n 30
   echo "sides: $(cat "$XDG_RUNTIME_DIR/frametop-hands/sides.json" 2>/dev/null || echo 'no sides.json')"
   for c in 5 6 7; do   # ft-hands' CPUs (HANDS_CPUS)

session/frametop.conf.example

@@ -42,6 +42,8 @@ POINTER_HEAD_DEADZONE=0.5  # keyboard clicks at the gaze (Meta+J, Meta+K): degre
 POINTER_KEY_TAP=0.25       # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
 GAZE_TRACKER=auto          # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
 GAZE_EYE=auto              # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
+#EYES_MODEL=classic        # our eye tracker (GAZE_TRACKER own): classic = the pupil finder | latest = the newest eye net, following new versions | a version's name (gaze/tracker/models.json) | a model folder of your own. Pick it on Frametop Input Settings' Gaze page, which downloads it
+EYES_MODELS_UPDATE=1       # our eye tracker: 1 = when it starts, check GitHub for the eye net list and fetch the net EYES_MODEL needs | 0 = only what's downloaded
 GAZE_FIT_WATCH=0           # gaze service, with our eye tracker installed: 1 = the put-on check (the quick check, when the headset goes on, first says which way to move it back to your saved fit; the fit check saves a good one) and the slip note (our tracker in use: a small note when the headset slipped) | 0 = off
 HANDS_MODE=off             # hand tracking (hands/run.sh install; install.sh sets frametop when it installs it), when it runs: frametop = while the Frametop desktop runs | always = whenever SteamVR runs, in games and the dashboard too | off = only by hand (ft-handsctl on). Frametop Input Settings' Hands page applies it at once
 HANDS_SWAP_SIDES=auto      # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them
@@ -52,6 +54,7 @@ HANDS_BRIGHT_ON=40         # hand tracking, auto: the colour frames' mean bright
 HANDS_BRIGHT_OFF=25        # ... and back to the mono cameras under this
 HANDS_COLOR_LEFT=color_video0  # hand tracking: which colour camera is passthrough_left (hands/tools/check_color.py tells)
 HANDS_COLOR_CROP=subtract  # hand tracking: how the colour calibration's crop applies, subtract | none (check_color.py tells)
+#HANDS_MODEL=latest        # hand tracking: latest = the fine-tuned models we recommend, following new versions (hands/models/student) | stock = Google's MediaPipe models | a version's name (hands/models.json). Pick it on Frametop Input Settings' Hands page, which downloads it
 #HANDS_MODELS=            # hand tracking: a folder of palm.ncnn.* and hand.ncnn.* to use instead of the fine-tuned models (hands/models/student; hands/models/ncnn has the stock ones)
 HANDS_MODELS_UPDATE=1      # hand tracking: 1 = when it starts, fetch newer fine-tuned models if GitHub's main (stable) or experimental branch pins them (hands/models.pin), without a new release | 0 = keep the ones it came with
 HANDS_MATTE=circle         # hand cutouts: circle = a round hole per hand | mesh = hand-shaped, from the skeleton | capsules = ft-hands' capsules (the old look)

uninstall.sh

@@ -120,6 +120,7 @@ main() {
     [ -e "$apps/$f.desktop" ] && entries+=("$apps/$f.desktop")
   done
   [ -e "$apps/frametop-handrec.desktop" ] && entries+=("$apps/frametop-handrec.desktop")
+  [ -e "$apps/frametop-eyerec.desktop" ] && entries+=("$apps/frametop-eyerec.desktop")
   entries+=("$apps"/frametop-profile-*.desktop)  # one per layout profile (layout/ft_layout.py)
   [ -L "$handsctl" ] || handsctl=
   [ -f "$eye_rate" ] && [ -f "$repo/gaze/eye-rate.py" ] || eye_rate=