mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-18 12:01:17 +02:00
Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
342bf78b05 | ||
|
|
b51fab9872 | ||
|
|
8fb00b263c | ||
|
|
4937caf310 | ||
|
|
128e7a4c94 | ||
|
|
1a439f05c2 | ||
|
|
016d2b8c2d | ||
|
|
42ec68ed51 | ||
|
|
fa49fab5bf | ||
|
|
ce9e476ec7 |
No files matched your search
@@ -73,7 +73,7 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a
|
||||
Browse the shots you take in the headset and save them to your Pictures folder.
|
||||
|
||||
**⌨️ Keyboard and trackpad**<br>
|
||||
Type and point in the headset from your computer or phone, through Valve's own input path, with nothing to install. Accents and emoji use KDE Connect, which Frame Control brings along and sets up the first time you type one.
|
||||
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Nothing else to install, anywhere.
|
||||
|
||||
</td>
|
||||
</tr>
|
||||
@@ -189,7 +189,7 @@ entry to `~/.ssh/config` and keys at `~/.ssh/id_ed25519_frame` and
|
||||
|
||||
This is a first public test, so reports are really useful, especially from
|
||||
Windows and Linux. The quickest way is **Report a problem** in the app (the
|
||||
speech-bubble button at the top, or **Help → Report a Problem…**). It adds
|
||||
warning-sign button at the top, or **Help → Report a Problem…**). It adds
|
||||
diagnostics with personal details removed, shows you exactly what's included,
|
||||
and sends it privately to the maintainer; nothing is published. Without the app,
|
||||
use the [feedback form](https://frame-control.pages.dev/feedback/). Please include:
|
||||
@@ -223,6 +223,7 @@ Frame's software fits together, all checked against a real headset and labelled
|
||||
| [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes |
|
||||
| [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing |
|
||||
| [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset |
|
||||
| [Eye tracking and heart rate](docs/tracking.md) | Our OpenXR → OSC bridge, BlueZ heart-rate panel and optional local session log; SlimeVR feasibility notes |
|
||||
| [AI agents and assistant](docs/agents.md) | Key-free MCP tools, human approvals, and an opt-in assistant panel |
|
||||
| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, the headset smoke test and a Windows test VM |
|
||||
| [Open questions](docs/open-questions.md) | What's still unchecked |
|
||||
|
||||
+1
-1
@@ -97,7 +97,7 @@ replies, screenshots and approval payloads are not sent to analytics.
|
||||
|
||||
## Assistant panel
|
||||
|
||||
Open **Settings** (the gear in the header) **→ Open assistant settings**, or `http://127.0.0.1:47810/assistant`.
|
||||
Open **Tools → Open assistant**, or `http://127.0.0.1:47810/assistant`.
|
||||
To put the same page in the headset, with the HTTP server still running:
|
||||
|
||||
```sh
|
||||
|
||||
+3
-14
@@ -3,9 +3,8 @@
|
||||
Frame Control can manage more than one Steam Frame, and each headset can be
|
||||
reached at more than one address: a LAN IP at home, another at the office, its
|
||||
mDNS name (`frame.local`), its Tailscale IP or MagicDNS name. The **Devices**
|
||||
tab (key 5) lists them. The connection pill in the header says whether the one in
|
||||
use is connected and how (Tailscale, or the network's name); click it to add,
|
||||
edit or reorder that headset's addresses, or to see each step of connecting.
|
||||
tab (key 5) lists them, and the connection pill in the header shows what the
|
||||
app is doing to reach the one in use, step by step, as it happens.
|
||||
|
||||
The code is in three modules, all stdlib-only Python on your computer:
|
||||
|
||||
@@ -58,7 +57,7 @@ and ranks them:
|
||||
5. addresses that only ever worked on other networks;
|
||||
6. Tailscale addresses while Tailscale is off.
|
||||
|
||||
Your order (on the Devices tab, or in the pill's dialog) breaks ties. The best-ranked address that answers
|
||||
Your order on the Devices tab breaks ties. The best-ranked address that answers
|
||||
wins; one that answers first waits up to 0.35 s for a better-ranked one that is
|
||||
still trying. If SSH to the winner fails in a way another address could fix
|
||||
(a different device answered there, or the link dropped), the next one that
|
||||
@@ -73,16 +72,6 @@ status --json`, including the Mac app's own CLI) with buttons to add their
|
||||
MagicDNS name or IP. **Find on this network** asks mDNS for SteamOS devkit
|
||||
services and checks `ALIAS.local` and `frame.local`.
|
||||
|
||||
**The pill's dialog** lists the same addresses, with what each one answered, and
|
||||
can add, edit, reorder and remove them without leaving the page you're on. When
|
||||
the headset reports a LAN IP on the same network as this computer and that IP
|
||||
isn't saved, it offers to add it. The offer puts the address first in the list,
|
||||
so on that network it wins over the Tailscale name; away from home the Tailscale
|
||||
name still leads. A new or edited address is tested straight away. While
|
||||
connected, **Reconnect** applies your changes now rather than at the next
|
||||
connection: it tries the addresses again, ranked as above, and the best-ranked
|
||||
one that answers promptly wins. It doesn't pick a particular address.
|
||||
|
||||
## Networks
|
||||
|
||||
A network is told apart by its default gateway: the router's IP address plus its
|
||||
|
||||
@@ -58,7 +58,7 @@ not been verified and is not guaranteed.
|
||||
|
||||
## Casting
|
||||
|
||||
**Cast to this screen** (Home → **Right now**) starts the existing headset Live view and requests full
|
||||
**Cast headset view** starts the existing headset Live view and requests full
|
||||
screen where supported. Show that screen to people in the room, or use the
|
||||
computer/phone's own screen mirroring. It creates no new stream transport,
|
||||
public URL or LAN server. iPhone uses the inline viewer if full screen is not
|
||||
|
||||
+10
-16
@@ -17,21 +17,15 @@ python3 ui/server.py # anywhere: then open http://127.0.0.1:47810
|
||||
|
||||
## Features
|
||||
|
||||
The window has six tabs: **Home** (what the headset's doing and what you'd do
|
||||
next, the headset view, screenshots, family and comfort, the keyboard and
|
||||
trackpad, and VR performance, folded away), **Games** (installed games,
|
||||
sideloaded titles, getting games), **Android** (apps, the catalogue, display
|
||||
settings, reports), **Tools** (in the headset: your computer's windows, the media
|
||||
player and the panel switcher; then sending files and text, Flatpaks, remote and
|
||||
power), **Devices** (your headsets and their addresses), and **Control** (the
|
||||
searchable Control Center, metrics, supported settings and headset guidance).
|
||||
Keys 1–6 switch between them. The gear in the header (key 7) opens **Settings**:
|
||||
privacy and updates, library artwork and the assistant. The battery chip opens
|
||||
the headset's details: storage, memory, temperature, Wi-Fi, uptime and SteamOS
|
||||
build. Files can be dropped anywhere in the window. A
|
||||
connection pill in the header always shows which headset and whether it's
|
||||
connected, and how (Tailscale, or the network's name). Click it to add, edit or
|
||||
reorder the headset's addresses, or to see each step of connecting. When the Frame can't be
|
||||
The window has six tabs: **Home** (headset view, status, screenshots),
|
||||
**Games** (installed games, sideloaded titles, getting games), **Android** (apps,
|
||||
the catalogue, display settings, reports), **Tools** (sending files and text,
|
||||
Flatpaks, remote and power), **Devices** (your headsets and their addresses),
|
||||
and **Control** (the searchable Control Center, metrics, supported settings and
|
||||
headset guidance). Keys 1–6 switch between them. Files can be dropped anywhere in the window. A
|
||||
connection pill in the header always shows which headset, which network this
|
||||
computer is on, the address in use or being tried, and each step of connecting
|
||||
as it happens; click it for the whole timeline. When the Frame can't be
|
||||
reached, a banner says why in plain words, what was tried, and counts down to
|
||||
the next try, filling everything in once it answers. Flatpak and Android installs run in the background; the bottom bar
|
||||
counts them while they run.
|
||||
@@ -245,5 +239,5 @@ Control implementations. MCP wraps this HTTP API without API keys. Changes
|
||||
require a separate user approval; power also retains its password prompt. The
|
||||
assistant uses a user-chosen endpoint and sends nothing until the user opts in
|
||||
for a message. Screenshot context is separately opt-in. Model replies cannot
|
||||
operate the headset. Settings → Open assistant settings opens the page; the linked guide
|
||||
operate the headset. Tools → Open assistant opens the page; the linked guide
|
||||
covers putting it in a Chromium panel on the Frame.
|
||||
@@ -20,6 +20,15 @@ SteamVR (vrserver, vrcompositor, vrdashboard) ← renders the room + pane
|
||||
Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 3056000
|
||||
```
|
||||
|
||||
## Tracking additions (verified 2026-09-28)
|
||||
|
||||
On SteamOS 0.4.1, build `20260925.6191901`, SteamVR exposes combined gaze
|
||||
through `XR_EXT_eye_gaze_interaction` in a headless OpenXR 1.0 session. Our
|
||||
reader obtained valid tracked samples and sent OSC to a configured loopback
|
||||
receiver. BlueZ LE discovery works; GTK4/GI can render our heart-rate panel.
|
||||
No BLE strap or SlimeVR trackers were attached. See [tracking](tracking.md)
|
||||
for the evidence, privacy defaults and untested integration boundaries.
|
||||
|
||||
## Facts worth knowing
|
||||
|
||||
| Fact | Where it matters |
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 19 KiB |
+2
-2
@@ -114,8 +114,8 @@ don't.
|
||||
|
||||
## Family and comfort
|
||||
|
||||
The shared Home card sets session limits, breaks and check-ins; **Cast to this
|
||||
screen** is in Home's **Right now** card. **Enable / test notifications** requests iOS notification
|
||||
The shared Home card sets session limits, breaks and check-ins, and offers
|
||||
**Cast headset view**. **Enable / test notifications** requests iOS notification
|
||||
permission and sends a local test. These are local notifications, not APNs push;
|
||||
iOS background suspension can interrupt phone alerts. The headset timer still
|
||||
runs. See [the behavior and verification limits](family-comfort.md).
|
||||
+1
-1
@@ -125,7 +125,7 @@ a limit on the number of floating panels.
|
||||
separate from Frame Control. Public reports describe some Proton support;
|
||||
Windows-only does not by itself prove a Frame app cannot run. Local status
|
||||
and sources are in [VR utilities](vr-utilities.md).
|
||||
- **Our performance HUD:** Home → VR performance (unfold it) → Open HUD in
|
||||
- **Our performance HUD:** Home → VR comfort and performance → Open HUD in
|
||||
headset creates its own gamescope panel using built-in tools. It needs no
|
||||
third-party overlay app. [Metrics and verification](vr-utilities.md).
|
||||
|
||||
|
||||
+2
-2
@@ -132,8 +132,8 @@ runs.
|
||||
|
||||
## Report a problem
|
||||
|
||||
**Report a problem** is the speech-bubble button in the header, also in the
|
||||
Settings page (**Privacy & updates**) and under **Help → Report a Problem…**. It sends the report
|
||||
**Report a problem** is the warning-sign button in the header, also in the
|
||||
Privacy panel and under **Help → Report a Problem…**. It sends the report
|
||||
privately to Frame Control's PostHog project as a `problem_report` event, the
|
||||
same way as the analytics above, so only the maintainer can read it and
|
||||
nothing is published. It works whatever the analytics settings are, because
|
||||
|
||||
@@ -65,8 +65,6 @@ flow, not evidence of an anonymous download API. It is not implemented here.
|
||||
- `search()` raises a user-readable `SourceError` with the browse URL (zero
|
||||
limit returns no rows). It does not invent app results or report a false
|
||||
“no matching games”. The aggregate search UI should surface this source error.
|
||||
In the app, Discover doesn't link to it; the **Sources** dialog lists SideQuest
|
||||
with a link to its website instead of an on/off switch.
|
||||
- `details()` accepts a numeric listing id and returns its canonical page link,
|
||||
`downloadable: False`, empty versions/tags/headsets and the `images` shape
|
||||
`{icon: None, banner: None, screenshots: []}`. Name is explicitly a listing id;
|
||||
|
||||
+4
-7
@@ -151,13 +151,10 @@ two fingers to scroll, two-finger tap to right-click) plus a text field that
|
||||
types on the Frame. On a computer, clicking the pad passes your mouse and
|
||||
keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac).
|
||||
|
||||
Since Control (below), the pad goes through **gamescope's own input socket**
|
||||
([`ui/frame_touch.py`](../ui/frame_touch.py)), so it reaches whichever panel has
|
||||
focus, Steam's own menus included, and needs nothing installed on the Frame.
|
||||
Only characters that aren't on a US keyboard (accents, emoji) go through
|
||||
**KDE Connect** as below; it starts the first time you type one. (Before
|
||||
Control, everything went through KDE Connect, which is why the notes below
|
||||
describe it in detail.)
|
||||
It goes through **KDE Connect**, the first-party route (KDE makes the Frame's
|
||||
desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py)
|
||||
on the Frame, which talks KDE Connect's own LAN protocol to the Frame's
|
||||
`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking.
|
||||
|
||||
**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901):
|
||||
|
||||
|
||||
@@ -187,6 +187,13 @@ refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
|
||||
accepted. The fake now refuses them the same way, and Frame Control makes ids
|
||||
Steam accepts.
|
||||
|
||||
## Tracking protocols and fake BlueZ
|
||||
|
||||
`tests/test_tracking.py` exercises our gaze conversion, OSC sender, HRS parser
|
||||
and BlueZ lifecycle with an in-memory fake object tree. It runs in the normal
|
||||
unit suite without Bluetooth, GTK or OpenXR. Real Frame results and the absent
|
||||
strap/tracker boundaries are recorded in [tracking](tracking.md).
|
||||
|
||||
## Windows test VM
|
||||
|
||||
The unit tests run on Windows in CI, but the app itself doesn't. Anything that
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
# Eye tracking and heart rate
|
||||
|
||||
Frame Control's own tools run on the Frame, using OpenXR and BlueZ. No
|
||||
VRCFaceTracking, LunaHR, Pulsoid or other tracking app is required. This is a
|
||||
command-line first version; it does not add a desktop app tab.
|
||||
|
||||
## What was checked
|
||||
|
||||
**Verified 2026-09-28**, on a real aarch64 Frame running SteamOS 0.4.1,
|
||||
BUILD_ID `20260925.6191901`:
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| OpenXR gaze | SteamVR advertises `XR_EXT_eye_gaze_interaction`, `XR_MND_headless` and `XR_KHR_convert_timespec_time`. `supportsEyeGazeInteraction=1`. A headless session reached FOCUSED and produced 269 valid, tracked orientations in the first ten-second probe. |
|
||||
| Our gaze → OSC bridge | A separate ten-second run produced 280 valid samples and 280 correctly padded 44-byte `/tracking/eye/CenterPitchYaw` messages at an explicitly configured loopback receiver. Only counters and packet-layout checks were retained. |
|
||||
| Bluetooth stack | BlueZ active, adapter powered, central/peripheral roles available. LE discovery started and stopped successfully. No pairing or adapter power settings changed. |
|
||||
| Our heart-rate panel | GTK4/GI runs on the stock image. A **synthetic 72 BPM** notification displayed in our X11 window, tagged `STEAM_GAME=2000000027`. Window capture checked; no real heart-rate measurement was taken. |
|
||||
| SlimeVR, separate feasibility check | Native aarch64 server v21.1.0 ran with an isolated Temurin 21 JRE, created its driver sockets and accepted a local TCP connection on port 21110. Driver v6.0.0 loaded with all shared libraries resolved; `HmdDriverFactory("IServerTrackedDeviceProvider_004")` returned a non-null provider and error 0. |
|
||||
|
||||

|
||||
|
||||
The image is a capture of our own Frame window using a fake notification,
|
||||
not a real sensor reading.
|
||||
|
||||
**Untested:** a real BLE strap's notifications, physical fit/contact behaviour,
|
||||
end-to-end heart-rate display/OSC/log with a strap, avatar response in VRChat,
|
||||
gaze accuracy/calibration, coexistence with every immersive app, in-headset
|
||||
panel placement, SlimeVR tracker/calibration data and the SlimeVR driver running
|
||||
inside SteamVR. No trackers or strap are attached. The driver was loaded in a
|
||||
separate process; it was **not registered or activated in SteamVR**. Steam and
|
||||
SteamVR were not stopped or restarted.
|
||||
|
||||
**Verified blocker resolved:** importing `tkinter` fails because `libtk8.6.so`
|
||||
is absent. The panel uses the installed GTK4/GI bindings instead. The Frame's
|
||||
OpenXR headers advertise a newer API version than the runtime accepts; our
|
||||
reader requests OpenXR 1.0 explicitly.
|
||||
|
||||
## Install our tools
|
||||
|
||||
From this checkout on your computer, while the Frame is awake:
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py install
|
||||
```
|
||||
|
||||
This copies our Python code and compiles our small C OpenXR reader into
|
||||
`~/.local/share/frame-control/tracking/` on the Frame. It uses the Frame's
|
||||
existing compiler, OpenXR headers/loader, Python, dbus-python, GI and GTK4.
|
||||
Nothing is downloaded, and no sudo, driver registration, system setting,
|
||||
service or autostart is added. `FRAME_ALIAS` can select another SSH alias.
|
||||
The desktop app/server keeps its existing stdlib-only dependency set.
|
||||
|
||||
## Eye tracking → OSC
|
||||
|
||||
Start with a ten-second capability/data-availability check:
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py gaze --seconds 10
|
||||
```
|
||||
|
||||
This prints support, session-state numbers and sample counters. It opens no
|
||||
OSC socket and prints no gaze coordinates. Exit 0 means at least one valid
|
||||
sample, 3 means no valid sample was observed, and 1 means an API/runtime error.
|
||||
If there are no valid samples, wake/wear the headset and check its tracking
|
||||
setup; a successful capability check alone does not prove usable gaze.
|
||||
|
||||
To send to VRChat running **on the Frame**, explicitly enable OSC in VRChat
|
||||
and choose its local UDP endpoint:
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py gaze --seconds 3600 --osc 127.0.0.1 9000
|
||||
```
|
||||
|
||||
For a receiver on another computer, replace `127.0.0.1` with that computer's
|
||||
IP address and choose its listening port. Addresses are IP literals (IPv4 or
|
||||
IPv6); there is no discovery or default destination. Loopback here always
|
||||
means **the Frame**, not the computer running the SSH command. OSC uses
|
||||
unencrypted UDP: configure only a receiver you intend to receive this data.
|
||||
|
||||
**Documented:** [VRChat's eye OSC interface](https://docs.vrchat.com/docs/osc-eye-tracking)
|
||||
accepts `/tracking/eye/CenterPitchYaw` with two floats in degrees, positive down
|
||||
and right. We locate OpenXR's combined gaze pose relative to VIEW (the head),
|
||||
rotate its -Z forward vector and convert that direction to these angles.
|
||||
Only active, orientation-valid **and tracked** samples are sent, at up to
|
||||
30 Hz. No eyelid/blink, individual-eye or face values are invented. We do not
|
||||
send neutral gaze on tracking loss; VRChat's documented timeout restores its
|
||||
automatic eye behaviour after input stops.
|
||||
|
||||
**Privacy:** gaze is personal data. It stays in process memory and a private
|
||||
pipe between our reader and bridge. There is no gaze log option, telemetry,
|
||||
OSC receiver or raw gaze on stdout/stderr. Only an explicit `--osc IP PORT`
|
||||
opens an output socket. Runtime diagnostics and validity counters are not
|
||||
measurements. Stop with Ctrl-C or let `--seconds` expire (maximum 24 hours).
|
||||
A lost headless session ends the run; it does not silently reconnect.
|
||||
|
||||
## BLE heart rate → our panel, OSC and optional log
|
||||
|
||||
First discover/pair your strap in SteamOS's Bluetooth settings. Select that
|
||||
strap's Bluetooth address explicitly; our tool does not scan for or connect
|
||||
to arbitrary nearby devices.
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py heart \
|
||||
--device AA:BB:CC:DD:EE:FF --panel --seconds 3600
|
||||
```
|
||||
|
||||
This uses BlueZ's standard Heart Rate Service (`180d`) and Heart Rate
|
||||
Measurement (`2a37`) notifications. It finds the characteristic only beneath
|
||||
the selected device's HRS service. The reader handles 8- and 16-bit BPM,
|
||||
contact flags and optional energy/RR fields; energy and RR intervals are
|
||||
validated for length but discarded. Zero BPM, reported loss of skin contact,
|
||||
malformed packets and readings older than five seconds are not shown as a
|
||||
current measurement. A disconnect stops the run; reconnect and start again.
|
||||
This is a social/fitness readout, not a medical monitor.
|
||||
|
||||
The panel is our GTK4 window on gamescope's X display. Use SteamVR's panel
|
||||
controls to float/dock it (see [panels](panels.md)). **Stop**, closing the panel,
|
||||
Ctrl-C, SSH hangup or the duration limit ends our subscription. A connection
|
||||
that was already open when we started is preserved; a connection we opened
|
||||
is disconnected on exit. No Bluetooth power or pairing state is changed.
|
||||
|
||||
Add either output explicitly:
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py heart \
|
||||
--device AA:BB:CC:DD:EE:FF --panel --seconds 3600 \
|
||||
--osc 127.0.0.1 9000 --address /avatar/parameters/HeartRate \
|
||||
--log /home/steamos/heart-session.csv
|
||||
```
|
||||
|
||||
The OSC value is integer BPM. `HeartRate` is a chosen avatar parameter, **not a
|
||||
built-in VRChat heart-rate feature**; your avatar/receiver must define the
|
||||
matching parameter. `--address` can select another literal OSC path. The local
|
||||
panel works without OSC, a log or an avatar integration.
|
||||
|
||||
The optional CSV contains only `unix_seconds,bpm`. It is created privately
|
||||
(mode 0600), refuses existing files/symlinks, and lives **on the Frame** at the
|
||||
path you specify. Nothing is logged by default, and heart-rate values are not
|
||||
printed to the terminal. Delete your session file when you no longer need it.
|
||||
|
||||
### Checking heart rate against a reference
|
||||
|
||||
`scripts/heart-check.py` runs on your computer. `listen` shows our OSC
|
||||
readings live as they arrive, so you can watch them next to another device:
|
||||
|
||||
```sh
|
||||
python3 scripts/heart-check.py listen --port 9000 --out ours.csv
|
||||
```
|
||||
|
||||
Point the Frame at it with `--osc <your computer's IP> 9000`. `compare` lines
|
||||
up two recordings by time and reports the mean difference, bias, the share
|
||||
within ±5 BPM and the delay between them. It passes when the mean difference
|
||||
is at most 5 BPM, at least 80% of reference readings are matched and nothing
|
||||
was shown while the sensor reported lost skin contact:
|
||||
|
||||
```sh
|
||||
python3 scripts/heart-check.py compare ours.csv reference.csv
|
||||
python3 scripts/heart-check.py compare ours.csv ~/Downloads/export.zip
|
||||
```
|
||||
|
||||
The reference can be a CSV (`time,bpm[,flags]`, time in unix seconds or ISO
|
||||
8601) or an Apple Health export (`export.zip` or `export.xml`). Only heart-rate
|
||||
records within the recording's time range are read. Everything stays on your
|
||||
computer.
|
||||
|
||||
`scripts/heart-test-strap.swift` turns a Mac into a synthetic strap. It
|
||||
advertises the standard Heart Rate Service and sends a fixed, known sequence
|
||||
(8-bit and 16-bit values and a skin-contact loss), printing each sent value, so
|
||||
`compare` can check that the Frame shows exactly what was sent. It needs
|
||||
Bluetooth permission for the process that runs it. **Untested on 2026-09-29:**
|
||||
it compiled, but on this Mac, launched from an agent session, macOS never
|
||||
delivered a Bluetooth state and no permission prompt appeared, so it never
|
||||
advertised.
|
||||
|
||||
## Pulse from the eye cameras (experimental)
|
||||
|
||||
The Frame has no heart-rate sensor. **Verified 2026-09-29** (SteamOS 0.4.1,
|
||||
build `20260925.6191901`): its sensors are an ambient light/proximity sensor
|
||||
(`vcnl4000`), a hall sensor (`als31300`), two passthrough cameras
|
||||
(`arcimx616`), two tracking cameras (`og01a1b`) and two IR eye cameras
|
||||
(`og0ve10`). There is no optical heart-rate (PPG) sensor.
|
||||
|
||||
The experiment asks whether the eye cameras can see a pulse anyway. With each
|
||||
heartbeat, the blood volume in the skin around the eye changes slightly and
|
||||
its IR reflectance changes with it. This is camera-based photoplethysmography;
|
||||
near-IR works, though the signal is weaker than in green light.
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py pulse --seconds 60 --show
|
||||
```
|
||||
|
||||
How it works:
|
||||
|
||||
- **Capture (verified).** SteamVR ships `eyetracking --calib N`, which saves
|
||||
both eye cameras for N seconds as 400×400 8-bit IR PNGs with a monotonic
|
||||
timestamp per frame, at about 90 fps per eye. SteamVR's live eye tracker,
|
||||
part of `steamvr.service`, gets its frames from the DSP and stops its
|
||||
cameras when the headset is off. Unworn captures ran alongside it: its PID
|
||||
and log were unchanged and our OpenXR gaze session still started
|
||||
afterwards. **Untested:** whether the capture and the live tracker coexist
|
||||
while the headset is worn and tracking.
|
||||
- **Privacy.** Each image is reduced to a 16×16 grid of patch averages as
|
||||
soon as it is complete, then deleted. Three worker processes do this beside
|
||||
the capture. If more than 900 images (about five seconds) ever wait, we stop
|
||||
reading them and delete them undecoded until the capture ends, and report
|
||||
an error. The capture directory is removed on exit, even after errors. No image is kept or leaves the Frame. The estimate
|
||||
is printed only with `--show`, and sent or saved only with `--osc` or
|
||||
`--log`, as for the strap.
|
||||
- **Estimate.** Patch traces are averaged down to 15 Hz and turned into
|
||||
relative change. A 2-second moving median removes drift and blinks.
|
||||
Patches with frequent spikes (the eyeball and eyelid) are dropped, as are
|
||||
dark or saturated ones. The 20% of patches with the clearest rhythm between
|
||||
42 and 180 BPM are combined in the frequency domain. Output is an overall
|
||||
estimate plus one estimate per second over 15-second windows. A result
|
||||
counts as **clear** only when the top patches agree and the combined signal
|
||||
stands out from the noise. Otherwise the command exits 3 and sends no OSC.
|
||||
`--log` still records the per-second estimates, so a comparison shows how
|
||||
far off an unclear result was.
|
||||
The thresholds are provisional until checked on real wearers.
|
||||
|
||||
**Verified on the Frame, unworn, 2026-09-29:** captures of 3,600-5,400 eye
|
||||
frames never had more than 8 images on disk, finished a few seconds after the
|
||||
capture ended and left no capture directory. **The estimator alone gave a
|
||||
false "clear" pulse.** With nobody wearing the headset, five runs reported a
|
||||
steady, self-consistent rhythm (90, 90, 93, 94 and 96 BPM; patch agreement
|
||||
100%, signal/noise 0.63-0.70), and earlier runs reported 127-129 BPM at lower
|
||||
signal/noise. It is a periodic camera or illumination artifact, and its
|
||||
frequency drifts between runs. A wearer-less scene cannot contain a pulse, so
|
||||
the signal/noise gate cannot tell this artifact from one. Because of that,
|
||||
`pulse` reads the Frame's proximity sensor (`vcnl4000`) before and after the
|
||||
capture. It reads about 3 unworn (**verified**). If either reading is below 20
|
||||
the result is never called clear, nothing is sent over OSC, and the command
|
||||
exits 3. **Inferred, unmeasured:** that a worn reading is well above 20; the
|
||||
cut-off is provisional until someone wears the headset. If the sensor can't be
|
||||
read, the guard is skipped. Confirming a real pulse also needs a reference
|
||||
(below).
|
||||
|
||||
**Do not interrupt the capture. Verified on the Frame, 2026-09-29:** sending
|
||||
SIGTERM to `eyetracking --calib` left the DSP service's eye camera (OV6211)
|
||||
stuck "streaming": its log had no "Stopping streaming" line, and every later
|
||||
request failed with "Failed to start streaming". Head tracking kept working.
|
||||
Clearing it needs the DSP service restarted or the Frame rebooted, so `pulse`
|
||||
never signals the tool. After Ctrl-C or a failure it keeps deleting images
|
||||
until the tool ends by itself (at most the `--seconds` plus a few seconds),
|
||||
and only kills a tool that overruns by 30 s, with a warning that the eye
|
||||
cameras may need a reboot. So an interrupted run can take a while to return.
|
||||
|
||||
**Verified on synthetic data** (unit tests): a 0.3% brightness pulse in a
|
||||
third of the patches, with noise, drift, blinks and eye movement, is
|
||||
recovered within 1.5 BPM at 58, 72 and 115 BPM; noise and blinks alone are
|
||||
not reported as a pulse. **Not yet verified:** whether a real wearer's eye
|
||||
images contain a usable pulse, and how accurate it is. That needs someone
|
||||
wearing the headset and a reference, as below.
|
||||
|
||||
### Comparing with an Apple Watch
|
||||
|
||||
1. On the watch, start a workout (for example **Other**) so it measures heart
|
||||
rate every few seconds rather than occasionally.
|
||||
2. Put the Frame on, sit still and look ahead. Run:
|
||||
|
||||
```sh
|
||||
python3 scripts/tracking-on-frame.py pulse --seconds 120 --show \
|
||||
--log /home/steamos/pulse.csv
|
||||
```
|
||||
|
||||
The per-second estimates print at the end. Compare them with what the
|
||||
watch showed.
|
||||
3. End the workout. On the iPhone, open Health → your picture → **Export All
|
||||
Health Data**, and AirDrop `export.zip` to the Mac.
|
||||
4. On the Mac:
|
||||
|
||||
```sh
|
||||
scp frame:pulse.csv . && ssh frame rm pulse.csv
|
||||
python3 scripts/heart-check.py compare pulse.csv ~/Downloads/export.zip
|
||||
```
|
||||
|
||||
This first version analyses after the capture ends, because the method must
|
||||
prove itself before a live panel is worth building. The Apple Watch is a
|
||||
reference, not ground truth: in workouts it is typically within a few BPM of
|
||||
a chest strap when you are still.
|
||||
|
||||
## SlimeVR: feasibility only
|
||||
|
||||
SlimeVR is an independent application stack. Neither of our features installs,
|
||||
launches or depends on it. Users who want it can follow
|
||||
[SlimeVR's setup documentation](https://docs.slimevr.dev/server/index.html).
|
||||
The consented upstream releases tested were
|
||||
[server v21.1.0](https://github.com/SlimeVR/SlimeVR-Server/releases/tag/v21.1.0)
|
||||
and [driver v6.0.0](https://github.com/SlimeVR/SlimeVR-OpenVR-Driver/releases/tag/v6.0.0),
|
||||
under SlimeVR's MIT/Apache-2.0 licensing.
|
||||
|
||||
**Verified layout, read-only:** the Frame's registered runtime is `/opt/steamvr`;
|
||||
its native driver is `drivers/cv/bin/linuxarm64/driver_cv.so`, with a
|
||||
`drivers/cv/driver.vrdrivermanifest`. Frame controller manifests/resources are
|
||||
under `drivers/frame_controller/`. Configuration is under
|
||||
`~/.config/openvr/config/`, not the Steam client's config directory. The
|
||||
SlimeVR release also uses `slimevr/bin/linuxarm64/driver_slimevr.so` plus its
|
||||
manifest. Nothing in those installed SteamVR directories was changed.
|
||||
|
||||
**Inferred:** the matching ABI/layout and standalone factory success make
|
||||
SteamVR integration plausible. They do not prove successful driver `Init`,
|
||||
server/driver IPC, tracking, or calibration. That needs a separate integration
|
||||
check with hardware and an agreed SteamVR restart. No Java executable was on
|
||||
PATH for this check, so an isolated JRE was used. SlimeVR's server opens LAN
|
||||
listeners; our temporary server was stopped and the temporary downloads,
|
||||
configuration and logs were removed. It is not left installed or running.
|
||||
|
||||
## Tests and remaining checks
|
||||
|
||||
```sh
|
||||
python3 -m unittest discover -s tests
|
||||
```
|
||||
|
||||
`tests/test_tracking.py` covers HRS packet parsing, contact/staleness, OSC
|
||||
padding/types and a real loopback socket, quaternion signs, opt-in networking,
|
||||
private/exclusive logging and a fake BlueZ object tree. The fake checks service
|
||||
ownership, notification routing, delayed GATT discovery and connection cleanup.
|
||||
It does not pretend to be a physical strap or a real OpenXR runtime.
|
||||
|
||||
Before calling hardware support complete, attach a strap and check BPM against
|
||||
its own display/reference, loss of contact, disconnect/reconnect, Stop, OSC and
|
||||
CSV together. Check avatar eyes while looking up/down/left/right in a supported
|
||||
VRChat session. No third-party tracking app is needed for either test.
|
||||
|
||||
Independent review attempt: `devin -p --model swe-2-max` with the frozen diff,
|
||||
contribution standards and read-only instructions returned no output for ten
|
||||
minutes. It was terminated with exit 143. No completed review or actual model
|
||||
identity was returned; hardware checks and independent review remain follow-up
|
||||
work before making the draft ready.
|
||||
+1
-1
@@ -94,7 +94,7 @@ A new shortcut is rolled back if artwork fails; failure is never reported as
|
||||
an installed app with a blank tile.
|
||||
|
||||
Artwork preference is **SteamGridDB → source images → generated fallback**.
|
||||
Set the optional free key in Frame Control's **Settings → Library artwork**, or
|
||||
Set the optional free key in Frame Control's **Library artwork settings**, or
|
||||
`STEAMGRIDDB_API_KEY` (`FRAME_STEAMGRIDDB_API_KEY` also works). Environment
|
||||
settings override the saved key. Without a key there are no provider calls or
|
||||
warnings. Saved keys stay in host app data, mode 0600 on POSIX, and are never
|
||||
|
||||
@@ -11,7 +11,7 @@ checks below. The PR stays draft.
|
||||
|
||||
## Our performance HUD
|
||||
|
||||
On **Home → VR performance** (folded until you open it), the app shows a timestamped sample
|
||||
On **Home → VR comfort and performance**, the app shows a timestamped sample
|
||||
with each status refresh (30 seconds, or Refresh). **Open HUD in headset**
|
||||
starts our text HUD as a gamescope panel, refreshed every two seconds. In the
|
||||
SteamVR dashboard, select **Frame Control HUD**, then Float in World or dock
|
||||
@@ -26,7 +26,7 @@ it twice reuses the existing process.
|
||||
| Compositor CPU | OpenVR compositor render CPU milliseconds, not game CPU time. |
|
||||
| System CPU | `/proc/stat` busy-time delta across the sample, with guest time counted once and iowait treated as idle. |
|
||||
| GPU clock | `3d00000.gpu/cur_freq`, converted from Hz to MHz; frequency is not load. |
|
||||
| Hottest sensor / battery | Existing thermal-zone and battery sysfs reads from `frame_status.py`. In the headset HUD only: the app shows them once, in the battery menu at the top. |
|
||||
| Hottest sensor / battery | Existing thermal-zone and battery sysfs reads from `frame_status.py`. |
|
||||
|
||||
OpenVR uses background application mode, which does not start SteamVR or keep
|
||||
it running. This mode also returned live timing in a read-only device probe.
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ is required.** Chromium and immersive WebXR are not in this playback path.
|
||||
|
||||
## Use it
|
||||
|
||||
In **Tools → Media player** (or **Play a video or photo** on Home), send a file, choose its layout and press
|
||||
In **Tools → Media in the headset**, send a file, choose its layout and press
|
||||
**Play**. **Theatre** gives it a larger screen and an 85% black surround.
|
||||
**Stop** removes both. Refresh reads the library and the player's state.
|
||||
The screen follows your head; it isn't a saved world-space panel.
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
/* Frame Control's OpenXR gaze source. No values on stdout/stderr or disk.
|
||||
* The Python bridge supplies a private pipe with --fd; standalone probes only
|
||||
* report counters. Uses a headless session, never submits frames or takes focus. */
|
||||
#define XR_USE_TIMESPEC
|
||||
#include <time.h>
|
||||
#include <openxr/openxr.h>
|
||||
#include <openxr/openxr_platform.h>
|
||||
#include <signal.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static volatile sig_atomic_t stopped;
|
||||
static void stop(int sig) { (void)sig; stopped = 1; }
|
||||
#define CHECK(call) do { result = (call); if (XR_FAILED(result)) { \
|
||||
fprintf(stderr, "%s failed (%d)\n", #call, result); goto cleanup; } } while (0)
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
int seconds = 10, fd = -1, running = 0, rc = 1;
|
||||
unsigned active = 0, valid = 0, samples = 0;
|
||||
for (int i = 1; i < argc; i++) {
|
||||
if (!strcmp(argv[i], "--seconds") && i+1 < argc) seconds = atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--fd") && i+1 < argc) fd = atoi(argv[++i]);
|
||||
else { fprintf(stderr, "usage: gaze [--seconds 1..86400] [--fd private-pipe]\n"); return 2; }
|
||||
}
|
||||
if (seconds < 1 || seconds > 86400 || (fd != -1 && fd < 3)) return 2;
|
||||
FILE *out = fd == -1 ? NULL : fdopen(fd, "w");
|
||||
if (fd != -1 && !out) return 2;
|
||||
signal(SIGINT, stop); signal(SIGTERM, stop); signal(SIGHUP, stop); signal(SIGPIPE, SIG_IGN);
|
||||
XrResult result;
|
||||
XrInstance instance = XR_NULL_HANDLE;
|
||||
XrSession session = XR_NULL_HANDLE;
|
||||
XrActionSet set = XR_NULL_HANDLE;
|
||||
XrSpace gaze = XR_NULL_HANDLE, view = XR_NULL_HANDLE;
|
||||
const char *extensions[] = {"XR_EXT_eye_gaze_interaction", "XR_MND_headless", "XR_KHR_convert_timespec_time"};
|
||||
XrInstanceCreateInfo create = {.type = XR_TYPE_INSTANCE_CREATE_INFO};
|
||||
strcpy(create.applicationInfo.applicationName, "Frame Control gaze");
|
||||
create.applicationInfo.apiVersion = XR_MAKE_VERSION(1, 0, 0);
|
||||
create.enabledExtensionCount = 3; create.enabledExtensionNames = extensions;
|
||||
CHECK(xrCreateInstance(&create, &instance));
|
||||
XrSystemGetInfo get = {.type = XR_TYPE_SYSTEM_GET_INFO, .formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY};
|
||||
XrSystemId system;
|
||||
CHECK(xrGetSystem(instance, &get, &system));
|
||||
XrSystemEyeGazeInteractionPropertiesEXT eye = {.type = XR_TYPE_SYSTEM_EYE_GAZE_INTERACTION_PROPERTIES_EXT};
|
||||
XrSystemProperties props = {.type = XR_TYPE_SYSTEM_PROPERTIES, .next = &eye};
|
||||
CHECK(xrGetSystemProperties(instance, system, &props));
|
||||
printf("supportsEyeGazeInteraction=%u\n", eye.supportsEyeGazeInteraction);
|
||||
if (!eye.supportsEyeGazeInteraction) goto cleanup;
|
||||
XrSessionCreateInfo sc = {.type = XR_TYPE_SESSION_CREATE_INFO, .systemId = system};
|
||||
CHECK(xrCreateSession(instance, &sc, &session));
|
||||
XrActionSetCreateInfo asc = {.type = XR_TYPE_ACTION_SET_CREATE_INFO};
|
||||
strcpy(asc.actionSetName, "gaze"); strcpy(asc.localizedActionSetName, "Gaze");
|
||||
CHECK(xrCreateActionSet(instance, &asc, &set));
|
||||
XrActionCreateInfo ac = {.type = XR_TYPE_ACTION_CREATE_INFO, .actionType = XR_ACTION_TYPE_POSE_INPUT};
|
||||
strcpy(ac.actionName, "gaze_pose"); strcpy(ac.localizedActionName, "Gaze pose");
|
||||
XrAction action;
|
||||
CHECK(xrCreateAction(set, &ac, &action));
|
||||
XrPath profile, input;
|
||||
CHECK(xrStringToPath(instance, "/interaction_profiles/ext/eye_gaze_interaction", &profile));
|
||||
CHECK(xrStringToPath(instance, "/user/eyes_ext/input/gaze_ext/pose", &input));
|
||||
XrActionSuggestedBinding binding = {action, input};
|
||||
XrInteractionProfileSuggestedBinding suggested = {.type = XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING,
|
||||
.interactionProfile = profile, .countSuggestedBindings = 1, .suggestedBindings = &binding};
|
||||
CHECK(xrSuggestInteractionProfileBindings(instance, &suggested));
|
||||
XrSessionActionSetsAttachInfo attach = {.type = XR_TYPE_SESSION_ACTION_SETS_ATTACH_INFO, .countActionSets = 1, .actionSets = &set};
|
||||
CHECK(xrAttachSessionActionSets(session, &attach));
|
||||
XrActionSpaceCreateInfo space = {.type = XR_TYPE_ACTION_SPACE_CREATE_INFO, .action = action, .poseInActionSpace.orientation.w = 1};
|
||||
CHECK(xrCreateActionSpace(session, &space, &gaze));
|
||||
XrReferenceSpaceCreateInfo ref = {.type = XR_TYPE_REFERENCE_SPACE_CREATE_INFO, .referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW,
|
||||
.poseInReferenceSpace.orientation.w = 1};
|
||||
CHECK(xrCreateReferenceSpace(session, &ref, &view));
|
||||
PFN_xrConvertTimespecTimeToTimeKHR convert;
|
||||
CHECK(xrGetInstanceProcAddr(instance, "xrConvertTimespecTimeToTimeKHR", (PFN_xrVoidFunction *)&convert));
|
||||
struct timespec start, now;
|
||||
clock_gettime(CLOCK_MONOTONIC, &start);
|
||||
while (!stopped) {
|
||||
clock_gettime(CLOCK_MONOTONIC, &now);
|
||||
if (now.tv_sec - start.tv_sec >= seconds) break;
|
||||
XrEventDataBuffer event = {.type = XR_TYPE_EVENT_DATA_BUFFER};
|
||||
while ((result = xrPollEvent(instance, &event)) == XR_SUCCESS) {
|
||||
if (event.type == XR_TYPE_EVENT_DATA_SESSION_STATE_CHANGED) {
|
||||
XrSessionState state = ((XrEventDataSessionStateChanged *)&event)->state;
|
||||
printf("sessionState=%d\n", state); fflush(stdout);
|
||||
if (state == XR_SESSION_STATE_READY && !running) {
|
||||
XrSessionBeginInfo begin = {.type = XR_TYPE_SESSION_BEGIN_INFO, .primaryViewConfigurationType = XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO};
|
||||
CHECK(xrBeginSession(session, &begin)); running = 1;
|
||||
} else if (state == XR_SESSION_STATE_STOPPING) {
|
||||
CHECK(xrEndSession(session)); running = 0; stopped = 1;
|
||||
} else if (state == XR_SESSION_STATE_EXITING || state == XR_SESSION_STATE_LOSS_PENDING) stopped = 1;
|
||||
} else if (event.type == XR_TYPE_EVENT_DATA_INSTANCE_LOSS_PENDING) stopped = 1;
|
||||
event.type = XR_TYPE_EVENT_DATA_BUFFER;
|
||||
}
|
||||
if (XR_FAILED(result)) goto cleanup;
|
||||
if (running && !stopped) {
|
||||
XrActiveActionSet activeSet = {set, XR_NULL_PATH};
|
||||
XrActionsSyncInfo sync = {.type = XR_TYPE_ACTIONS_SYNC_INFO, .countActiveActionSets = 1, .activeActionSets = &activeSet};
|
||||
CHECK(xrSyncActions(session, &sync));
|
||||
if (result != XR_SUCCESS) {
|
||||
struct timespec delay = {.tv_nsec = 33333333};
|
||||
nanosleep(&delay, NULL);
|
||||
continue; /* No stale gaze when the runtime denies focus. */
|
||||
}
|
||||
XrActionStateGetInfo ag = {.type = XR_TYPE_ACTION_STATE_GET_INFO, .action = action};
|
||||
XrActionStatePose pose = {.type = XR_TYPE_ACTION_STATE_POSE};
|
||||
CHECK(xrGetActionStatePose(session, &ag, &pose));
|
||||
samples++;
|
||||
if (pose.isActive) {
|
||||
active++;
|
||||
XrTime time; CHECK(convert(instance, &now, &time));
|
||||
XrSpaceLocation location = {.type = XR_TYPE_SPACE_LOCATION};
|
||||
CHECK(xrLocateSpace(gaze, view, time, &location));
|
||||
XrSpaceLocationFlags needed = XR_SPACE_LOCATION_ORIENTATION_VALID_BIT | XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT;
|
||||
if ((location.locationFlags & needed) == needed) {
|
||||
valid++;
|
||||
if (out) {
|
||||
XrQuaternionf q = location.pose.orientation;
|
||||
if (fprintf(out, "%g %g %g %g\n", q.x, q.y, q.z, q.w) < 0 || fflush(out)) goto cleanup;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
struct timespec delay = {.tv_nsec = 33333333}; nanosleep(&delay, NULL);
|
||||
}
|
||||
printf("samples=%u active=%u valid=%u\n", samples, active, valid);
|
||||
rc = valid ? 0 : 3; /* Distinguish a working session from observed gaze. */
|
||||
cleanup:
|
||||
if (view) xrDestroySpace(view);
|
||||
if (gaze) xrDestroySpace(gaze);
|
||||
if (session) xrDestroySession(session);
|
||||
if (set) xrDestroyActionSet(set);
|
||||
if (instance) xrDestroyInstance(instance);
|
||||
if (out) fclose(out);
|
||||
return rc;
|
||||
}
|
||||
@@ -0,0 +1,481 @@
|
||||
"""Experimental pulse estimate from the Frame's IR eye-tracking cameras.
|
||||
|
||||
Skin brightens and darkens very slightly with each heartbeat as blood volume
|
||||
changes (photoplethysmography). The eye cameras film the skin around each eye
|
||||
under steady IR light at about 90 frames per second, so that rhythm may be
|
||||
visible in the average brightness of small patches of skin.
|
||||
|
||||
Capture uses SteamVR's own `eyetracking --calib` mode, which writes PNG pairs
|
||||
to /tmp. Each image is reduced to a grid of patch averages the moment it is
|
||||
complete, then deleted; the capture directory is removed on exit. No image
|
||||
leaves the Frame or outlives the run. This is an experiment, not a medical
|
||||
measurement.
|
||||
"""
|
||||
from array import array
|
||||
import bisect
|
||||
import cmath
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
ET_DIR = Path("/opt/steamvr/tools/eyetracking")
|
||||
ET_BIN = ET_DIR / "bin/linuxarm64/eyetracking"
|
||||
ET_WEIGHTS = ET_DIR / "resources/et_dsp_20250610_03136.weights"
|
||||
GRID = 16 # 16 × 16 patches of 25 × 25 pixels on the 400 × 400 image
|
||||
RATE = 15.0 # analysis sample rate, Hz; the band of interest ends at 3 Hz
|
||||
LOW, HIGH = 42.0, 180.0 # BPM search band
|
||||
# The Frame's proximity sensor (vcnl4000) reads about 3 with nobody wearing it.
|
||||
# A worn reading has not been measured yet, so the cut-off is provisional.
|
||||
IIO = Path("/sys/bus/iio/devices")
|
||||
WORN_MIN = 20.0
|
||||
WINDOW = 15.0 # seconds per windowed estimate
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- capture ---
|
||||
|
||||
def grid_means(pixels, width, height, stride, channels, grid=GRID):
|
||||
"""Average of channel 0 in each of grid × grid equal patches."""
|
||||
bw, bh = width // grid, height // grid
|
||||
sums = [0] * (grid * grid)
|
||||
for y in range(bh * grid):
|
||||
start = y * stride
|
||||
row = pixels[start:start + width * channels:channels]
|
||||
base = (y // bh) * grid
|
||||
for bx in range(grid):
|
||||
sums[base + bx] += sum(row[bx * bw:(bx + 1) * bw])
|
||||
area = bw * bh
|
||||
return [s / area for s in sums]
|
||||
|
||||
|
||||
def load_grid(path):
|
||||
import gi
|
||||
gi.require_version("GdkPixbuf", "2.0")
|
||||
from gi.repository import GdkPixbuf
|
||||
image = GdkPixbuf.Pixbuf.new_from_file(str(path))
|
||||
return grid_means(image.read_pixel_bytes().get_data(), image.get_width(), image.get_height(),
|
||||
image.get_rowstride(), image.get_n_channels())
|
||||
|
||||
|
||||
def reduce_file(loader, path):
|
||||
"""Reduce one image to its patch grid and delete it, whatever happens."""
|
||||
try:
|
||||
return loader(path)
|
||||
finally:
|
||||
os.unlink(path)
|
||||
|
||||
|
||||
class Capture:
|
||||
"""Run SteamVR's eye-camera capture and reduce frames as they arrive."""
|
||||
NAME = re.compile(r"^(left|right)_(\d+)\.png$")
|
||||
# Only SteamVR's own capture directories are read and removed.
|
||||
PREFIX = "/tmp/etcalib_"
|
||||
# Printed by the capture tool; its output is only flushed when it exits.
|
||||
WRITING = re.compile(r"Writing capture to: (\S+)")
|
||||
# About five seconds of frames (~65 MB in RAM-backed /tmp). If reduction
|
||||
# falls further behind than this, the capture stops rather than letting
|
||||
# eye images pile up.
|
||||
MAX_BACKLOG = 900
|
||||
|
||||
def __init__(self, seconds, runner=subprocess.Popen, loader=load_grid, workers=3):
|
||||
self.seconds, self.runner, self.loader, self.workers = seconds, runner, loader, workers
|
||||
self.grids = {"left": {}, "right": {}}
|
||||
self.directory = None
|
||||
self.pool = None
|
||||
self.submitted = set()
|
||||
self.futures = {}
|
||||
self.new = set() # every capture directory that appeared during our run
|
||||
self.before = None # capture directories that existed before the run
|
||||
self.log = None
|
||||
|
||||
def candidates(self):
|
||||
parent, stem = os.path.split(self.PREFIX)
|
||||
return {Path(entry.path) for entry in os.scandir(parent)
|
||||
if entry.name.startswith(stem) and entry.is_dir(follow_symlinks=False)}
|
||||
|
||||
def run(self):
|
||||
# The capture tool's output goes to a private temporary file, read for
|
||||
# failure messages. It is block-buffered, so the capture directory is
|
||||
# found by watching for a new one rather than waiting for its name.
|
||||
import tempfile
|
||||
self.before = before = self.candidates()
|
||||
process = None
|
||||
with tempfile.TemporaryFile("w+") as log:
|
||||
self.log = log
|
||||
try:
|
||||
if self.workers:
|
||||
# SteamVR pins its eye tracker to cores 0-1; decoding runs beside it.
|
||||
import concurrent.futures
|
||||
import multiprocessing
|
||||
self.pool = concurrent.futures.ProcessPoolExecutor(
|
||||
self.workers, mp_context=multiprocessing.get_context("fork"))
|
||||
process = self.runner([str(ET_BIN), "-b", "CDSP", "-w", str(ET_WEIGHTS), "--calib", str(self.seconds)],
|
||||
cwd=str(ET_BIN.parent), stdout=log, stderr=subprocess.STDOUT)
|
||||
deadline = time.monotonic() + self.seconds + 30
|
||||
while True:
|
||||
# Checked on every poll: a second capture directory means
|
||||
# we can't tell which images are ours, so stop.
|
||||
self.new |= self.candidates() - before
|
||||
if len(self.new) > 1:
|
||||
raise RuntimeError("another eye-camera capture is running")
|
||||
if not self.directory and self.new:
|
||||
self.directory = next(iter(self.new))
|
||||
finished = process.poll() is not None
|
||||
self.reduce(final=finished)
|
||||
if finished:
|
||||
break
|
||||
if time.monotonic() > deadline:
|
||||
raise RuntimeError("eye-camera capture did not finish")
|
||||
time.sleep(0.02)
|
||||
log.seek(0)
|
||||
text = log.read()
|
||||
if process.returncode or not self.directory:
|
||||
reason = "cameras unavailable" if "Failed to" in text else f"exit {process.returncode}"
|
||||
raise RuntimeError(f"eye-camera capture failed ({reason})")
|
||||
named = self.written(log)
|
||||
if named and named != self.directory:
|
||||
raise RuntimeError("the capture wrote somewhere else; not using those images")
|
||||
return self.frames()
|
||||
finally:
|
||||
# Each step runs even if an earlier one failed: eye images must
|
||||
# be removed whatever else went wrong.
|
||||
# Ctrl-C and SIGTERM (raised as KeyboardInterrupt) are held
|
||||
# until the images are gone, then re-raised.
|
||||
interrupted, failed = None, None
|
||||
for step in (lambda: self.finish(process),
|
||||
lambda: self.pool and self.pool.shutdown(wait=True, cancel_futures=True),
|
||||
self.adopt_reported):
|
||||
try:
|
||||
step()
|
||||
except BaseException as error:
|
||||
if isinstance(error, Exception):
|
||||
failed = failed or error
|
||||
else:
|
||||
interrupted = interrupted or error
|
||||
self.log = None
|
||||
self.remove()
|
||||
if interrupted:
|
||||
raise interrupted
|
||||
if failed:
|
||||
raise RuntimeError(f"the eye-camera capture did not stop cleanly: {failed}")
|
||||
|
||||
def finish(self, process):
|
||||
"""Let the capture tool end by itself, deleting its images meanwhile.
|
||||
|
||||
Never signal it: stopping the tool early leaves the headset's eye
|
||||
camera stuck streaming until the DSP service restarts (verified on the
|
||||
Frame with SIGTERM). Its run is bounded by `seconds`, so waiting is
|
||||
short. Only if it overruns by a wide margin is it killed."""
|
||||
if not process:
|
||||
return
|
||||
interrupted = None
|
||||
give_up = time.monotonic() + self.seconds + 30
|
||||
while True:
|
||||
try:
|
||||
if process.poll() is not None:
|
||||
break
|
||||
self.discard()
|
||||
if time.monotonic() > give_up:
|
||||
process.kill()
|
||||
process.wait()
|
||||
raise RuntimeError("the eye-camera capture had to be killed; "
|
||||
"the eye cameras may need a reboot to stream again")
|
||||
time.sleep(0.05)
|
||||
except KeyboardInterrupt as error: # keep cleaning up until it ends
|
||||
interrupted = interrupted or error
|
||||
if interrupted:
|
||||
raise interrupted
|
||||
|
||||
def discard(self):
|
||||
"""Delete the eye images written so far, without reading them."""
|
||||
if self.before is not None:
|
||||
self.new |= self.candidates() - self.before
|
||||
for directory in self.new | ({self.directory} if self.directory else set()):
|
||||
if str(directory).startswith(self.PREFIX) and directory.is_dir() and not directory.is_symlink():
|
||||
for entry in os.scandir(directory):
|
||||
if self.NAME.match(entry.name):
|
||||
try:
|
||||
os.unlink(entry.path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def adopt_reported(self):
|
||||
"""The directory the tool reported is ours by its own account."""
|
||||
named = self.written(self.log)
|
||||
if named and str(named).startswith(self.PREFIX):
|
||||
self.new.add(named)
|
||||
|
||||
def written(self, log):
|
||||
"""The directory the capture tool reported, once its output is flushed."""
|
||||
log.seek(0)
|
||||
match = self.WRITING.search(log.read())
|
||||
return Path(match.group(1)) if match else None
|
||||
|
||||
def reduce(self, final=False):
|
||||
"""Reduce and delete every complete image; an image is complete once a
|
||||
later one of the same eye exists, or the capture has ended."""
|
||||
if not self.directory or not self.directory.is_dir():
|
||||
return
|
||||
pending = {"left": [], "right": []}
|
||||
for entry in os.scandir(self.directory):
|
||||
match = self.NAME.match(entry.name)
|
||||
if match:
|
||||
pending[match.group(1)].append((int(match.group(2)), entry.path))
|
||||
if sum(len(files) for files in pending.values()) > self.MAX_BACKLOG:
|
||||
raise RuntimeError("eye-image processing fell behind; capture abandoned")
|
||||
for eye, files in pending.items():
|
||||
files.sort()
|
||||
ready = files if final else files[:-1]
|
||||
for index, path in ready:
|
||||
if path in self.submitted:
|
||||
continue
|
||||
self.submitted.add(path)
|
||||
if self.pool:
|
||||
self.futures[(eye, index)] = self.pool.submit(reduce_file, self.loader, path)
|
||||
else:
|
||||
self.grids[eye][index] = array("f", reduce_file(self.loader, path))
|
||||
for key, future in list(self.futures.items()):
|
||||
if final or future.done():
|
||||
self.grids[key[0]][key[1]] = array("f", future.result())
|
||||
del self.futures[key]
|
||||
|
||||
def frames(self):
|
||||
"""[(monotonic seconds, eye, grid)] joined with the capture metadata."""
|
||||
meta = json.loads((self.directory / "meta.json").read_text())
|
||||
frames = []
|
||||
for pair in meta["frames"]:
|
||||
for eye in ("left", "right"):
|
||||
info = pair.get(eye) or {}
|
||||
match = self.NAME.match(Path(info.get("fname", "")).name)
|
||||
grid = self.grids[eye].get(int(match.group(2))) if match else None
|
||||
if info.get("valid") and grid is not None:
|
||||
frames.append((float(info["tsMono"]), eye, grid))
|
||||
return frames
|
||||
|
||||
def remove(self):
|
||||
"""Remove every capture directory that appeared during the run. Eye
|
||||
images must not outlive it, so a failure to delete is an error."""
|
||||
left = []
|
||||
if self.before is not None:
|
||||
try:
|
||||
self.new |= self.candidates() - self.before # even if interrupted before the first poll
|
||||
except OSError:
|
||||
pass
|
||||
for directory in self.new | ({self.directory} if self.directory else set()):
|
||||
if str(directory).startswith(self.PREFIX) and directory.is_dir() and not directory.is_symlink():
|
||||
try:
|
||||
shutil.rmtree(directory)
|
||||
except OSError:
|
||||
left.append(str(directory))
|
||||
if left:
|
||||
raise RuntimeError("could not delete eye images in " + ", ".join(sorted(left)))
|
||||
|
||||
|
||||
# --------------------------------------------------------------- analysis ---
|
||||
|
||||
def fft(values):
|
||||
"""In-place iterative radix-2 FFT of a list whose length is a power of 2."""
|
||||
n = len(values)
|
||||
a = list(values)
|
||||
j = 0
|
||||
for i in range(1, n):
|
||||
bit = n >> 1
|
||||
while j & bit:
|
||||
j ^= bit
|
||||
bit >>= 1
|
||||
j |= bit
|
||||
if i < j:
|
||||
a[i], a[j] = a[j], a[i]
|
||||
size = 2
|
||||
while size <= n:
|
||||
step = cmath.exp(-2j * math.pi / size)
|
||||
half = size // 2
|
||||
for start in range(0, n, size):
|
||||
w = 1
|
||||
for k in range(start, start + half):
|
||||
t = w * a[k + half]
|
||||
a[k + half] = a[k] - t
|
||||
a[k] += t
|
||||
w *= step
|
||||
size *= 2
|
||||
return a
|
||||
|
||||
|
||||
def resample(times, values, rate=RATE):
|
||||
"""Average samples into 1/rate bins from the first sample; empty bins are
|
||||
filled from the previous bin."""
|
||||
if not times:
|
||||
return []
|
||||
start = times[0]
|
||||
count = int((times[-1] - start) * rate) + 1
|
||||
sums, counts = [0.0] * count, [0] * count
|
||||
for t, v in zip(times, values):
|
||||
i = min(int((t - start) * rate), count - 1)
|
||||
sums[i] += v
|
||||
counts[i] += 1
|
||||
out, last = [], None
|
||||
for s, c in zip(sums, counts):
|
||||
last = s / c if c else last
|
||||
out.append(last)
|
||||
first = next(v for v in out if v is not None)
|
||||
return [first if v is None else v for v in out]
|
||||
|
||||
|
||||
def clean(signal, rate=RATE):
|
||||
"""Relative change with slow drift removed; None if the patch is mostly
|
||||
blinks or eye movement. Spikes (blinks, saccades) become gaps at the
|
||||
local level rather than clipped steps, which would add false rhythm."""
|
||||
mean = sum(signal) / len(signal)
|
||||
x = [v / mean - 1 for v in signal]
|
||||
half = int(rate) # 2-second centred moving median removes drift and blinks
|
||||
trend = []
|
||||
for i in range(len(x)):
|
||||
chunk = sorted(x[max(0, i - half):i + half + 1])
|
||||
trend.append(chunk[len(chunk) // 2])
|
||||
residual = [v - m for v, m in zip(x, trend)]
|
||||
mad = sorted(abs(v) for v in residual)[len(residual) // 2] or 1e-9
|
||||
limit = 4 * 1.4826 * mad
|
||||
spikes = sum(1 for v in residual if abs(v) > limit)
|
||||
if spikes > 0.05 * len(residual):
|
||||
return None
|
||||
return [v if abs(v) <= limit else 0.0 for v in residual]
|
||||
|
||||
|
||||
def spectrum(signal, rate=RATE):
|
||||
"""[(bpm, power)] within the search band, Hann-windowed and zero-padded."""
|
||||
n = len(signal)
|
||||
size = 1
|
||||
while size < max(n * 4, 256):
|
||||
size *= 2
|
||||
window = [0.5 - 0.5 * math.cos(2 * math.pi * i / (n - 1)) for i in range(n)] if n > 1 else [1.0]
|
||||
padded = [s * w for s, w in zip(signal, window)] + [0.0] * (size - n)
|
||||
result = fft(padded)
|
||||
out = []
|
||||
for k in range(size // 2):
|
||||
bpm = k * rate / size * 60
|
||||
if LOW <= bpm <= HIGH:
|
||||
out.append((bpm, abs(result[k]) ** 2))
|
||||
return out
|
||||
|
||||
|
||||
def peak(spec):
|
||||
"""Peak BPM with parabolic interpolation between spectral bins."""
|
||||
i = max(range(len(spec)), key=lambda k: spec[k][1])
|
||||
if 0 < i < len(spec) - 1:
|
||||
a, b, c = spec[i - 1][1], spec[i][1], spec[i + 1][1]
|
||||
denominator = a - 2 * b + c
|
||||
offset = 0.5 * (a - c) / denominator if denominator else 0.0
|
||||
return spec[i][0] + offset * (spec[1][0] - spec[0][0])
|
||||
return spec[i][0]
|
||||
|
||||
|
||||
def snr(spec, bpm, width=4.0):
|
||||
"""Power near the pulse and its first harmonic against the rest of the band."""
|
||||
near = sum(p for f, p in spec if abs(f - bpm) <= width or abs(f - 2 * bpm) <= width)
|
||||
rest = sum(p for f, p in spec) - near
|
||||
if near <= 0:
|
||||
return 0.0
|
||||
return near / rest if rest > 0 else float("inf")
|
||||
|
||||
|
||||
def normalised(spec):
|
||||
total = sum(p for _, p in spec) or 1.0
|
||||
return [p / total for _, p in spec]
|
||||
|
||||
|
||||
def usable(values):
|
||||
"""Patches that are neither dark nor saturated for the whole recording."""
|
||||
mean = sum(values) / len(values)
|
||||
return 8 <= mean <= 245
|
||||
|
||||
|
||||
def estimate(times, patches, rate=RATE, share=0.2, window=WINDOW):
|
||||
"""Estimate pulse from patch brightness traces.
|
||||
|
||||
times: monotonic seconds per frame. patches: {name: [brightness per frame]}.
|
||||
Selects the share of usable patches with the clearest periodic signal,
|
||||
combines their spectra and reports the overall and windowed estimates.
|
||||
"""
|
||||
cleaned = {}
|
||||
for name, values in patches.items():
|
||||
if usable(values):
|
||||
signal = clean(resample(times, values, rate), rate)
|
||||
if signal is not None and any(signal): # flat patches carry no rhythm
|
||||
cleaned[name] = signal
|
||||
if not cleaned or len(next(iter(cleaned.values()))) < rate * 8:
|
||||
raise ValueError("need at least 8 seconds of usable eye-camera frames")
|
||||
quality = []
|
||||
for name, signal in cleaned.items():
|
||||
spec = spectrum(signal, rate)
|
||||
own = peak(spec)
|
||||
quality.append((snr(spec, own), name, own, spec))
|
||||
quality.sort(reverse=True)
|
||||
chosen = quality[:max(4, int(len(quality) * share))]
|
||||
combined = [sum(values) for values in zip(*(normalised(spec) for _, _, _, spec in chosen))]
|
||||
bins = [f for f, _ in chosen[0][3]]
|
||||
bpm = peak(list(zip(bins, combined)))
|
||||
top = chosen[:8]
|
||||
agree = sum(1 for _, _, own, _ in top if abs(own - bpm) <= 5) / len(top)
|
||||
series = []
|
||||
samples = int(window * rate)
|
||||
length = len(next(iter(cleaned.values())))
|
||||
for end in range(samples, length + 1, int(rate)):
|
||||
specs = [spectrum(cleaned[name][end - samples:end], rate) for _, name, _, _ in chosen]
|
||||
total = [sum(values) for values in zip(*(normalised(s) for s in specs))]
|
||||
window_bins = [f for f, _ in specs[0]]
|
||||
series.append((times[0] + end / rate, peak(list(zip(window_bins, total)))))
|
||||
return {
|
||||
"bpm": bpm,
|
||||
"agreement": agree,
|
||||
"patches": len(chosen),
|
||||
"usable": len(cleaned),
|
||||
"snr": snr(list(zip(bins, combined)), bpm),
|
||||
"series": series,
|
||||
}
|
||||
|
||||
|
||||
def analyse(frames):
|
||||
"""Estimate from Capture.frames(); both eyes' patches are analysed together
|
||||
on a shared time base, each sample taken from that eye's nearest frame."""
|
||||
times = sorted({t for t, _, _ in frames})
|
||||
patches = {}
|
||||
for eye in ("left", "right"):
|
||||
eye_frames = sorted((t, grid) for t, e, grid in frames if e == eye)
|
||||
if not eye_frames:
|
||||
continue
|
||||
eye_times = [t for t, _ in eye_frames]
|
||||
nearest = []
|
||||
for t in times:
|
||||
i = bisect.bisect_left(eye_times, t)
|
||||
if i == len(eye_times) or (i > 0 and t - eye_times[i - 1] <= eye_times[i] - t):
|
||||
i -= 1
|
||||
nearest.append(i)
|
||||
for k in range(len(eye_frames[0][1])):
|
||||
patches[f"{eye}{k}"] = [eye_frames[i][1][k] for i in nearest]
|
||||
return estimate(times, patches)
|
||||
|
||||
|
||||
def proximity(root=None):
|
||||
"""The headset's proximity reading, or None if it can't be read."""
|
||||
try:
|
||||
for device in sorted(Path(root or IIO).glob("iio:device*")):
|
||||
if (device / "name").read_text().strip() == "vcnl4000":
|
||||
return float((device / "in_proximity_raw").read_text())
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def worn(reading):
|
||||
"""False only when the sensor says the headset is not on a face."""
|
||||
return reading is None or reading >= WORN_MIN
|
||||
|
||||
|
||||
def reliable(result):
|
||||
"""Whether the estimate is clear enough to show as a reading. Thresholds
|
||||
are provisional until checked against a reference on a real wearer."""
|
||||
return result["agreement"] >= 0.75 and result["snr"] >= 0.5
|
||||
@@ -0,0 +1,438 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Frame-local tracking tools. No network destination or data log by default."""
|
||||
import argparse
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import signal
|
||||
import socket
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
|
||||
class TrackingError(RuntimeError):
|
||||
"""A safe, actionable status message containing no sensor data."""
|
||||
|
||||
|
||||
HRS = "0000180d-0000-1000-8000-00805f9b34fb"
|
||||
MEASUREMENT = "00002a37-0000-1000-8000-00805f9b34fb"
|
||||
DEVICE = "org.bluez.Device1"
|
||||
SERVICE = "org.bluez.GattService1"
|
||||
CHARACTERISTIC = "org.bluez.GattCharacteristic1"
|
||||
|
||||
|
||||
def heart_rate(data):
|
||||
"""Validate the Bluetooth HRS measurement, returning BPM/contact only.
|
||||
|
||||
Energy and RR intervals are checked for length but never retained.
|
||||
None means contact is supported and the strap reports no skin contact.
|
||||
"""
|
||||
data = bytes(data)
|
||||
if len(data) < 2 or data[0] & 0xe0:
|
||||
raise ValueError("invalid HRS measurement")
|
||||
flags = data[0]
|
||||
size = 2 if flags & 1 else 1
|
||||
end = 1 + size + (2 if flags & 8 else 0)
|
||||
if len(data) < end:
|
||||
raise ValueError("truncated HRS measurement")
|
||||
extra = len(data) - end
|
||||
if (flags & 16 and (extra < 2 or extra % 2)) or (not flags & 16 and extra):
|
||||
raise ValueError("invalid HRS optional fields")
|
||||
if flags & 4 and not flags & 2:
|
||||
return None
|
||||
bpm = int.from_bytes(data[1:1 + size], "little")
|
||||
return bpm if bpm else None
|
||||
|
||||
|
||||
def gaze_angles(quaternion):
|
||||
"""OpenXR head-relative -Z forward → VRChat degrees, down/right positive."""
|
||||
if len(quaternion) != 4 or not all(math.isfinite(v) for v in quaternion):
|
||||
raise ValueError("invalid gaze orientation")
|
||||
norm = math.sqrt(sum(v * v for v in quaternion))
|
||||
if not 0.9 < norm < 1.1:
|
||||
raise ValueError("invalid gaze orientation")
|
||||
x, y, z, w = (v / norm for v in quaternion)
|
||||
# Rotate OpenXR's forward vector (0, 0, -1) into VIEW space.
|
||||
dx, dy, dz = -2 * (x*z + w*y), 2 * (w*x - y*z), 2 * (x*x + y*y) - 1
|
||||
return math.degrees(math.atan2(-dy, math.hypot(dx, dz))), math.degrees(math.atan2(dx, -dz))
|
||||
|
||||
|
||||
def osc_message(address, values):
|
||||
if not address.startswith("/") or any(c.isspace() or c in '\0#*,?[]{}' for c in address):
|
||||
raise ValueError("OSC address must be a literal path")
|
||||
def string(value):
|
||||
encoded = value.encode("utf-8") + b"\0"
|
||||
return encoded + b"\0" * (-len(encoded) % 4)
|
||||
tags, payload = ",", b""
|
||||
for value in values:
|
||||
if type(value) is int:
|
||||
tags += "i"
|
||||
payload += struct.pack(">i", value)
|
||||
else:
|
||||
if not math.isfinite(value):
|
||||
raise ValueError("OSC value must be finite")
|
||||
tags += "f"
|
||||
payload += struct.pack(">f", value)
|
||||
return string(address) + string(tags) + payload
|
||||
|
||||
|
||||
class Osc:
|
||||
def __init__(self, endpoint=None):
|
||||
self.sock = None
|
||||
self.target = None
|
||||
if endpoint:
|
||||
import ipaddress
|
||||
address = ipaddress.ip_address(endpoint[0])
|
||||
port = int(endpoint[1])
|
||||
if address.is_unspecified or address.is_multicast or not 1 <= port <= 65535:
|
||||
raise ValueError("OSC needs a unicast IP address and port 1..65535")
|
||||
self.sock = socket.socket(socket.AF_INET6 if address.version == 6 else socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.target = (str(address), port)
|
||||
|
||||
def send(self, address, values):
|
||||
if self.sock:
|
||||
self.sock.sendto(osc_message(address, values), self.target)
|
||||
|
||||
def close(self):
|
||||
if self.sock:
|
||||
self.sock.close()
|
||||
|
||||
|
||||
class HeartSession:
|
||||
def __init__(self, osc, address, log=None, clock=time.monotonic):
|
||||
self.osc, self.address, self.clock = osc, address, clock
|
||||
self.bpm, self.updated = None, None
|
||||
self.log = None
|
||||
if log:
|
||||
# Exclusive creation refuses existing files and symlinks; mode is
|
||||
# private even with a permissive process umask.
|
||||
fd = os.open(log, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
self.log = os.fdopen(fd, "w")
|
||||
self.log.write("unix_seconds,bpm\n")
|
||||
|
||||
def notification(self, data):
|
||||
self.bpm = heart_rate(data)
|
||||
self.updated = self.clock()
|
||||
if self.bpm is not None:
|
||||
self.osc.send(self.address, [self.bpm])
|
||||
if self.log:
|
||||
self.log.write(f"{time.time():.3f},{self.bpm}\n")
|
||||
self.log.flush()
|
||||
|
||||
def current(self):
|
||||
if self.updated is None or self.clock() - self.updated > 5:
|
||||
return None
|
||||
return self.bpm
|
||||
|
||||
def close(self):
|
||||
if self.log:
|
||||
self.log.close()
|
||||
|
||||
|
||||
class BluezHeart:
|
||||
"""One explicitly selected, already discovered strap; no ambient scan."""
|
||||
def __init__(self, bus, interface, address, on_value):
|
||||
self.bus, self.interface, self.on_value = bus, interface, on_value
|
||||
self.device = self.characteristic = None
|
||||
self.connected_here = False
|
||||
self.notifying = False
|
||||
self.match = None
|
||||
objects = self.objects()
|
||||
matches = [path for path, interfaces in objects.items()
|
||||
if str(interfaces.get(DEVICE, {}).get("Address", "")).upper() == address.upper()]
|
||||
if len(matches) != 1:
|
||||
raise TrackingError("Strap not found uniquely in BlueZ; pair/discover it in SteamOS Bluetooth settings first")
|
||||
self.device = matches[0]
|
||||
self.match = bus.add_signal_receiver(self.changed, signal_name="PropertiesChanged",
|
||||
dbus_interface="org.freedesktop.DBus.Properties",
|
||||
bus_name="org.bluez", path_keyword="path")
|
||||
try:
|
||||
if not objects[self.device][DEVICE].get("Connected"):
|
||||
self.call(self.device, DEVICE).Connect(timeout=20)
|
||||
self.connected_here = True
|
||||
except Exception:
|
||||
self.close()
|
||||
raise
|
||||
|
||||
def objects(self):
|
||||
return self.call("/", "org.freedesktop.DBus.ObjectManager").GetManagedObjects()
|
||||
|
||||
def call(self, path, kind):
|
||||
return self.interface(self.bus.get_object("org.bluez", path), kind)
|
||||
|
||||
def subscribe(self):
|
||||
objects = self.objects()
|
||||
if not objects.get(self.device, {}).get(DEVICE, {}).get("ServicesResolved"):
|
||||
return False
|
||||
services = {p for p, obj in objects.items() if str(obj.get(SERVICE, {}).get("UUID", "")).lower() == HRS
|
||||
and obj[SERVICE].get("Device") == self.device}
|
||||
for path, obj in objects.items():
|
||||
props = obj.get(CHARACTERISTIC, {})
|
||||
if props.get("Service") in services and str(props.get("UUID", "")).lower() == MEASUREMENT:
|
||||
if "notify" not in props.get("Flags", []):
|
||||
raise TrackingError("Heart-rate characteristic does not support notifications")
|
||||
self.characteristic = path
|
||||
self.call(path, CHARACTERISTIC).StartNotify()
|
||||
self.notifying = True
|
||||
return True
|
||||
raise TrackingError("Selected device has no standard Heart Rate Service measurement")
|
||||
|
||||
def changed(self, kind, changes, invalidated, path=None):
|
||||
if kind == CHARACTERISTIC and path == self.characteristic and "Value" in changes:
|
||||
self.on_value(changes["Value"])
|
||||
elif kind == DEVICE and path == self.device and "Connected" in changes and not changes["Connected"]:
|
||||
self.on_value(None)
|
||||
|
||||
def close(self):
|
||||
try:
|
||||
if self.notifying:
|
||||
self.call(self.characteristic, CHARACTERISTIC).StopNotify()
|
||||
finally:
|
||||
if self.match:
|
||||
self.match.remove()
|
||||
if self.connected_here:
|
||||
self.call(self.device, DEVICE).Disconnect()
|
||||
|
||||
|
||||
def run_gaze(args, osc):
|
||||
binary = Path(__file__).with_name("gaze")
|
||||
command = [str(binary), "--seconds", str(args.seconds)]
|
||||
if not args.osc:
|
||||
return subprocess.call(command)
|
||||
read_fd, write_fd = os.pipe()
|
||||
process = None
|
||||
try:
|
||||
process = subprocess.Popen(command + ["--fd", str(write_fd)], pass_fds=(write_fd,))
|
||||
os.close(write_fd)
|
||||
write_fd = None
|
||||
with os.fdopen(read_fd) as source:
|
||||
read_fd = None
|
||||
for line in source:
|
||||
try:
|
||||
angles = gaze_angles([float(v) for v in line.split()])
|
||||
except ValueError:
|
||||
continue
|
||||
osc.send("/tracking/eye/CenterPitchYaw", angles)
|
||||
return process.wait()
|
||||
finally:
|
||||
if read_fd is not None:
|
||||
os.close(read_fd)
|
||||
if write_fd is not None:
|
||||
os.close(write_fd)
|
||||
if process and process.poll() is None:
|
||||
process.terminate()
|
||||
try:
|
||||
process.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
|
||||
|
||||
class HeartPanel:
|
||||
"""Our GTK panel, using the Frame's existing GTK4/GI platform libraries."""
|
||||
def __init__(self):
|
||||
os.environ["GDK_BACKEND"] = "x11"
|
||||
import gi
|
||||
gi.require_version("Gtk", "4.0")
|
||||
gi.require_version("GdkX11", "4.0")
|
||||
from gi.repository import Gtk, Gdk, GdkX11, GLib
|
||||
Gtk.init()
|
||||
self.running = True
|
||||
self.window = Gtk.Window(title="Frame Control · Heart rate")
|
||||
self.window.set_default_size(480, 320)
|
||||
self.window.connect("close-request", self.stop)
|
||||
Gtk.Settings.get_default().set_property("gtk-application-prefer-dark-theme", True)
|
||||
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=16)
|
||||
box.set_valign(Gtk.Align.CENTER)
|
||||
box.set_halign(Gtk.Align.CENTER)
|
||||
box.set_size_request(440, -1)
|
||||
for side in ("top", "bottom", "start", "end"):
|
||||
getattr(box, "set_margin_" + side)(24)
|
||||
self.window.set_child(box)
|
||||
title = Gtk.Label(label="Heart rate")
|
||||
title.add_css_class("title-2")
|
||||
box.append(title)
|
||||
self.reading = Gtk.Label(label="—")
|
||||
self.reading.add_css_class("reading")
|
||||
box.append(self.reading)
|
||||
self.status = Gtk.Label(label="Waiting for strap")
|
||||
box.append(self.status)
|
||||
button = Gtk.Button(label="Stop")
|
||||
button.connect("clicked", self.stop)
|
||||
box.append(button)
|
||||
css = Gtk.CssProvider()
|
||||
css.load_from_data(b".reading { font-size: 144px; font-weight: 700; }")
|
||||
Gtk.StyleContext.add_provider_for_display(Gdk.Display.get_default(), css, Gtk.STYLE_PROVIDER_PRIORITY_APPLICATION)
|
||||
self.window.present()
|
||||
context = GLib.MainContext.default()
|
||||
while context.pending():
|
||||
context.iteration(False)
|
||||
try:
|
||||
xid = GdkX11.X11Surface.get_xid(self.window.get_surface())
|
||||
subprocess.run(["xprop", "-id", str(xid), "-f", "STEAM_GAME", "32c", "-set", "STEAM_GAME", "2000000027"],
|
||||
check=True, stdout=subprocess.DEVNULL)
|
||||
except Exception:
|
||||
self.window.destroy()
|
||||
raise
|
||||
|
||||
def stop(self, *args):
|
||||
self.running = False
|
||||
return True
|
||||
|
||||
def update(self, bpm):
|
||||
self.reading.set_label(str(bpm) if bpm is not None else "—")
|
||||
self.status.set_label("beats per minute" if bpm is not None else "Waiting for strap")
|
||||
|
||||
def close(self):
|
||||
self.window.destroy()
|
||||
|
||||
|
||||
def run_heart(args, osc):
|
||||
import dbus
|
||||
from dbus.mainloop.glib import DBusGMainLoop
|
||||
from gi.repository import GLib
|
||||
DBusGMainLoop(set_as_default=True)
|
||||
session = HeartSession(osc, args.address, args.log)
|
||||
reader, root = None, None
|
||||
failure = []
|
||||
def value(data):
|
||||
if data is None:
|
||||
session.bpm = None
|
||||
failure.append("Strap disconnected; reconnect and start again")
|
||||
return
|
||||
try:
|
||||
session.notification(data)
|
||||
except ValueError:
|
||||
session.bpm = None
|
||||
except OSError:
|
||||
failure.append("OSC or session log write failed")
|
||||
try:
|
||||
reader = BluezHeart(dbus.SystemBus(), dbus.Interface, args.device, value)
|
||||
context = GLib.MainContext.default()
|
||||
deadline = time.monotonic() + 20
|
||||
while not reader.subscribe():
|
||||
if time.monotonic() > deadline:
|
||||
raise TrackingError("Timed out waiting for the strap's GATT services")
|
||||
while context.pending():
|
||||
context.iteration(False)
|
||||
time.sleep(0.1)
|
||||
end = time.monotonic() + args.seconds
|
||||
print("Heart-rate notifications started; readings stay local unless OSC or a log was selected.")
|
||||
if args.panel:
|
||||
root = HeartPanel()
|
||||
running = lambda: root.running if root else True
|
||||
while running() and time.monotonic() < end and not failure:
|
||||
while context.pending():
|
||||
context.iteration(False)
|
||||
if root:
|
||||
root.update(session.current())
|
||||
time.sleep(0.05)
|
||||
if failure:
|
||||
raise TrackingError(failure[0])
|
||||
return 0
|
||||
finally:
|
||||
if root:
|
||||
root.close()
|
||||
try:
|
||||
if reader:
|
||||
reader.close()
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def run_pulse(args, osc):
|
||||
"""Experimental: estimate pulse from the IR eye cameras (see pulse.py)."""
|
||||
import pulse
|
||||
if not pulse.ET_BIN.exists():
|
||||
raise TrackingError("SteamVR's eye-tracking tool is not installed on this Frame")
|
||||
print(f"Capturing {args.seconds} s from the eye cameras. Wear the headset and keep still.", flush=True)
|
||||
readings = [pulse.proximity()]
|
||||
try:
|
||||
frames = pulse.Capture(args.seconds).run()
|
||||
readings.append(pulse.proximity())
|
||||
except RuntimeError as error: # capture status only; no image data
|
||||
raise TrackingError(str(error))
|
||||
print(f"Captured {len(frames)} eye frames; images already deleted. Analysing...", flush=True)
|
||||
try:
|
||||
result = pulse.analyse(frames)
|
||||
except ValueError as error:
|
||||
raise TrackingError(str(error))
|
||||
on_face = all(pulse.worn(reading) for reading in readings)
|
||||
if not on_face:
|
||||
# Unworn, the cameras still show a steady periodic artifact that looks
|
||||
# like a pulse (verified on the Frame), so it is never called clear.
|
||||
print("The proximity sensor says the headset is not being worn; no pulse can be read.")
|
||||
clear = on_face and pulse.reliable(result)
|
||||
offset = time.time() - time.monotonic() # the capture's timestamps are CLOCK_MONOTONIC
|
||||
if args.log:
|
||||
fd = os.open(args.log, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(fd, "w") as log:
|
||||
log.write("unix_seconds,bpm\n")
|
||||
for when, bpm in result["series"]:
|
||||
log.write(f"{when + offset:.3f},{bpm:.1f}\n")
|
||||
if clear:
|
||||
osc.send(args.address, [round(result["bpm"])])
|
||||
if args.show:
|
||||
print(f"Estimate: {result['bpm']:.1f} BPM ({'clear' if clear else 'NOT clear'}; "
|
||||
f"patch agreement {result['agreement']:.0%}, signal/noise {result['snr']:.2f}, "
|
||||
f"{result['patches']} of {result['usable']} usable patches)")
|
||||
print("Per-second estimates (15 s windows): "
|
||||
+ " ".join(str(round(bpm)) for _, bpm in result["series"]))
|
||||
else:
|
||||
print("A clear pulse was found." if clear else "No clear pulse was found.")
|
||||
return 0 if clear else 3
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
commands = parser.add_subparsers(dest="command", required=True)
|
||||
gaze = commands.add_parser("gaze", help="headless OpenXR; prints counters only without --osc")
|
||||
heart = commands.add_parser("heart", help="standard BLE HRS from an explicitly selected strap")
|
||||
pulse = commands.add_parser("pulse", help="experimental pulse estimate from the IR eye cameras")
|
||||
for command in (gaze, heart, pulse):
|
||||
command.add_argument("--osc", nargs=2, metavar=("IP", "PORT"), help="explicit UDP destination; no default")
|
||||
for command in (gaze, heart):
|
||||
command.add_argument("--seconds", type=int, default=10, help="bounded run, 1..86400 seconds (default: 10)")
|
||||
pulse.add_argument("--seconds", type=int, default=60, help="capture length, 20..300 seconds (default: 60)")
|
||||
heart.add_argument("--device", required=True, help="strap Bluetooth address already discovered by BlueZ")
|
||||
heart.add_argument("--panel", action="store_true", help="show our panel on gamescope DISPLAY=:0")
|
||||
for command in (heart, pulse):
|
||||
command.add_argument("--address", default="/avatar/parameters/HeartRate", help="integer BPM OSC parameter")
|
||||
command.add_argument("--log", help="new private CSV file; disabled by default")
|
||||
pulse.add_argument("--show", action="store_true", help="print the estimate and per-second series")
|
||||
args = parser.parse_args()
|
||||
if not 1 <= args.seconds <= 86400:
|
||||
parser.error("--seconds must be 1..86400")
|
||||
if args.command == "pulse" and not 20 <= args.seconds <= 300:
|
||||
parser.error("pulse --seconds must be 20..300")
|
||||
def interrupted(signum, frame):
|
||||
raise KeyboardInterrupt
|
||||
signal.signal(signal.SIGTERM, interrupted)
|
||||
if hasattr(signal, "SIGHUP"):
|
||||
signal.signal(signal.SIGHUP, interrupted)
|
||||
osc = None
|
||||
try:
|
||||
if args.command in ("heart", "pulse"):
|
||||
osc_message(args.address, [0])
|
||||
if getattr(args, "panel", False):
|
||||
os.environ["DISPLAY"] = ":0"
|
||||
osc = Osc(args.osc)
|
||||
run = {"gaze": run_gaze, "heart": run_heart, "pulse": run_pulse}[args.command]
|
||||
return run(args, osc)
|
||||
except TrackingError as error:
|
||||
print(str(error))
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
return 130
|
||||
except Exception as error:
|
||||
# Never dump notifications, gaze, BLE addresses or exception payloads.
|
||||
print(f"Tracking stopped ({type(error).__name__}). Check the device, runtime and selected output.")
|
||||
return 1
|
||||
finally:
|
||||
if osc:
|
||||
osc.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,274 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check Frame Control's heart-rate readings against a reference, on your computer.
|
||||
|
||||
python3 scripts/heart-check.py listen --port 9000 --out ours.csv
|
||||
python3 scripts/heart-check.py compare ours.csv reference.csv
|
||||
python3 scripts/heart-check.py compare ours.csv ~/Downloads/export.zip
|
||||
|
||||
`listen` receives our integer-BPM OSC messages (send them here with
|
||||
`tracking-on-frame.py heart --osc <this computer's IP> 9000`) and shows each
|
||||
reading live, so you can watch it next to a reference such as your Apple
|
||||
Watch. `--out` also records `unix_seconds,bpm` to a new private file.
|
||||
|
||||
`compare` lines up two recordings by time and reports how far apart they are.
|
||||
The reference can be:
|
||||
- a CSV whose first column is a time (unix seconds or ISO 8601) and whose
|
||||
second is BPM, such as our own log, `listen --out`, or the test strap's
|
||||
output (a third `flags` column marks skin-contact loss as "no reading");
|
||||
- an Apple Health export (`export.zip` or `export.xml`, from Health → your
|
||||
profile → Export All Health Data). Only heart-rate records within the
|
||||
recording's time range are read.
|
||||
|
||||
Everything stays on this computer. Nothing is uploaded.
|
||||
"""
|
||||
import argparse
|
||||
import bisect
|
||||
import csv
|
||||
from datetime import datetime
|
||||
import io
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
import zipfile
|
||||
from xml.etree import ElementTree
|
||||
|
||||
ADDRESS = "/avatar/parameters/HeartRate"
|
||||
|
||||
|
||||
def parse_osc(packet):
|
||||
"""Return (address, values) for a single OSC message with i/f arguments."""
|
||||
def string(offset):
|
||||
end = packet.index(b"\0", offset)
|
||||
return packet[offset:end].decode("utf-8"), (end + 4) & ~3
|
||||
address, offset = string(0)
|
||||
tags, offset = string(offset)
|
||||
if not tags.startswith(","):
|
||||
raise ValueError("not an OSC message")
|
||||
values = []
|
||||
for tag in tags[1:]:
|
||||
if tag not in "if" or offset + 4 > len(packet):
|
||||
raise ValueError("unsupported OSC argument")
|
||||
values.append(struct.unpack(">i" if tag == "i" else ">f", packet[offset:offset + 4])[0])
|
||||
offset += 4
|
||||
return address, values
|
||||
|
||||
|
||||
def listen(port, address, out, seconds, clock=time.time, stream=sys.stdout):
|
||||
log = None
|
||||
if out:
|
||||
log = os.fdopen(os.open(out, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "w")
|
||||
log.write("unix_seconds,bpm\n")
|
||||
received = 0
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
|
||||
sock.bind(("0.0.0.0", port))
|
||||
sock.settimeout(0.5)
|
||||
print(f"Listening for {address} on UDP port {port}. Ctrl-C stops.", file=stream, flush=True)
|
||||
end = clock() + seconds if seconds else None
|
||||
try:
|
||||
while end is None or clock() < end:
|
||||
try:
|
||||
packet = sock.recv(1024)
|
||||
except socket.timeout:
|
||||
continue
|
||||
try:
|
||||
path, values = parse_osc(packet)
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
continue
|
||||
if path != address or len(values) != 1:
|
||||
continue
|
||||
now = clock()
|
||||
bpm = int(values[0])
|
||||
received += 1
|
||||
print(f"{time.strftime('%H:%M:%S', time.localtime(now))} {bpm:3d} bpm", file=stream, flush=True)
|
||||
if log:
|
||||
log.write(f"{now:.3f},{bpm}\n")
|
||||
log.flush()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
if log:
|
||||
log.close()
|
||||
return received
|
||||
|
||||
|
||||
def parse_time(text):
|
||||
text = text.strip()
|
||||
if re.fullmatch(r"\d+(\.\d+)?", text):
|
||||
return float(text)
|
||||
# Apple Health uses "2026-09-29 09:12:03 +1000".
|
||||
text = re.sub(r" ([+-]\d{2}):?(\d{2})$", r"\1\2", text).replace("Z", "+0000")
|
||||
for pattern in ("%Y-%m-%d %H:%M:%S%z", "%Y-%m-%dT%H:%M:%S%z", "%Y-%m-%dT%H:%M:%S.%f%z"):
|
||||
try:
|
||||
return datetime.strptime(text, pattern).timestamp()
|
||||
except ValueError:
|
||||
pass
|
||||
raise ValueError(f"unrecognised time {text!r}")
|
||||
|
||||
|
||||
def read_csv(path):
|
||||
"""[(time, bpm or None)], sorted. A header row is skipped."""
|
||||
samples = []
|
||||
with open(path, newline="") as source:
|
||||
for row in csv.reader(source):
|
||||
if len(row) < 2:
|
||||
continue
|
||||
try:
|
||||
when, bpm = parse_time(row[0]), int(float(row[1]))
|
||||
except (ValueError, OverflowError):
|
||||
continue # header or unparseable line
|
||||
try:
|
||||
flags = int(float(row[2])) if len(row) > 2 else None
|
||||
except (ValueError, OverflowError):
|
||||
flags = None # an unrecognised flag leaves the reading as it is
|
||||
if flags is not None and flags & 4 and not flags & 2:
|
||||
bpm = None # contact supported and not detected: no reading
|
||||
samples.append((when, bpm))
|
||||
return sorted(samples, key=lambda s: s[0])
|
||||
|
||||
|
||||
def read_health(path, start, end):
|
||||
"""Heart-rate records from an Apple Health export between start and end."""
|
||||
samples = []
|
||||
def scan(source):
|
||||
for _, element in ElementTree.iterparse(source):
|
||||
if element.tag == "Record" and element.get("type") == "HKQuantityTypeIdentifierHeartRate":
|
||||
try:
|
||||
when = parse_time(element.get("startDate") or "")
|
||||
value = round(float(element.get("value") or ""))
|
||||
except (ValueError, OverflowError):
|
||||
continue
|
||||
if start <= when <= end:
|
||||
samples.append((when, value))
|
||||
element.clear()
|
||||
if zipfile.is_zipfile(path):
|
||||
with zipfile.ZipFile(path) as archive:
|
||||
names = [n for n in archive.namelist() if n.endswith("/export.xml") or n == "export.xml"]
|
||||
if not names:
|
||||
raise ValueError("no export.xml in that archive; use Health's Export All Health Data")
|
||||
with archive.open(names[0]) as source:
|
||||
scan(source)
|
||||
else:
|
||||
with open(path, "rb") as source:
|
||||
scan(source)
|
||||
return sorted(samples)
|
||||
|
||||
|
||||
def read_any(path, start=None, end=None):
|
||||
path = str(path)
|
||||
if path.endswith((".zip", ".xml")):
|
||||
return read_health(path, start, end)
|
||||
return read_csv(path)
|
||||
|
||||
|
||||
def value_at(samples, when, hold, times=None):
|
||||
"""Our reading at a moment: the latest sample no older than `hold` seconds.
|
||||
`times` is the samples' time column, if the caller has already built it."""
|
||||
times = times if times is not None else [s[0] for s in samples]
|
||||
i = bisect.bisect_right(times, when) - 1
|
||||
if i < 0 or when - times[i] > hold:
|
||||
return None
|
||||
return samples[i][1]
|
||||
|
||||
|
||||
def compare(ours, reference, max_lag=10.0, hold=5.0):
|
||||
"""Compare our readings with the reference at each reference moment.
|
||||
|
||||
`lag` is the delay added to reference times before looking up ours (our
|
||||
readings arrive after the sensor's). The best lag within ±max_lag is used.
|
||||
"""
|
||||
real = [(t, b) for t, b in reference if b is not None]
|
||||
if not real or not any(b is not None for _, b in ours):
|
||||
raise ValueError("both recordings need at least one reading")
|
||||
best = None
|
||||
ours_times = [t for t, _ in ours]
|
||||
steps = int(max_lag * 4)
|
||||
for step in range(-steps, steps + 1):
|
||||
lag = step / 4
|
||||
pairs = [(value_at(ours, t + lag, hold, ours_times), b) for t, b in real]
|
||||
pairs = [(o, r) for o, r in pairs if o is not None]
|
||||
if not pairs:
|
||||
continue
|
||||
error = sum(abs(o - r) for o, r in pairs) / len(pairs)
|
||||
key = (-len(pairs), error, abs(lag))
|
||||
if best is None or key < best[0]:
|
||||
best = (key, lag, pairs)
|
||||
if best is None:
|
||||
raise ValueError("the recordings do not overlap in time")
|
||||
_, lag, pairs = best
|
||||
differences = [o - r for o, r in pairs]
|
||||
# While the reference says "no reading" (lost skin contact), we must not
|
||||
# produce new readings. Count ours that arrive during such a stretch.
|
||||
gaps = [t for t, b in reference if b is None]
|
||||
reference_times = [t for t, _ in reference]
|
||||
shown_in_gaps = 0
|
||||
for t, bpm in ours:
|
||||
i = bisect.bisect_right(reference_times, t - lag) - 1
|
||||
if bpm is not None and i >= 0 and reference[i][1] is None:
|
||||
shown_in_gaps += 1
|
||||
return {
|
||||
"reference_readings": len(real),
|
||||
"matched": len(pairs),
|
||||
"lag_seconds": lag,
|
||||
"mean_abs_error": sum(abs(d) for d in differences) / len(differences),
|
||||
"bias": sum(differences) / len(differences),
|
||||
"max_abs_error": max(abs(d) for d in differences),
|
||||
"within_5": sum(1 for d in differences if abs(d) <= 5) / len(differences),
|
||||
"exact": sum(1 for d in differences if d == 0) / len(differences),
|
||||
"no_contact_moments": len(gaps),
|
||||
"shown_during_no_contact": shown_in_gaps,
|
||||
}
|
||||
|
||||
|
||||
def report(result, tolerance, stream=sys.stdout):
|
||||
print(f"Reference readings: {result['reference_readings']}, matched: {result['matched']}", file=stream)
|
||||
print(f"Best alignment: ours {result['lag_seconds']:+.2f} s after the reference", file=stream)
|
||||
print(f"Mean absolute difference: {result['mean_abs_error']:.2f} BPM "
|
||||
f"(bias {result['bias']:+.2f}, worst {result['max_abs_error']})", file=stream)
|
||||
print(f"Within ±5 BPM: {result['within_5']:.0%}; identical: {result['exact']:.0%}", file=stream)
|
||||
if result["no_contact_moments"]:
|
||||
print(f"No-contact moments: {result['no_contact_moments']}, where we showed a stale "
|
||||
f"reading: {result['shown_during_no_contact']}", file=stream)
|
||||
coverage = result["matched"] / result["reference_readings"]
|
||||
passed = (result["mean_abs_error"] <= tolerance and coverage >= 0.8
|
||||
and not result["shown_during_no_contact"])
|
||||
print(("PASS" if passed else "FAIL") + f" (mean difference ≤ {tolerance} BPM, ≥80% of reference "
|
||||
f"readings matched, nothing shown without contact)", file=stream)
|
||||
return passed
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
commands = parser.add_subparsers(dest="command", required=True)
|
||||
heard = commands.add_parser("listen", help="show and optionally record our OSC readings live")
|
||||
heard.add_argument("--port", type=int, default=9000)
|
||||
heard.add_argument("--address", default=ADDRESS)
|
||||
heard.add_argument("--out", help="new private CSV file")
|
||||
heard.add_argument("--seconds", type=float, default=0, help="stop after this long (default: until Ctrl-C)")
|
||||
check = commands.add_parser("compare", help="compare our recording with a reference")
|
||||
check.add_argument("ours", type=Path)
|
||||
check.add_argument("reference", type=Path)
|
||||
check.add_argument("--tolerance", type=float, default=5.0, help="allowed mean difference in BPM (default 5)")
|
||||
check.add_argument("--max-lag", type=float, default=10.0, help="largest time offset to search, seconds")
|
||||
args = parser.parse_args(argv)
|
||||
if args.command == "listen":
|
||||
count = listen(args.port, args.address, args.out, args.seconds)
|
||||
print(f"Received {count} readings.")
|
||||
return 0 if count else 3
|
||||
try:
|
||||
ours = read_csv(args.ours)
|
||||
if not ours:
|
||||
raise ValueError("our recording has no readings")
|
||||
reference = read_any(args.reference, ours[0][0] - 60, ours[-1][0] + 60)
|
||||
result = compare(ours, reference, args.max_lag)
|
||||
except (ValueError, OSError, csv.Error, ElementTree.ParseError, zipfile.BadZipFile) as error:
|
||||
print(f"Could not compare: {error}")
|
||||
return 1
|
||||
return 0 if report(result, args.tolerance) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,100 @@
|
||||
// A synthetic Bluetooth heart-rate strap for testing, run on the Mac.
|
||||
//
|
||||
// swiftc -O scripts/heart-test-strap.swift -o /tmp/heart-test-strap
|
||||
// /tmp/heart-test-strap [seconds] # default 60
|
||||
//
|
||||
// Advertises the standard Heart Rate Service (180d) as "FC Test Strap" and,
|
||||
// while a central is subscribed, sends one Heart Rate Measurement (2a37) per
|
||||
// second from a fixed, known sequence. Each sent value is printed to stdout as
|
||||
// `unix_seconds,bpm,flags` so scripts/heart-check.py can compare what the
|
||||
// Frame received with what was sent. Every value is synthetic; this is not a
|
||||
// sensor. macOS asks for Bluetooth permission the first time it runs.
|
||||
import CoreBluetooth
|
||||
import Foundation
|
||||
|
||||
let hrs = CBUUID(string: "180D")
|
||||
let measurement = CBUUID(string: "2A37")
|
||||
|
||||
/// The known sequence: an 8-bit ramp, 16-bit encodings, a skin-contact loss
|
||||
/// (which must show as no reading) and a recovery.
|
||||
func packet(_ second: Int) -> (bpm: Int, flags: UInt8) {
|
||||
switch second % 60 {
|
||||
case 0..<30: return (60 + second % 60 * 4, 0x06) // 60…176, contact detected
|
||||
case 30..<40: return (180 - (second % 60 - 30) * 3, 0x07) // 16-bit value
|
||||
case 40..<45: return (150, 0x04) // contact lost
|
||||
default: return (72 + (second % 60 - 45), 0x06)
|
||||
}
|
||||
}
|
||||
|
||||
final class Strap: NSObject, CBPeripheralManagerDelegate {
|
||||
let seconds: Int
|
||||
var manager: CBPeripheralManager!
|
||||
var characteristic: CBMutableCharacteristic!
|
||||
var subscribed = false
|
||||
var sent = 0
|
||||
|
||||
init(seconds: Int) {
|
||||
self.seconds = seconds
|
||||
super.init()
|
||||
manager = CBPeripheralManager(delegate: self, queue: nil)
|
||||
}
|
||||
|
||||
func peripheralManagerDidUpdateState(_ peripheral: CBPeripheralManager) {
|
||||
guard peripheral.state == .poweredOn else {
|
||||
FileHandle.standardError.write("Bluetooth state \(peripheral.state.rawValue)\n".data(using: .utf8)!)
|
||||
if peripheral.state == .unauthorized || peripheral.state == .unsupported || peripheral.state == .poweredOff {
|
||||
FileHandle.standardError.write("Bluetooth unavailable (state \(peripheral.state.rawValue))\n".data(using: .utf8)!)
|
||||
exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
characteristic = CBMutableCharacteristic(type: measurement, properties: [.notify], value: nil, permissions: [])
|
||||
let service = CBMutableService(type: hrs, primary: true)
|
||||
service.characteristics = [characteristic]
|
||||
peripheral.add(service)
|
||||
}
|
||||
|
||||
func peripheralManager(_ peripheral: CBPeripheralManager, didAdd service: CBService, error: Error?) {
|
||||
if let error { fail("add service: \(error.localizedDescription)") }
|
||||
peripheral.startAdvertising([CBAdvertisementDataLocalNameKey: "FC Test Strap",
|
||||
CBAdvertisementDataServiceUUIDsKey: [hrs]])
|
||||
}
|
||||
|
||||
func peripheralManagerDidStartAdvertising(_ peripheral: CBPeripheralManager, error: Error?) {
|
||||
if let error { fail("advertise: \(error.localizedDescription)") }
|
||||
FileHandle.standardError.write("Advertising FC Test Strap\n".data(using: .utf8)!)
|
||||
Timer.scheduledTimer(withTimeInterval: 1, repeats: true) { _ in self.tick() }
|
||||
}
|
||||
|
||||
func peripheralManager(_ peripheral: CBPeripheralManager, central: CBCentral, didSubscribeTo characteristic: CBCharacteristic) {
|
||||
subscribed = true
|
||||
FileHandle.standardError.write("Central subscribed\n".data(using: .utf8)!)
|
||||
}
|
||||
|
||||
func peripheralManager(_ peripheral: CBPeripheralManager, central: CBCentral, didUnsubscribeFrom characteristic: CBCharacteristic) {
|
||||
subscribed = false
|
||||
FileHandle.standardError.write("Central unsubscribed\n".data(using: .utf8)!)
|
||||
}
|
||||
|
||||
func tick() {
|
||||
guard subscribed else { return }
|
||||
if sent >= seconds { exit(0) }
|
||||
let (bpm, flags) = packet(sent)
|
||||
var bytes: [UInt8] = [flags, UInt8(bpm & 0xff)]
|
||||
if flags & 1 != 0 { bytes.append(UInt8(bpm >> 8)) }
|
||||
if manager.updateValue(Data(bytes), for: characteristic, onSubscribedCentrals: nil) {
|
||||
print(String(format: "%.3f,%d,%d", Date().timeIntervalSince1970, bpm, flags))
|
||||
fflush(stdout)
|
||||
sent += 1
|
||||
}
|
||||
}
|
||||
|
||||
func fail(_ message: String) -> Never {
|
||||
FileHandle.standardError.write("\(message)\n".data(using: .utf8)!)
|
||||
exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
let seconds = CommandLine.arguments.count > 1 ? Int(CommandLine.arguments[1]) ?? 60 : 60
|
||||
let strap = Strap(seconds: seconds)
|
||||
RunLoop.main.run()
|
||||
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Install or run our local-only tracking tools on the Frame.
|
||||
|
||||
python3 scripts/tracking-on-frame.py install
|
||||
python3 scripts/tracking-on-frame.py gaze --seconds 10
|
||||
python3 scripts/tracking-on-frame.py gaze --seconds 3600 --osc 127.0.0.1 9000
|
||||
python3 scripts/tracking-on-frame.py heart --device AA:BB:CC:DD:EE:FF --panel
|
||||
python3 scripts/tracking-on-frame.py pulse --seconds 60 --show # experimental
|
||||
|
||||
FRAME_ALIAS overrides the SSH alias (default: frame). Runs in the foreground;
|
||||
Ctrl-C stops the reader. No service, autostart, sudo or SteamVR settings changes.
|
||||
"""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
|
||||
REMOTE = '"$HOME/.local/share/frame-control/tracking"'
|
||||
INSTALL = '''set -eu
|
||||
base="$HOME/.local/share/frame-control/tracking"
|
||||
mkdir -p "$base"
|
||||
stage=$(mktemp -d "$base/.install.XXXXXX")
|
||||
trap 'rm -rf "$stage"' EXIT
|
||||
tar -xf - -C "$stage"
|
||||
cc -O2 -Wall -Wextra -Werror "$stage/gaze.c" \\
|
||||
-L/opt/steamvr/bin/linuxarm64 -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \\
|
||||
-lopenxr_loader -o "$stage/gaze"
|
||||
chmod 700 "$stage/gaze" "$stage/tracking.py"
|
||||
chmod 600 "$stage/pulse.py"
|
||||
mv "$stage/gaze" "$stage/tracking.py" "$stage/pulse.py" "$base/"
|
||||
echo 'Installed Frame Control tracking tools (no service started).'
|
||||
'''
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in ("install", "gaze", "heart", "pulse"):
|
||||
print(__doc__)
|
||||
return 2
|
||||
host = os.environ.get("FRAME_ALIAS", "frame")
|
||||
if not host or host.startswith("-"):
|
||||
raise ValueError("invalid SSH alias")
|
||||
ssh = ["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=10", host]
|
||||
if sys.argv[1] == "install":
|
||||
import tempfile
|
||||
source = Path(__file__).resolve().parents[1] / "frame" / "tracking"
|
||||
with tempfile.TemporaryFile() as archive:
|
||||
with tarfile.open(fileobj=archive, mode="w") as tar:
|
||||
for name in ("gaze.c", "tracking.py", "pulse.py"):
|
||||
tar.add(source / name, arcname=name)
|
||||
archive.seek(0)
|
||||
return subprocess.call(ssh + ["bash -c " + shlex.quote(INSTALL)], stdin=archive)
|
||||
# Allocate a tty so SSH forwards Ctrl-C and hangup to the foreground process.
|
||||
command = 'exec python3 ' + REMOTE + '/tracking.py ' + shlex.join(sys.argv[1:])
|
||||
return subprocess.call(ssh[:-1] + ["-tt", host, command])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -64,7 +64,7 @@ function sharedPage({productionApi=false} = {}) {
|
||||
const request=deferred();request.path=path;request.opts=opts;requests.push(request);return request.promise;
|
||||
};
|
||||
Object.assign(context,{renderPill(){},renderConnDlg(){},renderBanner(){},viaText(){},netLabel(){},
|
||||
resetShots(){},closeDevPop(){},loadDevices(){},renderDevices(){},renderTests(){},reloadAll(){},checkContactPrompt(){}});
|
||||
resetShots(){},loadDevices(){},renderDevices(){},renderTests(){},reloadAll(){},checkContactPrompt(){}});
|
||||
vm.runInContext(`const UI_KEY = 'test-only';
|
||||
const link = {s:null,device:'frame-a',phase:'connected',rev:0,net:null};
|
||||
const disp = {}, gm = {seq:0}, cat = {}, wi = {};
|
||||
@@ -443,20 +443,20 @@ test('every linked workflow selects its real page and section, preserving shortc
|
||||
const start=html.indexOf('const PAGES ='),end=html.indexOf('window.addEventListener("hashchange", showPage);',start);
|
||||
const ids=new Set([...html.matchAll(/\bid="([^"]+)"/g)].map(m=>m[1]));
|
||||
let activePage,scrolled;
|
||||
const nodes=['home','games','android','tools','devices','control','settings'].map(page=>({dataset:{page},classList:{toggle(key,on){if(on)activePage=page;}}}));
|
||||
const nodes=['home','games','android','tools','devices','control'].map(page=>({dataset:{page},classList:{toggle(key,on){if(on)activePage=page;}}}));
|
||||
const sections=new Map();
|
||||
const context={location:{hash:'#control'},document:{querySelectorAll:s=>s==='.page'?nodes:[],title:''},
|
||||
window:{scrollTo(){}},$:id=>{
|
||||
if(!ids.has(id))return null;
|
||||
if(!sections.has(id))sections.set(id,{hidden:false,classList:{toggle(){}},scrollIntoView(){scrolled=id;}});
|
||||
if(!sections.has(id))sections.set(id,{hidden:false,scrollIntoView(){scrolled=id;}});
|
||||
return sections.get(id);
|
||||
},live:false,toggleLive(){},scrollToY(){},loadMacView(){},loadPanelSwitcher(){}};
|
||||
},live:false,toggleLive(){},loadMacView(){},loadPanelSwitcher(){}};
|
||||
const androidStart=html.indexOf('function showAndroidTab(');
|
||||
const androidEnd=html.indexOf("document.querySelectorAll('[data-store-tab]').forEach(button => button.onclick",androidStart);
|
||||
vm.createContext(context);vm.runInContext(html.slice(androidStart,androidEnd)+html.slice(start,end)+'\n'+html.match(/^PAGES\.push\([^\n]+/m)[0],context);
|
||||
vm.createContext(context);vm.runInContext(html.slice(androidStart,androidEnd)+html.slice(start,end)+'\nPAGES.push("devices", "control");',context);
|
||||
vm.runInContext('showPage()',context);assert.equal(activePage,'control');
|
||||
const expected={view:'home',shots:'home',vrTools:'home',comfort:'home',pad:'home',library:'games',getgames:'games',sideloaded:'games',android:'android',display:'android',androidLibrary:'android',appStore:'android',
|
||||
power:'tools',transfer:'tools',media:'tools',apps:'tools',artworkSettings:'settings',privacy:'settings',panels:'tools',macview:'tools',devices:'devices'};
|
||||
power:'tools',transfer:'tools',media:'tools',apps:'tools',artworkSettings:'tools',privacy:'tools',panels:'tools',macview:'tools',devices:'devices'};
|
||||
for(const item of cc.catalog) for(const [href] of item.links||[]) {
|
||||
if(!href.startsWith('#')||href==='#connection')continue;
|
||||
const id=href.slice(1);assert.ok(ids.has(id)||['android','devices'].includes(id),href);
|
||||
@@ -468,5 +468,4 @@ test('every linked workflow selects its real page and section, preserving shortc
|
||||
}
|
||||
assert.equal(vm.runInContext('PAGES[4]',context),'devices');
|
||||
assert.equal(vm.runInContext('PAGES[5]',context),'control');
|
||||
assert.equal(vm.runInContext('PAGES[6]',context),'settings');
|
||||
});
|
||||
@@ -19,8 +19,8 @@ function grab(name) {
|
||||
fail(`unbalanced ${name}`);
|
||||
}
|
||||
const names = ["esc", "storeName", "storeHue", "storeInitial", "storeText", "storeOpenSource", "sourcePage", "storeImage",
|
||||
"GLYPH_VR", "GLYPH_APP", "RUNS", "storeRuns", "storeKind", "storeArt", "storeIcon", "sourceMark", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
|
||||
"updateInstall", "openStoreApp", "loadOfferDetails", "installStoreApp"];
|
||||
"storeArt", "storeIcon", "sourceMark", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
|
||||
"updateInstall", "openStoreApp", "loadOfferDetails"];
|
||||
const code = names.map(grab).join("");
|
||||
|
||||
// A fake page: elements by id. The install panel's elements exist only while
|
||||
@@ -34,10 +34,8 @@ function page(api = async () => ({})) {
|
||||
const document = { activeElement: null, querySelector: () => null };
|
||||
const console = { error() {} };
|
||||
const calls = [];
|
||||
const fns = new Function("$", "sourceState", "document", "console", "api", "runJob", "loadAndroid", "act", "window", code + "; return { renderAppDetail, openStoreApp };")(
|
||||
$, sourceState, document, console, (...a) => { calls.push(a); return api(...a); },
|
||||
async (title, key, work) => work(), () => {}, async (title, work) => work(),
|
||||
{ open() { fail("an in-app installation opened an external page"); } });
|
||||
const fns = new Function("$", "sourceState", "document", "console", "api", code + "; return { renderAppDetail, openStoreApp };")(
|
||||
$, sourceState, document, console, (...a) => { calls.push(a); return api(...a); });
|
||||
return { els, sourceState, calls, ...fns };
|
||||
}
|
||||
|
||||
@@ -70,7 +68,7 @@ const offer = { source: "fdroid", id: "org.example.brush", source_name: "F-Droid
|
||||
{
|
||||
const p = page(async () => { throw new Error("offline"); });
|
||||
p.sourceState.apps = [{ name: "Broken", offers: [] }];
|
||||
await p.openStoreApp(0, true);
|
||||
await p.openStoreApp(0);
|
||||
if (!p.els.appDetail.open || !p.els.detailBody.innerHTML.includes('role="alert"')) fail("broken listing didn't open with the error");
|
||||
if (p.calls.length) fail("asked for details of a listing that couldn't render");
|
||||
}
|
||||
@@ -92,18 +90,4 @@ const offer = { source: "fdroid", id: "org.example.brush", source_name: "F-Droid
|
||||
if (!p.els.detailBody.innerHTML.includes('id="detailTitle">Open Brush<')) fail("page lost after a failed details request");
|
||||
if (!p.els.detailInstallNote.textContent.includes("couldn't be refreshed")) fail("no note after a failed details request");
|
||||
}
|
||||
// The featured Install starts a new or failed job, while a job already running or
|
||||
// installed opens its details without another install or a launch command.
|
||||
for (const [job, expected] of [[null, 1], [{ done: false }, 0],
|
||||
[{ done: true, result: { package: offer.package } }, 0], [{ done: true, error: "offline" }, 1]]) {
|
||||
const p = page(async path => path === "/api/sources/install" ? { package: offer.package } : {});
|
||||
p.sourceState.apps = [{ name: "Open Brush", offers: [{ ...offer }] }];
|
||||
if (job) p.sourceState.jobs.set(`${offer.source}:${offer.id}`, job);
|
||||
await p.openStoreApp(0, true);
|
||||
if (!p.els.appDetail.open || !p.els.detailBody.innerHTML.includes('id="detailInstall"')) fail("hero installation didn't open details");
|
||||
const installs = p.calls.filter(([path]) => path === "/api/sources/install");
|
||||
if (installs.length !== expected) fail(`expected ${expected} installation requests, got ${installs.length}`);
|
||||
if (installs.some(([, body]) => body.source !== offer.source || body.id !== offer.id)) fail("installed a different offer");
|
||||
if (p.calls.some(([path]) => path === "/api/android")) fail("hero Install launched an already installed app");
|
||||
}
|
||||
console.log("ok");
|
||||
@@ -56,48 +56,6 @@ const log = ()=>{}, toast = ()=>{};
|
||||
assert.equal($('sessionMinutes').disabled,false);
|
||||
assert.equal($('sessionCancel').disabled,true);
|
||||
})().catch(e=>{console.error(e);process.exitCode=1;});
|
||||
'''
|
||||
result = subprocess.run(['node', '-e', setup + code + checks], capture_output=True, text=True)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
|
||||
def test_a_finished_session_leaves_the_defaults(self):
|
||||
"""A one-minute test session that has ended mustn't fill the form with 1 / 0 / 0."""
|
||||
page = (ROOT / 'ui/index.html').read_text(encoding='utf-8')
|
||||
code = page[page.index('let comfortBusy ='):page.index('async function pollComfort()')]
|
||||
setup = r'''
|
||||
const assert = require('node:assert/strict');
|
||||
const elements = new Map();
|
||||
const $ = id => {
|
||||
// The form's own defaults, as in the page's HTML.
|
||||
const value = {sessionMinutes:'30', breakMinutes:'20', stillMinutes:'30'}[id] || '';
|
||||
if (!elements.has(id)) elements.set(id, {textContent:'', hidden:true, disabled:false, value, defaultValue:value,
|
||||
checked:true, defaultChecked:true, type: /Alert$/.test(id) ? 'checkbox' : 'number'});
|
||||
return elements.get(id);
|
||||
};
|
||||
const window = {frameApp:{notify:async()=>{}}};
|
||||
const log = ()=>{}, toast = ()=>{};
|
||||
'''
|
||||
checks = r'''
|
||||
renderComfort({id:'old', active:false, time:100, options:{minutes:1,breakMinutes:0,stillMinutes:0,batteryAlert:false,heatAlert:false},
|
||||
events:[{id:'old:1',kind:'finished',time:90,message:'Session ended'}]});
|
||||
assert.equal($('sessionMinutes').value, '30');
|
||||
assert.equal($('breakMinutes').value, '20');
|
||||
assert.equal($('batteryAlert').checked, true);
|
||||
assert.match($('comfortEvents').textContent, /Session ended/);
|
||||
assert.notEqual($('comfortEvents').textContent, 'Session ended'); // it carries a time
|
||||
renderComfort({id:'new', active:true, time:200, remaining:600, options:{minutes:45,breakMinutes:15,stillMinutes:20,batteryAlert:true,heatAlert:false}, events:[]});
|
||||
assert.equal($('sessionMinutes').value, 45); // a running session shows its own settings
|
||||
assert.equal($('heatAlert').checked, false);
|
||||
// The page stays open while that session ends: the next one starts from the defaults again.
|
||||
renderComfort({id:'new', active:false, time:900, options:{minutes:45,breakMinutes:15,stillMinutes:20,batteryAlert:true,heatAlert:false},
|
||||
events:[{id:'new:1',kind:'finished',time:899,message:'Session ended'}]});
|
||||
assert.equal($('sessionMinutes').value, '30');
|
||||
assert.equal($('breakMinutes').value, '20');
|
||||
assert.equal($('heatAlert').checked, true);
|
||||
renderComfort({id:'new', active:false, time:905, options:{minutes:45}, events:[]});
|
||||
$('sessionMinutes').value = '50'; // what's typed for the next session
|
||||
renderComfort({id:'new', active:false, time:910, options:{minutes:45}, events:[]});
|
||||
assert.equal($('sessionMinutes').value, '50'); // later polls leave it alone
|
||||
'''
|
||||
result = subprocess.run(['node', '-e', setup + code + checks], capture_output=True, text=True)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
|
||||
@@ -323,20 +323,6 @@ class Pins(Base):
|
||||
|
||||
|
||||
class Registry(Base):
|
||||
def test_an_address_added_first_wins_on_its_own_network(self):
|
||||
# The page's "Add 192.168.x.x" offer: the headset is reached over Tailscale, which has
|
||||
# worked here before. The LAN address has to go ahead of it to be used at home.
|
||||
d = self.reg.add_device("frame-4", hosts=["frame.tail1234.ts.net"])
|
||||
self.reg.record_success(d["id"], "frame.tail1234.ts.net", "n-home", 6.0)
|
||||
self.reg.add_address(d["id"], "192.168.1.40", kind="lan", first=True)
|
||||
self.reg.record_success(d["id"], "192.168.1.40", "n-home", 1.0) # Test now found it
|
||||
addrs = self.reg.get(d["id"])["addresses"]
|
||||
self.assertEqual([a["host"] for a in addrs], ["192.168.1.40", "frame.tail1234.ts.net"])
|
||||
at_home = [a["host"] for a, _ in fd.order_addresses(addrs, "n-home", True)]
|
||||
self.assertEqual(at_home[0], "192.168.1.40")
|
||||
away = [a["host"] for a, _ in fd.order_addresses(addrs, "n-cafe", True)]
|
||||
self.assertEqual(away[0], "frame.tail1234.ts.net") # elsewhere Tailscale still leads
|
||||
|
||||
def test_address_editing(self):
|
||||
d = self.reg.add_device("frame-3", hosts=["192.168.1.40"])
|
||||
a = self.reg.add_address(d["id"], "frame-3.local", label="mDNS")
|
||||
|
||||
@@ -18,7 +18,6 @@ import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest import mock
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
@@ -632,257 +631,3 @@ class PageQueue(unittest.TestCase):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class PadEvents(unittest.TestCase):
|
||||
"""The trackpad and key row go through gamescope; accents and emoji go the KDE Connect way."""
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
import server
|
||||
cls.pad = staticmethod(server.pad_events)
|
||||
|
||||
def test_moves_clicks_and_scroll(self):
|
||||
out, extra = self.pad([{"dx": 3, "dy": -2}, {"singleclick": True}, {"singlehold": True},
|
||||
{"singlerelease": True}, {"scroll": True, "dy": 1}, {"rightclick": True}])
|
||||
self.assertEqual(out[0], {"dx": 3, "dy": -2})
|
||||
self.assertEqual(out[1:3], [{"button": "left", "down": True}, {"button": "left", "down": False}])
|
||||
self.assertEqual(out[3:5], [{"button": "left", "down": True}, {"button": "left", "down": False}])
|
||||
self.assertEqual(out[5], {"scroll": [0, -15]}) # up = content follows the finger
|
||||
self.assertEqual(out[6:], [{"button": "right", "down": True}, {"button": "right", "down": False}])
|
||||
self.assertEqual(extra, [])
|
||||
|
||||
def test_keys_and_modifiers(self):
|
||||
out, _ = self.pad([{"specialKey": 12}, {"specialKey": 1, "ctrl": True}, {"key": "c", "ctrl": True}])
|
||||
self.assertEqual(out[:2], [{"key": 28, "down": True}, {"key": 28, "down": False}])
|
||||
self.assertEqual([e["key"] for e in out[2:6]], [29, 14, 14, 29]) # Ctrl+Backspace
|
||||
self.assertEqual([(e["key"], e["down"]) for e in out[6:]],
|
||||
[(29, True), (46, True), (46, False), (29, False)]) # Ctrl+C
|
||||
|
||||
def test_text_stays_ascii_through_gamescope(self):
|
||||
out, slow = self.pad([{"key": "Hi "}, {"key": "there"}])
|
||||
self.assertEqual(out, [{"text": "Hi there"}])
|
||||
self.assertEqual(slow, [])
|
||||
|
||||
def test_keyboard_goes_to_kde_whole_and_in_order(self):
|
||||
events = [{"key": "a"}, {"key": "é"}, {"specialKey": 12}, {"key": "c", "ctrl": True}, {"dx": 2, "dy": 1}]
|
||||
out, slow = self.pad(events, kde=True)
|
||||
self.assertEqual(out, [{"dx": 2, "dy": 1}]) # the pointer still goes through gamescope
|
||||
self.assertEqual(slow, events[:4]) # the keyboard as sent, in order
|
||||
|
||||
def test_shifted_key_with_modifier(self):
|
||||
out, _ = self.pad([{"key": "A", "ctrl": True}])
|
||||
self.assertEqual([e["key"] for e in out], [29, 42, 30, 30, 42, 29])
|
||||
|
||||
|
||||
class PadDelivery(unittest.TestCase):
|
||||
"""remote_input's promises: "sent" means the whole batch is taken; accents go once, in order."""
|
||||
|
||||
class Agent:
|
||||
def __init__(self, sent=True, state="ready"):
|
||||
self.sent, self.state, self.got = sent, state, []
|
||||
|
||||
def send(self, events):
|
||||
if events and self.sent:
|
||||
self.got.append(events)
|
||||
return {"state": self.state, "sent": self.sent and bool(events)}
|
||||
|
||||
def setUp(self):
|
||||
import server
|
||||
self.s = server
|
||||
self.saved = (server._touch, server._input, server.KDE_WAIT)
|
||||
server._touch, server._input = self.Agent(), self.Agent()
|
||||
self.reset()
|
||||
|
||||
def tearDown(self):
|
||||
self.s._touch, self.s._input, self.s.KDE_WAIT = self.saved
|
||||
self.settle()
|
||||
self.reset()
|
||||
|
||||
def reset(self):
|
||||
self.s._kde_queue.clear()
|
||||
self.s._kde_until = self.s._typed_until = 0.0
|
||||
self.s._kde_worker[0] = None
|
||||
|
||||
def settle(self):
|
||||
worker = self.s._kde_worker[0]
|
||||
if worker:
|
||||
worker.join(5)
|
||||
|
||||
def typed(self):
|
||||
return [e for batch in self.s._input.got for e in batch]
|
||||
|
||||
def test_accent_only_batch_is_acknowledged_and_typed_once(self):
|
||||
status = self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.assertTrue(status["sent"]) # so the page doesn't send it again
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
|
||||
def test_mixed_text_stays_in_one_ordered_stream(self):
|
||||
self.s.remote_input({"events": [{"key": "aéb"}, {"specialKey": 12}]})
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "aéb"}, {"specialKey": 12}])
|
||||
self.assertEqual(self.s._touch.got, []) # none of it split off through gamescope
|
||||
|
||||
def test_batch_gamescope_didnt_take_is_not_queued(self):
|
||||
self.s._touch = self.Agent(sent=False, state="starting")
|
||||
status = self.s.remote_input({"events": [{"dx": 1, "dy": 1}, {"key": "é"}]})
|
||||
self.assertFalse(status["sent"]) # the page tries the whole batch again...
|
||||
self.assertEqual(self.s._kde_queue, []) # ...so the accent isn't queued twice
|
||||
|
||||
def test_waits_for_kde_connect_then_sends_without_more_input(self):
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
time.sleep(0.2)
|
||||
self.assertEqual(self.s._kde_queue, [{"key": "é"}])
|
||||
self.s._input.sent = True
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
|
||||
def test_later_ascii_waits_behind_a_pending_accent(self):
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.s.remote_input({"events": [{"key": "x"}]})
|
||||
self.assertEqual(self.s._touch.got, []) # "x" can't overtake the "é"
|
||||
self.assertEqual(self.s._kde_queue, [{"key": "é"}, {"key": "x"}])
|
||||
self.s._input.sent = True
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
|
||||
|
||||
def test_queue_is_bounded_and_dropped_on_error(self):
|
||||
self.s._input = self.Agent(sent=False, state="error")
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.KDE_WAIT = 0.3
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.assertLessEqual(len(self.s._kde_queue), self.s.KDE_QUEUE_LIMIT)
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
|
||||
def test_ascii_just_after_an_accent_stays_behind_it(self):
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.settle()
|
||||
self.s.remote_input({"events": [{"key": "x"}]}) # KDE Connect may still be typing the é
|
||||
self.settle()
|
||||
self.assertEqual(self.s._touch.got, [])
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
|
||||
|
||||
def test_accent_just_after_gamescope_text_waits_for_it(self):
|
||||
self.s.remote_input({"events": [{"key": "x" * 50}]})
|
||||
self.assertEqual(self.s._touch.got, [[{"text": "x" * 50}]])
|
||||
start = time.time()
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.settle()
|
||||
self.assertGreater(time.time() - start, 0.6) # 50 keys at 16 ms, plus the margin
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
|
||||
def test_typing_time_counts_every_key_transition(self):
|
||||
t = self.s.typing_seconds
|
||||
self.assertAlmostEqual(t([{"text": "ab"}]), 4 * 0.008)
|
||||
self.assertAlmostEqual(t([{"text": "A"}]), 4 * 0.008) # shift down, key down, key up, shift up
|
||||
self.assertAlmostEqual(t([{"text": "x" * 500}]), 8.0) # a long paste
|
||||
self.assertAlmostEqual(t([{"key": 29, "down": True}, {"dx": 3, "dy": 0}]), 0.008)
|
||||
|
||||
def test_the_margin_is_added_once_not_per_request(self):
|
||||
for _ in range(10):
|
||||
self.s.remote_input({"events": [{"key": "ab"}]})
|
||||
# ten requests of 4 transitions each: the work queues up, with no margin added per request
|
||||
self.assertLess(self.s._typed_until - time.time(), 10 * 4 * 0.008 + 0.05)
|
||||
|
||||
def test_trimming_while_sending_doesnt_drop_unsent_events(self):
|
||||
slow = self.Agent()
|
||||
gate = threading.Event()
|
||||
send = slow.send
|
||||
|
||||
def held(events):
|
||||
gate.wait(5)
|
||||
return send(events)
|
||||
slow.send = held
|
||||
self.s._input = slow
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
time.sleep(0.2) # the worker has taken its 150 and is mid-send
|
||||
self.s.remote_input({"events": [{"key": "ü"}] * 150})
|
||||
gate.set()
|
||||
self.settle()
|
||||
sent = self.typed()
|
||||
self.assertEqual(sent[:150], [{"key": "é"}] * 150)
|
||||
self.assertEqual(sent[150:], [{"key": "ü"}] * 150) # none of the new ones went missing
|
||||
|
||||
def test_input_queued_as_the_worker_finishes_is_still_sent(self):
|
||||
for _ in range(40):
|
||||
self.reset()
|
||||
self.s._input.got.clear()
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
time.sleep(0.002)
|
||||
self.s.remote_input({"events": [{"key": "ü"}]})
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "ü"}])
|
||||
|
||||
|
||||
class PadRoute(unittest.TestCase):
|
||||
def setUp(self):
|
||||
import server
|
||||
self.s = server
|
||||
self.agents = mock.patch.multiple(server, _touch=mock.Mock(), _input=mock.Mock())
|
||||
self.agents.start()
|
||||
self.addCleanup(self.agents.stop)
|
||||
self.s._touch.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s._input.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s.reset_pad_route()
|
||||
self.s._touch.reset_mock(); self.s._input.reset_mock()
|
||||
self.addCleanup(self.s.reset_pad_route)
|
||||
|
||||
def switch(self):
|
||||
with mock.patch.multiple(self.s, FRAME="frame-a", HOST_OPTS=[], MUX=[], SSH=[], macview=None), \
|
||||
mock.patch.multiple(self.s.frame_android, FRAME="frame-a", SSH_OPTS=[]):
|
||||
self.s.route("frame-b", [])
|
||||
|
||||
def test_switch_stops_both_input_agents_and_discards_old_queue(self):
|
||||
self.s._kde_queue.append({"key": "old"})
|
||||
self.s._kde_until = self.s._typed_until = time.time() + 60
|
||||
cancel = self.s._kde_cancel
|
||||
self.switch()
|
||||
self.s._input.stop.assert_called_once()
|
||||
self.s._touch.stop.assert_called_once()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.assertEqual((self.s._kde_until, self.s._typed_until), (0, 0))
|
||||
self.assertTrue(cancel.is_set())
|
||||
|
||||
def test_switch_cancels_delayed_and_retrying_text_before_new_input(self):
|
||||
for delaying in (True, False):
|
||||
with self.subTest(delaying=delaying):
|
||||
waiting, cancel = threading.Event(), threading.Event()
|
||||
class ObservedCancel:
|
||||
def wait(self, timeout):
|
||||
waiting.set()
|
||||
return cancel.wait(timeout)
|
||||
def set(self):
|
||||
cancel.set()
|
||||
self.s.reset_pad_route()
|
||||
self.s._kde_cancel = ObservedCancel()
|
||||
self.s._typed_until = time.time() + 60 if delaying else 0
|
||||
self.s._input.reset_mock()
|
||||
self.s._input.send.return_value = {"state": "starting", "sent": False}
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
worker = self.s._kde_worker[0]
|
||||
self.assertTrue(waiting.wait(1), "worker never reached its cancellable delay")
|
||||
self.switch()
|
||||
worker.join(1)
|
||||
self.assertFalse(worker.is_alive())
|
||||
attempts = self.s._input.send.call_count
|
||||
self.assertEqual(attempts, 0 if delaying else 1)
|
||||
self.s._input.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s.remote_input({"events": [{"key": "ü"}]})
|
||||
next_worker = self.s._kde_worker[0]
|
||||
if next_worker:
|
||||
next_worker.join(1)
|
||||
self.assertFalse(next_worker.is_alive())
|
||||
self.assertEqual(self.s._input.send.call_count, attempts + 1)
|
||||
self.assertEqual(self.s._input.send.call_args.args[0], [{"key": "ü"}])
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
@@ -897,47 +897,6 @@ class Connecting(unittest.TestCase):
|
||||
self.assertEqual(self.routes, [])
|
||||
self.assertTrue(all("ControlPath=none" in c for c in self.calls() if "-G" not in c))
|
||||
|
||||
def test_a_test_started_earlier_cant_overwrite_a_newer_one(self):
|
||||
d = self.device("nothing.invalid")
|
||||
entered = {1: threading.Event(), 2: threading.Event()}
|
||||
release = {1: threading.Event(), 2: threading.Event()}
|
||||
calls = []
|
||||
|
||||
def probe(host, port, update=None):
|
||||
calls.append(host)
|
||||
n = len(calls)
|
||||
entered[n].set()
|
||||
release[n].wait(10)
|
||||
return {"state": "refused", "detail": f"test {n}", "ip": None, "rtt_ms": None}
|
||||
|
||||
tests = [threading.Thread(target=self.link.test, args=(d["id"],), daemon=True) for _ in range(2)]
|
||||
with mock.patch.object(fl, "probe", probe):
|
||||
try:
|
||||
tests[0].start()
|
||||
self.assertTrue(entered[1].wait(5), "the first test never probed")
|
||||
tests[1].start()
|
||||
self.assertTrue(entered[2].wait(5), "the second test never probed")
|
||||
# The first finishes while the second is still probing: it mustn't show its
|
||||
# rows or mark the second done.
|
||||
release[1].set()
|
||||
tests[0].join(10)
|
||||
self.assertFalse(tests[0].is_alive())
|
||||
running = self.link.snapshot()["tests"][d["id"]]
|
||||
self.assertFalse(running["done"])
|
||||
self.assertEqual(running["rows"][0]["detail"], "Waiting")
|
||||
release[2].set()
|
||||
tests[1].join(10)
|
||||
self.assertFalse(tests[1].is_alive())
|
||||
finally:
|
||||
for e in release.values():
|
||||
e.set()
|
||||
for t in tests:
|
||||
if t.ident: # started
|
||||
t.join(10)
|
||||
result = self.link.snapshot()["tests"][d["id"]]
|
||||
self.assertTrue(result["done"])
|
||||
self.assertEqual(result["rows"][0]["detail"], "test 2")
|
||||
|
||||
def test_switching_to_a_headset_that_never_answers_stops_using_the_last_one(self):
|
||||
self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
|
||||
@@ -56,19 +56,18 @@ class LinkUI(unittest.TestCase):
|
||||
r = subprocess.run(["node", "-e", script], capture_output=True, text=True)
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
|
||||
def test_the_pill_keeps_the_failure_detail_in_its_tooltip(self):
|
||||
# The connection redesign keeps the state and retry countdown in the pill,
|
||||
# with the full failure explanation in its tooltip and details dialog.
|
||||
def test_the_pill_leads_with_the_reason(self):
|
||||
# The header pill cut off "Offline: checking the headset's identity failed · retry in…":
|
||||
# the reason comes first, in a few words, and only the countdown gets cut.
|
||||
self.run_page(r'''
|
||||
link.s = changed;
|
||||
renderPill();
|
||||
assert.match($('conn').innerHTML, /<b>Office Frame<\/b><span>Offline<span class="via"> · retrying in 12 s<\/span>/);
|
||||
assert.ok($('conn').title.includes(changed.error.message));
|
||||
assert.match($('conn').innerHTML, /<b>Office Frame<\/b><span>SSH identity changed · retry in 12 s<\/span>/);
|
||||
assert.equal($('conn').title, 'Office Frame is offline: SSH identity changed · retry in 12 s. Click for details.');
|
||||
// Without a short reason: the stage that failed.
|
||||
link.s = {...changed, retry_at: null, error: {stage: 'identity', message: 'Something.'}};
|
||||
renderPill();
|
||||
assert.match($('conn').innerHTML, /<span>Offline<\/span>/);
|
||||
assert.ok($('conn').title.includes('Something.'));
|
||||
assert.match($('conn').innerHTML, /<span>Checking the headset's identity failed<\/span>/);
|
||||
''')
|
||||
|
||||
def test_the_banner_never_doubles_a_full_stop(self):
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
"""The page's inline scripts share one global scope, so a second top-level function or
|
||||
variable with a name already used replaces the first everywhere, silently."""
|
||||
import html.parser
|
||||
import json
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
|
||||
# Compiles the scripts as one strict-mode block. There, functions are block scoped like let
|
||||
# and const, so V8 itself rejects a name declared twice in the shared scope (however it's
|
||||
# indented or declared: function, class, let, const, destructuring), while helpers with the
|
||||
# same name inside different functions stay legal. var is the exception (declaring one twice
|
||||
# is allowed), so the page doesn't use var at all: Node's own copy of the acorn parser finds
|
||||
# real var declarations, not the word in comments, strings or CSS var().
|
||||
DUPLICATES = r'''
|
||||
const vm = require('vm');
|
||||
const scripts = JSON.parse(require('fs').readFileSync(0, 'utf8'));
|
||||
try { new vm.Script('"use strict"; {\n' + scripts.join('\n;\n') + '\n}'); console.log('ok'); }
|
||||
catch (e) { console.log(e.message); }
|
||||
'''
|
||||
VARS = r'''
|
||||
const scripts = JSON.parse(require('fs').readFileSync(0, 'utf8'));
|
||||
let acorn, walk;
|
||||
try {
|
||||
acorn = require('internal/deps/acorn/acorn/dist/acorn');
|
||||
walk = require('internal/deps/acorn/acorn-walk/dist/walk');
|
||||
} catch { console.log('no parser'); process.exit(0); }
|
||||
for (const code of scripts) {
|
||||
let found = null;
|
||||
try {
|
||||
walk.simple(acorn.parse(code, { ecmaVersion: 'latest' }), {
|
||||
VariableDeclaration(n) { if (n.kind === 'var' && !found) found = code.slice(n.start, n.end).slice(0, 40); },
|
||||
});
|
||||
} catch (e) { console.log(e.message); process.exit(0); }
|
||||
if (found) { console.log('declares with var: ' + found); process.exit(0); }
|
||||
}
|
||||
console.log('ok');
|
||||
'''
|
||||
|
||||
|
||||
def node(script, scripts, *flags):
|
||||
r = subprocess.run(['node', *flags, '-e', script], input=json.dumps(scripts), capture_output=True, text=True)
|
||||
return r.stdout.strip() or r.stderr.strip()
|
||||
|
||||
|
||||
def check(scripts):
|
||||
"""'ok', or V8's message for the first name declared twice."""
|
||||
return node(DUPLICATES, scripts)
|
||||
|
||||
|
||||
def check_vars(scripts):
|
||||
"""'ok', or the first var declaration. Skips where Node doesn't include acorn."""
|
||||
out = node(VARS, scripts, '--expose-internals')
|
||||
if out == 'no parser':
|
||||
raise unittest.SkipTest("this Node doesn't include acorn")
|
||||
return out
|
||||
|
||||
|
||||
class ClassicScripts(html.parser.HTMLParser):
|
||||
"""The page's inline classic scripts: no src, and no type other than a JavaScript one (a
|
||||
module has its own scope, and JSON data isn't code). The types a browser runs as script:
|
||||
https://mimesniff.spec.whatwg.org/#javascript-mime-type"""
|
||||
JS = {'', 'application/ecmascript', 'application/javascript', 'application/x-ecmascript',
|
||||
'application/x-javascript', 'text/ecmascript', 'text/javascript', 'text/javascript1.0',
|
||||
'text/javascript1.1', 'text/javascript1.2', 'text/javascript1.3', 'text/javascript1.4',
|
||||
'text/javascript1.5', 'text/jscript', 'text/livescript', 'text/x-ecmascript', 'text/x-javascript'}
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.scripts, self.current = [], None
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
if tag == 'script':
|
||||
a = dict(attrs)
|
||||
inline = 'src' not in a and (a.get('type') or '').strip().lower() in self.JS
|
||||
self.current = [] if inline else None
|
||||
|
||||
def handle_data(self, data):
|
||||
if self.current is not None:
|
||||
self.current.append(data)
|
||||
|
||||
def handle_endtag(self, tag):
|
||||
if tag == 'script' and self.current is not None:
|
||||
self.scripts.append(''.join(self.current))
|
||||
if tag == 'script':
|
||||
self.current = None
|
||||
|
||||
|
||||
def classic_scripts(page):
|
||||
p = ClassicScripts()
|
||||
p.feed(page)
|
||||
return p.scripts
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which('node'), 'Node parses the page scripts')
|
||||
class PageScripts(unittest.TestCase):
|
||||
def test_no_top_level_name_is_declared_twice(self):
|
||||
scripts = classic_scripts((ROOT / 'ui/index.html').read_text(encoding='utf-8'))
|
||||
self.assertGreaterEqual(len(scripts), 2)
|
||||
self.assertEqual(check(scripts), 'ok')
|
||||
|
||||
def test_the_page_declares_nothing_with_var(self):
|
||||
scripts = classic_scripts((ROOT / 'ui/index.html').read_text(encoding='utf-8'))
|
||||
self.assertEqual(check_vars(scripts), 'ok')
|
||||
|
||||
def test_the_check_finds_what_it_should(self):
|
||||
twice = {
|
||||
'indented function': ['function loadPanels() {}', ' async function loadPanels() {}'],
|
||||
'class': ['class Panel {}', 'class Panel {}'],
|
||||
'destructured': ['const { a, b } = {};', 'let [b] = [];'],
|
||||
'later declarator': ['let x = 1;', 'const y = 2, x = 3;'],
|
||||
'function and const': ['function f() {}', 'const f = 1;'],
|
||||
}
|
||||
for what, scripts in twice.items():
|
||||
with self.subTest(what):
|
||||
self.assertIn('already been declared', check(scripts))
|
||||
helpers = ['function a() { function help() {} }', 'function b() { const help = 1; }']
|
||||
self.assertEqual(check(helpers), 'ok')
|
||||
|
||||
def test_the_var_check_finds_what_it_should(self):
|
||||
# var: legal to declare twice, so not allowed at all; however it's written.
|
||||
for code in ['var x = 1;', 'var/*c*/x = 1;', 'var {x} = {x: 1};', 'function f() { var y; }']:
|
||||
with self.subTest(code):
|
||||
self.assertIn('declares with var', check_vars([code]))
|
||||
# The word var elsewhere is fine.
|
||||
text = ['// var x\nconst a = "var y", b = `var ${a}`, c = "color: var(--blue)", d = {}.var;']
|
||||
self.assertEqual(check_vars(text), 'ok')
|
||||
|
||||
def test_only_inline_classic_scripts_are_checked(self):
|
||||
page = ('<script>let a;</script><script type="module">export const m = 1;</script>'
|
||||
'<script type="application/json">{"b": 1}</script><script src = "x.js"></script>'
|
||||
'<script data-src="y" type="text/javascript">let c;</script>'
|
||||
'<script type="text/ecmascript">let d;</script><script type=" Application/X-JavaScript ">let e;</script>')
|
||||
self.assertEqual(classic_scripts(page), ['let a;', 'let c;', 'let d;', 'let e;'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -1,55 +0,0 @@
|
||||
"""The page calls showPage() while its first script is still running, before names declared
|
||||
later (in that script or the second one) exist. Touching one of them there throws, and the
|
||||
rest of the page's setup never runs: opening Frame Control at #games did exactly that."""
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
STUBS = ['$', 'toggleLive', 'scrollToY', 'loadMacView', 'loadPanels', 'loadPanelSwitcher', 'loadTitles', 'showAndroidTab']
|
||||
|
||||
RUN = r'''
|
||||
const vm = require('vm');
|
||||
const {code, hashes} = JSON.parse(require('fs').readFileSync(0, 'utf8'));
|
||||
const failures = [];
|
||||
for (const hash of hashes) {
|
||||
const context = vm.createContext({
|
||||
location: {hash}, document: {querySelectorAll: () => [], title: ''}, live: false,
|
||||
});
|
||||
try { vm.runInContext(code, context); }
|
||||
catch (e) { failures.push(`${hash}: ${e.message}`); }
|
||||
}
|
||||
console.log(JSON.stringify(failures));
|
||||
'''
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which('node'), 'Node runs the page code')
|
||||
class PageStartup(unittest.TestCase):
|
||||
def test_opening_any_page_or_section_at_startup_runs(self):
|
||||
page = (ROOT / 'ui/index.html').read_text(encoding='utf-8')
|
||||
first, second = re.findall(r'<script>(.*?)</script>', page, re.S)[:2]
|
||||
tables = first[first.index('const PAGES ='):first.index('let page =')]
|
||||
start = first.index('function showPage(')
|
||||
show = first[start:first.index('\n}\n', start) + 3]
|
||||
# Everything declared after the startup call is still uninitialised when it runs.
|
||||
call = re.search(r'^showPage\(\);', first, re.M).end()
|
||||
later = re.findall(r'^(?:const|let)\s+(\w+)', first[call:] + second, re.M)
|
||||
later = [n for n in dict.fromkeys(later) if n not in STUBS and n not in ('PAGES', 'SECTION_PAGE', 'page')]
|
||||
self.assertIn('link', later) # the name that broke #games
|
||||
stubs = ''.join(f'function {n}() {{ return {{ classList: {{ toggle() {{}} }}, scrollIntoView() {{}} }}; }}\n'
|
||||
for n in STUBS if n != '$')
|
||||
code = ('const $ = id => id === "nowhere" ? null : { classList: { toggle() {} }, scrollIntoView() {} };\n' + stubs + tables + 'let page = "home";\n' + show +
|
||||
'showPage();\n' + ''.join(f'let {n};\n' for n in later))
|
||||
hashes = ['', '#home', '#games', '#android', '#tools', '#settings', '#devices', '#control', '#nowhere']
|
||||
hashes += ['#' + k for k in re.findall(r'(\w+): "', tables)]
|
||||
r = subprocess.run(['node', '-e', RUN], input=json.dumps({'code': code, 'hashes': hashes}),
|
||||
capture_output=True, text=True)
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertEqual(json.loads(r.stdout), [])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,472 @@
|
||||
"""Eye-camera pulse estimate and the heart-rate comparison tool; no headset needed."""
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import random
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import zipfile
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load(name, path):
|
||||
spec = importlib.util.spec_from_file_location(name, ROOT / path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
pulse = load("pulse", "frame/tracking/pulse.py")
|
||||
check = load("heart_check", "scripts/heart-check.py")
|
||||
|
||||
|
||||
def synthetic(bpm, pulsing, seconds=40, fps=30, patches=48, noise=0.004, seed=7):
|
||||
"""Patch brightness with a faint pulse in `pulsing` patches, plus sensor
|
||||
noise, slow drift, blinks in some patches and eye movement in others."""
|
||||
rng = random.Random(seed)
|
||||
times = [i / fps for i in range(seconds * fps)]
|
||||
values = {}
|
||||
for k in range(patches):
|
||||
base, amplitude, trace = 40 + k, 0.003 if k < pulsing else 0.0, []
|
||||
for t in times:
|
||||
v = base * (1 + amplitude * math.sin(2 * math.pi * bpm / 60 * t)
|
||||
+ noise * rng.gauss(0, 1) + 0.02 * math.sin(0.1 * t + k))
|
||||
if k % 8 == 7 and int(t * 10) % 47 == 0:
|
||||
v *= 0.5 # blink
|
||||
if k % 8 == 6 and int(t * 10) % 23 == 0:
|
||||
v *= 1.3 # frequent eye movement
|
||||
trace.append(v)
|
||||
values[k] = trace
|
||||
return times, values
|
||||
|
||||
|
||||
class Estimate(unittest.TestCase):
|
||||
def test_finds_pulse(self):
|
||||
for bpm in (58, 72, 115):
|
||||
with self.subTest(bpm=bpm):
|
||||
result = pulse.estimate(*synthetic(bpm, 16))
|
||||
self.assertAlmostEqual(result["bpm"], bpm, delta=1.5)
|
||||
self.assertTrue(pulse.reliable(result))
|
||||
self.assertTrue(all(abs(b - bpm) <= 2 for _, b in result["series"]))
|
||||
|
||||
def test_noise_and_blinks_are_not_a_pulse(self):
|
||||
result = pulse.estimate(*synthetic(72, 0))
|
||||
self.assertFalse(pulse.reliable(result))
|
||||
|
||||
def test_frequent_spike_patches_are_dropped(self):
|
||||
times, values = synthetic(72, 16)
|
||||
result = pulse.estimate(times, values)
|
||||
self.assertLess(result["usable"], len(values))
|
||||
|
||||
def test_needs_enough_frames(self):
|
||||
with self.assertRaises(ValueError):
|
||||
pulse.estimate(*synthetic(72, 16, seconds=5))
|
||||
times, values = synthetic(72, 16)
|
||||
with self.assertRaises(ValueError):
|
||||
pulse.estimate(times, {k: [0.0] * len(times) for k in values}) # all dark
|
||||
|
||||
def test_series_times(self):
|
||||
times, values = synthetic(72, 16, seconds=20)
|
||||
series = pulse.estimate(times, values)["series"]
|
||||
self.assertAlmostEqual(series[0][0], pulse.WINDOW, delta=0.1)
|
||||
self.assertEqual(len(series), 20 - int(pulse.WINDOW) + 1)
|
||||
|
||||
def test_flat_patches_do_not_rank_first(self):
|
||||
times, values = synthetic(72, 16)
|
||||
values["flat"] = [100.0] * len(times)
|
||||
result = pulse.estimate(times, values)
|
||||
self.assertAlmostEqual(result["bpm"], 72, delta=1.5)
|
||||
self.assertEqual(pulse.snr([(60.0, 0.0), (61.0, 0.0)], 60), 0.0)
|
||||
|
||||
def test_analyse_uses_nearest_frame(self):
|
||||
times, values = synthetic(72, 4, patches=4, seconds=10)
|
||||
# Right-eye frames 1 ms before each left frame: nearest is that frame.
|
||||
frames = [(t, "left", [values[k][i] for k in range(4)]) for i, t in enumerate(times)]
|
||||
frames += [(t - 0.001, "right", [float(i)] * 4) for i, t in enumerate(times)]
|
||||
seen = {}
|
||||
original = pulse.estimate
|
||||
with patch.object(pulse, "estimate", lambda t, p: seen.update(p) or original(t, p)):
|
||||
pulse.analyse(frames)
|
||||
self.assertEqual(seen["right0"][:5], [0.0, 0.0, 1.0, 1.0, 2.0])
|
||||
|
||||
def test_fft_matches_dft(self):
|
||||
signal = [math.sin(i) + (i % 3) for i in range(16)]
|
||||
fast = pulse.fft(signal)
|
||||
for k in range(16):
|
||||
slow = sum(signal[n] * complex(math.cos(2 * math.pi * k * n / 16), -math.sin(2 * math.pi * k * n / 16))
|
||||
for n in range(16))
|
||||
self.assertAlmostEqual(abs(fast[k] - slow), 0, places=9)
|
||||
|
||||
def test_grid_means(self):
|
||||
# 4 × 4 image in RGB with padding: left half 10, right half 30 (channel 0).
|
||||
width, height, channels, stride = 4, 4, 3, 16
|
||||
pixels = bytearray(stride * height)
|
||||
for y in range(height):
|
||||
for x in range(width):
|
||||
pixels[y * stride + x * channels] = 10 if x < 2 else 30
|
||||
pixels[y * stride + x * channels + 1] = 255 # other channels ignored
|
||||
self.assertEqual(pulse.grid_means(bytes(pixels), width, height, stride, channels, grid=2),
|
||||
[10, 30, 10, 30])
|
||||
|
||||
def test_analyse_combines_both_eyes(self):
|
||||
times, values = synthetic(80, 16, patches=16)
|
||||
frames = []
|
||||
for i, t in enumerate(times):
|
||||
frames.append((t, "left", [values[k][i] for k in range(16)]))
|
||||
frames.append((t + 0.001, "right", [values[k][i] for k in range(16)]))
|
||||
result = pulse.analyse(frames)
|
||||
self.assertAlmostEqual(result["bpm"], 80, delta=1.5)
|
||||
self.assertEqual(result["usable"] % 2, 0)
|
||||
|
||||
|
||||
class FakeCaptureTool:
|
||||
"""Stands in for `eyetracking --calib`: writes PNG names over a few polls."""
|
||||
def __init__(self, directory, frames=6, fail=False):
|
||||
self.directory, self.frames, self.fail = Path(directory), frames, fail
|
||||
self.polls = 0
|
||||
self.returncode = None
|
||||
self.stdout = None
|
||||
self.announce = None # a different directory to report, if set
|
||||
|
||||
def __call__(self, command, cwd, stdout, stderr):
|
||||
self.command = command
|
||||
self.stdout = stdout
|
||||
if self.fail:
|
||||
stdout.write("Failed to initialize cameras\n")
|
||||
stdout.flush()
|
||||
self.returncode = 1
|
||||
return self
|
||||
# Real output is block-buffered until exit, so nothing is printed here.
|
||||
stdout.flush()
|
||||
self.directory.mkdir()
|
||||
return self
|
||||
|
||||
def poll(self):
|
||||
if self.returncode is not None:
|
||||
return self.returncode
|
||||
if self.polls < self.frames:
|
||||
for eye in ("left", "right"):
|
||||
(self.directory / f"{eye}_{self.polls}.png").write_bytes(b"png")
|
||||
self.polls += 1
|
||||
return None
|
||||
meta = {"frames": [{eye: {"fname": str(self.directory / f"{eye}_{i}.png"), "frameNum": i + 10,
|
||||
"tsMono": 100 + i / 90, "valid": i != 2} for eye in ("left", "right")}
|
||||
for i in range(self.frames)]}
|
||||
(self.directory / "meta.json").write_text(json.dumps(meta))
|
||||
# The real tool's buffered output only appears once it exits.
|
||||
self.stdout.write(f"Writing capture to: {self.announce or self.directory}\nCaptured images\n")
|
||||
self.stdout.flush()
|
||||
self.returncode = 0
|
||||
return 0
|
||||
|
||||
def terminate(self):
|
||||
raise AssertionError("the capture tool must never be signalled")
|
||||
|
||||
kill = terminate
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return self.returncode
|
||||
|
||||
|
||||
class WornCheck(unittest.TestCase):
|
||||
def sensor(self, name, raw):
|
||||
root = Path(self.enterContext(tempfile.TemporaryDirectory()))
|
||||
device = root / "iio:device2"
|
||||
device.mkdir()
|
||||
(device / "name").write_text(name + "\n")
|
||||
(device / "in_proximity_raw").write_text(raw + "\n")
|
||||
return root
|
||||
|
||||
def test_reads_the_proximity_sensor(self):
|
||||
self.assertEqual(pulse.proximity(self.sensor("vcnl4000", "3.250000000")), 3.25)
|
||||
|
||||
def test_unreadable_sensor_does_not_block(self):
|
||||
self.assertIsNone(pulse.proximity(self.sensor("other", "9")))
|
||||
self.assertIsNone(pulse.proximity(self.sensor("vcnl4000", "junk")))
|
||||
self.assertIsNone(pulse.proximity(Path("/nonexistent")))
|
||||
self.assertTrue(pulse.worn(None))
|
||||
|
||||
def test_low_reading_means_unworn(self):
|
||||
self.assertFalse(pulse.worn(3.1))
|
||||
self.assertTrue(pulse.worn(pulse.WORN_MIN))
|
||||
|
||||
|
||||
class CaptureLifecycle(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.root = Path(tempfile.mkdtemp())
|
||||
self.directory = self.root / "etcalib_test"
|
||||
(self.root / "etcalib_older").mkdir() # an earlier capture is not ours
|
||||
self.loaded = []
|
||||
|
||||
def tearDown(self):
|
||||
import shutil
|
||||
shutil.rmtree(self.root, ignore_errors=True)
|
||||
|
||||
def loader(self, path):
|
||||
self.loaded.append(Path(path).name)
|
||||
self.assertTrue(Path(path).exists())
|
||||
return [float(len(self.loaded))]
|
||||
|
||||
def capture(self, tool):
|
||||
class TestCapture(pulse.Capture):
|
||||
# A temporary directory stands in for /tmp/etcalib_*.
|
||||
PREFIX = str(self.root / "etcalib_")
|
||||
capture = TestCapture(20, runner=tool, loader=self.loader, workers=0)
|
||||
with patch.object(pulse.time, "sleep", lambda s: None):
|
||||
return capture, capture.run()
|
||||
|
||||
def test_reduces_deletes_and_joins_metadata(self):
|
||||
tool = FakeCaptureTool(self.directory)
|
||||
capture, frames = self.capture(tool)
|
||||
self.assertEqual(sorted(self.loaded), sorted(f"{e}_{i}.png" for e in ("left", "right") for i in range(6)))
|
||||
self.assertFalse(self.directory.exists()) # images and metadata removed
|
||||
self.assertTrue((self.root / "etcalib_older").exists())
|
||||
self.assertEqual(len(frames), 10) # frame 2 invalid in both eyes
|
||||
self.assertEqual(tool.command[-2:], ["--calib", "20"])
|
||||
self.assertTrue(all(isinstance(t, float) and eye in ("left", "right") for t, eye, _ in frames))
|
||||
|
||||
def test_camera_failure(self):
|
||||
tool = FakeCaptureTool(self.directory, fail=True)
|
||||
with self.assertRaises(RuntimeError) as raised:
|
||||
self.capture(tool)
|
||||
self.assertIn("cameras unavailable", str(raised.exception))
|
||||
|
||||
def test_backlog_abandons_capture_and_removes_images(self):
|
||||
class Stalled(FakeCaptureTool):
|
||||
def poll(self):
|
||||
if self.polls > 8:
|
||||
self.returncode = 0 # the bounded run ends by itself
|
||||
return 0
|
||||
for i in range(20):
|
||||
for eye in ("left", "right"):
|
||||
(self.directory / f"{eye}_{self.polls * 20 + i}.png").write_bytes(b"png")
|
||||
self.polls += 1
|
||||
return None
|
||||
tool = Stalled(self.directory)
|
||||
class TestCapture(pulse.Capture):
|
||||
PREFIX = str(self.root / "etcalib_")
|
||||
MAX_BACKLOG = 50
|
||||
capture = TestCapture(20, runner=tool, loader=self.loader, workers=0)
|
||||
capture.reduce = lambda final=False, original=capture.reduce: (
|
||||
original(final) if tool.polls > 3 else None) # reduction stalls
|
||||
with patch.object(pulse.time, "sleep", lambda s: None), self.assertRaises(RuntimeError) as raised:
|
||||
capture.run()
|
||||
self.assertIn("fell behind", str(raised.exception))
|
||||
self.assertEqual(tool.returncode, 0) # left to end by itself, never signalled
|
||||
self.assertFalse(self.directory.exists()) # no image left behind
|
||||
|
||||
def test_second_capture_directory_stops_and_removes_both(self):
|
||||
foreign = self.root / "etcalib_foreign"
|
||||
class Racing(FakeCaptureTool):
|
||||
def poll(self):
|
||||
if self.polls == 2:
|
||||
foreign.mkdir()
|
||||
(foreign / "left_0.png").write_bytes(b"png")
|
||||
return super().poll()
|
||||
tool = Racing(self.directory)
|
||||
with self.assertRaises(RuntimeError) as raised:
|
||||
self.capture(tool)
|
||||
self.assertIn("another eye-camera capture", str(raised.exception))
|
||||
self.assertFalse(self.directory.exists())
|
||||
self.assertFalse(foreign.exists()) # no eye image outlives the run
|
||||
self.assertTrue((self.root / "etcalib_older").exists())
|
||||
|
||||
def test_images_elsewhere_are_not_used_but_are_removed(self):
|
||||
tool = FakeCaptureTool(self.directory)
|
||||
tool.announce = self.root / "etcalib_reported"
|
||||
tool.announce.mkdir()
|
||||
(tool.announce / "left_0.png").write_bytes(b"png")
|
||||
with self.assertRaises(RuntimeError) as raised:
|
||||
self.capture(tool)
|
||||
self.assertIn("somewhere else", str(raised.exception))
|
||||
self.assertFalse(tool.announce.exists())
|
||||
self.assertFalse(self.directory.exists())
|
||||
|
||||
def test_interrupt_during_cleanup_still_removes_images(self):
|
||||
class Stubborn(FakeCaptureTool):
|
||||
interrupts = 0
|
||||
def poll(self):
|
||||
if self.polls == 2 and self.interrupts < 2:
|
||||
self.interrupts += 1
|
||||
raise KeyboardInterrupt # Ctrl-C mid-capture, and again while waiting
|
||||
return super().poll()
|
||||
tool = Stubborn(self.directory)
|
||||
with self.assertRaises(KeyboardInterrupt):
|
||||
self.capture(tool)
|
||||
self.assertEqual(tool.returncode, 0) # still let finish, never signalled
|
||||
self.assertFalse(self.directory.exists())
|
||||
|
||||
def test_overrunning_tool_is_killed_as_a_last_resort(self):
|
||||
class Hung(FakeCaptureTool):
|
||||
killed = False
|
||||
def poll(self):
|
||||
(self.directory / "left_0.png").write_bytes(b"png")
|
||||
return None if not self.killed else -9
|
||||
def kill(self):
|
||||
self.killed = True
|
||||
tool = Hung(self.directory)
|
||||
clock = iter(range(0, 10000, 20))
|
||||
with patch.object(pulse.time, "monotonic", lambda: next(clock)), \
|
||||
self.assertRaises(RuntimeError) as raised:
|
||||
self.capture(tool)
|
||||
self.assertIn("may need a reboot", str(raised.exception))
|
||||
self.assertTrue(tool.killed)
|
||||
self.assertFalse(self.directory.exists())
|
||||
|
||||
def test_cleanup_tries_every_directory(self):
|
||||
capture = pulse.Capture(20)
|
||||
capture.PREFIX = str(self.root / "etcalib_")
|
||||
first, second = self.root / "etcalib_a", self.root / "etcalib_b"
|
||||
for directory in (first, second):
|
||||
directory.mkdir()
|
||||
(directory / "left_0.png").write_bytes(b"png")
|
||||
capture.new = {first, second}
|
||||
real = pulse.shutil.rmtree
|
||||
def flaky(path):
|
||||
if Path(path) == first:
|
||||
raise OSError("busy")
|
||||
real(path)
|
||||
with patch.object(pulse.shutil, "rmtree", flaky), self.assertRaises(RuntimeError) as raised:
|
||||
capture.remove()
|
||||
self.assertIn("etcalib_a", str(raised.exception))
|
||||
self.assertFalse(second.exists())
|
||||
|
||||
def test_removes_a_directory_seen_only_at_the_end(self):
|
||||
capture = pulse.Capture(20)
|
||||
capture.PREFIX = str(self.root / "etcalib_")
|
||||
capture.before = capture.candidates()
|
||||
late = self.root / "etcalib_late"
|
||||
late.mkdir()
|
||||
(late / "left_0.png").write_bytes(b"png")
|
||||
capture.remove()
|
||||
self.assertFalse(late.exists())
|
||||
self.assertTrue((self.root / "etcalib_older").exists())
|
||||
|
||||
def test_never_adopts_directories_without_a_baseline(self):
|
||||
capture = pulse.Capture(20)
|
||||
capture.PREFIX = str(self.root / "etcalib_")
|
||||
capture.remove()
|
||||
self.assertTrue((self.root / "etcalib_older").exists())
|
||||
|
||||
def test_only_removes_capture_directories(self):
|
||||
capture = pulse.Capture(20)
|
||||
capture.directory = self.root
|
||||
capture.remove()
|
||||
self.assertTrue(self.root.exists())
|
||||
|
||||
|
||||
class HeartCheck(unittest.TestCase):
|
||||
def write(self, name, text):
|
||||
path = Path(self.enterContext(tempfile.TemporaryDirectory())) / name
|
||||
path.write_text(text)
|
||||
return path
|
||||
|
||||
def test_osc_round_trip(self):
|
||||
tracking = load("tracking", "frame/tracking/tracking.py")
|
||||
self.assertEqual(check.parse_osc(tracking.osc_message("/avatar/parameters/HeartRate", [72])),
|
||||
("/avatar/parameters/HeartRate", [72]))
|
||||
address, values = check.parse_osc(tracking.osc_message("/x", [1.5, -2.0]))
|
||||
self.assertEqual((address, values), ("/x", [1.5, -2.0]))
|
||||
with self.assertRaises(ValueError):
|
||||
check.parse_osc(b"/x\0\0,s\0\0abc\0")
|
||||
|
||||
def test_listen_records_readings(self):
|
||||
tracking = load("tracking", "frame/tracking/tracking.py")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
out = Path(directory) / "ours.csv"
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as probe:
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
import threading
|
||||
def send():
|
||||
import time
|
||||
time.sleep(0.3)
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sender:
|
||||
sender.sendto(tracking.osc_message(check.ADDRESS, [72]), ("127.0.0.1", port))
|
||||
sender.sendto(tracking.osc_message("/other", [1]), ("127.0.0.1", port))
|
||||
threading.Thread(target=send).start()
|
||||
shown = io.StringIO()
|
||||
count = check.listen(port, check.ADDRESS, str(out), 1.5, stream=shown)
|
||||
self.assertEqual(count, 1)
|
||||
self.assertIn("72 bpm", shown.getvalue())
|
||||
self.assertEqual(out.read_text().splitlines()[1].split(",")[1], "72")
|
||||
self.assertEqual(out.stat().st_mode & 0o777, 0o600)
|
||||
|
||||
def test_compare_finds_lag_and_contact_loss(self):
|
||||
reference = [(1000.0 + i, 60 + i, 6) for i in range(40)] + [(1040.0 + i, 150, 4) for i in range(5)]
|
||||
ours = [(1002.0 + i, 60 + i) for i in range(40)] # 2 s late, nothing during contact loss
|
||||
ref = self.write("ref.csv", "unix_seconds,bpm,flags\n" + "".join(f"{t},{b},{f}\n" for t, b, f in reference))
|
||||
mine = self.write("ours.csv", "unix_seconds,bpm\n" + "".join(f"{t},{b}\n" for t, b in ours))
|
||||
result = check.compare(check.read_csv(mine), check.read_any(ref))
|
||||
self.assertEqual(result["lag_seconds"], 2.0)
|
||||
self.assertEqual(result["mean_abs_error"], 0)
|
||||
self.assertEqual(result["shown_during_no_contact"], 0)
|
||||
self.assertTrue(check.report(result, 5, stream=io.StringIO()))
|
||||
# A stale reading sent during contact loss fails the check.
|
||||
stale = check.read_csv(mine) + [(1043.0, 99)]
|
||||
result = check.compare(stale, check.read_any(ref))
|
||||
self.assertEqual(result["shown_during_no_contact"], 1)
|
||||
self.assertFalse(check.report(result, 5, stream=io.StringIO()))
|
||||
|
||||
def test_compare_against_apple_health_export(self):
|
||||
records = "".join(
|
||||
f'<Record type="HKQuantityTypeIdentifierHeartRate" unit="count/min" '
|
||||
f'startDate="2026-09-29 12:00:{s:02d} +1000" endDate="2026-09-29 12:00:{s:02d} +1000" value="{70 + s % 3}"/>'
|
||||
for s in range(0, 60, 5))
|
||||
other = '<Record type="HKQuantityTypeIdentifierStepCount" startDate="2026-09-29 12:00:00 +1000" value="9"/>'
|
||||
xml = f'<?xml version="1.0"?><HealthData>{other}{records}</HealthData>'
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
archive = Path(directory) / "export.zip"
|
||||
with zipfile.ZipFile(archive, "w") as z:
|
||||
z.writestr("apple_health_export/export.xml", xml)
|
||||
start = check.parse_time("2026-09-29 12:00:00 +1000")
|
||||
samples = check.read_any(archive, start - 60, start + 120)
|
||||
self.assertEqual(len(samples), 12)
|
||||
self.assertEqual(samples[0], (start, 70))
|
||||
ours = [(start + i + 1.0, 71) for i in range(60)]
|
||||
result = check.compare(ours, samples)
|
||||
self.assertLessEqual(result["mean_abs_error"], 1)
|
||||
|
||||
def test_unusual_flags_and_missing_export(self):
|
||||
path = self.write("ref.csv", "time,bpm,flags\n1000,70,6.0\n1001,71,yes\n1002,72,4\n")
|
||||
self.assertEqual(check.read_csv(path), [(1000.0, 70), (1001.0, 71), (1002.0, None)])
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
archive = Path(directory) / "other.zip"
|
||||
with zipfile.ZipFile(archive, "w") as z:
|
||||
z.writestr("notes.txt", "x")
|
||||
with self.assertRaises(ValueError):
|
||||
check.read_any(archive, 0, 1)
|
||||
|
||||
def test_compare_reports_bad_input_without_traceback(self):
|
||||
good = self.write("ref.csv", "1000,70\n")
|
||||
shown = io.StringIO()
|
||||
with patch("sys.stdout", shown):
|
||||
self.assertEqual(check.main(["compare", str(good.parent / "missing.csv"), str(good)]), 1)
|
||||
self.assertEqual(check.main(["compare", str(self.write("ours.csv", "1000,inf\n1001,70\n")),
|
||||
str(self.write("bad.xml", "<not closed"))]), 1)
|
||||
self.assertEqual(shown.getvalue().count("Could not compare"), 2)
|
||||
|
||||
def test_health_records_with_non_finite_values_are_skipped(self):
|
||||
record = '<Record type="HKQuantityTypeIdentifierHeartRate" startDate="2026-09-29 12:00:00 +1000" value="%s"/>'
|
||||
path = self.write("export.xml", "<HealthData>" + record % "inf" + record % "72" + "</HealthData>")
|
||||
start = check.parse_time("2026-09-29 12:00:00 +1000")
|
||||
self.assertEqual(check.read_health(path, start - 1, start + 1), [(start, 72)])
|
||||
|
||||
def test_time_formats(self):
|
||||
self.assertEqual(check.parse_time("1700000000.5"), 1700000000.5)
|
||||
self.assertEqual(check.parse_time("2023-11-14T22:13:20Z"), 1700000000)
|
||||
self.assertEqual(check.parse_time("2023-11-15 08:13:20 +1000"), 1700000000)
|
||||
with self.assertRaises(ValueError):
|
||||
check.parse_time("yesterday")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -150,14 +150,6 @@ class Apply(unittest.TestCase):
|
||||
self.assertFalse(self.t.STALE[0])
|
||||
self.assertEqual(self.said[-1][1].get("stale"), None)
|
||||
|
||||
def test_stale_survives_a_bare_release(self):
|
||||
gs = FakeGamescope()
|
||||
panel = self.t.apply(gs, {"button": "left", "down": True, "window": 99, "display": ":1"}, None)
|
||||
self.assertTrue(self.t.STALE[0])
|
||||
self.t.apply(gs, {"button": "left", "down": False}, panel) # the page's release names no panel
|
||||
self.assertTrue(self.t.STALE[0])
|
||||
self.assertEqual(self.said[-1][1].get("stale"), True)
|
||||
|
||||
def test_same_window_id_on_the_other_display_is_another_panel(self):
|
||||
gs = FakeGamescope()
|
||||
self.t.apply(gs, {"fx": 0.5, "fy": 0.5, "window": 7, "display": ":0"}, None)
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
"""Tracking protocols and fake-Frame BlueZ lifecycle; no headset or strap needed."""
|
||||
import importlib.util
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import stat
|
||||
import struct
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
SPEC = importlib.util.spec_from_file_location("tracking", Path(__file__).resolve().parents[1] / "frame/tracking/tracking.py")
|
||||
t = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(t)
|
||||
|
||||
|
||||
class Protocols(unittest.TestCase):
|
||||
def test_hrs_formats(self):
|
||||
self.assertEqual(t.heart_rate(b"\x00\x48"), 72)
|
||||
self.assertEqual(t.heart_rate(b"\x01\x2c\x01"), 300)
|
||||
self.assertEqual(t.heart_rate(b"\x1e\x48\x01\x00\x00\x04\x00\x04"), 72)
|
||||
self.assertEqual(t.heart_rate(b"\x02\x48"), 72) # contact not supported
|
||||
self.assertIsNone(t.heart_rate(b"\x04\x48")) # no contact
|
||||
self.assertIsNone(t.heart_rate(b"\x00\x00"))
|
||||
|
||||
def test_hrs_malformed(self):
|
||||
for packet in (b"", b"\x00", b"\x01\x48", b"\x08\x48\x00", b"\x10\x48",
|
||||
b"\x10\x48\x00", b"\x00\x48\x01", b"\xe0\x48"):
|
||||
with self.subTest(packet=packet), self.assertRaises(ValueError):
|
||||
t.heart_rate(packet)
|
||||
|
||||
def test_gaze_coordinates(self):
|
||||
self.assertEqual(t.gaze_angles([0, 0, 0, 1]), (0, 0))
|
||||
angle = math.radians(15)
|
||||
pitch, yaw = t.gaze_angles([math.sin(angle), 0, 0, math.cos(angle)])
|
||||
self.assertAlmostEqual(pitch, -30) # OpenXR +X rotation looks up
|
||||
self.assertAlmostEqual(yaw, 0)
|
||||
pitch, yaw = t.gaze_angles([0, -math.sin(angle), 0, math.cos(angle)])
|
||||
self.assertAlmostEqual(pitch, 0)
|
||||
self.assertAlmostEqual(yaw, 30) # right
|
||||
|
||||
def test_invalid_gaze(self):
|
||||
for pose in ([0, 0, 0, 0], [math.nan, 0, 0, 1], [0, 0, 0], [0, 0, 0, math.inf]):
|
||||
with self.assertRaises(ValueError):
|
||||
t.gaze_angles(pose)
|
||||
|
||||
def test_osc_wire(self):
|
||||
self.assertEqual(t.osc_message('/x', [72]), b'/x\0\0,i\0\0' + struct.pack('>i', 72))
|
||||
self.assertEqual(t.osc_message('/x', [1.0, -2.0]), b'/x\0\0,ff\0' + struct.pack('>ff', 1, -2))
|
||||
for address in ('x', '/x\0y', '/x y', '/x*'):
|
||||
with self.assertRaises(ValueError):
|
||||
t.osc_message(address, [1])
|
||||
|
||||
def test_default_never_opens_socket(self):
|
||||
with patch.object(t.socket, 'socket') as create:
|
||||
osc = t.Osc()
|
||||
osc.send('/x', [72])
|
||||
osc.close()
|
||||
create.assert_not_called()
|
||||
|
||||
def test_only_configured_endpoint(self):
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as receiver:
|
||||
receiver.bind(('127.0.0.1', 0))
|
||||
receiver.settimeout(1)
|
||||
osc = t.Osc(receiver.getsockname())
|
||||
try:
|
||||
osc.send('/tracking/eye/CenterPitchYaw', [0.0, 30.0])
|
||||
self.assertEqual(receiver.recv(1024), t.osc_message('/tracking/eye/CenterPitchYaw', [0.0, 30.0]))
|
||||
finally:
|
||||
osc.close()
|
||||
|
||||
def test_endpoint_validation(self):
|
||||
for endpoint in [('example.org', 9000), ('0.0.0.0', 9000), ('224.0.0.1', 9000), ('127.0.0.1', 0), ('::1', 65536)]:
|
||||
with self.assertRaises(ValueError):
|
||||
t.Osc(endpoint)
|
||||
|
||||
def test_heart_staleness_contact_and_log(self):
|
||||
now = [0]
|
||||
osc = Mock()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'session.csv'
|
||||
session = t.HeartSession(osc, '/hr', path, lambda: now[0])
|
||||
self.assertIsNone(session.current())
|
||||
session.notification(b'\x00\x48')
|
||||
self.assertEqual(session.current(), 72)
|
||||
osc.send.assert_called_once_with('/hr', [72])
|
||||
now[0] = 6
|
||||
self.assertIsNone(session.current())
|
||||
session.notification(b'\x04\x48')
|
||||
self.assertIsNone(session.current())
|
||||
self.assertEqual(osc.send.call_count, 1)
|
||||
session.close()
|
||||
self.assertEqual(path.read_text().splitlines()[0], 'unix_seconds,bpm')
|
||||
self.assertEqual(len(path.read_text().splitlines()), 2)
|
||||
if os.name != 'nt':
|
||||
self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600)
|
||||
with self.assertRaises(FileExistsError):
|
||||
t.HeartSession(osc, '/hr', path)
|
||||
|
||||
def test_no_log_by_default(self):
|
||||
with patch.object(t.os, 'open') as create:
|
||||
session = t.HeartSession(Mock(), '/hr')
|
||||
session.notification(b'\x00\x48')
|
||||
session.close()
|
||||
create.assert_not_called()
|
||||
|
||||
|
||||
class FakeBluez(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.device = '/org/bluez/hci0/dev_TEST'
|
||||
self.service = self.device + '/service1'
|
||||
self.char = self.service + '/char1'
|
||||
self.objects = {
|
||||
self.device: {t.DEVICE: {'Address': 'AA:BB:CC:DD:EE:FF', 'Connected': False, 'ServicesResolved': True}},
|
||||
self.service: {t.SERVICE: {'UUID': t.HRS, 'Device': self.device}},
|
||||
self.char: {t.CHARACTERISTIC: {'UUID': t.MEASUREMENT, 'Service': self.service, 'Flags': ['notify']}},
|
||||
}
|
||||
self.api = Mock()
|
||||
self.api.GetManagedObjects.side_effect = lambda: self.objects
|
||||
self.bus = Mock()
|
||||
self.interface = Mock(return_value=self.api)
|
||||
self.values = Mock()
|
||||
|
||||
def reader(self):
|
||||
return t.BluezHeart(self.bus, self.interface, 'AA:BB:CC:DD:EE:FF', self.values)
|
||||
|
||||
def test_subscribe_receive_and_cleanup(self):
|
||||
reader = self.reader()
|
||||
self.api.Connect.assert_called_once()
|
||||
self.assertTrue(reader.subscribe())
|
||||
self.api.StartNotify.assert_called_once()
|
||||
reader.changed(t.CHARACTERISTIC, {'Value': [0, 72]}, [], self.char)
|
||||
self.values.assert_called_once_with([0, 72])
|
||||
reader.changed(t.CHARACTERISTIC, {'Value': [0, 73]}, [], '/other/strap')
|
||||
self.assertEqual(self.values.call_count, 1)
|
||||
reader.close()
|
||||
self.api.StopNotify.assert_called_once()
|
||||
self.api.Disconnect.assert_called_once()
|
||||
self.bus.add_signal_receiver.return_value.remove.assert_called_once()
|
||||
|
||||
def test_preserve_existing_connection(self):
|
||||
self.objects[self.device][t.DEVICE]['Connected'] = True
|
||||
reader = self.reader()
|
||||
reader.subscribe()
|
||||
reader.close()
|
||||
self.api.Connect.assert_not_called()
|
||||
self.api.Disconnect.assert_not_called()
|
||||
|
||||
def test_only_selected_device_service(self):
|
||||
self.objects[self.service][t.SERVICE]['Device'] = '/other/device'
|
||||
reader = self.reader()
|
||||
try:
|
||||
with self.assertRaises(RuntimeError):
|
||||
reader.subscribe()
|
||||
finally:
|
||||
reader.close()
|
||||
self.api.StartNotify.assert_not_called()
|
||||
|
||||
def test_wait_for_services(self):
|
||||
self.objects[self.device][t.DEVICE]['ServicesResolved'] = False
|
||||
reader = self.reader()
|
||||
self.assertFalse(reader.subscribe())
|
||||
reader.close()
|
||||
self.api.StartNotify.assert_not_called()
|
||||
self.api.StopNotify.assert_not_called()
|
||||
|
||||
def test_disconnect_notification(self):
|
||||
reader = self.reader()
|
||||
reader.changed(t.DEVICE, {'Connected': 0}, [], self.device) # dbus.Boolean behaves as int
|
||||
self.values.assert_called_once_with(None)
|
||||
reader.close()
|
||||
|
||||
def test_failed_notify_cleans_connection(self):
|
||||
reader = self.reader()
|
||||
self.api.StartNotify.side_effect = RuntimeError('failure')
|
||||
with self.assertRaises(RuntimeError):
|
||||
reader.subscribe()
|
||||
reader.close()
|
||||
self.api.StopNotify.assert_not_called()
|
||||
self.api.Disconnect.assert_called_once()
|
||||
|
||||
def test_unknown_device_does_not_connect_or_scan(self):
|
||||
self.objects.clear()
|
||||
with self.assertRaises(RuntimeError):
|
||||
self.reader()
|
||||
self.api.Connect.assert_not_called()
|
||||
self.api.StartDiscovery.assert_not_called()
|
||||
+2
-4
@@ -740,9 +740,7 @@ class Registry:
|
||||
return a
|
||||
raise DeviceError(f"{host} isn't one of this headset's addresses")
|
||||
|
||||
def add_address(self, device_id, host, kind=None, label="", first=False):
|
||||
"""Add an address at the end of the list, or at the front (first=True), where the
|
||||
user's order makes it win over the others that work on the same network."""
|
||||
def add_address(self, device_id, host, kind=None, label=""):
|
||||
with self._changing():
|
||||
d = self._find(device_id)
|
||||
a = new_address(host, kind, label)
|
||||
@@ -750,7 +748,7 @@ class Registry:
|
||||
raise DeviceError(f"{a['host']} is already on the list")
|
||||
if len(d["addresses"]) >= 32:
|
||||
raise DeviceError("That's enough addresses for one headset")
|
||||
d["addresses"].insert(0 if first else len(d["addresses"]), a)
|
||||
d["addresses"].append(a)
|
||||
self.save()
|
||||
return copy.deepcopy(a)
|
||||
|
||||
|
||||
+1
-7
@@ -356,7 +356,6 @@ class Link:
|
||||
self.version = 0
|
||||
self.stopped = False
|
||||
self.kicks = [] # reasons someone asked for a (re)connect
|
||||
self.test_gen = {} # device id -> its newest test of the addresses (see test())
|
||||
self.busy = False # the loop is handling kicks
|
||||
self.state = {"phase": "idle", "reason": None, "device": None, "network": None, "stages": [],
|
||||
"probes": [], "via": None, "error": None, "retry_at": None, "attempt": 0,
|
||||
@@ -1307,13 +1306,9 @@ class Link:
|
||||
started = now()
|
||||
rows = [{"host": a["host"], "kind": a["kind"], "state": "waiting", "detail": "Waiting", "ip": None,
|
||||
"rtt_ms": None, "ssh": None} for a in device["addresses"]]
|
||||
with self.cond:
|
||||
gen = self.test_gen[device_id] = self.test_gen.get(device_id, 0) + 1
|
||||
|
||||
def put(**fields):
|
||||
with self.cond:
|
||||
if gen != self.test_gen[device_id]:
|
||||
return # a newer test has started: its results are the ones to show
|
||||
self.state["tests"][device_id] = dict({"started": started, "done": False, "rows": rows}, **fields)
|
||||
self.version += 1
|
||||
self.cond.notify_all()
|
||||
@@ -1622,8 +1617,7 @@ def devices_action(link, body, open_setup, busy=lambda: 0):
|
||||
raise frame_devices.DeviceError(f"Removed, but couldn't edit ~/.ssh/config: {e}")
|
||||
msg = f"Removed {d['name']}" + (f" and its '{d['alias']}' entry in ~/.ssh/config" if removed else "")
|
||||
elif action == "address-add":
|
||||
a = reg.add_address(did, body.get("host"), body.get("kind") or None, body.get("label") or "",
|
||||
first=body.get("first") is True)
|
||||
a = reg.add_address(did, body.get("host"), body.get("kind") or None, body.get("label") or "")
|
||||
if is_active and link.state["phase"] == "failed":
|
||||
link.kick("retry")
|
||||
msg = f"Added {a['host']}"
|
||||
|
||||
+1
-3
@@ -506,10 +506,8 @@ def apply(gs, event, panel):
|
||||
say("ready", focus=panel.get("window"), display=panel.get("display"), stale=True) # the page re-syncs
|
||||
STALE[0] = True
|
||||
return panel
|
||||
release = event.get("down") is False and ("button" in event or "key" in event)
|
||||
if STALE[0] and not stale and not (release and "window" not in event):
|
||||
if STALE[0] and not stale:
|
||||
# Anything that goes through (a trackpad move names no panel) means caught up: stop re-syncing.
|
||||
# A bare release doesn't: the page leaves the panel off releases, so it says nothing about focus.
|
||||
STALE[0] = False
|
||||
say("ready", focus=(panel or {}).get("window"), display=(panel or {}).get("display"))
|
||||
if "fx" in event and panel and panel.get("window"):
|
||||
|
||||
+293
-646
File diff suppressed because it is too large.
Load diff
+6
-187
@@ -63,7 +63,6 @@ import frame_report # noqa: E402
|
||||
import frame_store # noqa: E402
|
||||
import frame_telemetry # noqa: E402
|
||||
import frame_titles # noqa: E402
|
||||
import frame_touch # noqa: E402
|
||||
import frame_webinstall # noqa: E402
|
||||
import frame_vr # noqa: E402
|
||||
import frame_utilities # noqa: E402
|
||||
@@ -116,9 +115,6 @@ def route(alias, host_opts):
|
||||
with _route_lock:
|
||||
moved = alias != FRAME
|
||||
if moved:
|
||||
reset_pad = globals().get("reset_pad_route")
|
||||
if reset_pad:
|
||||
reset_pad() # cancel old input before any agent can start on the new route
|
||||
frame_catalog._env.clear() # the SteamOS and Lepton builds reports record are per headset
|
||||
FRAME = frame_android.FRAME = alias
|
||||
HOST_OPTS = list(host_opts)
|
||||
@@ -129,6 +125,9 @@ def route(alias, host_opts):
|
||||
mv = globals().get("macview")
|
||||
if mv:
|
||||
mv.retarget(alias, host_opts) # its tunnel is its own ssh: it must follow the headset too
|
||||
agent = globals().get("_input")
|
||||
if moved and agent:
|
||||
agent.stop() # typing and pointing mustn't go on reaching the headset switched away from
|
||||
|
||||
|
||||
LINK = None # the connector (frame_link.Link); None on the Frame itself
|
||||
@@ -1169,192 +1168,12 @@ def licenses():
|
||||
return notices
|
||||
|
||||
|
||||
# KDE Connect's specialKey numbers -> Linux key codes (KEY_BACKSPACE, KEY_TAB, ...).
|
||||
PAD_KEYS = {1: 14, 2: 15, 4: 105, 5: 103, 6: 106, 7: 108, 8: 104, 9: 109, 10: 102, 11: 107, 12: 28, 13: 111, 14: 1,
|
||||
**{21 + i: code for i, code in enumerate([59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 87, 88])}}
|
||||
PAD_MODS = (("ctrl", 29), ("alt", 56), ("shift", 42), ("super", 125))
|
||||
PAD_CLICKS = {"singleclick": "left", "rightclick": "right", "middleclick": "middle"}
|
||||
PAD_NOTCH = 15 # scroll units for one wheel notch
|
||||
|
||||
|
||||
def typed_in_kde(events):
|
||||
"""Whether these events must go the KDE Connect way: they hold an accent or emoji, or
|
||||
earlier ones are queued, being sent or still being typed there (typing must stay in
|
||||
order, so it can't be split between the two ways)."""
|
||||
with _kde_lock:
|
||||
waiting = bool(_kde_queue) or time.time() < _kde_until
|
||||
return waiting or any(ch not in frame_touch.ASCII for e in events for ch in e.get("key", ""))
|
||||
|
||||
|
||||
def pad_events(events, kde=False):
|
||||
"""The trackpad's and key row's events as (gamescope events, KDE Connect events).
|
||||
|
||||
Gamescope's own input reaches every panel and needs nothing installed, but types only
|
||||
US-keyboard characters. With kde, the keyboard goes to KDE Connect as it was sent, so
|
||||
text stays in order; the pointer always goes through gamescope.
|
||||
"""
|
||||
ascii_keys = frame_touch.ASCII
|
||||
out, slow = [], []
|
||||
|
||||
def tap(code, mods):
|
||||
down = [{"key": c, "down": True} for c in mods]
|
||||
out.extend(down + [{"key": code, "down": True}, {"key": code, "down": False}]
|
||||
+ [{"key": c, "down": False} for c in reversed(mods)])
|
||||
for e in events:
|
||||
mods = [code for name, code in PAD_MODS if e.get(name)]
|
||||
if "dx" in e or "dy" in e:
|
||||
if not e.get("scroll"):
|
||||
out.append({k: e[k] for k in ("dx", "dy") if k in e})
|
||||
if e.get("scroll"):
|
||||
out.append({"scroll": [0, -(e.get("dy") or 0) * PAD_NOTCH]})
|
||||
for flag, button in PAD_CLICKS.items():
|
||||
if e.get(flag):
|
||||
out += [{"button": button, "down": True}, {"button": button, "down": False}]
|
||||
if e.get("doubleclick"):
|
||||
out += [{"button": "left", "down": d} for d in (True, False, True, False)]
|
||||
if e.get("singlehold"):
|
||||
out.append({"button": "left", "down": True})
|
||||
if e.get("singlerelease"):
|
||||
out.append({"button": "left", "down": False})
|
||||
if kde and ("key" in e or "specialKey" in e):
|
||||
slow.append({k: e[k] for k in ("key", "specialKey", "ctrl", "alt", "shift", "super") if k in e})
|
||||
continue
|
||||
if "specialKey" in e and e["specialKey"] in PAD_KEYS:
|
||||
tap(PAD_KEYS[e["specialKey"]], mods)
|
||||
if "key" in e:
|
||||
for ch in e["key"]:
|
||||
if ch not in ascii_keys:
|
||||
continue # only reached when kde is wrong; the caller checks typed_in_kde first
|
||||
elif mods:
|
||||
code, shifted = ascii_keys[ch]
|
||||
tap(code, mods + ([42] if shifted and 42 not in mods else []))
|
||||
else:
|
||||
if out and "text" in out[-1] and len(out[-1]["text"]) < 100:
|
||||
out[-1]["text"] += ch
|
||||
else:
|
||||
out.append({"text": ch})
|
||||
return out, slow
|
||||
|
||||
|
||||
def remote_input(body):
|
||||
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent.
|
||||
|
||||
"sent" in the answer means the whole batch is taken (the page keeps it and tries again
|
||||
if not), so the keyboard's share is queued only once that's so.
|
||||
"""
|
||||
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent."""
|
||||
events = body.get("events", [])
|
||||
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
|
||||
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
|
||||
events = [input_event(e) for e in events]
|
||||
touch, slow = pad_events(events, typed_in_kde(events))
|
||||
status = _touch.send(touch)
|
||||
if touch and not status.get("sent"):
|
||||
return status
|
||||
if touch:
|
||||
_note_typed(touch)
|
||||
if slow:
|
||||
_queue_for_kde(slow)
|
||||
return {**status, "sent": True}
|
||||
|
||||
|
||||
# Typing KDE Connect is to do: it starts the first time it's needed, so this waits for it.
|
||||
# Nothing on the Frame says when typed text has landed, so each way holds the other back
|
||||
# for as long as what it was given should take (a short margin on a time estimate).
|
||||
_kde_queue = []
|
||||
_kde_lock = threading.Lock()
|
||||
_pad_send_lock = threading.Lock()
|
||||
_kde_worker = [None]
|
||||
_kde_until = 0.0 # keys typed through KDE Connect should have landed by then
|
||||
_typed_until = 0.0 # when gamescope's agent will have typed everything it's been given
|
||||
_kde_route = object()
|
||||
_kde_cancel = threading.Event()
|
||||
KDE_QUEUE_LIMIT = 200
|
||||
KDE_WAIT = 90 # seconds to wait for KDE Connect before giving up on what's queued
|
||||
TYPING_SETTLE = 0.6 # seconds past the estimate, once
|
||||
|
||||
|
||||
def reset_pad_route():
|
||||
"""Discard input for the headset being left, including batches waiting for an agent."""
|
||||
global _kde_route, _kde_cancel, _kde_until, _typed_until
|
||||
with _pad_send_lock, _kde_lock:
|
||||
_kde_cancel.set()
|
||||
_kde_cancel = threading.Event()
|
||||
_kde_route = object()
|
||||
_kde_queue.clear()
|
||||
_kde_worker[0] = None
|
||||
_kde_until = _typed_until = 0.0
|
||||
_input.stop()
|
||||
_touch.stop()
|
||||
|
||||
|
||||
def typing_seconds(touch):
|
||||
"""How long gamescope's agent takes over these: it sleeps 8 ms after every key transition."""
|
||||
transitions = 0
|
||||
for e in touch:
|
||||
if "text" in e:
|
||||
transitions += sum(2 + 2 * frame_touch.ASCII[ch][1] for ch in e["text"] if ch in frame_touch.ASCII)
|
||||
elif "key" in e:
|
||||
transitions += 1
|
||||
return transitions * 0.008
|
||||
|
||||
|
||||
def _note_typed(touch):
|
||||
global _typed_until
|
||||
seconds = typing_seconds(touch)
|
||||
if seconds:
|
||||
with _kde_lock: # the end of the work, which queues up; the margin is added once, where it's checked
|
||||
_typed_until = max(_typed_until, time.time()) + seconds
|
||||
|
||||
|
||||
def _queue_for_kde(events):
|
||||
with _kde_lock:
|
||||
_kde_queue.extend(events)
|
||||
del _kde_queue[:-KDE_QUEUE_LIMIT]
|
||||
if _kde_worker[0] is None:
|
||||
_kde_worker[0] = threading.Thread(target=_drain_kde, args=(_kde_route, _kde_cancel), daemon=True)
|
||||
_kde_worker[0].start()
|
||||
|
||||
|
||||
def _drain_kde(route, cancel):
|
||||
"""The only sender, in order. A batch leaves the queue while it's being sent (the queue
|
||||
can be trimmed meanwhile) and goes back to the front if it didn't go."""
|
||||
global _kde_until
|
||||
deadline = time.time() + KDE_WAIT
|
||||
while True:
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
batch = _kde_queue[:INPUT_BATCH_LIMIT]
|
||||
del _kde_queue[:len(batch)]
|
||||
if not batch:
|
||||
_kde_worker[0] = None # retired under the lock, so a new request starts another
|
||||
return
|
||||
wait = _typed_until + TYPING_SETTLE - time.time()
|
||||
_kde_until = time.time() + 3600 # held while in flight
|
||||
if wait > 0 and cancel.wait(wait):
|
||||
return # a switch cancels the delay and its old-headset batch
|
||||
# Serialize sending with route resets without blocking callers from queuing input.
|
||||
with _pad_send_lock:
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
status = _input.send(batch)
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
if status.get("sent"):
|
||||
_kde_until = time.time() + len(batch) * 0.03 + TYPING_SETTLE
|
||||
deadline = time.time() + KDE_WAIT
|
||||
continue
|
||||
_kde_until = 0.0
|
||||
if status.get("state") == "error" or time.time() > deadline:
|
||||
_kde_queue.clear() # not coming; don't hold it forever
|
||||
_kde_worker[0] = None
|
||||
return
|
||||
_kde_queue[:0] = batch
|
||||
del _kde_queue[:-KDE_QUEUE_LIMIT] # as when queuing: the oldest go first
|
||||
if cancel.wait(0.4):
|
||||
return
|
||||
return _input.send([input_event(e) for e in events])
|
||||
|
||||
|
||||
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
|
||||
@@ -2926,7 +2745,7 @@ class Handler(BaseHTTPRequestHandler):
|
||||
elif path == "/api/touch":
|
||||
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
|
||||
elif path == "/api/input":
|
||||
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
|
||||
self.send_json(_input.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_input.status))
|
||||
elif path == "/api/job":
|
||||
self.send_json(job_status(url.query))
|
||||
elif path == "/api/queue":
|
||||
|
||||
Reference in new issue
Block a user