mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-24 19:01:43 +02:00
* Live view: a Desktop view that stays still, and Control to tap on the Frame The live view gets a second source and a way to use the Frame from it: - Desktop: the app panel in use in the headset, streamed from its own window (x11grab of gamescope's redirected window), so it doesn't move as the wearer looks around. A picker shows any other panel, view only. - Control: on the Desktop view a tap or click lands exactly where you put it; drag is a mouse drag, press and hold right-clicks, two fingers scroll, and on a computer the mouse, wheel and keyboard work directly. On the headset view the view is a trackpad. A text field and key row type from a phone. Input goes through gamescope's own EIS socket (the way Steam feeds Remote Play input) with the libei already on the image: ui/frame_touch.py, over the same long-lived ssh machinery as the keyboard agent, nothing to install. It reaches the panel that has focus on either X display, which the KDE Connect route can't. Verified on the Frame and from the iPhone app in the Simulator. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Control: fixes from review - Keys held on the Frame are released with buttons when Control stops or the view loses focus; keys for the Frame no longer trigger Frame Control's own shortcuts. - Taps only act when the picture on screen is the panel in use; positions, presses, keys, text and scrolls name their panel (display and window: ids repeat across :0 and :1, told apart by pid), and the Frame drops them if focus has moved on. Releases always go. - While connecting, a tap keeps its position; on an error only releases wait and retries back off; trimming a long queue never drops a release. - Lifting one of two scrolling fingers ends the scroll; a cancelled touch isn't a tap; clicks and holds on the bars around the picture do nothing. - A capture loop from before a Live restart can't stop the new video. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Control: close the targeting gaps from the second review - The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope packs ":1" into the first value), so a window id repeated across :0 and :1 can't be mistaken; the pid is only the fallback. - Presses, keys, text and scrolls read focus afresh on the Frame; only moves use a reading up to a second old. - A gesture remembers the panel it started on and does nothing more if that stops being the one in use; a press with no panel to aim at isn't sent. - Opening a screenshot clears the panel Control would act on; switching to another app releases held keys and buttons. - Trimming keeps a click with its position; the error backoff holds for new input too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Control: fixes from the SWE-2 Max review - The Frame side tracks keys as well as buttons and lets go of both when the session ends. - A stale tap tells the page, which re-reads the panels at once. - A paused input device waits instead of ending the session; only a disconnect does. An OS error on one event skips it. - Presses check focus with two property reads and do the full lookup only when it changed. - Writes to an agent's stdin are serialized, so two devices sending at once can't tear a line (the keyboard agent too). - Connecting gives up with a message after 15 s instead of hanging on "Connecting…"; text goes in 100-character pieces so releases don't wait behind a long paste; a cancelled mouse gesture releases what's held. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Control: stale clears, pauses reconverge, pastes split per request - The Frame says it has caught up as soon as an aimed event lands after a stale one, so the page stops re-reading the panels. - After a device pause it lets go of everything it holds (releases that arrived while paused were dropped), and waits for the device once per batch, not once per event. - The quick focus check no longer freshens the panel geometry's age. - Each request carries at most about 100 characters of text. - Turning Control off while it connects doesn't report an error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * frame_touch: build the socket path on the Frame, so Windows can import it for tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * frame_touch: any event that goes through clears the stale flag A trackpad move names no panel, so waiting for an aimed event could leave the page re-reading panels for the rest of the session; a release still aimed at the old panel doesn't count. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Trackpad and key row go through gamescope's input; KDE Connect only for accents and emoji The pad reaches whichever panel has focus (Steam's menus too), needs nothing installed, and no longer clamps to 1280x720. Characters that aren't on a US keyboard start KDE Connect the first time they're typed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Touch: a bare release no longer clears the stale-focus notice before the page has seen it The page leaves the panel off releases, so a release says nothing about focus. Found in review of the stale-latch fix. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Pad: keep typing in order and acknowledge it once, with accents through one KDE Connect queue Review found accent-only input repeated forever (sent was false), text split across two transports out of order, accents stuck until the next keypress, duplicated on retry, and racy. A batch with an accent now sends its whole keyboard through KDE Connect; later keys follow while any are queued; the server owns one ordered, bounded queue drained by a single thread; sent means the whole batch is taken. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Pad: drain the KDE Connect queue without losing, stranding or reordering input Second review found the queue could be trimmed under an in-flight send (deleting unsent events), the worker could retire as a request enqueued, and ASCII could overtake an accent still being typed (or the reverse). The worker now takes its batch out while sending and puts it back on failure, retires under the lock, and each way waits out the other's estimated typing time. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Pad: count gamescope's real typing time, with the settle margin added once Review found the estimate used 12 ms a character where the agent takes 16 ms (32 shifted), and added the margin on every request so it piled up. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Document gamescope pad verification and shared-device limits * Prevent Unicode input startup from deadlocking a headset switch * Record real focused-panel click and ASCII key delivery * Retain accepted Unicode input under queue saturation and failure --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: SAPHID <saphid@users.noreply.github.com>
486 lines
28 KiB
Markdown
486 lines
28 KiB
Markdown
# Testing
|
||
|
||
Frame Control is tested in three layers, from fast and fake to slow and real.
|
||
A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/steam-frame/issues/6)).
|
||
|
||
| Layer | Runs | Needs | Covers |
|
||
|---|---|---|---|
|
||
| Unit tests (`tests/*.py`) | `python3 -m unittest discover -s tests` | Nothing | Parsing, validation, request guards; SSH and HTTP are mocked |
|
||
| Fake Frame (`tests/e2e`) | `scripts/e2e.sh` | Linux with Docker | The real server and scripts against a container that behaves like a Frame |
|
||
| Headset smoke test | `scripts/frame-smoke.sh` | A Frame on the `frame` alias | Install, launch and remove on the real device, recorded with its BUILD_ID |
|
||
| [Windows test VM](#windows-test-vm) | `scripts/windows-vm.sh` | A Linux machine with KVM and Docker | The Windows build on a real Windows desktop, against a real Frame when needed |
|
||
|
||
## Unit tests
|
||
|
||
The gamescope keyboard/trackpad route has focused suites in `test_input.py`,
|
||
`test_touch.py` and `test_service_regressions.py`. They cover pad translation,
|
||
ASCII/modifier keys, Unicode queuing/retries, stale panel targets, release
|
||
handling, route cancellation, and intended-headset start/status guards.
|
||
`tests/page/pad_queue.mjs` and `tests/page/ctrl_gestures.mjs` execute the page's
|
||
real queue and gesture functions in Node. These are offline tests; their
|
||
acknowledgements do not prove that a real application received input.
|
||
See [the 2026-10-24 PR #66 evidence](evidence/pad-gamescope-2026-10-24.md)
|
||
for the real focused-panel click/key observation, cleanup and remaining limits.
|
||
|
||
The read-only power/heat sampler has fixture and transport tests in
|
||
`tests/test_power_sample.py`. Run
|
||
`python3 -m unittest discover -s tests -p test_power_sample.py -v`.
|
||
See [power-and-heat.md](power-and-heat.md) for sampling, smoke-lock behavior and
|
||
real-device verification limits.
|
||
|
||
```sh
|
||
python3 -m unittest discover -s tests
|
||
```
|
||
|
||
The Control Center is covered by `tests/test_control_center.py` and its Node
|
||
behavior suite `tests/control_center.cjs` (also run by unittest discovery).
|
||
They exercise production search, metrics, navigation and action handlers with
|
||
mocked API/DOM boundaries: volume Apply/mute/readback, offline and failed reads,
|
||
stale samples, selected-headset capture, duplicate actions and validation before
|
||
SSH. Integration regressions run the production shared refresh, Home renderer,
|
||
host callback and Control Center bootstrap: a pre-command status request must be
|
||
drained before audio readback starts, and delayed startup retains sample receipt
|
||
age, failed-read state and early host capabilities. The actual Flatpak install
|
||
redraw cannot revive a failed sample; a new successful poll restores metrics and
|
||
audio after either redraw or delayed mount. Draft regressions cover a pending headset
|
||
switch and mute followed by Apply. Static asset HTTP tests and desktop/iPhone bundle checks catch missing CSS
|
||
or JavaScript. A focused run is:
|
||
|
||
```sh
|
||
python3 -m unittest discover -s tests -p test_control_center.py -v
|
||
node --test tests/control_center.cjs
|
||
node --check ui/control-center.js
|
||
```
|
||
|
||
Responsive layout, actual browser rendering and keyboard/touch behavior still
|
||
need browser verification. No Control Center test connects to a real headset.
|
||
|
||
About 120 tests, a few seconds, on Python 3.9 and newer. GitHub Actions runs
|
||
them on macOS, Windows and Linux. They don't pick up `tests/e2e`.
|
||
|
||
## The fake Frame
|
||
|
||
Frame doctor has focused offline tests in `tests/test_frame_doctor.py` and
|
||
fake-Frame fault tests in `tests/e2e/test_doctor.py`:
|
||
|
||
```sh
|
||
python3 -m unittest discover -s tests -p test_frame_doctor.py -v
|
||
scripts/e2e.sh test_doctor
|
||
```
|
||
|
||
They cover privacy filtering, host-key preservation, restart safety and missing
|
||
journal evidence, plus asleep, full-title-storage, SSH/devkit-off and missing-key
|
||
faults. The plan/report tests cover exact-evidence advice, blocked plans,
|
||
private exclusive baselines, malformed input, target mismatches and comparisons
|
||
without repair claims. The fake-Frame storage test records a full-disk baseline,
|
||
clears only the fake fault and compares a fresh run. The fake's doctor-only
|
||
`systemctl show` responses assume stable restart counters and inactive keep-awake; unsupported queries fail. These tests do not
|
||
exercise a real SteamOS user manager or touch a headset.
|
||
|
||
`tests/fakeframe/` builds a container that stands in for the headset, and a
|
||
second one for the computer Frame Control runs on. `scripts/e2e.sh` builds
|
||
both, starts them with `docker compose`, runs `tests/e2e` in the host
|
||
container and takes everything down, exiting with the tests' status:
|
||
|
||
```sh
|
||
scripts/e2e.sh # everything, about 2 minutes plus the first build
|
||
scripts/e2e.sh test_titles # one module
|
||
scripts/e2e.sh test_faults.Faults.test_disk_full # one test
|
||
FAKEFRAME_KEEP=1 scripts/e2e.sh # leave it running afterwards
|
||
```
|
||
|
||
It needs a Linux host with Docker and `docker compose`, and zsh. The images
|
||
are `fakeframe-frame` and `fakeframe-host`; the compose project, network and
|
||
volumes are `fakeframe-e2e*`. CI runs it on a native arm64 runner
|
||
(`ubuntu-24.04-arm`, the `e2e` job in `.github/workflows/checks.yml`).
|
||
|
||
The host container exists because OpenSSH reads `~/.ssh/config` from the
|
||
passwd home directory, not `$HOME`. There, `ssh frame` reaches the fake Frame
|
||
through the same `Host frame` block `ui/frame_connect.py` writes, the
|
||
repository is mounted read-only at `/repo`, and each test module starts the
|
||
real `ui/server.py` (Python 3.9) and talks to it over HTTP with the headers
|
||
its guards want.
|
||
|
||
### What's real and what's fake
|
||
|
||
| On the fake Frame | |
|
||
|---|---|
|
||
| Arch Linux (`archlinux:base`, or Valve's Holo Core aarch64 preview on arm64), user `steamos`, `/etc/os-release` with BUILD_ID 20260922.6101926 | Real OS, Frame's identity |
|
||
| `sshd` with key and password logins, `rsync`, `python3` | Real |
|
||
| Valve's steamos-devkit-service on port 32000 and its hooks, vendored unmodified in `tests/fakeframe/steamos-devkit-service` | Real; only its `dbus` import (for mDNS through systemd-resolved) is a stand-in that logs the registration |
|
||
| Valve's devkit-utils, copied over by Frame Control itself | Real |
|
||
| **fakesteam**: `~/.steam/steam.pid`, `steam.token` and the `steam.pipe` FIFO; answers `approve-ssh-key`, `create-shortcut`, `run-game`, `list-shortcuts` and `delete-shortcut` with the response files devkit-utils waits for; takes `steam://rungameid`, `install` and `store` URLs. Games it really runs start under a stand-in `reaper SteamLaunch AppId=<id> -- <command>` (bash with that command line), so force quit finds a real process tree | Fake |
|
||
| DevTools on `127.0.0.1:8080` with a `SharedJSContext` target. The JavaScript Frame Control sends runs for real in Node against stand-in `SteamClient`, `appStore` and `downloadsStore` objects (`cef_shim.js`), so async functions, optional chaining and `Map`s behave as in Steam's CEF. `SteamUIStore.RunningApps` lists launches that haven't exited; `TerminateApp` ends only launches that never ran a process, so a running program needs Frame Control's signals | The JS engine is real; the objects are fake |
|
||
| `steam`, `wpctl`, `flatpak`, `podman`, `nmcli`, `qdbus6`, `gamescopectl`, SteamOS's `steamos-enable-sshd` helper, and Lepton's launcher | Stubs that record their calls |
|
||
| Battery, charger and thermal zones under `/sys/class` | Files the supervisor writes. `/sys` is read-only in a container and Docker's AppArmor profile refuses writes under it, so each folder is a volume mounted twice: over `/sys/class/...` for `frame_status.py` to read, and under `/var/lib/fakeframe/sys` for the supervisor to write |
|
||
| `vrserver` and `plasmashell` | Renamed `sleep` processes, so the status page and the clipboard find them |
|
||
|
||
Every fake behaviour copied from the device has a comment citing the doc or
|
||
observation and the BUILD_ID it came from; anything not seen on a headset is
|
||
marked as a guess. The fake keeps its state in `/var/lib/fakeframe/state.json`
|
||
(shortcuts, devkit titles, compat tool mapping, launches, pairing requests,
|
||
Lepton instances, volume, Flatpaks, clipboard) and logs stub calls to
|
||
`calls.jsonl` beside it.
|
||
|
||
Native programs really run: a launched aarch64 title executes on an arm64
|
||
host, and an x86-64 one on x86-64 (the container shares the host's kernel).
|
||
Proton titles are recorded with the command Steam would run, not run.
|
||
|
||
### Fault switches
|
||
|
||
`fakeframe-ctl` works over SSH (`ssh frame fakeframe-ctl help`) and from the
|
||
host container (`FAKEFRAME_CTL=http://fakeframe:9999`), so a test can flip a
|
||
switch while SSH is down:
|
||
|
||
| Command | Effect |
|
||
|---|---|
|
||
| `pairing on\|off` | Steam's **Pair new host** screen open or not; off gives the device's 403 text |
|
||
| `answer approve\|deny\|timeout` | How the pairing prompt is answered |
|
||
| `steam on\|off` | Steam client running (pid file, pipe, DevTools) |
|
||
| `sleep on\|off` | Headset asleep: ports 22 and 32000 accept and never answer, so SSH times out |
|
||
| `sshd on\|off` | sshd stopped: new connections are refused, open ones stay |
|
||
| `devkit-service on\|off` | Port 32000 closed |
|
||
| `disk-full on\|off` | Fills the small (64 MB) filesystem on `~/devkit-game` |
|
||
| `runtime NAME installed\|missing` | Proton, the Steam Linux Runtimes, Lepton |
|
||
| `battery KEY=VALUE...` | e.g. `capacity=15 status=Discharging current_now=-900000` |
|
||
| `keys harness\|none`, `authorized-keys` | Set or read `~/.ssh/authorized_keys` |
|
||
| `reset`, `state`, `calls [TOOL]` | Start over; read the state and call log |
|
||
|
||
### What the fake can't show
|
||
|
||
- Rendering: the headset view, desktop capture content, live video, SteamVR,
|
||
gamescope and panels. The capture stub returns a placeholder PNG.
|
||
- Proton and FEX: whether a Windows or x86-64 program actually runs.
|
||
- Android: there's no Android in the Lepton stand-in, so no ADB, display
|
||
settings, probes or app crashes.
|
||
- The real Steam client's UI and anything it does that isn't modelled, and
|
||
mDNS discovery.
|
||
- `sudo` and the power buttons, Tailscale, and the Windows and macOS sides of
|
||
the app (the host container is Linux, so the `rsync` paths are tested and the
|
||
`scp` fallback isn't).
|
||
|
||
## Headset smoke test
|
||
|
||
**Documented shared-device procedure:** before a test installs, launches or
|
||
stops an application, acquire `ssh frame 'mkdir /tmp/frame-test.lock'`. If it
|
||
fails, leave that lock alone and continue offline work. Only the thread that
|
||
acquired it releases it with `ssh frame 'rmdir /tmp/frame-test.lock'`, after
|
||
cleanup. Keep each device session to a few minutes.
|
||
|
||
Check battery capacity and charging state under `/sys/class/power_supply`
|
||
before and after; keep capacity above 20%. Stop only processes started by the
|
||
test, remove temporary installs and profiles, and restore the prior dashboard
|
||
state. Leave Steam and SteamVR running. Do not reboot or change global settings.
|
||
Record the build, actual interaction results, cleanup and any unworn-headset
|
||
limits alongside screenshots or logs. These are caller responsibilities; the
|
||
smoke script below does not acquire this shared lock itself.
|
||
|
||
[Packaged Desktop Capture evidence, 2026-10-24](evidence/desktop-capture-packaged-2026-10-24.md)
|
||
records v0.4.17 GUI button captures of a real mapped nested desktop on SteamOS
|
||
0.5.6, a bounded owned user unit, and cleanup back to the original Game Mode
|
||
state. Check X window mapping/focus and SteamVR overlay visibility separately:
|
||
successful capture of a mapped window does not prove the desktop is visible
|
||
to a wearer. That run's overlay stayed `not_visible`; #73 remains unverified
|
||
for its reported visible-desktop/0.4.5 condition. Only state/log metadata is public.
|
||
|
||
```sh
|
||
scripts/frame-smoke.sh # needs `ssh frame` to work without a password
|
||
scripts/frame-smoke.sh --pair # also pairs a throwaway key: approve it in the headset
|
||
```
|
||
|
||
It checks `properties.json` and the status, then installs, launches and
|
||
removes three tiny titles built from bytes by `tests/smoke/tiny_programs.py`
|
||
(an ARM64 and an x86-64 static Linux program that sleep for ten seconds, and
|
||
an x86-64 `.exe` that exits at once). A launch passes only with fresh evidence:
|
||
the ARM64 program running, the `.exe` started (its process or Steam's log),
|
||
and the x86-64 program running or Steam logging that its runtime isn't
|
||
installed, which is what the Frame does today. Steam's log lines about each
|
||
title are kept.
|
||
|
||
Everything it installs is removed again, also after a failure: the titles and
|
||
their Steam shortcuts, a paired key, and `~/devkit-utils` if it wasn't there
|
||
before (if it was, it stays, synced to this checkout as Frame Control always
|
||
does). A cleanup that fails counts as a failed step. Results go to
|
||
`tests/smoke/results/<time>-<BUILD_ID>.json` (not committed) with a summary on
|
||
screen; it exits 0 when every step passed, 1 if one failed, 2 if the headset
|
||
isn't reachable.
|
||
|
||
Smoke process cleanup runs before and after title removal and during the
|
||
initial/final leftover sweeps. It targets only the reserved `fc_smoke_arm64`,
|
||
`fc_smoke_x86_64`, `fc_smoke_exe` IDs and their legacy `fc-smoke-` equivalents.
|
||
On the target, `tests/smoke/stop_processes.py` requires the SSH user's uid and
|
||
an exact title directory in `/proc` cwd/exe or an individual absolute command
|
||
argument; similarly named folders and shell command substrings do not count.
|
||
Deleted cwd/exe paths still count, so a process left after file removal can be
|
||
found. It sends SIGTERM, waits up to three seconds, then uses SIGKILL with
|
||
another three-second bound, rescanning for survivors. PID handles prevent PID
|
||
reuse from signalling another process. Linux Python 3.9+ with pidfd support
|
||
is required; missing support, signal errors and survivors fail the
|
||
cleanup step. Stopped PIDs are recorded in the results JSON.
|
||
If the initial leftover cleanup fails, the run skips installs and pairing and
|
||
records the failure instead of overwriting an active test title.
|
||
|
||
This deliberately does not stop shared Wine/Proton services by name or prefix,
|
||
Steam, SteamVR, process groups or arbitrary descendants. A process that drops
|
||
all title-path evidence cannot safely be attributed and is outside this
|
||
check. Unreadable `/proc` fields provide no ownership evidence; other readable
|
||
fields are still checked, and a wholly unattributable process is left alone.
|
||
Real Frame verification, 2026-10-24, SteamOS build `20261015.6141207`: the
|
||
fixed Python smoke entry point passed all 13 steps under the shared device lock.
|
||
The `fc_smoke_exe` launch ran under Proton Experimental (ARM64), and its
|
||
remove step stopped five test-owned wrapper/Proton PIDs before removing the
|
||
title; the following process sweep found none. The final cleanup found no
|
||
test titles or processes, the test desktop was closed, and the battery
|
||
remained at 99%. The `scripts/frame-smoke.sh` zsh wrapper could not run
|
||
on lxso2 because zsh is absent; `python3 tests/smoke/frame_smoke.py` ran
|
||
the same entry point instead.
|
||
The local regression suite runs real Linux processes in temporary fake title
|
||
directories (including deleted folders and a TERM-resistant wrapper); it checks
|
||
exit, escalation and survival of unrelated processes without SSH or a headset:
|
||
|
||
```sh
|
||
python3 -m unittest discover -s tests -p test_smoke_cleanup.py -v
|
||
```
|
||
|
||
`--pair` asks the devkit service to pair a new RSA key, which needs someone
|
||
in the headset to open **Settings → Developer → Pair new host** and approve
|
||
it; the key is checked and then taken out of `authorized_keys` again.
|
||
|
||
## When the device disagrees with the fake
|
||
|
||
The fake is only as good as what's been seen on a headset. When the smoke
|
||
test (or anyone) finds the Frame doing something else:
|
||
|
||
1. Record what the device did, with the date and BUILD_ID, in the doc that
|
||
covers it (`docs/sideloading.md`, `docs/ssh.md` and so on).
|
||
2. Change the fake to match, with a comment citing that observation. The
|
||
behaviours are in `tests/fakeframe/rootfs/usr/local/lib/fakeframe/`
|
||
(`fakesteam.py` for Steam, `cef_shim.js` for DevTools, `init.py` for the
|
||
switches, the stubs in `rootfs/usr/local/bin`).
|
||
3. Run `scripts/e2e.sh`. If the app is wrong, the tests now fail the way the
|
||
device did; fix the app and add a unit test.
|
||
|
||
For example, on 2026-09-27 the smoke test found that Steam's `create-shortcut`
|
||
refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
|
||
accepted. The fake now refuses them the same way, and Frame Control makes ids
|
||
Steam accepts.
|
||
|
||
## PC panel host tests
|
||
|
||
PC in the headset (Experimental, Windows only). `tests/test_pcview.py` checks
|
||
that only Windows gets the PC host and that it starts nothing until turned on,
|
||
the Tools link and card wording, source-bound tickets, reconnect/Stop
|
||
revocation, input release, source-coordinate mapping and use of the existing
|
||
panel launcher. `tests/test_pc_controller.py` checks the C rate controller
|
||
against a frozen trace of the Mac's Swift one.
|
||
|
||
The `PC host libraries` workflow builds the native adapter on Windows (and
|
||
the unshipped Linux adapter on Ubuntu x64 and ARM64), then runs a real x264
|
||
synthetic stream through the HTTP/WebSocket agent. x264 is GPL, so only these
|
||
CI builds include it (`desktop/build.py --with-x264`); the release workflow
|
||
tests the installer's own bundle with its bundled Python and checks x264 isn't
|
||
in it. `FRAME_PC_REQUIRE_NATIVE=1` and `FRAME_PC_REQUIRE_X264=1` make a missing
|
||
bundle or encoder fail CI instead of skipping.
|
||
|
||
These automated CI checks do not capture a desktop, exercise Media Foundation
|
||
or a GPU, or check laser alignment. A [Windows test VM](#windows-test-vm) run
|
||
on 2026-10-24 captured its actual desktop with Desktop Duplication, encoded
|
||
with Media Foundation and streamed to a real Frame. It found MSYS2 disables
|
||
WGC: an element name alone does not prove its optional capture properties
|
||
exist. Native property-probe tests and source-selection regressions now cover
|
||
that boundary. [Dated evidence and limits](evidence/windows-vm-2026-10-24.md);
|
||
[what's verified](pc-in-headset.md#whats-verified).
|
||
|
||
## Windows test VM
|
||
|
||
Windows CI runs the unit tests and checks installer contents. Desktop capture,
|
||
installer launch and interactive behavior need a Windows machine. This is a Windows 11 VM in a
|
||
[dockur/windows](https://github.com/dockur/windows) container on a Linux machine
|
||
with KVM, driven from the Mac with `scripts/windows-vm.sh`.
|
||
|
||
Setting it up, once, on the Linux machine:
|
||
|
||
- **Windows comes from Microsoft.** The container downloads the Windows 11
|
||
image from Microsoft on first start. Windows runs unactivated, which is fine
|
||
for testing. Don't use activation workarounds or third-party Windows images.
|
||
- **Publish its ports on loopback only.** Map `127.0.0.1:2222:22` (SSH),
|
||
`127.0.0.1:8006:8006` (the web console) and, if you need it,
|
||
`127.0.0.1:13389:3389`. The Mac reaches them through `ssh -J`. Use
|
||
`restart: "no"` and `stop_grace_period: 2m` so it only runs when someone is
|
||
testing, and a `docker stop` shuts Windows down cleanly.
|
||
- **Give it SSH on first sign-in.** The container runs `/oem/install.bat`
|
||
once. Have it add the OpenSSH Server capability, start `sshd`, set PowerShell
|
||
as its default shell, and put a dedicated public key (for example
|
||
`~/.ssh/id_ed25519_winvm` on the Mac) in
|
||
`C:\ProgramData\ssh\administrators_authorized_keys`.
|
||
- Give it 4 cores, 8 GB of memory and a 32 GB disk. That's enough for the app
|
||
and the tests. Inspect free memory/disk, KVM access and existing VMs before
|
||
starting; leave other VMs alone. The first install can take tens of minutes
|
||
(the 2026-10-24 run took about 16 minutes to reach the desktop), rather than
|
||
the 1–2 minute startup of an already installed guest.
|
||
- For PC streaming, check the actual bundled capture properties. MSYS2's
|
||
GStreamer build disables WGC: this app falls back to Desktop Duplication for
|
||
whole screens and explains why individual windows are unavailable. A VM
|
||
capture test does not prove physical GPU encoding or laser alignment.
|
||
|
||
Using it, with `WINVM_HOST` set to the Linux machine's ssh alias:
|
||
|
||
```sh
|
||
scripts/windows-vm.sh up # start it and wait for SSH (1-2 minutes)
|
||
scripts/windows-vm.sh put Frame-Control-Setup-x64.exe
|
||
scripts/windows-vm.sh ps 'Start-Process "$env:USERPROFILE\Downloads\Frame-Control-Setup-x64.exe" /S -Wait'
|
||
scripts/windows-vm.sh shot screen.png # what's on its screen
|
||
scripts/windows-vm.sh click 723 359 # screen pixels, as in the screenshot
|
||
scripts/windows-vm.sh keys s t e a m o s ret # QEMU key names
|
||
scripts/windows-vm.sh down # shut Windows down
|
||
```
|
||
|
||
- **Clicks go through a scheduled task.** Commands over SSH run in a
|
||
session with no desktop, so `click` and `scroll` write the position to a
|
||
file, and a scheduled task running as the signed-in user replays it. The
|
||
VM's screen must be signed in; it is after `up`. Clicks and scrolls sent at
|
||
the same time run one after another. QEMU's own `mouse_move` is relative
|
||
and drifts, so the script doesn't use it.
|
||
- **Screenshots may not show the pointer.** Check the result of a click (a
|
||
menu that opens, a button that changes) rather than the pointer's position.
|
||
- **Windows' `ssh` waits for stdin.** The script closes it for every command.
|
||
Do the same if you run `ssh` in the VM by hand.
|
||
- **Starting the app.** Run Frame Control in the signed-in session, not over
|
||
SSH. Use its Start-menu shortcut via `click`, or a scheduled task like the
|
||
one `click` uses.
|
||
|
||
**Testing against a real Frame.** The VM reaches the headset on the LAN like
|
||
any other computer. Run **Set Up Connection** in the VM's Frame Control once;
|
||
that makes the VM's keys. So the VM doesn't keep access between test runs,
|
||
afterwards take the lines it added out of the headset's
|
||
`~/.ssh/authorized_keys`: the ones matching the VM's
|
||
`~/.ssh/id_ed25519_frame.pub` and, if pairing made one,
|
||
`~/.ssh/id_rsa_frame_devkit.pub`. Check that `ssh -o BatchMode=yes frame true`
|
||
in the VM now fails. Then, around each run:
|
||
|
||
```sh
|
||
scripts/windows-vm.sh frame-key add # let the VM's key into the headset
|
||
# ... test ...
|
||
scripts/windows-vm.sh frame-key remove # and take it out again
|
||
```
|
||
|
||
`add` appends the VM's key tagged `windows-vm-test`, unless the headset
|
||
already trusts that key through another line. `remove` deletes only the exact
|
||
line `add` wrote, so it doesn't undo what Set Up Connection did, and it leaves
|
||
the file alone if it can't rewrite it. Follow the shared-device
|
||
procedure in [Headset smoke test](#headset-smoke-test) before installing or
|
||
launching anything on the headset.
|
||
|
||
**Verified 2026-09-30:** Windows 11 Pro 25H2 (build 26200) against a Frame on
|
||
BUILD_ID 20260925.6191901. The script was used to install Frame Control 0.4.0,
|
||
connect it to the Frame and follow Remote Desktop through to the Frame's
|
||
desktop ([what it does](streaming.md#a-see-and-control-the-frame-from-the-mac)).
|
||
`frame-key` left `authorized_keys` byte-for-byte as it was.
|
||
|
||
## Force quit
|
||
|
||
`tests/test_force_quit.py` feeds `ui/frame_running.py` made-up process tables:
|
||
the `reaper SteamLaunch AppId=` parsing, naming from Steam, manifests, Android
|
||
`meta.json` and devkit folders, and a Frame's worth of Steam, gamescope, SteamVR,
|
||
sshd and session processes that must never be signalled. Processes that ignore
|
||
SIGTERM, survive SIGKILL or exit and have their pid reused check the escalation.
|
||
The Control Center's confirmation, duplicate and headset-switch handling run in
|
||
`tests/control_center.cjs`. `tests/e2e/test_running.py` force quits real process
|
||
trees on the fake Frame. None of it has run on a headset.
|
||
|
||
## Owned media player
|
||
|
||
`tests/test_media.py` covers layout evidence and overrides, OU eye ordering,
|
||
hardware-decoder command construction, malformed splats, stereo parallax and
|
||
fake-Frame library/process ownership. It also covers optional-surround refusal
|
||
at creation, setup, first upload and retry after standby, fatal screen refusal,
|
||
and best-effort overlay teardown with shutdown.
|
||
Run `python3 -m unittest discover -s tests -p test_media.py -v`.
|
||
`tests/e2e/test_media_transfer.py` checks the real
|
||
HTTP/SSH upload and library listing without pretending the fake renders VR.
|
||
Real decode timings and captured stereo output are recorded in
|
||
[vr-video.md](vr-video.md). Generated media only; no external player required.
|
||
|
||
**Verified 2026-09-28**, real Frame BUILD_ID 20260925.6191901: `~/.local`
|
||
and `~/.local/share` are `steamos:steamos`, mode 0755. The fake supervisor
|
||
sets those parent owners too; previously its root-created Steam manifests
|
||
left the parents root-owned and incorrectly prevented user runtime installs.
|
||
|
||
## Agent interfaces
|
||
|
||
`tests/test_agent.py` exercises MCP stdio, exact-action human approvals and the
|
||
assistant against an in-process HTTP endpoint with canned responses (no keys or
|
||
external calls). `tests/e2e/test_agents.py` runs the MCP/HTTP/SSH path against the
|
||
fake Frame for approved installs, clipboard and file transfer. Headset Chromium
|
||
rendering and real screenshots still need a device; see [agent evidence](agents.md#evidence-and-limits).
|
||
|
||
## Family and comfort
|
||
|
||
`tests/test_comfort.py` uses an injected clock, fake headset sensor readings and
|
||
actions, plus a Node fake of Steam's Home API. It covers warnings before Home,
|
||
late/suspended sessions, cancellation, failed actions, duplicate alerts, reboot
|
||
invalidation, per-zone thermal trips and shared on-headset state. The server
|
||
guards reject invalid session settings before SSH. See
|
||
[real-device evidence and limits](family-comfort.md#verification).
|
||
|
||
## Panel switcher
|
||
|
||
`tests/test_panels.py` supplies fake-Frame `vrcmd --overlays` output, checks
|
||
main-panel filtering (including hidden panels), revalidates closed panels before
|
||
focus, and drives the headset helper's real loopback HTTP server to test access
|
||
keys, Host/Origin guards, malformed requests, offline errors and Close. It runs
|
||
in the normal unit suite without OpenVR or a headset. The fixture format comes
|
||
from SteamVR 2.18.1, BUILD_ID `20260925.6191901`; it does not simulate rendering.
|
||
|
||
On the Frame, run `python3 -` over SSH with `ui/frame_panels.py` on stdin to
|
||
list panels. `--focus <key>` rechecks the list and requests focus. In Frame
|
||
Control, **Tools → Panel switcher → Open in headset** exercises installation,
|
||
Chromium rendering and the same helper through HTTP. Close the switcher after
|
||
testing. [The recorded device checks](panels.md#frame-controls-panel-switcher)
|
||
cover actual focus, HTTP guards and an OpenXR sample transition, and separately
|
||
identify the unverified Steam-game, spatial layout, reboot and laser behaviors.
|
||
|
||
## Backup and recovery kit
|
||
|
||
`tests/test_backup.py` runs the headset-side script `ui/frame_backup_remote.py`
|
||
for real against a temporary folder laid out like the Frame's home, with only
|
||
SSH faked: path selection and the secrets it must never copy, the manifest and
|
||
checksums, restore checks after a reset, restore writing only its own files,
|
||
unsafe archives and backup names, and the exact Game Mode command.
|
||
`tests/e2e/test_recovery_kit.py` backs up and restores a Proton save on the fake
|
||
Frame through the server. See [backup-and-recovery.md](backup-and-recovery.md#verification).
|
||
A full backup and an identical-file Proton save restore were checked on a
|
||
real Frame running build `20261015.6141207`; an after-reset restore and
|
||
Android data restore remain unverified. See the linked device evidence.
|
||
|
||
## Game storage
|
||
|
||
`python3 -m unittest discover -s tests -p test_storage.py -v` exercises the
|
||
read-only metadata probe, guarded HTTP endpoint and Node renderer/navigation
|
||
regressions. All headset reads are temporary fixtures or mocked SSH. See
|
||
[game storage](game-storage.md#verification-and-limits) for coverage and device
|
||
verification limits. This suite does not move games or contact a headset.
|
||
|
||
## Battery monitoring
|
||
|
||
`tests/test_battery.py` covers real file parsing with temporary sysfs-like
|
||
files, source preference, missing/invalid sensors and permission errors.
|
||
`tests/control_center.cjs` covers charging states and optional health fields,
|
||
alongside existing stale/offline handling. No headset is used. See
|
||
[battery monitoring](battery-health.md) for sources, units and limitations.
|
||
|
||
## Host-key recovery presentation
|
||
|
||
`python3 -m unittest discover -s tests -p test_link_ui.py -v` runs the page's
|
||
connection/banner/selection functions in Node and checks the recovery CSS guard.
|
||
`test_link.py` covers connection failures and short probe reasons with fake SSH
|
||
and local sockets. Neither suite uses a real headset.
|
||
|
||
Browser verification on 2026-10-24 used the current page's markup, styles and
|
||
recovery renderers with synthetic changed-key state and two headsets at
|
||
1280×800. It caught a later `.item .s` rule overriding the earlier wrapping
|
||
rule, which the original source-only assertion missed. Check computed wrapping,
|
||
the complete explanation, tooltip punctuation, Refresh bounds and selection
|
||
after **Use this headset** when changing these styles. This fixture does not
|
||
exercise Electron integration or a real SSH identity change.
|