Compare commits

..
Author SHA1 Message Date
saphid ae9df35442 Add Android phone companion with automatic screenshot downloads 2026-10-24 21:22:57 +11:00
saphid ce27c0e8f1 Add durable foreground iOS screenshot downloads and Photos preference 2026-10-24 16:43:44 +11:00
saphid 5277232367 Merge remote-tracking branch 'origin/main' into feat/phone-screenshot-autosync-108
# Conflicts:
#	README.md
#	ui/frame_macview.py
#	ui/index.html
#	ui/mac-view.html
2026-10-24 16:31:05 +11:00
saphid 3fdc6c46c3 Document Mac capture and live-video verification 2026-10-17 13:01:09 +11:00
saphid 9a6b48dea5 iPhone: record physical feature tests and remaining checks 2026-10-16 20:36:44 +11:00
saphid a3e2b70555 iPhone: record approved beta and phone test blocker 2026-10-16 20:11:41 +11:00
saphid 092aae24bd iPhone: document cloud signing access and the public beta link 2026-10-16 19:33:28 +11:00
saphid 7adf0805d7 iPhone: declare encryption compliance and document TestFlight releases
Declare no non-exempt encryption in the generated app, document upload and tester setup, and record the physical iPhone connection check and remaining tests. Refs #100.
2026-10-16 19:29:45 +11:00
saphidandClaude Opus 5.5 7f23acb03f iPhone: show/hide toggle on the Developer Mode password; TestFlight upload script
The setup screen's password field gets an eye button (44 pt target) that
swaps between hidden and visible text and keeps focus.

ios/scripts/testflight.sh archives with automatic signing and uploads to
App Store Connect using an API key (ASC_KEY_ID, ASC_ISSUER_ID,
ASC_KEY_PATH); the build number is the UTC time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:24:37 +11:00
saphid d9454ef099 Improve Frame Control website onboarding and audit fixes 2026-09-30 19:56:02 +10:00
saphidandClaude Opus 5.5 71a3a96dd2 Mac space: fix separated windows landing on the wrong display; steadier laser; room-matched displays; text size
From Alex's first try in the headset:
- Separated windows could end up on another window's display, so panels
  showed the wrong window or froze. New virtual displays were slotted in
  next to the main one, pushing earlier ones along while a window was being
  placed; and displays under 800x600 points never got their 2x mode.
  Now each display goes left of the leftmost one, the window is placed
  only once the display is in its 2x mode and checked afterwards, and
  displays are at least 800x600 (probed: smaller ones have no usable 2x
  mode on macOS 26). Verified: detach A, detach B, dock A, detach A again,
  each panel streams its own window at 2x.
- Laser: One Euro smoothing on the ray; a click uses where it pointed
  80 ms before the trigger pull, and the Mac pointer holds still until the
  ray moves 1.2 degrees, so clicks don't become drags.
- The Mac lays the separated windows' displays out left/right of the main
  one as their panels sit in the room, so the Mac's own pointer crosses
  into the panel beside it. Verified in the preview.
- Text size: A-/A+ on window panels send the window's own Cmd+-/Cmd+=;
  automatic by default, following distance after a move. WebGL 2 with
  mipmaps and anisotropic filtering, so distant panels don't shimmer.
- The agent logs each stream's start/end and Separate mode's steps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:09:15 +10:00
saphidandClaude Opus 5.5 952b660bf7 Mac space: launch Chromium XR with OpenXR on and the seccomp filter off (as its own launcher does)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:45:26 +10:00
saphidandClaude Opus 5.5 a4d840a148 Mac space (first version): the Mac's screen in the room, windows pulled out as their own panels
- docs/mac-space.md: what Vision Pro's Mac Virtual Display does, what the
  Frame allows (another process can't move SteamVR's panels: verified,
  PermissionDenied), and why a WebXR page comes first.
- ui/mac-stream.js: the stream protocol shared by the viewers (connect,
  clock sync, decode, acks/reports, reconnect, input); mac-view.html uses it.
- ui/mac-space.html: WebXR (passthrough behind) or an in-page preview.
  Desktop panel with a Vision Pro-style bar and close button; laser clicks,
  thumbstick scrolls, squeeze or the bar grabs; dragging a window's title
  bar off the desktop opens it on its own display as a new panel that
  follows the pointer; dropping it back sends it home.
- Agent: space tickets -> a space key (/space/key) that lists windows and
  displays (/space/windows, now with positions) and opens any stream;
  Stop All revokes it. Serves /space and /mac-stream.js.
- Frame Control: "Open Mac space" in the Tools card.

Verified on the Mac (preview in headless Chrome, lab agent): the main
display streams into the panel; a test window detached onto its own
display and streamed; dropped back, it returned to its exact frame and the
display was removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:24:36 +10:00
154 changed files with 7030 additions and 5995 deletions

No files matched your search

+1 -1
View File
@@ -7,7 +7,7 @@ body:
value: |
**Before you start:** read [CONTRIBUTING.md](https://github.com/saphid/frame-control/blob/main/CONTRIBUTING.md).
Issues from new contributors are auto-closed by default. A maintainer reviews them and reopens worthwhile ones. The [website feedback form](https://frame-control.pages.dev/feedback/) skips that queue.
Issues from new contributors are auto-closed by default. A maintainer reviews them and reopens worthwhile ones. The [website feedback form](https://framecontrol.fun/feedback/) skips that queue.
Keep this short. If it doesn't fit on one screen, it's too long. Write in your own voice.
+1 -1
View File
@@ -1,5 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: Feedback form (no GitHub account needed, skips the queue)
url: https://frame-control.pages.dev/feedback/
url: https://framecontrol.fun/feedback/
about: Bugs, ideas and questions from the website become issues here without being auto-closed.
+22
View File
@@ -0,0 +1,22 @@
name: Android phone
on:
pull_request:
paths: ['android/**', 'ui/**', 'scripts/**', 'frame/**', 'apk-catalog/**', 'ios/scripts/make_frame_bundle.py', '.github/workflows/android.yml']
push:
branches: [main, 'feat/phone-screenshot-autosync-108']
paths: ['android/**', 'ui/**', 'scripts/**', 'frame/**', 'apk-catalog/**', 'ios/scripts/make_frame_bundle.py', '.github/workflows/android.yml']
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
- uses: android-actions/setup-android@v3
- uses: gradle/actions/setup-gradle@v4
- run: sdkmanager 'platforms;android-36' 'build-tools;36.0.0'
- run: ./android/gradlew -p android assembleDebug assembleDebugAndroidTest testDebugUnitTest lintDebug --no-daemon
+1 -1
View File
@@ -104,7 +104,7 @@ jobs:
'',
`Maintainers review auto-closed issues regularly and reopen worthwhile ones. Issues that do not meet the quality bar in [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md) will not be reopened or receive a reply.`,
'',
'Just want to report a bug or share an idea? The [website feedback form](https://frame-control.pages.dev/feedback/) skips this queue.',
'Just want to report a bug or share an idea? The [website feedback form](https://framecontrol.fun/feedback/) skips this queue.',
'',
'If a maintainer replies `lgtmi` on one of your issues, your future issues will stay open. If a maintainer replies `lgtm`, your future issues and PRs will stay open. The command must be at the start of the reply (optionally after one or more `@username` mentions) or at the end.',
'',
+4 -28
View File
@@ -26,9 +26,7 @@ jobs:
# "view || create" could race and make duplicate drafts. The draft is tied to
# the pushed tag (the tag must already exist on GitHub, then tag_name is set
# explicitly), so scripts/publish-release.sh and `gh release upload` find it
# by tag. Editing a draft can make GitHub show an untagged-* name again; the
# upload step below recovers that exact draft if tag lookup fails. It is
# created with the REST API so the id comes straight back: the
# by tag. It is created with the REST API so the id comes straight back: the
# release list can lag a fresh draft (v0.4.2's first run found nothing and
# PATCHed releases/ with an empty id). tests/test_release_workflow.py runs
# this step against tests/fakegh.
@@ -42,13 +40,8 @@ jobs:
GH_REPO: ${{ github.repository }}
run: |
tag="${GITHUB_REF_NAME}"
# A rerun finds the draft even if GitHub temporarily shows untagged-*.
id=$(gh api --paginate "repos/$GH_REPO/releases?per_page=100" \
--jq ".[] | select(.tag_name == \"$tag\" or (.draft and (.tag_name | startswith(\"untagged-\")) and .name == \"Frame Control ${tag#v}\")) | .id")
if [ -n "$id" ] && ! [[ "$id" =~ ^[0-9]+$ ]]; then
echo "expected one release for $tag; found: $id" >&2
exit 1
fi
# A rerun finds the draft an earlier attempt made.
id=$(gh api --paginate "repos/$GH_REPO/releases?per_page=100" --jq ".[] | select(.tag_name == \"$tag\") | .id" | head -n 1)
if [ -z "$id" ]; then
# Given a missing tag, GitHub would create it on the default branch.
gh api "repos/$GH_REPO/git/ref/tags/$tag" >/dev/null \
@@ -142,25 +135,8 @@ jobs:
shell: bash
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
run: |
# A release-notes edit can make GitHub expose a draft as untagged-*.
# Keep the normal upload path; recover only that lookup failure.
if result=$(gh release upload "${GITHUB_REF_NAME}" ${{ matrix.files }} --clobber 2>&1); then
printf '%s\n' "$result"
exit 0
fi
printf '%s\n' "$result" >&2
case "$result" in *"release not found"*) ;; *) exit 1 ;; esac
tag="${GITHUB_REF_NAME}"
ids=$(gh api --paginate "repos/$GH_REPO/releases?per_page=100" \
--jq ".[] | select(.draft and .name == \"Frame Control ${tag#v}\" and (.tag_name == \"$tag\" or (.tag_name | startswith(\"untagged-\")))) | .id")
if ! [[ "$ids" =~ ^[0-9]+$ ]]; then
echo "expected one matching draft for $tag; found: ${ids:-none}" >&2
exit 1
fi
gh api -X PATCH "repos/$GH_REPO/releases/$ids" -f tag_name="$tag" >/dev/null
gh release upload "$tag" ${{ matrix.files }} --clobber
gh release upload "${GITHUB_REF_NAME}" ${{ matrix.files }} --clobber
# Tagged builds are attached to the release above; only other builds need artifacts.
- uses: actions/upload-artifact@v4
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
-75
View File
@@ -2,81 +2,6 @@
## Unreleased
- Use gamescope's input for the keyboard and trackpad, following the focused
headset panel without installing an input service. Start KDE Connect only for
accents and emoji, keeping queued text ordered and cancelling it on headset
switches. Retain intended-headset checks before starting either control and
prevent Unicode helper startup from deadlocking with a headset switch.
- Keep accepted keyboard text when the Unicode queue is full: reject new batches
before sending their clicks or keys, and count in-flight text toward the limit.
Retain unsent text after helper failures/timeouts, report the error and resume
it with Try again. Clean up failed workers without touching a new headset's queue.
[Current verification and device limits](docs/evidence/pad-gamescope-2026-10-24.md).
- Keep media playback running when SteamVR refuses the optional theatre dark
surround, including a retry after standby. Log the refusal and play without
the surround. Overlay teardown attempts every owned handle and always calls
OpenVR shutdown even if a handle fails to close.
- Check the desktop server's single-instance lock before starting background
downloads, so a second server exits cleanly with the existing "already
running" message.
- Recheck saved spatial layouts on SteamOS 0.5.6 / SteamVR 2.18.2 (issue #31).
Read-only probes expose some panel geometry, but safe restoration of
Steam/gamescope docking and positions remains unverified. Record exact API
results and a reproducible probe; saved layouts remain unavailable.
- Record packaged v0.4.17 Desktop Capture succeeding on a mapped nested KDE
compositor on a real Frame with SteamOS 0.5.6. Headset overlay visibility was
not established; this is evidence, not a new fix or resolution of issue #73.
- Correct the Return to Game Mode help text: Frame Control selects the switch
supported by the headset's SteamOS build, including `steamosctl` on current
Frames. The button's behavior is unchanged.
- Distinguish an unreadable gamescope panel list from a confirmed empty list in
Desktop Capture and Control (issue #73). Failed discovery now asks users to
check SteamVR and report the exact message and SteamOS version, rather than
claiming no panel is open. This does not establish a new capture fix.
- Add an opt-in desktop low-battery alert without starting a family session.
Existing status reads confirm low charge and drain; recovery hysteresis and a
ten-minute cooldown limit repeat alerts. Off on page open/headset switch;
pauses with hidden-page status refreshes. No extra polling or headset changes.
- Clarify first-time desktop downloads, installation and opening on the website
and README, including CPU choices, unsigned-app warnings, pairing fallback
and the source-only iOS build. Keep section navigation available on mobile.
- Treat headset aliases and other CSV metadata as plain text in power and heat
exports, so spreadsheet formula prefixes cannot run when a report is opened.
- Add desktop Tools → Power & heat report: bounded read-only one/five-minute
observations with progress, stop and HTML/JSON/CSV downloads. Received samples
survive stop, disconnect and headset switch, labeled incomplete and tied to the
original headset. Shared sampler/report modules ship with the app; no power or
thermal controls are added. Download before replacing a report or quitting.
- Fix Return to Game Mode misidentifying full Plasma Desktop Mode when its VR
overlay also runs gamescope (issue #142). Session targets now determine the
mode; conflicting or incomplete evidence stays unknown. Use SteamOS's
`steamosctl` switch when available, with older selector commands retained.
- Export read-only power and heat observations to an offline accessible HTML
report with JSON/CSV downloads. `--html FILE` limits sessions to ten minutes;
Ctrl+C saves received samples with an explicit incomplete state. Units,
unknown counts and sources remain visible; no headset controls are added.
- Add a guarded `frame-doctor.sh --plan` with read-only, evidence-specific next
steps and explicit local baseline reports for before/after diagnostics (issue
#34). Restart risk blocks plans/comparisons; no automatic repairs, password
changes or headset security changes are performed.
- Recover a tagged release upload when GitHub temporarily displays its draft
as `untagged-…` after an edit, without creating a duplicate draft or
silently retrying unrelated upload failures.
- Import standalone APK/OBB backups from XAPK/APKS/APKM archives through the
existing preview and game-data setup flow. Unsafe archives, split sets and
Android/data restores are refused with guidance. This does not establish RE4
+46 -62
View File
@@ -7,19 +7,19 @@
**Manage your Valve Steam Frame from your computer.**<br>
See what the headset sees, install games and Android apps, move files and text across, and check battery and status, all over SSH.
[![Latest release](https://img.shields.io/github/v/release/saphid/frame-control?label=release&color=1a9fff)](https://github.com/saphid/frame-control/releases/latest)
[![Latest release](https://img.shields.io/github/v/release/saphid/steam-frame?label=release&color=1a9fff)](https://github.com/saphid/steam-frame/releases/latest)
[![Platforms](https://img.shields.io/badge/macOS%20%7C%20Windows%20%7C%20Linux-2a475e?label=runs%20on)](#install)
[![Checks](https://img.shields.io/github/actions/workflow/status/saphid/frame-control/checks.yml?branch=main&label=checks)](https://github.com/saphid/frame-control/actions/workflows/checks.yml)
[![Checks](https://img.shields.io/github/actions/workflow/status/saphid/steam-frame/checks.yml?branch=main&label=checks)](https://github.com/saphid/steam-frame/actions/workflows/checks.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-66c0f4)](LICENSE)
[**Website**](https://frame-control.pages.dev) · [**Download**](#install) · [Trailer](#trailer) · [Features](#features) · [Set up the headset](#set-up-the-headset) · [Feedback](#feedback) · [Docs](#going-further)
[**Get started**](https://framecontrol.fun) · [**Download**](https://framecontrol.fun/#download) · [Trailer](#trailer) · [Features](#features) · [Set up the headset](#set-up-the-headset) · [Feedback](#feedback) · [Docs](#going-further)
<br>
<img src="docs/img/frame-control.png" alt="Frame Control's Games tab: installed games, sideloaded titles, and your Steam library with Frame ratings" width="900">
<a id="trailer"></a>
<a href="https://github.com/saphid/frame-control/releases/download/trailer/frame-control-trailer.mp4"><img src="docs/img/trailer.jpg" alt="Watch the Frame Control trailer" width="900"></a>
<a href="https://github.com/saphid/steam-frame/releases/download/trailer/frame-control-trailer.mp4"><img src="docs/img/trailer.jpg" alt="Watch the Frame Control trailer" width="900"></a>
<sub>The trailer: 66 seconds, with sound. Downloads the MP4 from the trailer release.</sub>
@@ -75,8 +75,8 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a
**📸 Screenshots and recordings**<br>
Browse the shots you take in the headset and save them to your Pictures folder, or have new ones and your Steam clips copied over automatically. Record what the headset shows, with its sound, or export Steam's clips at any length, as .mp4 files in your Videos folder ([not tested on a headset yet](docs/recording.md)).
**⌨️ Keyboard and trackpad**<br>
Type and point in the headset from your computer or phone, through Valve's own input path, with nothing to install. Accents and emoji use KDE Connect, which Frame Control brings along and sets up the first time you type one. Copy text to the headset's clipboard and back again. Press T to open it.
**⌨️ Type on the headset**<br>
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Copy text to the headset's clipboard and back again. Nothing else to install, anywhere. Press T to open it.
</td>
</tr>
@@ -116,30 +116,32 @@ already ships (sideloading a game copies Valve's own devkit scripts to
## Install
**Install Frame Control on your computer, not on the headset.** The website is
for downloads and instructions; it does not connect to your Frame in a browser.
Choose the operating system and processor of the computer you will use.
**New here? [Get Frame Control and follow the setup guide on the website](https://framecontrol.fun/#download).**
Install it on your **computer, not the headset**. You don't need to clone this
repository or install Python. Download for your computer, install and open the
app, then choose **Set Up Connection** with the Frame awake on the same network.
- **macOS:** open the `.dmg`, drag Frame Control to Applications, then open it there.
- **Windows:** run the installer and launch Frame Control from Start, or extract the portable `.zip` and run `Frame Control.exe`.
- **Linux:** make the AppImage executable and open it, or install the `.deb` and launch Frame Control.
If the app is blocked, see the first-launch notes below before proceeding.
| | Download | Needs |
|---|---|---|
| **macOS** (Apple Silicon) | [Frame-Control-mac-arm64.dmg](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-mac-arm64.dmg) | Nothing extra |
| **Windows** 10 / 11 (x64) | [Frame-Control-Setup-x64.exe](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-Setup-x64.exe) · [portable .zip](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-win-x64.zip) | OpenSSH Client (normally installed) |
| **Linux** (x64) | [AppImage](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-linux-x86_64.AppImage) · [.deb](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-linux-amd64.deb) | `ssh` (most desktops have it) |
| **Linux** (arm64) | [AppImage](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-linux-arm64.AppImage) · [.deb](https://github.com/saphid/frame-control/releases/latest/download/Frame-Control-linux-arm64.deb) | `ssh`, and `adb` for Android apps (`sudo apt install adb`) |
| **macOS** (Apple Silicon) | [Frame-Control-mac-arm64.dmg](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-mac-arm64.dmg) | Nothing extra |
| **Windows** 10 / 11 (x64) | [Frame-Control-Setup-x64.exe](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-Setup-x64.exe) · [portable .zip](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-win-x64.zip) | Nothing extra |
| **Linux** (x64) | [AppImage](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-x86_64.AppImage) · [.deb](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-amd64.deb) | `ssh` (most desktops have it) |
| **Linux** (arm64) | [AppImage](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-arm64.AppImage) · [.deb](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-arm64.deb) | `ssh`, and `adb` for Android apps (`sudo apt install adb`) |
**Which build?** On a Mac, check Apple menu → About This Mac: the download is
for Apple Silicon (M-series); no Intel Mac build is published. On Windows,
check Settings → System → About → System type: the release is x64, not a native
Windows ARM build. On Linux, `uname -m` reports `x86_64` (x64) or `aarch64`
(arm64). Use `.deb` on Debian/Ubuntu, or AppImage on other distributions.
**iPhone and iPad:** the same features from your phone, with nothing to install on
a computer. Build it from [`ios/`](ios) in Xcode; see [docs/iphone.md](docs/iphone.md).
**iPhone and iPad:** no App Store, TestFlight or `.ipa` download is provided in
the current release. It runs from your phone without a computer after setup.
Build it from [`ios/`](ios) in Xcode on a Mac with your own signing team; see
[docs/iphone.md](docs/iphone.md).
**Android phones (development):** a native companion is in [`android/`](android).
It can download new screenshots during a user-started sync session, with Gallery
saving enabled by default. See [build instructions and current testing limits](docs/android-phone.md).
The app brings its own Python and `adb`. macOS includes SSH; Windows needs the
OpenSSH Client optional feature (normally installed).
The app brings its own Python and `adb`; SSH is built into macOS and Windows.
From 0.4 it updates itself: when a new version is published, a banner offers
**Update and restart**. It sends anonymous usage statistics, which you can turn
off. Sharing compatibility results and error details is opt-in. See
@@ -152,7 +154,7 @@ distribution's. If you already have `adb`, the app uses yours.
There's no paid Apple developer account behind it, so macOS says the app is
damaged or can't be checked. Drag it to Applications, then clear the download
quarantine once, only for a download you trust from the releases linked above:
quarantine once:
```sh
xattr -dr com.apple.quarantine "/Applications/Frame Control.app"
@@ -181,44 +183,26 @@ If it complains about FUSE, install `libfuse2` (Ubuntu 24.04+: `libfuse2t64`),
or run it with `--appimage-extract-and-run`.
</details>
## Open the app
- **macOS:** open the `.dmg`, drag **Frame Control** to **Applications**, then
open it from Applications. For the unsigned-app warning, use the macOS steps
above. Allow local network access and control of Terminal when prompted.
- **Windows:** run the installer, then open **Frame Control** from Start. For
the portable download, extract the entire `.zip` first and run
`Frame Control.exe` inside it. See the SmartScreen note above.
- **Linux:** install the `.deb` with your package installer and launch
**Frame Control** from your applications menu; or make the AppImage executable
and run it as described above. Install the OpenSSH client if `ssh` is missing.
## Set up the headset
Keep the Frame awake and connect it and your computer to the same trusted local
network. Guest Wi-Fi/client isolation can prevent discovery and connections.
You type one password on the headset, once. Everything else happens on your
computer.
1. **On the Frame:** Steam Settings → System → **Enable Developer Mode**, then
Developer → **Set User Password**. Choose a password that isn't easy to guess:
it also protects `sudo` and remote desktop, even after SSH keys are set up.
2. **On your computer:** open Frame Control and choose **Set Up Connection…**.
If you dismissed the first-run prompt, use **Frame → Set Up Connection…**.
Setup opens a terminal and tries to find the headset. If it can't, use its IP
address from Quick Settings or Steam Settings → Internet. Windows/Linux setup
prompts for it; macOS exits instead. On macOS, return to the app →
**Devices & connection → + Add a headset…**, set SSH alias to `frame` for
the default headset, fill in **Address**, then choose **Set Up…**.
3. **Pair or enter the password:** setup first tries Valve's devkit pairing.
Open Steam Settings → Developer → **Pair new host** in the headset and approve
only the request you just started. The service and pairing-mode step are
verified on a Frame; approval and key installation through that flow remain
unverified. If pairing fails or times out, setup falls back to asking for the
Developer Mode password in the computer's terminal. Password typing may show
no characters; press Enter when done. See [SSH details](docs/ssh.md#password-free-pairing-steamos-devkit-service).
4. After setup succeeds, return to Frame Control. It uses SSH keys for later
connections whenever the Frame is awake and reachable. If it stays offline,
check Developer Mode, the address and network isolation, then run setup again.
For connections away from home, see [Tailscale](docs/tailscale.md).
in the Developer section, **Set User Password**. Pick something short:
you'll type it once more on your computer and then never again.
2. **On your computer:** open Frame Control. It offers to **Set Up
Connection**, which finds the headset, creates an SSH key, and asks for that
password once in a terminal window. If it can't find the Frame, type the
IP address from the Frame's Quick Settings.
Before asking for the password it tries Valve's SteamOS devkit pairing: in
the headset, open Steam Settings → Developer → **Pair new host** and approve
the request, and no password is needed. (The service and the pairing-mode
step are verified on a Frame; the approval itself isn't yet. See
[SSH](docs/ssh.md#password-free-pairing-steamos-devkit-service).)
3. That's it. The app now reaches the headset whenever it's awake and on the
same network. For anywhere else, see [Tailscale](docs/tailscale.md).
**What it changes:** only what you click. Installs go to your user account on
the Frame (`--user` Flatpaks, Lepton instances, Steam downloads, sideloaded
@@ -265,9 +249,9 @@ Frame's software fits together, all checked against a real headset and labelled
| [VR mods and custom songs](docs/mods.md) | UEVR for Unreal games from Frame Control (install, play in VR, remove), the per-game table, Beat Saber blockers |
| [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes |
| [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing |
| [Frame Control for Android phones](docs/android-phone.md) | Development build, pairing, screenshot downloads and foreground sync limits |
| [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset |
| [AI agents and assistant](docs/agents.md) | Key-free MCP tools, human approvals, and an opt-in assistant panel |
| [Power and heat measurements](docs/power-and-heat.md) | Read-only sampling and offline HTML reports with JSON/CSV downloads |
| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, the headset smoke test and a Windows test VM |
| [Open questions](docs/open-questions.md) | What's still unchecked |
@@ -283,8 +267,8 @@ Frame's software fits together, all checked against a real headset and labelled
loopback `127.0.0.1:8080`) only through SSH tunnels. The compatibility
database key (maintainer-only) is never written to the repo.
- `steamos` has `sudo`, protected by the same Developer Mode password. Once
you've switched to key auth, that password still protects `sudo` and RDP,
so pick one that isn't trivially guessable.
you've switched to key auth, a short password still protects `sudo` and
RDP, so pick one that isn't trivially guessable.
- Don't port-forward 22, 3389, or 5555–5599 from your router. For remote access,
use Tailscale: `scripts/tailscale-on-frame.sh` (no sudo). In its userspace mode
**every** Frame port is reachable from your tailnet, including Steam's DevTools
+5
View File
@@ -0,0 +1,5 @@
.gradle/
**/build/
local.properties
*.keystore
*.jks
+35
View File
@@ -0,0 +1,35 @@
plugins { id 'com.android.application' }
android {
namespace 'com.saphid.framecontrol'
compileSdk 36
defaultConfig {
applicationId 'com.saphid.framecontrol'
minSdk 29
targetSdk 36
versionCode 1
versionName '0.1.0'
testInstrumentationRunner 'androidx.test.runner.AndroidJUnitRunner'
}
compileOptions { sourceCompatibility JavaVersion.VERSION_17; targetCompatibility JavaVersion.VERSION_17 }
sourceSets.main.assets.srcDir layout.buildDirectory.dir('generated/frameAssets').get().asFile
}
def frameAssets = tasks.register('frameAssets', Exec) {
workingDir rootProject.projectDir.parentFile
commandLine 'python3', 'android/scripts/make_assets.py', layout.buildDirectory.dir('generated/frameAssets').get().asFile
inputs.files(fileTree(rootProject.file('../ui')), fileTree(rootProject.file('../scripts')), fileTree(rootProject.file('../frame')), fileTree(rootProject.file('../apk-catalog')) { exclude 'data/**' })
inputs.file(rootProject.file('../ios/scripts/make_frame_bundle.py'))
inputs.file(rootProject.file('scripts/make_assets.py'))
outputs.dir layout.buildDirectory.dir('generated/frameAssets')
}
tasks.named('preBuild').configure { dependsOn frameAssets }
dependencies {
implementation 'com.github.mwiede:jsch:2.28.8'
implementation 'com.google.code.gson:gson:2.13.2'
implementation 'androidx.core:core:1.17.0'
testImplementation 'junit:junit:4.13.2'
androidTestImplementation 'androidx.test:runner:1.7.0'
androidTestImplementation 'androidx.test.ext:junit:1.3.0'
}
@@ -0,0 +1,409 @@
package com.saphid.framecontrol;
import static org.junit.Assert.*;
import android.app.Activity;
import android.app.Instrumentation;
import android.content.*;
import android.database.Cursor;
import android.graphics.*;
import android.net.Uri;
import android.provider.MediaStore;
import android.view.View;
import android.view.ViewGroup;
import android.widget.Button;
import androidx.test.ext.junit.runners.AndroidJUnit4;
import androidx.test.platform.app.InstrumentationRegistry;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.*;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.*;
import org.junit.runner.RunWith;
@RunWith(AndroidJUnit4.class)
public class ScreenshotDeviceTest {
private Context context;
private final Set<Uri> cleanup = new LinkedHashSet<>();
@Before
public void setup() {
context = InstrumentationRegistry.getInstrumentation().getTargetContext();
}
@After
public void cleanup() {
for (Uri uri : cleanup) context.getContentResolver().delete(uri, null, null);
}
private String name(String headset, SyncEngine.Shot shot) throws Exception {
String file = shot.id.substring(shot.id.lastIndexOf('/') + 1);
int dot = file.lastIndexOf('.');
return file.substring(0, dot)
+ "_"
+ FrameConnection.hash(headset + "/" + shot.id).substring(0, 20)
+ file.substring(dot);
}
private Uri row(String headset, SyncEngine.Shot shot) throws Exception {
Uri collection = MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY);
@SuppressWarnings("deprecation")
Uri includingPending = MediaStore.setIncludePending(collection);
try (Cursor c =
context
.getContentResolver()
.query(
includingPending,
new String[] {"_id"},
MediaStore.Images.Media.DISPLAY_NAME
+ "=? AND "
+ MediaStore.Images.Media.RELATIVE_PATH
+ "=?",
new String[] {name(headset, shot), "Pictures/Frame Control/"},
null)) {
if (c != null && c.moveToFirst()) {
Uri uri = ContentUris.withAppendedId(collection, c.getLong(0));
cleanup.add(uri);
assertFalse("Duplicate Gallery row", c.moveToNext());
return uri;
}
return null;
}
}
@Test
public void actualGalleryOriginalAndAppOnlySurviveRelaunch() throws Exception {
String headset = "instrumented-" + UUID.randomUUID();
File root = new File(context.getFilesDir(), "instrumented/" + headset);
Object gate = new Object();
Bitmap bitmap = Bitmap.createBitmap(48, 32, Bitmap.Config.ARGB_8888);
bitmap.eraseColor(Color.rgb(25, 184, 143));
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bytes);
bitmap.recycle();
byte[] png = bytes.toByteArray();
SyncEngine.Transport transport =
new SyncEngine.Transport() {
public void download(SyncEngine.Shot shot, File target) throws Exception {
Files.write(target.toPath(), png);
}
public boolean validImage(File f) {
Bitmap b = BitmapFactory.decodeFile(f.getPath());
if (b == null) return false;
b.recycle();
return true;
}
};
GalleryExporter exporter = new GalleryExporter(context, headset, gate);
SyncEngine engine = new SyncEngine(root, true);
engine.reconcile(Collections.emptyList(), transport, exporter);
SyncEngine.Shot shot =
new SyncEngine.Shot("999999999991/108/20261024000000_1.png", png.length, 1792800000);
engine.reconcile(List.of(shot), transport, exporter);
assertNull(row(headset, shot));
engine.reconcile(List.of(shot), transport, exporter);
Uri saved = row(headset, shot);
assertNotNull(saved);
try (InputStream in = context.getContentResolver().openInputStream(saved)) {
assertArrayEquals(png, read(in));
}
assertEquals("saved", engine.items().get(0).destination);
engine = new SyncEngine(root, true);
engine.reconcile(List.of(shot), transport, exporter);
assertEquals(saved, row(headset, shot));
engine.setGallery(false);
SyncEngine.Shot second =
new SyncEngine.Shot("999999999991/108/20261024000001_1.png", png.length, 1792800001);
engine.reconcile(List.of(shot, second), transport, exporter);
engine.reconcile(List.of(shot, second), transport, exporter);
engine.setGallery(true);
engine = new SyncEngine(root, true);
engine.reconcile(List.of(shot, second), transport, exporter);
assertNull(row(headset, second));
assertTrue(
engine.items().stream()
.anyMatch(
i ->
i.shot.id.equals(second.id)
&& i.destination.equals("appOnly")
&& i.file.exists()));
}
@Test
public void interruptedMediaStoreRowIsRewrittenWithoutDuplicate() throws Exception {
String headset = "pending-" + UUID.randomUUID();
SyncEngine.Shot shot =
new SyncEngine.Shot("999999999992/108/20261024000002_1.png", 4, 1792800002);
ContentValues values = new ContentValues();
values.put(MediaStore.Images.Media.DISPLAY_NAME, name(headset, shot));
values.put(MediaStore.Images.Media.RELATIVE_PATH, "Pictures/Frame Control/");
values.put(MediaStore.Images.Media.MIME_TYPE, "image/png");
values.put(MediaStore.Images.Media.IS_PENDING, 1);
Uri pending =
context
.getContentResolver()
.insert(
MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY), values);
assertNotNull(pending);
cleanup.add(pending);
File file = File.createTempFile("gallery-proof", ".png", context.getCacheDir());
byte[] original = {1, 2, 3, 4};
Files.write(file.toPath(), original);
GalleryExporter exporter = new GalleryExporter(context, headset, new Object());
assertTrue(exporter.export(shot, file, () -> true));
assertEquals(pending, row(headset, shot));
assertTrue(exporter.export(shot, file, () -> true));
assertEquals(pending, row(headset, shot));
try (InputStream in = context.getContentResolver().openInputStream(pending)) {
assertArrayEquals(original, read(in));
}
file.delete();
}
@Test
public void galleryOptOutLatchesPreviouslyPairedLibraries() throws Exception {
assertFalse("Instrumentation must own a stopped product", SyncService.running());
File files = Files.createTempDirectory(context.getCacheDir().toPath(), "libraries-").toFile();
String preferences = "library-test-" + UUID.randomUUID();
Context isolated =
new ContextWrapper(context) {
@Override
public File getFilesDir() {
return files;
}
@Override
public SharedPreferences getSharedPreferences(String name, int mode) {
return context.getSharedPreferences(preferences, mode);
}
};
try {
SyncEngine.Transport transfer =
new SyncEngine.Transport() {
public void download(SyncEngine.Shot shot, File target) throws Exception {
Files.write(target.toPath(), new byte[] {1, 2, 3, 4});
}
public boolean validImage(File file) {
return true;
}
};
SyncEngine.Exporter unavailable = (shot, file, eligible) -> false;
SyncEngine.Shot shot = new SyncEngine.Shot("123/108/20261024120000_1.png", 4, 100);
File a = SyncService.library(isolated, "headset-a");
File b = SyncService.library(isolated, "headset-b");
for (File root : List.of(a, b)) {
SyncEngine engine = new SyncEngine(root, true);
engine.reconcile(Collections.emptyList(), transfer, unavailable);
engine.reconcile(List.of(shot), transfer, unavailable);
engine.reconcile(List.of(shot), transfer, unavailable);
assertEquals("pending", engine.items().get(0).destination);
}
assertTrue(
SyncService.prefs(isolated)
.edit()
.putString("pin", "headset-b")
.putBoolean("gallery", true)
.commit());
SyncService.gallery(isolated, false);
SyncService.gallery(isolated, true);
for (File root : List.of(a, b)) {
SyncEngine engine = new SyncEngine(root, true);
assertEquals("appOnly", engine.items().get(0).destination);
engine.reconcile(
List.of(shot),
transfer,
(s, f, eligible) -> {
fail("Opted-out items must never be exported after re-pairing");
return false;
});
}
} finally {
SyncService.prefs(isolated).edit().clear().commit();
try (java.util.stream.Stream<java.nio.file.Path> paths = Files.walk(files.toPath())) {
for (java.nio.file.Path path :
paths.sorted(Comparator.reverseOrder()).collect(java.util.stream.Collectors.toList()))
Files.delete(path);
}
}
}
@Test
public void changingHeadsetsRebindsShareToTheSelectedOriginal() throws Exception {
assertFalse(SyncService.running());
String aPin = "switch-a-" + UUID.randomUUID(), bPin = "switch-b-" + UUID.randomUUID();
File a = SyncService.library(context, aPin), b = SyncService.library(context, bPin);
SharedPreferences prefs = SyncService.prefs(context);
String previousPin = prefs.getString("pin", null);
boolean previousGallery = prefs.getBoolean("gallery", true);
Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation();
AtomicReference<Intent> shared = new AtomicReference<>();
Instrumentation.ActivityMonitor monitor =
new Instrumentation.ActivityMonitor() {
@Override
public Instrumentation.ActivityResult onStartActivity(Intent intent) {
if (!Intent.ACTION_CHOOSER.equals(intent.getAction())) return null;
shared.set(intent.getParcelableExtra(Intent.EXTRA_INTENT));
return new Instrumentation.ActivityResult(Activity.RESULT_CANCELED, null);
}
};
Activity activity = null;
try {
SyncEngine.Shot shot = new SyncEngine.Shot("123/108/20261024120000_1.png", 4, 100);
for (File root : List.of(a, b)) {
byte[] original = root.equals(a) ? new byte[] {1, 2, 3, 4} : new byte[] {5, 6, 7, 8};
SyncEngine.Transport transfer =
new SyncEngine.Transport() {
public void download(SyncEngine.Shot s, File file) throws Exception {
Files.write(file.toPath(), original);
}
public boolean validImage(File file) {
return true;
}
};
SyncEngine engine = new SyncEngine(root, false);
engine.reconcile(Collections.emptyList(), transfer, (s, f, e) -> false);
engine.reconcile(List.of(shot), transfer, (s, f, e) -> false);
engine.reconcile(List.of(shot), transfer, (s, f, e) -> false);
}
assertTrue(prefs.edit().putString("pin", aPin).putBoolean("gallery", false).commit());
activity =
instrumentation.startActivitySync(
new Intent(context, MainActivity.class)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TASK));
instrumentation.addMonitor(monitor);
instrumentation.waitForIdleSync();
Activity screen = activity;
Runnable share =
() -> {
Button button = shareButton(screen.findViewById(android.R.id.content));
assertNotNull("The downloaded original must be reachable", button);
assertTrue(button.performClick());
};
instrumentation.runOnMainSync(share);
assertNotNull(shared.get());
Uri first = shared.get().getParcelableExtra(Intent.EXTRA_STREAM);
try (InputStream in = context.getContentResolver().openInputStream(first)) {
assertArrayEquals(new byte[] {1, 2, 3, 4}, read(in));
}
assertTrue(prefs.edit().putString("pin", bPin).commit());
long deadline = System.nanoTime() + java.util.concurrent.TimeUnit.SECONDS.toNanos(5);
Uri selected;
do {
instrumentation.runOnMainSync(share);
selected = shared.get().getParcelableExtra(Intent.EXTRA_STREAM);
if (!first.equals(selected)) break;
Thread.sleep(100);
} while (System.nanoTime() < deadline);
assertNotEquals("Share must no longer target the previous headset", first, selected);
try (InputStream in = context.getContentResolver().openInputStream(selected)) {
assertArrayEquals(new byte[] {5, 6, 7, 8}, read(in));
}
} finally {
instrumentation.removeMonitor(monitor);
Activity screen = activity;
if (screen != null) instrumentation.runOnMainSync(screen::finish);
SharedPreferences.Editor restore = prefs.edit().putBoolean("gallery", previousGallery);
if (previousPin == null) restore.remove("pin");
else restore.putString("pin", previousPin);
assertTrue(restore.commit());
for (File root : List.of(a, b)) {
if (!root.exists()) continue;
try (java.util.stream.Stream<java.nio.file.Path> paths = Files.walk(root.toPath())) {
for (java.nio.file.Path path :
paths.sorted(Comparator.reverseOrder()).collect(java.util.stream.Collectors.toList()))
Files.delete(path);
}
}
}
}
private static Button shareButton(View view) {
if (view instanceof Button && "Open / share original".contentEquals(((Button) view).getText()))
return (Button) view;
if (view instanceof ViewGroup) {
ViewGroup group = (ViewGroup) view;
for (int i = 0; i < group.getChildCount(); i++) {
Button found = shareButton(group.getChildAt(i));
if (found != null) return found;
}
}
return null;
}
/** Optional real-headset provisioning. Only this test APK exposes the hook. */
@Test
public void provisionRealHeadsetIfRequested() throws Exception {
android.os.Bundle args = InstrumentationRegistry.getArguments();
Assume.assumeTrue("Optional real-headset provisioning", "true".equals(args.getString("live")));
String host = args.getString("host"), pin = args.getString("pin");
assertNotNull(host);
assertNotNull(pin);
assertEquals(
"Probe must return the expected host key before authentication",
pin,
FrameConnection.probe(host, 22, "steamos"));
String publicKey = new PhoneIdentity().authorizedKey(SyncService.install(context));
Files.write(
new File(context.getFilesDir(), "live-test-public.pub").toPath(),
(publicKey + "\n").getBytes(StandardCharsets.US_ASCII));
assertTrue(
SyncService.prefs(context)
.edit()
.putString("host", host)
.putString("user", "steamos")
.putInt("port", 22)
.putString("pin", pin)
.commit());
}
@Test
public void verifyRealHeadsetIfRequested() throws Exception {
android.os.Bundle args = InstrumentationRegistry.getArguments();
String id = args.getString("verifyId");
Assume.assumeNotNull(id);
String pin = SyncService.prefs(context).getString("pin", null);
assertNotNull(pin);
SyncEngine engine =
new SyncEngine(
SyncService.library(context, pin),
SyncService.prefs(context).getBoolean("gallery", true));
SyncEngine.Item item =
engine.items().stream().filter(i -> i.shot.id.equals(id)).findFirst().orElseThrow();
assertTrue(item.file.exists());
String expected = args.getString("sha256");
assertNotNull(expected);
assertEquals(expected, sha(Files.readAllBytes(item.file.toPath())));
boolean gallery = "true".equals(args.getString("gallery"));
Uri saved = row(pin, item.shot);
cleanup.clear();
if (gallery) {
assertEquals("saved", item.destination);
assertNotNull(saved);
try (InputStream in = context.getContentResolver().openInputStream(saved)) {
assertEquals(expected, sha(read(in)));
}
} else {
assertEquals("appOnly", item.destination);
assertNull(saved);
}
}
private static String sha(byte[] bytes) throws Exception {
StringBuilder out = new StringBuilder();
for (byte b : java.security.MessageDigest.getInstance("SHA-256").digest(bytes))
out.append(String.format(java.util.Locale.ROOT, "%02x", b & 255));
return out.toString();
}
private static byte[] read(InputStream input) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] b = new byte[4096];
int n;
while ((n = input.read(b)) != -1) out.write(b, 0, n);
return out.toByteArray();
}
}
+19
View File
@@ -0,0 +1,19 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_CONNECTED_DEVICE" />
<application android:label="Frame Control" android:theme="@style/AppTheme"
android:icon="@drawable/ic_frame" android:allowBackup="false" android:dataExtractionRules="@xml/data_extraction"
android:networkSecurityConfig="@xml/network_security" android:supportsRtl="true">
<activity android:name=".MainActivity" android:exported="true" android:configChanges="orientation|screenSize|keyboardHidden">
<intent-filter><action android:name="android.intent.action.MAIN" /><category android:name="android.intent.category.LAUNCHER" /></intent-filter>
</activity>
<service android:name=".SyncService" android:exported="false" android:foregroundServiceType="connectedDevice" />
<provider android:name="androidx.core.content.FileProvider" android:authorities="${applicationId}.files" android:exported="false" android:grantUriPermissions="true">
<meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/file_paths" />
</provider>
</application>
</manifest>
@@ -0,0 +1,414 @@
package com.saphid.framecontrol;
import android.content.Context;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import com.jcraft.jsch.*;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.*;
import java.util.concurrent.*;
import java.util.regex.*;
/** All headset traffic goes through one pinned, authenticated SSH connection. */
final class FrameConnection implements AutoCloseable, SyncEngine.Transport {
private final Context context;
private Session session;
private ChannelExec server;
private PipedOutputStream serverInput;
private int localPort;
private String apiKey;
private static final String HOST_ALGORITHMS = "ecdsa-sha2-nistp256,rsa-sha2-512,rsa-sha2-256";
FrameConnection(Context context) {
this.context = context.getApplicationContext();
}
static String fingerprint(String pin) throws Exception {
return "SHA256:"
+ Base64.getEncoder()
.withoutPadding()
.encodeToString(
MessageDigest.getInstance("SHA-256").digest(Base64.getDecoder().decode(pin)));
}
static String hash(String value) throws Exception {
byte[] bytes =
MessageDigest.getInstance("SHA-256").digest(value.getBytes(StandardCharsets.UTF_8));
StringBuilder result = new StringBuilder();
for (byte b : bytes) result.append(String.format(Locale.ROOT, "%02x", b & 255));
return result.toString();
}
static String quote(String s) {
return "'" + s.replace("'", "'\\''") + "'";
}
private static Session session(JSch jsch, String host, int port, String user) throws Exception {
if (host.trim().isEmpty()
|| host.length() > 253
|| host.contains("/")
|| host.chars().anyMatch(Character::isWhitespace)
|| port < 1
|| port > 65535
|| !user.matches("[a-zA-Z0-9_-]{1,32}"))
throw new IOException("Check the headset address, port and username.");
Session s = jsch.getSession(user, host, port);
s.setConfig("StrictHostKeyChecking", "yes");
s.setConfig("server_host_key", HOST_ALGORITHMS);
s.setConfig("PreferredAuthentications", "publickey");
s.setTimeout(20000);
s.setServerAliveInterval(15000);
s.setServerAliveCountMax(2);
return s;
}
private static final class PinRepository implements HostKeyRepository {
String observed;
final String expected;
PinRepository(String expected) {
this.expected = expected;
}
@Override
public int check(String host, byte[] key) {
observed = Base64.getEncoder().encodeToString(key);
return observed.equals(expected) ? OK : CHANGED;
}
@Override
public void add(HostKey k, UserInfo u) {}
@Override
public void remove(String h, String t) {}
@Override
public void remove(String h, String t, byte[] k) {}
@Override
public String getKnownHostsRepositoryID() {
return "Frame Control pinned headset";
}
@Override
public HostKey[] getHostKey() {
return new HostKey[0];
}
@Override
public HostKey[] getHostKey(String h, String t) {
return new HostKey[0];
}
}
/** Intentionally refuses the key exchange before any authentication is sent. */
static String probe(String host, int port, String user) throws Exception {
JSch jsch = new JSch();
PinRepository pin = new PinRepository(null);
jsch.setHostKeyRepository(pin);
Session s = session(jsch, host, port, user);
try {
s.connect(10000);
} catch (JSchException e) {
if (pin.observed == null) throw e;
} finally {
s.disconnect();
}
return pin.observed;
}
static void pair(String host, int port, String user, String pin, String password, String install)
throws Exception {
JSch jsch = new JSch();
jsch.setHostKeyRepository(new PinRepository(pin));
Session s = session(jsch, host, port, user);
s.setConfig("PreferredAuthentications", "password,keyboard-interactive");
s.setPassword(password);
s.setUserInfo(
new UserInfo() {
public String getPassphrase() {
return null;
}
public String getPassword() {
return password;
}
public boolean promptPassword(String x) {
return true;
}
public boolean promptPassphrase(String x) {
return false;
}
public boolean promptYesNo(String x) {
return false;
}
public void showMessage(String x) {}
});
try {
s.connect(15000);
String key = new PhoneIdentity().authorizedKey(install);
run(
s,
"umask 077; mkdir -p ~/.ssh && chmod 700 ~/.ssh && touch ~/.ssh/authorized_keys && chmod"
+ " 600 ~/.ssh/authorized_keys && (grep -qxF "
+ quote(key)
+ " ~/.ssh/authorized_keys || printf '%s\\n' "
+ quote(key)
+ " >> ~/.ssh/authorized_keys)",
10000);
} finally {
s.disconnect();
}
// Verify the new key before storing any pairing settings.
JSch verified = new JSch();
verified.setHostKeyRepository(new PinRepository(pin));
verified.addIdentity(new PhoneIdentity(), null);
Session check = session(verified, host, port, user);
try {
check.connect(15000);
} finally {
check.disconnect();
}
}
static String run(Session s, String command, int timeout) throws Exception {
ChannelExec channel = (ChannelExec) s.openChannel("exec");
channel.setCommand(command);
channel.setInputStream(null);
channel.setErrStream(
new OutputStream() {
public void write(int b) {}
public void write(byte[] b, int offset, int length) {}
});
InputStream input = channel.getInputStream();
long deadline = System.nanoTime() + TimeUnit.MILLISECONDS.toNanos(timeout);
ByteArrayOutputStream output = new ByteArrayOutputStream();
try {
channel.connect(timeout);
while (true) {
while (input.available() > 0) {
byte[] b = new byte[Math.min(input.available(), 8192)];
int n = input.read(b);
if (n > 0) output.write(b, 0, n);
if (output.size() > 2 * 1024 * 1024)
throw new IOException("Headset command returned too much data.");
}
if (channel.isClosed() && input.available() == 0) break;
if (System.nanoTime() > deadline) throw new IOException("The headset command timed out.");
Thread.sleep(25);
}
if (channel.getExitStatus() != 0)
throw new IOException("The headset command failed (" + channel.getExitStatus() + ").");
return output.toString(StandardCharsets.UTF_8.name());
} finally {
channel.disconnect();
}
}
void connect(String host, int port, String user, String pin, String install) throws Exception {
JSch jsch = new JSch();
jsch.setHostKeyRepository(new PinRepository(pin));
jsch.addIdentity(new PhoneIdentity(), null);
session = session(jsch, host, port, user);
session.connect(15000);
String version;
try (InputStream in = context.getAssets().open("frame-bundle.version")) {
version = new String(readBounded(in, 128), StandardCharsets.US_ASCII).trim();
}
if (!version.matches("[0-9a-f]{16}")) throw new IOException("Invalid bundled server version.");
String dir = ".cache/frame-control/" + version;
String present =
run(session, "test -f " + dir + "/ui/server.py && printf ready || true", 10000);
if (!present.equals("ready")) {
run(session, "mkdir -p .cache/frame-control", 10000);
String temp = dir + ".android-" + UUID.randomUUID();
ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");
try (InputStream in = context.getAssets().open("frame-bundle.tar.gz")) {
sftp.connect(10000);
sftp.put(in, temp + ".tar.gz");
} finally {
sftp.disconnect();
}
run(
session,
"mkdir "
+ temp
+ " && tar xzf "
+ temp
+ ".tar.gz -C "
+ temp
+ " && "
+ "(test -f "
+ dir
+ "/ui/server.py && rm -rf "
+ temp
+ " || mv "
+ temp
+ " "
+ dir
+ ") && rm -f "
+ temp
+ ".tar.gz",
60000);
}
apiKey = UUID.randomUUID().toString().replace("-", "");
server = (ChannelExec) session.openChannel("exec");
serverInput = new PipedOutputStream();
server.setInputStream(new PipedInputStream(serverInput));
server.setCommand(
"cd "
+ dir
+ " && FRAME_LOCAL=1 FRAME_UI_KEY="
+ apiKey
+ " FRAME_DEVICE="
+ quote(host)
+ " FRAME_CLIENT="
+ quote(install)
+ " exec python3 -I -u -B \"$PWD/ui/server.py\" --port 0 --exit-on-eof 2>&1");
InputStream input = server.getInputStream();
server.connect(10000);
ByteArrayOutputStream startup = new ByteArrayOutputStream();
Pattern portPattern = Pattern.compile("Frame Control on http://127\\.0\\.0\\.1:([0-9]+)\\s");
long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(40);
int portFound = 0;
while (System.nanoTime() < deadline) {
while (input.available() > 0) {
startup.write(input.read());
if (startup.size() > 65536) throw new IOException("The headset server did not start.");
}
Matcher m = portPattern.matcher(startup.toString(StandardCharsets.UTF_8.name()));
if (m.find()) {
portFound = Integer.parseInt(m.group(1));
break;
}
if (server.isClosed())
throw new IOException("The headset server stopped before it was ready.");
Thread.sleep(50);
}
if (portFound < 1 || portFound > 65535)
throw new IOException("The headset server did not answer in time.");
localPort = session.setPortForwardingL("127.0.0.1", 0, "127.0.0.1", portFound);
// Drain the session-bound server output; never log session keys or user data.
Thread drain =
new Thread(
() -> {
try {
byte[] b = new byte[4096];
while (input.read(b) != -1) {}
} catch (IOException ignored) {
}
},
"frame-server-output");
drain.setDaemon(true);
drain.start();
}
String page() {
return "http://127.0.0.1:" + localPort + "/?key=" + apiKey;
}
boolean connected() {
return session != null && session.isConnected() && server != null && !server.isClosed();
}
private HttpURLConnection request(String path) throws Exception {
HttpURLConnection c =
(HttpURLConnection) new URL("http://127.0.0.1:" + localPort + path).openConnection();
c.setConnectTimeout(10000);
c.setReadTimeout(20000);
c.setInstanceFollowRedirects(false);
c.setRequestProperty("X-Frame-UI", apiKey);
return c;
}
List<SyncEngine.Shot> shots() throws Exception {
HttpURLConnection c = request("/api/shots");
try (InputStream in = c.getInputStream()) {
byte[] body = readBounded(in, 2 * 1024 * 1024);
JsonArray array =
JsonParser.parseString(new String(body, StandardCharsets.UTF_8))
.getAsJsonObject()
.getAsJsonArray("shots");
List<SyncEngine.Shot> shots = new ArrayList<>();
for (JsonElement e : array) {
var s = e.getAsJsonObject();
shots.add(
new SyncEngine.Shot(
s.get("id").getAsString(), s.get("size").getAsLong(), s.get("time").getAsDouble()));
}
return shots;
} finally {
c.disconnect();
}
}
private static byte[] readBounded(InputStream input, int limit) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] b = new byte[8192];
int n;
while ((n = input.read(b)) != -1) {
if (out.size() + n > limit) throw new IOException("The headset response is too large.");
out.write(b, 0, n);
}
return out.toByteArray();
}
@Override
public void download(SyncEngine.Shot shot, File destination) throws Exception {
HttpURLConnection c =
request("/api/shots/image?id=" + URLEncoder.encode(shot.id, StandardCharsets.UTF_8.name()));
try (InputStream in = c.getInputStream();
FileOutputStream out = new FileOutputStream(destination)) {
byte[] b = new byte[65536];
long count = 0;
int n;
while ((n = in.read(b)) != -1) {
count += n;
if (count > shot.size) throw new IOException("Screenshot changed while downloading.");
out.write(b, 0, n);
}
out.getFD().sync();
} finally {
c.disconnect();
}
}
@Override
public boolean validImage(File file) {
android.graphics.BitmapFactory.Options options = new android.graphics.BitmapFactory.Options();
options.inJustDecodeBounds = true;
android.graphics.BitmapFactory.decodeFile(file.getPath(), options);
if (options.outWidth <= 0
|| options.outHeight <= 0
|| options.outWidth > 16384
|| options.outHeight > 16384
|| (long) options.outWidth * options.outHeight > 60_000_000) return false;
options.inJustDecodeBounds = false;
options.inSampleSize = Math.max(1, Math.max(options.outWidth, options.outHeight) / 2048);
android.graphics.Bitmap b = android.graphics.BitmapFactory.decodeFile(file.getPath(), options);
if (b == null) return false;
b.recycle();
return true;
}
@Override
public void close() {
try {
if (serverInput != null) serverInput.close();
} catch (IOException ignored) {
}
if (server != null) server.disconnect();
if (session != null) session.disconnect();
}
}
@@ -0,0 +1,96 @@
package com.saphid.framecontrol;
import android.content.*;
import android.database.Cursor;
import android.net.Uri;
import android.provider.MediaStore;
import java.io.*;
import java.util.function.BooleanSupplier;
/** Deterministic, app-owned MediaStore rows make interrupted exports retryable. */
final class GalleryExporter implements SyncEngine.Exporter {
private final Context context;
private final String headset;
private final Object settingsLock;
GalleryExporter(Context context, String headset, Object settingsLock) {
this.context = context.getApplicationContext();
this.headset = headset;
this.settingsLock = settingsLock;
}
@Override
public boolean export(SyncEngine.Shot shot, File file, BooleanSupplier eligible)
throws Exception {
String name = shot.id.substring(shot.id.lastIndexOf('/') + 1);
int dot = name.lastIndexOf('.');
name =
name.substring(0, dot)
+ "_"
+ FrameConnection.hash(headset + "/" + shot.id).substring(0, 20)
+ name.substring(dot);
String folder = "Pictures/Frame Control/";
ContentResolver resolver = context.getContentResolver();
Uri collection = MediaStore.Images.Media.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY);
// API 29 excludes even app-owned pending rows unless explicitly included.
@SuppressWarnings("deprecation")
Uri recoveryCollection = MediaStore.setIncludePending(collection);
Uri row = null;
boolean pending = true;
try (Cursor cursor =
resolver.query(
recoveryCollection,
new String[] {"_id", MediaStore.Images.Media.IS_PENDING},
MediaStore.Images.Media.DISPLAY_NAME
+ "=? AND "
+ MediaStore.Images.Media.RELATIVE_PATH
+ "=? AND "
+ MediaStore.Images.Media.OWNER_PACKAGE_NAME
+ "=?",
new String[] {name, folder, context.getPackageName()},
null)) {
if (cursor != null && cursor.moveToFirst()) {
row = ContentUris.withAppendedId(collection, cursor.getLong(0));
pending = cursor.getInt(1) != 0;
}
}
// A completed row is the commit receipt, including a crash before JSON save.
if (row != null && !pending) return true;
if (!eligible.getAsBoolean()) return false;
if (row == null) {
ContentValues values = new ContentValues();
values.put(MediaStore.Images.Media.DISPLAY_NAME, name);
values.put(MediaStore.Images.Media.RELATIVE_PATH, folder);
values.put(
MediaStore.Images.Media.MIME_TYPE, shot.id.endsWith(".png") ? "image/png" : "image/jpeg");
values.put(MediaStore.Images.Media.IS_PENDING, 1);
values.put(MediaStore.Images.Media.DATE_TAKEN, (long) (shot.time * 1000));
row = resolver.insert(collection, values);
if (row == null) throw new IOException("Gallery could not create the screenshot.");
}
try {
try (InputStream in = new FileInputStream(file);
OutputStream out = resolver.openOutputStream(row, "wt")) {
if (out == null) throw new IOException("Gallery could not open the screenshot.");
byte[] b = new byte[65536];
int n;
while ((n = in.read(b)) != -1) out.write(b, 0, n);
}
synchronized (settingsLock) {
if (!eligible.getAsBoolean()) {
resolver.delete(row, null, null);
return false;
}
ContentValues done = new ContentValues();
done.put(MediaStore.Images.Media.IS_PENDING, 0);
if (resolver.update(row, done, null, null) != 1)
throw new IOException("Gallery could not finish saving the screenshot.");
}
return true;
} catch (Exception e) {
// A pending row is retained and rewritten on retry. A completed row is
// discovered before any future insert, even after process death.
throw e;
}
}
}
@@ -0,0 +1,512 @@
package com.saphid.framecontrol;
import android.Manifest;
import android.app.*;
import android.content.*;
import android.content.pm.PackageManager;
import android.graphics.Color;
import android.net.Uri;
import android.os.*;
import android.text.InputType;
import android.view.*;
import android.webkit.*;
import android.widget.*;
import androidx.core.content.FileProvider;
import java.util.*;
import java.util.concurrent.*;
public final class MainActivity extends Activity {
private final ExecutorService network = Executors.newSingleThreadExecutor();
private final Handler handler = new Handler(Looper.getMainLooper());
private LinearLayout root, downloads;
private FrameLayout content;
private ScrollView companion;
private EditText host, port, user;
private TextView status, empty;
private Switch gallery;
private Button pair, connect, stop;
private WebView web;
private String loadedPage, librarySignature = "";
private int downloadLimit = 100;
private boolean busy, visible, controls;
private ValueCallback<Uri[]> fileSelection;
private static final int INK = Color.rgb(232, 240, 238),
MUTED = Color.rgb(164, 183, 179),
ACCENT = Color.rgb(124, 216, 195);
private final Runnable refresh =
new Runnable() {
public void run() {
if (!visible) return;
update();
handler.postDelayed(this, 1500);
}
};
private int dp(int n) {
return Math.round(getResources().getDisplayMetrics().density * n);
}
private TextView text(String value, int size, int color) {
TextView v = new TextView(this);
v.setText(value);
v.setTextSize(size);
v.setTextColor(color);
v.setPadding(0, dp(6), 0, dp(6));
return v;
}
private Button button(String label, Runnable action) {
Button b = new Button(this);
b.setText(label);
b.setAllCaps(false);
b.setOnClickListener(v -> action.run());
return b;
}
private EditText field(String label, String value, int type) {
companionBody.addView(text(label, 13, MUTED));
EditText v = new EditText(this);
v.setSingleLine(true);
v.setText(value);
v.setHint(label);
v.setContentDescription(label);
v.setInputType(type);
v.setTextColor(INK);
v.setSaveEnabled(false);
companionBody.addView(v);
return v;
}
private LinearLayout companionBody;
@Override
public void onCreate(Bundle state) {
super.onCreate(state);
root = new LinearLayout(this);
root.setOrientation(LinearLayout.VERTICAL);
root.setBackgroundColor(Color.rgb(17, 24, 25));
root.setPadding(dp(20), dp(12), dp(20), dp(12));
root.setOnApplyWindowInsetsListener(
(v, insets) -> {
if (Build.VERSION.SDK_INT >= 30) {
android.graphics.Insets i =
insets.getInsets(
WindowInsets.Type.systemBars() | WindowInsets.Type.displayCutout());
v.setPadding(dp(20) + i.left, dp(12) + i.top, dp(20) + i.right, dp(12) + i.bottom);
} else
v.setPadding(
dp(20) + insets.getSystemWindowInsetLeft(),
dp(12) + insets.getSystemWindowInsetTop(),
dp(20) + insets.getSystemWindowInsetRight(),
dp(12) + insets.getSystemWindowInsetBottom());
return insets;
});
root.addView(text("FRAME CONTROL", 13, ACCENT));
root.addView(text("Your headset, closer.", 27, INK));
status = text(SyncService.state(), 14, MUTED);
status.setAccessibilityLiveRegion(View.ACCESSIBILITY_LIVE_REGION_POLITE);
root.addView(status);
LinearLayout tabs = new LinearLayout(this);
tabs.addView(
button("Screenshots", () -> show(false)), new LinearLayout.LayoutParams(0, dp(48), 1));
tabs.addView(
button("Headset controls", () -> show(true)), new LinearLayout.LayoutParams(0, dp(48), 1));
root.addView(tabs);
content = new FrameLayout(this);
root.addView(content, new LinearLayout.LayoutParams(-1, 0, 1));
companion = new ScrollView(this);
companion.setFillViewport(true);
companionBody = new LinearLayout(this);
companionBody.setOrientation(LinearLayout.VERTICAL);
companion.addView(companionBody);
SharedPreferences p = SyncService.prefs(this);
companionBody.addView(text("Connect your Steam Frame", 20, INK));
companionBody.addView(
text(
"Enable Developer Mode and connect to the same network, or use your headset’s Tailscale"
+ " address.",
14,
MUTED));
host =
field(
"Headset address",
p.getString("host", "frame.local"),
InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_URI);
port = field("SSH port", Integer.toString(p.getInt("port", 22)), InputType.TYPE_CLASS_NUMBER);
user =
field(
"SSH username",
p.getString("user", "steamos"),
InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_VISIBLE_PASSWORD);
pair = button("Pair headset", this::probe);
companionBody.addView(pair);
LinearLayout actions = new LinearLayout(this);
connect = button("Start syncing", this::startSync);
stop = button("Stop", () -> stopService(new Intent(this, SyncService.class)));
actions.addView(connect, new LinearLayout.LayoutParams(0, dp(52), 1));
actions.addView(stop, new LinearLayout.LayoutParams(0, dp(52), 1));
companionBody.addView(actions);
companionBody.addView(text("Screenshot downloads", 20, INK));
gallery = new Switch(this);
gallery.setText("Save screenshots to Gallery");
gallery.setTextColor(INK);
gallery.setTextSize(16);
gallery.setMinHeight(dp(56));
gallery.setChecked(p.getBoolean("gallery", true));
gallery.setOnCheckedChangeListener(
(b, value) -> {
try {
SyncService.gallery(this, value);
librarySignature = "";
} catch (Exception e) {
gallery.setOnCheckedChangeListener(null);
gallery.setChecked(!value);
error(SyncService.message(e));
recreate();
}
});
companionBody.addView(gallery);
companionBody.addView(
text(
"New captures download automatically while sync is running. A notification keeps the"
+ " connection visible. Stop syncing to disconnect. Android may delay transfers in"
+ " battery-saving modes.",
14,
MUTED));
companionBody.addView(
text(
"Turning Gallery off keeps new downloads in this app. Turning it on again won’t export"
+ " earlier app-only downloads.",
14,
MUTED));
empty =
text(
"Pair and start syncing. Existing headset screenshots stay on the headset; new captures"
+ " will appear here.",
14,
MUTED);
companionBody.addView(empty);
downloads = new LinearLayout(this);
downloads.setOrientation(LinearLayout.VERTICAL);
companionBody.addView(downloads);
show(false);
setContentView(root);
}
private void show(boolean useControls) {
controls = useControls;
content.removeAllViews();
if (!controls) {
content.addView(companion, new FrameLayout.LayoutParams(-1, -1));
return;
}
String page = SyncService.page();
if (page == null) {
loadedPage = null;
if (web != null) web.loadUrl("about:blank");
content.addView(text("Start syncing to open your headset’s controls.", 17, MUTED));
return;
}
if (web == null) {
web = new WebView(this);
web.setBackgroundColor(Color.rgb(17, 24, 25));
WebSettings settings = web.getSettings();
settings.setJavaScriptEnabled(true);
settings.setDomStorageEnabled(true);
settings.setAllowFileAccess(false);
settings.setAllowContentAccess(false);
settings.setMixedContentMode(WebSettings.MIXED_CONTENT_NEVER_ALLOW);
settings.setSaveFormData(false);
web.setWebViewClient(
new WebViewClient() {
@Override
public boolean shouldOverrideUrlLoading(WebView view, WebResourceRequest request) {
Uri url = request.getUrl();
Uri origin = Uri.parse(loadedPage == null ? "" : loadedPage);
if ("http".equals(url.getScheme())
&& "127.0.0.1".equals(url.getHost())
&& url.getPort() == origin.getPort()) return false;
if (request.hasGesture()
&& ("https".equals(url.getScheme()) || "http".equals(url.getScheme()))) {
try {
startActivity(new Intent(Intent.ACTION_VIEW, url));
} catch (ActivityNotFoundException ignored) {
}
}
return true;
}
});
web.setWebChromeClient(
new WebChromeClient() {
@Override
public boolean onShowFileChooser(
WebView w, ValueCallback<Uri[]> callback, FileChooserParams params) {
if (fileSelection != null) fileSelection.onReceiveValue(null);
fileSelection = callback;
Intent choose =
new Intent(Intent.ACTION_OPEN_DOCUMENT)
.setType("*/*")
.addCategory(Intent.CATEGORY_OPENABLE);
choose.putExtra(
Intent.EXTRA_ALLOW_MULTIPLE,
params.getMode() == FileChooserParams.MODE_OPEN_MULTIPLE);
try {
startActivityForResult(choose, 110);
} catch (ActivityNotFoundException e) {
fileSelection.onReceiveValue(null);
fileSelection = null;
return false;
}
return true;
}
});
web.setDownloadListener(
(url, ua, disposition, mime, length) -> {
show(false);
Toast.makeText(
this,
"New screenshot originals are in Screenshot downloads.",
Toast.LENGTH_LONG)
.show();
});
}
content.addView(web, new FrameLayout.LayoutParams(-1, -1));
if (!page.equals(loadedPage)) {
loadedPage = page;
web.loadUrl(page);
}
}
@Override
protected void onActivityResult(int request, int result, Intent data) {
super.onActivityResult(request, result, data);
if (request == 110 && fileSelection != null) {
fileSelection.onReceiveValue(WebChromeClient.FileChooserParams.parseResult(result, data));
fileSelection = null;
}
}
private void update() {
if (!busy) status.setText(SyncService.state());
boolean paired = SyncService.prefs(this).contains("pin");
boolean running = SyncService.running();
connect.setEnabled(paired && !busy && !running);
pair.setEnabled(!busy && !running);
stop.setEnabled(!busy && running);
host.setEnabled(!running && !busy);
port.setEnabled(!running && !busy);
user.setEnabled(!running && !busy);
if (controls && !Objects.equals(loadedPage, SyncService.page())) show(true);
try {
List<SyncEngine.Item> items = SyncService.items(this);
StringBuilder signature = new StringBuilder();
for (var item : items)
signature
.append(item.file.getAbsolutePath())
.append('|')
.append(item.destination)
.append('|')
.append(item.file.exists())
.append('\n');
if (signature.toString().equals(librarySignature)) return;
librarySignature = signature.toString();
downloads.removeAllViews();
empty.setVisibility(items.isEmpty() ? View.VISIBLE : View.GONE);
int shown = 0;
for (SyncEngine.Item item : items) {
if (shown++ >= downloadLimit) {
downloads.addView(
button(
"Show 100 more downloads",
() -> {
downloadLimit += 100;
librarySignature = "";
update();
}));
break;
}
String label = item.shot.id.substring(item.shot.id.lastIndexOf('/') + 1);
LinearLayout card = new LinearLayout(this);
card.setOrientation(LinearLayout.VERTICAL);
card.setPadding(0, dp(12), 0, dp(12));
card.addView(text(label, 16, INK));
String where =
item.destination.equals("saved")
? "Saved to Gallery"
: item.destination.equals("appOnly") ? "In Frame Control" : "Gallery save pending";
if (!item.file.exists()) where = "Download pending";
card.addView(text(where + " · Game " + item.shot.id.split("/")[1], 13, MUTED));
if (item.file.exists()) card.addView(button("Open / share original", () -> share(item)));
downloads.addView(card);
}
} catch (Exception e) {
empty.setVisibility(View.VISIBLE);
empty.setText("Could not read screenshot downloads: " + SyncService.message(e));
}
}
private void share(SyncEngine.Item item) {
Uri uri = FileProvider.getUriForFile(this, getPackageName() + ".files", item.file);
Intent send =
new Intent(Intent.ACTION_SEND)
.setType(item.shot.id.endsWith(".png") ? "image/png" : "image/jpeg")
.putExtra(Intent.EXTRA_STREAM, uri)
.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION);
startActivity(Intent.createChooser(send, "Screenshot original"));
}
private void startSync() {
if (Build.VERSION.SDK_INT >= 33
&& checkSelfPermission(Manifest.permission.POST_NOTIFICATIONS)
!= PackageManager.PERMISSION_GRANTED) {
requestPermissions(new String[] {Manifest.permission.POST_NOTIFICATIONS}, 108);
return;
}
startForegroundService(new Intent(this, SyncService.class));
}
@Override
public void onRequestPermissionsResult(int request, String[] permissions, int[] grants) {
super.onRequestPermissionsResult(request, permissions, grants);
if (request == 108) {
if (grants.length > 0 && grants[0] == PackageManager.PERMISSION_GRANTED)
startForegroundService(new Intent(this, SyncService.class));
else
error(
"Allow notifications in Android Settings to start a visible screenshot sync session.");
}
}
private void probe() {
final String address = host.getText().toString().trim(),
username = user.getText().toString().trim();
final int sshPort;
try {
sshPort = Integer.parseInt(port.getText().toString());
} catch (NumberFormatException e) {
error("Enter an SSH port from 1 to 65535.");
return;
}
stopService(new Intent(this, SyncService.class));
busy = true;
pair.setEnabled(false);
status.setText("Checking headset identity…");
network.execute(
() -> {
try {
String pin = FrameConnection.probe(address, sshPort, username);
String fingerprint = FrameConnection.fingerprint(pin);
runOnUiThread(
() -> {
if (isFinishing() || isDestroyed()) return;
busy = false;
new AlertDialog.Builder(this)
.setTitle("Trust this headset?")
.setMessage(
address
+ ":"
+ sshPort
+ "\n\n"
+ fingerprint
+ "\n\n"
+ "Compare this SSH fingerprint with your headset before entering its"
+ " Developer Mode password.")
.setNegativeButton("Cancel", null)
.setPositiveButton(
"Trust and pair", (d, w) -> password(address, sshPort, username, pin))
.show();
});
} catch (Exception e) {
runOnUiThread(
() -> {
busy = false;
error(SyncService.message(e));
});
}
});
}
private void password(String address, int sshPort, String username, String pin) {
EditText password = new EditText(this);
password.setInputType(InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_PASSWORD);
password.setHint("Developer Mode password");
password.setSaveEnabled(false);
new AlertDialog.Builder(this)
.setTitle("Pair with " + address)
.setMessage(
"Used once to authorize this phone’s protected SSH key. The password is not saved.")
.setView(password)
.setNegativeButton("Cancel", (d, w) -> password.setText(""))
.setPositiveButton(
"Pair",
(d, w) -> {
String secret = password.getText().toString();
password.setText("");
busy = true;
status.setText("Pairing this phone…");
network.execute(
() -> {
try {
FrameConnection.pair(
address, sshPort, username, pin, secret, SyncService.install(this));
if (!SyncService.prefs(this)
.edit()
.putString("host", address)
.putInt("port", sshPort)
.putString("user", username)
.putString("pin", pin)
.commit()) throw new java.io.IOException("Could not save pairing.");
runOnUiThread(
() -> {
busy = false;
status.setText("Paired. Start syncing when you’re ready.");
update();
});
} catch (Exception e) {
runOnUiThread(
() -> {
busy = false;
error(SyncService.message(e));
});
}
});
})
.show();
}
private void error(String message) {
if (isFinishing() || isDestroyed()) return;
new AlertDialog.Builder(this)
.setTitle("Frame Control")
.setMessage(message)
.setPositiveButton("OK", null)
.show();
}
@Override
protected void onResume() {
super.onResume();
visible = true;
if (web != null) web.onResume();
handler.post(refresh);
}
@Override
protected void onPause() {
visible = false;
handler.removeCallbacks(refresh);
if (web != null) web.onPause();
super.onPause();
}
@Override
protected void onDestroy() {
network.shutdownNow();
if (fileSelection != null) fileSelection.onReceiveValue(null);
if (web != null) {
web.destroy();
web = null;
}
super.onDestroy();
}
}
@@ -0,0 +1,115 @@
package com.saphid.framecontrol;
import android.security.keystore.KeyGenParameterSpec;
import android.security.keystore.KeyProperties;
import com.jcraft.jsch.Identity;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.math.BigInteger;
import java.nio.charset.StandardCharsets;
import java.security.KeyPairGenerator;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.Signature;
import java.security.interfaces.RSAPublicKey;
import java.util.Base64;
/** The SSH private key is a non-exportable Android Keystore signing key. */
final class PhoneIdentity implements Identity {
private static final String ALIAS = "frame-control-ssh-v1";
private final PrivateKey key;
private final byte[] publicBlob;
PhoneIdentity() throws Exception {
KeyStore store = KeyStore.getInstance("AndroidKeyStore");
store.load(null);
if (!store.containsAlias(ALIAS)) {
KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA", "AndroidKeyStore");
generator.initialize(
new KeyGenParameterSpec.Builder(ALIAS, KeyProperties.PURPOSE_SIGN)
.setKeySize(3072)
.setDigests(KeyProperties.DIGEST_SHA256, KeyProperties.DIGEST_SHA512)
.setSignaturePaddings(KeyProperties.SIGNATURE_PADDING_RSA_PKCS1)
.build());
generator.generateKeyPair();
}
key = (PrivateKey) store.getKey(ALIAS, null);
RSAPublicKey publicKey = (RSAPublicKey) store.getCertificate(ALIAS).getPublicKey();
publicBlob =
blob(
"ssh-rsa".getBytes(StandardCharsets.US_ASCII),
integer(publicKey.getPublicExponent()),
integer(publicKey.getModulus()));
}
private static byte[] integer(BigInteger value) {
return value.toByteArray();
}
private static byte[] blob(byte[]... strings) throws Exception {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream(bytes);
for (byte[] s : strings) {
out.writeInt(s.length);
out.write(s);
}
return bytes.toByteArray();
}
String authorizedKey(String install) {
return "ssh-rsa "
+ Base64.getEncoder().encodeToString(publicBlob)
+ " frame-control-android-"
+ install;
}
@Override
public byte[] getPublicKeyBlob() {
return publicBlob.clone();
}
@Override
public byte[] getSignature(byte[] data) {
return getSignature(data, "rsa-sha2-512");
}
@Override
public byte[] getSignature(byte[] data, String algorithm) {
try {
if (!algorithm.equals("rsa-sha2-256") && !algorithm.equals("rsa-sha2-512")) return null;
Signature signer =
Signature.getInstance(
algorithm.equals("rsa-sha2-512") ? "SHA512withRSA" : "SHA256withRSA");
signer.initSign(key);
signer.update(data);
return blob(algorithm.getBytes(StandardCharsets.US_ASCII), signer.sign());
} catch (Exception e) {
return null;
}
}
@Override
public boolean setPassphrase(byte[] ignored) {
return true;
}
@Override
public String getAlgName() {
return "ssh-rsa";
}
@Override
public String getName() {
return ALIAS;
}
@Override
public boolean isEncrypted() {
return false;
}
@Override
public void clear() {
/* Keystore owns the key; disconnect never deletes it. */
}
}
@@ -0,0 +1,570 @@
package com.saphid.framecontrol;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.Strictness;
import com.google.gson.TypeAdapter;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonToken;
import com.google.gson.stream.JsonWriter;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.LinkOption;
import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.nio.file.StandardOpenOption;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.BooleanSupplier;
import java.util.regex.Pattern;
/** One reusable engine per root. The caller supplies an account/headset-specific root. */
public final class SyncEngine {
private static final long MAX_SIZE = 30L * 1024 * 1024;
private static final Pattern ID = Pattern.compile(
"^[0-9]{1,12}/[0-9]{1,20}/[0-9]{14}_[0-9]{1,4}\\.(jpg|png)$");
private static final Gson GSON = new GsonBuilder().registerTypeAdapter(Shot.class,
new TypeAdapter<Shot>() {
@Override public void write(JsonWriter writer, Shot shot) throws IOException {
writer.beginObject();
writer.name("id").value(shot.id);
writer.name("size").value(shot.size);
writer.name("time").value(shot.time);
writer.endObject();
}
@Override public Shot read(JsonReader reader) throws IOException {
String id = null;
Long size = null;
Double time = null;
Set<String> fields = new HashSet<>();
reader.beginObject();
while (reader.hasNext()) {
String field = reader.nextName();
if (!fields.add(field)) throw new IOException("Duplicate screenshot field");
switch (field) {
case "id":
if (reader.peek() != JsonToken.STRING) throw new IOException("Invalid id");
id = reader.nextString();
break;
case "size":
if (reader.peek() != JsonToken.NUMBER) throw new IOException("Invalid size");
size = reader.nextLong();
break;
case "time":
if (reader.peek() != JsonToken.NUMBER) throw new IOException("Invalid time");
time = reader.nextDouble();
break;
default: throw new IOException("Unknown screenshot field");
}
}
reader.endObject();
if (id == null || size == null || time == null) throw new IOException("Incomplete shot");
return new Shot(id, size, time);
}
}).create();
public static final class Shot {
public final String id;
public final long size;
public final double time;
public Shot(String id, long size, double time) {
this.id = id;
this.size = size;
this.time = time;
}
}
public interface Transport {
void download(Shot shot, File destination) throws Exception;
boolean validImage(File f) throws Exception;
}
public interface Exporter {
boolean export(Shot shot, File original, BooleanSupplier eligible) throws Exception;
}
public static final class Item {
public final Shot shot;
/** Future original path; may not exist while the shot is observing or queued. */
public final File file;
public final String destination;
private Item(Shot shot, File file, String destination) {
this.shot = shot;
this.file = file;
this.destination = destination;
}
}
// Boxed fields make missing journal fields distinguishable from legitimate zero values.
private static final class Ledger {
Integer version;
Boolean initialized;
Boolean gallery;
Map<String, Receipt> records;
}
private static final class Receipt {
Shot shot;
String state;
Integer observations;
String destination;
Receipt(Shot shot, String state, int observations, String destination) {
this.shot = shot;
this.state = state;
this.observations = observations;
this.destination = destination;
}
}
/** Only external callbacks and rejected transfer contents are safe to retry per item. */
private static final class ItemFailure extends IOException {
private static final long serialVersionUID = 1L;
ItemFailure(String message, Exception cause) {
super(message, cause);
}
}
private final Object lock = new Object();
private final Path root;
private final Path originals;
private final Path journal;
private Ledger ledger;
private long generation;
private long active;
public SyncEngine(File root, boolean gallery) throws Exception {
if (root == null || Files.isSymbolicLink(root.toPath())) {
throw new IOException("Unsafe sync root");
}
this.root = root.getCanonicalFile().toPath();
List<Path> missingDirectories = new ArrayList<>();
for (Path path = this.root; path != null && !Files.exists(path); path = path.getParent()) {
missingDirectories.add(path);
}
Files.createDirectories(this.root);
for (Path directory : missingDirectories) syncDirectory(directory.getParent());
originals = this.root.resolve("originals");
journal = this.root.resolve("receipts.json");
checkPath(originals);
checkPath(journal);
if (Files.exists(originals, LinkOption.NOFOLLOW_LINKS)
&& !Files.isDirectory(originals, LinkOption.NOFOLLOW_LINKS)) {
throw new IOException("Invalid originals directory");
}
if (Files.exists(journal, LinkOption.NOFOLLOW_LINKS)) {
if (!Files.isRegularFile(journal, LinkOption.NOFOLLOW_LINKS)) {
throw new IOException("Invalid sync journal");
}
// Never recover a damaged or incomplete journal by establishing another baseline.
try (JsonReader reader = new JsonReader(new InputStreamReader(
Files.newInputStream(journal), StandardCharsets.UTF_8))) {
reader.setStrictness(Strictness.STRICT);
ledger = GSON.fromJson(reader, Ledger.class);
if (reader.peek() != JsonToken.END_DOCUMENT) {
throw new IOException("Trailing sync journal data");
}
} catch (RuntimeException e) {
throw new IOException("Corrupt sync journal", e);
}
validateLedger();
setGallery(gallery);
} else {
if (Files.exists(originals, LinkOption.NOFOLLOW_LINKS)) {
try (java.util.stream.Stream<Path> paths = Files.walk(originals)) {
if (paths.anyMatch(p -> !Files.isDirectory(p, LinkOption.NOFOLLOW_LINKS))) {
throw new IOException("Originals exist without a sync journal");
}
}
}
ledger = emptyLedger();
ledger.gallery = gallery;
persist(ledger);
}
}
public void setGallery(boolean enabled) throws Exception {
synchronized (lock) {
if (ledger.gallery == enabled) return;
Ledger next = copy();
next.gallery = enabled;
if (!enabled) {
for (Receipt receipt : next.records.values()) {
if ("pending".equals(receipt.destination)) receipt.destination = "appOnly";
}
}
commit(next);
}
}
/** A second live reconcile is rejected. stop() cancels its generation and permits a new run. */
public void reconcile(List<Shot> shots, Transport transport, Exporter exporter) throws Exception {
final long run;
synchronized (lock) {
if (active != 0) throw new IllegalStateException("Sync already in progress");
run = ++generation;
active = run;
}
try {
Map<String, Shot> listing = listing(shots);
synchronized (lock) {
if (active != run) return;
Ledger next = copy();
if (!next.initialized) {
for (Shot shot : listing.values()) {
next.records.put(shot.id, new Receipt(shot, "baseline", 0, null));
}
next.initialized = true;
commit(next);
return;
}
// Missing observations break consecutiveness, including across restarts.
for (Receipt receipt : next.records.values()) {
if (!listing.containsKey(receipt.shot.id)
&& ("observing".equals(receipt.state) || "queued".equals(receipt.state))) {
receipt.observations = 0;
}
}
for (Shot shot : listing.values()) {
Receipt receipt = next.records.get(shot.id);
if (receipt == null) {
// Latch eligibility at discovery, before waiting for a stable second sample.
next.records.put(shot.id, new Receipt(shot, "observing", 1,
next.gallery ? "pending" : "appOnly"));
} else if ("observing".equals(receipt.state) || "queued".equals(receipt.state)) {
receipt.observations = same(receipt.shot, shot)
? Math.min(2, receipt.observations + 1) : 1;
receipt.shot = shot;
if (receipt.observations == 2 && "observing".equals(receipt.state)) {
receipt.state = "queued";
}
}
}
commit(next); // Queue and its disposition reach disk before any download.
}
List<String> ids;
synchronized (lock) {
if (active != run) return;
ids = new ArrayList<>(ledger.records.keySet());
}
ItemFailure failure = null;
for (String id : ids) {
Shot download = null;
synchronized (lock) {
if (active != run) return;
Receipt receipt = ledger.records.get(id);
if ("queued".equals(receipt.state) && receipt.observations == 2) {
download = receipt.shot;
}
}
try {
if (download != null) download(run, download, transport);
export(run, id, exporter);
} catch (ItemFailure e) {
if (failure == null) failure = e;
else failure.addSuppressed(e);
}
}
// Surface retryable errors to the service after independent items have progressed.
// Untagged local I/O/journal failures escape immediately and halt the run.
if (failure != null) throw failure;
} catch (Exception e) {
synchronized (lock) {
if (active != run) return; // A late failed callback belongs to a cancelled run.
}
throw e;
} finally {
synchronized (lock) {
if (active == run) active = 0;
}
}
}
/** Snapshot of all post-baseline receipts, including downloads still pending. */
public List<Item> items() {
synchronized (lock) {
List<Item> result = new ArrayList<>();
for (Receipt receipt : ledger.records.values()) {
if (!"baseline".equals(receipt.state)) {
result.add(new Item(receipt.shot, originals.resolve(receipt.shot.id).toFile(),
receipt.destination));
}
}
result.sort(Comparator.comparingDouble((Item item) -> item.shot.time).reversed()
.thenComparing(item -> item.shot.id));
return Collections.unmodifiableList(result);
}
}
public void stop() {
synchronized (lock) {
++generation;
active = 0;
}
}
private void download(long run, Shot shot, Transport transport) throws Exception {
Path temporary;
synchronized (lock) {
if (active != run) return;
temporary = Files.createTempFile(root, ".download-", ".part");
}
try {
try {
transport.download(shot, temporary.toFile());
} catch (InterruptedException e) {
throw e;
} catch (Exception e) {
throw new ItemFailure("Download failed: " + shot.id + ": " + e.getMessage(), e);
}
synchronized (lock) {
if (active != run) return;
}
checkTransfer(temporary, shot);
boolean valid;
try {
valid = transport.validImage(temporary.toFile());
} catch (InterruptedException e) {
throw e;
} catch (Exception e) {
throw new ItemFailure("Image validation failed: " + shot.id + ": " + e.getMessage(), e);
}
if (!valid) throw new ItemFailure("Invalid image: " + shot.id, null);
checkTransfer(temporary, shot);
try (FileChannel channel = FileChannel.open(temporary, StandardOpenOption.WRITE)) {
channel.force(true);
}
synchronized (lock) {
if (active != run) return;
Path target = originals.resolve(shot.id);
createParents(target.getParent());
checkPath(target);
Files.move(temporary, target, StandardCopyOption.ATOMIC_MOVE,
StandardCopyOption.REPLACE_EXISTING);
syncDirectory(target.getParent());
Ledger next = copy();
// Read the current destination: setGallery may have changed it during the transfer.
next.records.get(shot.id).state = "downloaded";
commit(next);
}
} finally {
Files.deleteIfExists(temporary);
}
}
private void export(long run, String id, Exporter exporter) throws Exception {
Shot shot;
File original;
synchronized (lock) {
if (!eligible(run, id)) return;
Receipt receipt = ledger.records.get(id);
shot = receipt.shot;
Path path = originals.resolve(id);
checkDownload(path, shot);
original = path.toFile();
}
boolean saved;
try {
saved = exporter.export(shot, original, () -> {
synchronized (lock) {
return eligible(run, id);
}
});
} catch (InterruptedException e) {
throw e;
} catch (Exception e) {
throw new ItemFailure("Gallery export failed: " + shot.id + ": " + e.getMessage(), e);
}
synchronized (lock) {
if (active != run || !saved) return;
Ledger next = copy();
// A true result means the exporter actually saved (or recovered) the MediaStore item.
next.records.get(id).destination = "saved";
commit(next);
}
}
private boolean eligible(long run, String id) {
Receipt receipt = ledger.records.get(id);
return active == run && ledger.gallery && receipt != null
&& "downloaded".equals(receipt.state) && "pending".equals(receipt.destination);
}
private static Map<String, Shot> listing(List<Shot> shots) throws IOException {
if (shots == null) throw new IOException("Missing screenshot listing");
Map<String, Shot> result = new LinkedHashMap<>();
for (Shot shot : shots) {
validateShot(shot);
Shot previous = result.put(shot.id, shot);
if (previous != null && !same(previous, shot)) {
throw new IOException("Conflicting screenshot identity: " + shot.id);
}
}
return result;
}
private static void validateShot(Shot shot) throws IOException {
if (shot == null || shot.id == null || !ID.matcher(shot.id).matches()
|| shot.size <= 0 || shot.size > MAX_SIZE || !Double.isFinite(shot.time)) {
throw new IOException("Invalid screenshot metadata");
}
}
private static boolean same(Shot a, Shot b) {
return a.size == b.size && Double.compare(a.time, b.time) == 0;
}
private void validateLedger() throws IOException {
if (ledger == null || ledger.version == null || ledger.version != 1
|| ledger.initialized == null || ledger.gallery == null || ledger.records == null
|| (!ledger.initialized && !ledger.records.isEmpty())) {
throw new IOException("Invalid sync journal header");
}
for (Map.Entry<String, Receipt> entry : ledger.records.entrySet()) {
Receipt receipt = entry.getValue();
if (receipt == null) throw new IOException("Invalid sync receipt");
validateShot(receipt.shot);
if (!entry.getKey().equals(receipt.shot.id) || receipt.observations == null
|| receipt.observations < 0 || receipt.observations > 2) {
throw new IOException("Invalid sync receipt identity");
}
boolean baseline = "baseline".equals(receipt.state);
boolean observing = "observing".equals(receipt.state);
boolean queued = "queued".equals(receipt.state);
boolean downloaded = "downloaded".equals(receipt.state);
if ((!baseline && !observing && !queued && !downloaded)
|| (baseline && receipt.destination != null)
|| (baseline && receipt.observations != 0)
|| (observing && receipt.observations > 1)
|| (downloaded && receipt.observations != 2)
|| ((observing || queued || downloaded) && !"pending".equals(receipt.destination)
&& !"appOnly".equals(receipt.destination) && !"saved".equals(receipt.destination))
|| ((observing || queued) && "saved".equals(receipt.destination))
|| (!ledger.gallery && "pending".equals(receipt.destination))) {
throw new IOException("Invalid sync receipt state");
}
Path path = originals.resolve(receipt.shot.id);
checkPath(path);
if (downloaded) checkDownload(path, receipt.shot);
}
}
private void checkDownload(Path path, Shot shot) throws IOException {
checkPath(path);
if (!Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS) || Files.size(path) != shot.size) {
throw new IOException("Incomplete original: " + shot.id);
}
}
private void checkTransfer(Path path, Shot shot) throws IOException {
checkPath(path); // Unsafe paths and local filesystem errors remain fatal.
BasicFileAttributes attributes;
try {
attributes = Files.readAttributes(path, BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS);
} catch (NoSuchFileException e) {
throw new ItemFailure("Missing downloaded image: " + shot.id, e);
}
if (!attributes.isRegularFile() || attributes.size() != shot.size) {
throw new ItemFailure("Incomplete downloaded image: " + shot.id, null);
}
}
private void checkPath(Path path) throws IOException {
if (!path.normalize().startsWith(root)) throw new IOException("Path escaped sync root");
Path current = root;
if (Files.isSymbolicLink(current)) throw new IOException("Symlink in sync root");
for (Path component : root.relativize(path)) {
current = current.resolve(component);
if (Files.isSymbolicLink(current)) throw new IOException("Symlink in sync path");
}
}
private void createParents(Path directory) throws IOException {
checkPath(directory);
Path current = root;
for (Path component : root.relativize(directory)) {
Path parent = current;
current = current.resolve(component);
if (!Files.exists(current, LinkOption.NOFOLLOW_LINKS)) {
Files.createDirectory(current);
syncDirectory(parent);
} else if (!Files.isDirectory(current, LinkOption.NOFOLLOW_LINKS)) {
throw new IOException("Invalid originals directory");
}
}
}
private Ledger copy() {
Ledger next = emptyLedger();
next.initialized = ledger.initialized;
next.gallery = ledger.gallery;
for (Map.Entry<String, Receipt> entry : ledger.records.entrySet()) {
Receipt receipt = entry.getValue();
next.records.put(entry.getKey(), new Receipt(receipt.shot, receipt.state,
receipt.observations, receipt.destination));
}
return next;
}
private static Ledger emptyLedger() {
Ledger result = new Ledger();
result.version = 1;
result.initialized = false;
result.records = new LinkedHashMap<>();
return result;
}
private void commit(Ledger next) throws IOException {
persist(next);
ledger = next;
}
private void persist(Ledger next) throws IOException {
checkPath(journal);
Path temporary = Files.createTempFile(root, ".receipts-", ".tmp");
try {
try (FileOutputStream output = new FileOutputStream(temporary.toFile());
Writer writer = new OutputStreamWriter(output, StandardCharsets.UTF_8)) {
GSON.toJson(next, writer);
writer.flush();
output.getFD().sync();
}
Files.move(temporary, journal, StandardCopyOption.ATOMIC_MOVE,
StandardCopyOption.REPLACE_EXISTING);
syncDirectory(root);
} finally {
Files.deleteIfExists(temporary);
}
}
private static void syncDirectory(Path directory) {
// Some Android filesystems reject directory channels. File contents were already fsynced;
// the rename is atomic either way. Do not report rollback after a committed rename.
try (FileChannel channel = FileChannel.open(directory, StandardOpenOption.READ)) {
channel.force(true);
} catch (IOException | UnsupportedOperationException ignored) {
// Best available portable Java directory durability.
}
}
}
@@ -0,0 +1,234 @@
package com.saphid.framecontrol;
import android.app.*;
import android.content.*;
import android.content.pm.ServiceInfo;
import android.os.*;
import java.io.File;
import java.util.*;
import java.util.concurrent.*;
/** Explicit user-started connected-device session, with a visible Stop action. */
public final class SyncService extends Service {
static final String CHANNEL = "headset-sync";
static final Object SETTINGS_LOCK = new Object();
private static volatile SyncService instance;
private static volatile String state = "Disconnected";
private static volatile String page;
private final ExecutorService worker = Executors.newSingleThreadExecutor();
private volatile boolean stopping;
private volatile FrameConnection connection;
private volatile SyncEngine engine;
static SharedPreferences prefs(Context c) {
return c.getSharedPreferences("frame", MODE_PRIVATE);
}
static String state() {
return state;
}
static String page() {
return page;
}
static boolean running() {
SyncService service = instance;
return service != null && !service.stopping;
}
static String install(Context c) {
SharedPreferences p = prefs(c);
String id = p.getString("install", null);
if (id == null) {
id = UUID.randomUUID().toString();
if (!p.edit().putString("install", id).commit())
throw new IllegalStateException("Could not save this phone's identity.");
}
return id;
}
static File library(Context c, String pin) throws Exception {
return new File(c.getFilesDir(), "screenshots/" + FrameConnection.hash(pin));
}
static void gallery(Context c, boolean value) throws Exception {
synchronized (SETTINGS_LOCK) {
SyncService service = instance;
// Journal first: a failed disk write must not appear as a saved setting.
String pin = prefs(c).getString("pin", null);
SyncEngine live = service == null ? null : service.engine;
if (live != null) live.setGallery(value);
if (!value) {
// The setting is global. Latch the opt-out in every retained library,
// including headsets that are not currently paired or connected.
File libraries = new File(c.getFilesDir(), "screenshots");
if (libraries.exists()) {
File[] roots = libraries.listFiles();
if (roots == null) throw new java.io.IOException("Could not read screenshot libraries.");
File activeRoot = live != null && pin != null ? library(c, pin) : null;
for (File root : roots) {
if (!root.getName().matches("[0-9a-f]{64}")) continue;
if (activeRoot != null && root.equals(activeRoot)) continue;
new SyncEngine(root, false).setGallery(false);
}
}
} else if (live == null && pin != null) {
new SyncEngine(library(c, pin), true).setGallery(true);
}
if (!prefs(c).edit().putBoolean("gallery", value).commit())
throw new java.io.IOException("Could not save the Gallery setting.");
}
}
static List<SyncEngine.Item> items(Context c) throws Exception {
SyncService service = instance;
if (service != null && service.engine != null) return service.engine.items();
String pin = prefs(c).getString("pin", null);
if (pin == null) return Collections.emptyList();
synchronized (SETTINGS_LOCK) {
return new SyncEngine(library(c, pin), prefs(c).getBoolean("gallery", true)).items();
}
}
@Override
public void onCreate() {
super.onCreate();
instance = this;
stopping = false;
NotificationManager manager = getSystemService(NotificationManager.class);
manager.createNotificationChannel(
new NotificationChannel(
CHANNEL, "Headset screenshot sync", NotificationManager.IMPORTANCE_LOW));
}
private Notification notification(String message) {
PendingIntent open =
PendingIntent.getActivity(
this,
0,
new Intent(this, MainActivity.class),
PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT);
PendingIntent stop =
PendingIntent.getService(
this,
1,
new Intent(this, SyncService.class).setAction("STOP"),
PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT);
return new Notification.Builder(this, CHANNEL)
.setSmallIcon(com.saphid.framecontrol.R.drawable.ic_frame)
.setContentTitle("Frame Control")
.setContentText(message)
.setContentIntent(open)
.setOngoing(true)
.addAction(new Notification.Action.Builder(null, "Stop syncing", stop).build())
.build();
}
private void status(String text) {
state = text;
getSystemService(NotificationManager.class).notify(108, notification(text));
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
if (intent != null && "STOP".equals(intent.getAction())) {
stopSelf();
return START_NOT_STICKY;
}
if (connection != null || stopping) return START_NOT_STICKY;
state = "Connecting to headset";
if (Build.VERSION.SDK_INT >= 34)
startForeground(
108, notification(state), ServiceInfo.FOREGROUND_SERVICE_TYPE_CONNECTED_DEVICE);
else startForeground(108, notification(state));
connection = new FrameConnection(this);
worker.execute(this::runSession);
return START_NOT_STICKY;
}
private void runSession() {
try {
SharedPreferences p = prefs(this);
String pin = p.getString("pin", null);
if (pin == null) throw new java.io.IOException("Pair with your headset first.");
synchronized (SETTINGS_LOCK) {
engine = new SyncEngine(library(this, pin), p.getBoolean("gallery", true));
}
while (!stopping) {
try {
FrameConnection active = new FrameConnection(this);
connection = active;
status("Connecting to headset");
active.connect(
p.getString("host", ""),
p.getInt("port", 22),
p.getString("user", "steamos"),
pin,
install(this));
if (stopping) {
active.close();
break;
}
page = active.page();
status("Syncing new screenshots");
GalleryExporter exporter = new GalleryExporter(this, pin, SETTINGS_LOCK);
while (!stopping && active.connected()) {
try {
engine.reconcile(active.shots(), active, exporter);
status("Syncing new screenshots");
} catch (Exception e) {
status("Screenshot pending · " + message(e));
}
Thread.sleep(4000);
}
if (!stopping) throw new java.io.IOException("Connection lost. Reconnecting…");
} catch (Exception e) {
page = null;
if (connection != null) connection.close();
if (stopping) break;
status("Waiting for headset · " + message(e));
Thread.sleep(10000);
}
}
} catch (InterruptedException ignored) {
Thread.currentThread().interrupt();
} catch (Exception e) {
state = message(e);
} finally {
page = null;
if (connection != null) connection.close();
stopSelf();
}
}
static String message(Exception e) {
String text = e.getMessage();
if (text == null || text.trim().isEmpty()) return "Try reconnecting.";
if (text.contains("HostKey")
|| text.contains("HostKey has been changed")
|| text.contains("reject HostKey")) return "Headset identity changed. Stop and pair again.";
if (text.contains("Auth fail")) return "Headset login refused. Pair again.";
return text.length() > 180 ? text.substring(0, 180) : text;
}
@Override
public void onDestroy() {
stopping = true;
page = null;
if (engine != null) engine.stop();
if (connection != null) connection.close();
worker.shutdownNow();
instance = null;
if (state.equals("Syncing new screenshots")
|| state.startsWith("Waiting")
|| state.startsWith("Connecting")) state = "Disconnected";
stopForeground(STOP_FOREGROUND_REMOVE);
super.onDestroy();
}
@Override
public IBinder onBind(Intent intent) {
return null;
}
}
@@ -0,0 +1,3 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android" android:width="48dp" android:height="48dp" android:viewportWidth="48" android:viewportHeight="48">
<path android:fillColor="#7CD8C3" android:pathData="M8,12L40,12L44,30L34,36L26,28L22,28L14,36L4,30ZM11,17L9,27L14,29L20,23L28,23L34,29L39,27L37,17Z" />
</vector>
@@ -0,0 +1,9 @@
<resources>
<style name="AppTheme" parent="android:style/Theme.Material.NoActionBar">
<item name="android:fontFamily">sans</item>
<item name="android:colorAccent">#7CD8C3</item>
<item name="android:windowLightStatusBar">false</item>
<item name="android:windowBackground">#111819</item>
<item name="android:navigationBarColor">#111819</item>
</style>
</resources>
@@ -0,0 +1,4 @@
<data-extraction-rules>
<cloud-backup><exclude domain="root" path="." /><exclude domain="file" path="." /><exclude domain="database" path="." /><exclude domain="sharedpref" path="." /><exclude domain="external" path="." /></cloud-backup>
<device-transfer><exclude domain="root" path="." /><exclude domain="file" path="." /><exclude domain="database" path="." /><exclude domain="sharedpref" path="." /><exclude domain="external" path="." /></device-transfer>
</data-extraction-rules>
@@ -0,0 +1 @@
<paths><files-path name="screenshots" path="screenshots/" /></paths>
@@ -0,0 +1,4 @@
<network-security-config>
<base-config cleartextTrafficPermitted="false" />
<domain-config cleartextTrafficPermitted="true"><domain includeSubdomains="false">127.0.0.1</domain><domain includeSubdomains="false">localhost</domain></domain-config>
</network-security-config>
@@ -0,0 +1,789 @@
package com.saphid.framecontrol;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
import static org.junit.Assert.*;
public class SyncEngineTest {
@Rule public TemporaryFolder temporary = new TemporaryFolder();
private static final String FIRST = "123/456/20261024120000_1.jpg";
private static final String SECOND = "123/456/20261024120001_2.png";
private static final SyncEngine.Shot A = shot(FIRST);
private static final SyncEngine.Shot B = shot(SECOND);
private static SyncEngine.Shot shot(String id) { return new SyncEngine.Shot(id, 4, 100.5); }
private static class Transfer implements SyncEngine.Transport {
final List<String> downloads = new ArrayList<>();
boolean valid = true;
boolean truncated;
@Override public void download(SyncEngine.Shot shot, File destination) throws Exception {
downloads.add(shot.id);
try (RandomAccessFile file = new RandomAccessFile(destination, "rw")) {
file.setLength(truncated ? shot.size - 1 : shot.size);
}
}
@Override public boolean validImage(File file) throws Exception { return valid; }
}
private static class Gallery implements SyncEngine.Exporter {
int attempts;
int saves;
boolean deny;
boolean fail;
@Override public boolean export(SyncEngine.Shot shot, File original,
java.util.function.BooleanSupplier eligible) throws Exception {
attempts++;
assertTrue(original.isFile());
if (fail) throw new IOException("Gallery unavailable");
if (deny || !eligible.getAsBoolean()) return false;
saves++;
return true;
}
}
private interface Checked { void run() throws Exception; }
private static IOException fails(Checked operation) throws Exception {
try {
operation.run();
} catch (IOException expected) {
return expected;
}
throw new AssertionError("Expected operation to reject invalid or unavailable state");
}
private static void reconcile(SyncEngine engine, Transfer transfer, Gallery gallery,
SyncEngine.Shot... shots) throws Exception {
engine.reconcile(Arrays.asList(shots), transfer, gallery);
}
private static void observed(SyncEngine engine, Transfer transfer, Gallery gallery,
SyncEngine.Shot shot) throws Exception {
reconcile(engine, transfer, gallery);
reconcile(engine, transfer, gallery, shot);
}
private static void assertDownloadPending(SyncEngine engine, String destination) {
assertEquals(1, engine.items().size());
assertEquals(destination, engine.items().get(0).destination);
assertFalse(engine.items().get(0).file.exists());
}
private static SyncEngine.Item item(SyncEngine engine, String id) {
return engine.items().stream().filter(i -> i.shot.id.equals(id)).findFirst().get();
}
private static void await(CountDownLatch latch) throws InterruptedException {
assertTrue("Timed out waiting for callback", latch.await(5, TimeUnit.SECONDS));
}
private static final class Async implements AutoCloseable {
private final AtomicReference<Throwable> failure = new AtomicReference<>();
private final Thread thread;
Async(Checked operation) {
thread = new Thread(() -> {
try { operation.run(); } catch (Throwable e) { failure.set(e); }
}, "sync-core-test");
thread.start();
}
void finish() throws Exception {
thread.join(5000);
assertFalse("Reconcile did not finish", thread.isAlive());
if (failure.get() != null) throw new AssertionError("Background reconcile failed", failure.get());
}
@Override public void close() throws Exception {
if (thread.isAlive()) thread.interrupt();
thread.join(5000);
assertFalse("Leaked reconcile thread", thread.isAlive());
}
}
@Test public void firstListingIsDurableBaselineAndNeverDownloadsHistory() throws Exception {
File root = temporary.newFolder();
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
SyncEngine engine = new SyncEngine(root, true);
reconcile(engine, transfer, gallery, A);
assertTrue(new File(root, "receipts.json").isFile());
engine = new SyncEngine(root, true);
reconcile(engine, transfer, gallery, A);
reconcile(engine, transfer, gallery, new SyncEngine.Shot(FIRST, 7, 999));
assertTrue(transfer.downloads.isEmpty());
assertTrue(engine.items().isEmpty());
}
@Test public void emptyBaselineRequiresTwoIdenticalNewObservations() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
assertTrue(transfer.downloads.isEmpty());
assertDownloadPending(engine, "pending");
reconcile(engine, transfer, gallery, A);
assertEquals(Collections.singletonList(FIRST), transfer.downloads);
assertEquals(1, gallery.saves);
assertEquals("saved", engine.items().get(0).destination);
assertEquals(4, engine.items().get(0).file.length());
}
@Test public void sizeAndMtimeChangesResetStabilityAndMissingPollBreaksIt() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
SyncEngine.Shot size = new SyncEngine.Shot(FIRST, 5, A.time);
SyncEngine.Shot time = new SyncEngine.Shot(FIRST, 5, A.time + 1);
reconcile(engine, transfer, gallery, size);
reconcile(engine, transfer, gallery, time);
reconcile(engine, transfer, gallery);
reconcile(engine, transfer, gallery, time);
assertTrue(transfer.downloads.isEmpty());
reconcile(engine, transfer, gallery, time);
assertEquals(1, transfer.downloads.size());
assertEquals(5, engine.items().get(0).file.length());
}
@Test public void identityIncludesAccountAndAppAndDuplicateRowsAreOnlyOneObservation() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
reconcile(engine, transfer, gallery, A);
SyncEngine.Shot otherAccount = shot("124/456/20261024120000_1.jpg");
SyncEngine.Shot otherApp = shot("123/457/20261024120000_1.jpg");
reconcile(engine, transfer, gallery, A, otherAccount, otherAccount, otherApp);
assertTrue(transfer.downloads.isEmpty());
reconcile(engine, transfer, gallery, otherAccount, otherApp);
assertEquals(2, transfer.downloads.size());
assertEquals(2, engine.items().size());
reconcile(engine, transfer, gallery, otherAccount, otherApp);
assertEquals(2, transfer.downloads.size());
}
@Test public void truncatedAndInvalidImagesStayQueuedUntilValidRetry() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
transfer.truncated = true;
fails(() -> reconcile(engine, transfer, gallery, A));
assertDownloadPending(engine, "pending");
assertFalse(new File(root, "originals/" + FIRST).exists());
transfer.truncated = false; transfer.valid = false;
fails(() -> reconcile(engine, transfer, gallery, A));
assertDownloadPending(engine, "pending");
assertEquals(0, gallery.attempts);
transfer.valid = true;
reconcile(engine, transfer, gallery, A);
assertEquals(3, transfer.downloads.size());
assertEquals("saved", engine.items().get(0).destination);
assertEquals(0, root.listFiles((dir, name) -> name.startsWith(".download-")).length);
}
@Test public void deniedExportRetriesWithoutDownloadingAndSavedReceiptDeduplicates() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
gallery.deny = true;
reconcile(engine, transfer, gallery, A);
assertEquals("pending", engine.items().get(0).destination);
engine = new SyncEngine(root, true);
gallery.deny = false;
reconcile(engine, transfer, gallery); // Export retry does not need the remote file still listed.
assertEquals(1, transfer.downloads.size());
assertEquals(1, gallery.saves);
reconcile(new SyncEngine(root, true), transfer, gallery, A);
assertEquals(2, gallery.attempts);
}
@Test public void thrownExportRetainsPendingReceiptAcrossRelaunch() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
gallery.fail = true;
fails(() -> reconcile(engine, transfer, gallery, A));
assertEquals("pending", engine.items().get(0).destination);
gallery.fail = false;
reconcile(new SyncEngine(root, true), transfer, gallery);
assertEquals(1, gallery.saves);
assertEquals(1, transfer.downloads.size());
}
@Test public void queuedShotAndStabilitySurviveRelaunch() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
transfer.truncated = true;
fails(() -> reconcile(engine, transfer, gallery, A));
SyncEngine relaunched = new SyncEngine(root, true);
transfer.truncated = false;
reconcile(relaunched, transfer, gallery, A);
assertEquals("appOnly", relaunched.items().get(0).destination);
assertEquals(0, gallery.attempts);
}
@Test public void firstObservationSurvivesRelaunch() throws Exception {
File root = temporary.newFolder();
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(new SyncEngine(root, false), transfer, gallery, A);
SyncEngine engine = new SyncEngine(root, false);
reconcile(engine, transfer, gallery, A);
assertEquals(1, engine.items().size());
}
@Test public void offThenOnNeverRetroExportsDownloadedOrQueuedItems() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
gallery.deny = true;
reconcile(engine, transfer, gallery, A);
engine.setGallery(false);
engine.setGallery(true);
gallery.deny = false;
reconcile(engine, transfer, gallery, A, B);
assertEquals("appOnly", engine.items().get(0).destination);
reconcile(engine, transfer, gallery, A, B);
assertEquals(1, gallery.saves); // Only the later B item.
assertEquals("appOnly", engine.items().stream().filter(i -> i.shot.id.equals(FIRST))
.findFirst().get().destination);
}
@Test public void switchingOffWhileDownloadBlockedChangesItsDisposition() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), true);
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Transfer transfer = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
entered.countDown(); await(release); super.download(shot, file);
}
};
Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
try (Async async = new Async(() -> reconcile(engine, transfer, gallery, A))) {
try {
await(entered);
engine.setGallery(false); // Must finish while network is blocked.
engine.setGallery(true);
} finally { release.countDown(); }
async.finish();
}
assertEquals("appOnly", engine.items().get(0).destination);
assertEquals(0, gallery.attempts);
}
@Test public void switchingOffWhileExporterBlockedInvalidatesEligibility() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), true);
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Gallery gallery = new Gallery() {
@Override public boolean export(SyncEngine.Shot shot, File original,
java.util.function.BooleanSupplier eligible) throws Exception {
assertTrue(eligible.getAsBoolean());
entered.countDown(); await(release);
assertFalse(eligible.getAsBoolean());
return false;
}
};
Transfer transfer = new Transfer();
observed(engine, transfer, gallery, A);
try (Async async = new Async(() -> reconcile(engine, transfer, gallery, A))) {
try { await(entered); engine.setGallery(false); engine.setGallery(true); }
finally { release.countDown(); }
async.finish();
}
assertEquals("appOnly", engine.items().get(0).destination);
}
@Test public void lateDownloadAfterStopCannotOverwriteFreshRunOrJournal() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Transfer slow = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
entered.countDown(); await(release);
Files.write(file.toPath(), new byte[]{9, 9, 9, 9});
}
};
Transfer fresh = new Transfer(); Gallery gallery = new Gallery();
observed(engine, fresh, gallery, A);
try (Async async = new Async(() -> reconcile(engine, slow, gallery, A))) {
byte[] journal;
try {
await(entered); engine.stop(); engine.setGallery(false);
reconcile(engine, fresh, gallery, A);
journal = Files.readAllBytes(new File(root, "receipts.json").toPath());
} finally { release.countDown(); }
async.finish();
assertArrayEquals(journal, Files.readAllBytes(new File(root, "receipts.json").toPath()));
}
assertArrayEquals(new byte[4], Files.readAllBytes(engine.items().get(0).file.toPath()));
assertEquals("appOnly", new SyncEngine(root, true).items().get(0).destination);
}
@Test public void lateExportAfterStopDoesNotMarkFreshAppOnlyReceiptSaved() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer();
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Gallery delayed = new Gallery() {
@Override public boolean export(SyncEngine.Shot shot, File original,
java.util.function.BooleanSupplier eligible) throws Exception {
entered.countDown(); await(release);
assertFalse(eligible.getAsBoolean());
return true; // Model a pre-stop external save whose callback arrived late.
}
};
observed(engine, transfer, delayed, A);
try (Async async = new Async(() -> reconcile(engine, transfer, delayed, A))) {
byte[] journal;
try {
await(entered); engine.stop(); engine.setGallery(false);
reconcile(engine, transfer, new Gallery(), A);
journal = Files.readAllBytes(new File(root, "receipts.json").toPath());
} finally { release.countDown(); }
async.finish();
assertArrayEquals(journal, Files.readAllBytes(new File(root, "receipts.json").toPath()));
}
assertEquals("appOnly", engine.items().get(0).destination);
}
@Test public void concurrentReconcileIsRejectedWithoutBlockingOnDownload() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Transfer slow = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
entered.countDown(); await(release); super.download(shot, file);
}
};
Gallery gallery = new Gallery(); observed(engine, slow, gallery, A);
try (Async async = new Async(() -> reconcile(engine, slow, gallery, A))) {
try {
await(entered);
try { reconcile(engine, new Transfer(), gallery, A); fail("Concurrent run accepted"); }
catch (IllegalStateException expected) { assertDownloadPending(engine, "appOnly"); }
} finally { release.countDown(); }
async.finish();
}
}
@Test public void corruptedAndIncompleteJournalsFailClosed() throws Exception {
for (String content : new String[]{"", "{", "null", "{}", "{\"gallery\":true}",
"{\"version\":2,\"initialized\":true,\"gallery\":true,\"records\":{}}",
"{\"version\":1,\"initialized\":true,\"gallery\":true,\"records\":{}} trailing"}) {
File root = temporary.newFolder();
File journal = new File(root, "receipts.json");
Files.write(journal.toPath(), content.getBytes(StandardCharsets.UTF_8));
fails(() -> new SyncEngine(root, true));
assertEquals(content, new String(Files.readAllBytes(journal.toPath()), StandardCharsets.UTF_8));
}
}
@Test public void damagedReceiptAndMissingOriginalFailClosed() throws Exception {
File root = temporary.newFolder();
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
SyncEngine engine = new SyncEngine(root, false);
observed(engine, transfer, gallery, A); reconcile(engine, transfer, gallery, A);
File journal = new File(root, "receipts.json");
String json = new String(Files.readAllBytes(journal.toPath()), StandardCharsets.UTF_8);
for (String damaged : new String[]{json.replace("downloaded", "unknown"),
json.replace("\"time\":100.5", "\"otherTime\":100.5"),
json.replace("\"size\":4", "\"size\":0"),
json.replace("\"time\":100.5", "\"time\":1e999")}) {
Files.write(journal.toPath(), damaged.getBytes(StandardCharsets.UTF_8));
fails(() -> new SyncEngine(root, false));
}
Files.write(journal.toPath(), json.getBytes(StandardCharsets.UTF_8));
Files.delete(engine.items().get(0).file.toPath());
fails(() -> new SyncEngine(root, false));
Files.delete(journal.toPath());
Files.write(new File(root, "originals/orphan").toPath(), new byte[]{1});
fails(() -> new SyncEngine(root, false));
}
@Test public void invalidListingCannotEstablishBaselineOrPartiallyAdvanceObservations() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
fails(() -> reconcile(engine, transfer, gallery, A, shot("../bad")));
reconcile(engine, transfer, gallery, A); // A is still baseline, not newly discovered.
reconcile(engine, transfer, gallery, A, B);
fails(() -> reconcile(engine, transfer, gallery, B, shot("../bad")));
assertTrue(transfer.downloads.isEmpty());
reconcile(engine, transfer, gallery, B);
assertEquals(Collections.singletonList(SECOND), transfer.downloads);
}
@Test public void metadataBoundsAndPathTraversalAreRejected() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
for (String id : new String[]{"../" + FIRST, "/" + FIRST, "123/../" + FIRST,
"123\\456\\20261024120000_1.jpg", "123/456/20261024120000_1.JPG",
"1234567890123/456/20261024120000_1.jpg", "123/123456789012345678901/20261024120000_1.jpg",
"123/456/2026102412000_1.jpg", "123/456/20261024120000_12345.jpg",
FIRST + "\n", FIRST + "/extra", "123/456/20261024120000_1.gif"}) {
fails(() -> reconcile(engine, transfer, gallery, shot(id)));
}
for (long size : new long[]{0, -1, 30L * 1024 * 1024 + 1, Long.MAX_VALUE}) {
fails(() -> reconcile(engine, transfer, gallery, new SyncEngine.Shot(FIRST, size, 1)));
}
for (double time : new double[]{Double.NaN, Double.POSITIVE_INFINITY, Double.NEGATIVE_INFINITY}) {
fails(() -> reconcile(engine, transfer, gallery, new SyncEngine.Shot(FIRST, 1, time)));
}
fails(() -> engine.reconcile(null, transfer, gallery));
fails(() -> reconcile(engine, transfer, gallery, (SyncEngine.Shot) null));
fails(() -> reconcile(engine, transfer, gallery, A, new SyncEngine.Shot(FIRST, 5, 1)));
assertTrue(transfer.downloads.isEmpty());
}
@Test public void exactMaximumMetadataAndThirtyMebibyteOriginalAreAccepted() throws Exception {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
SyncEngine.Shot maximum = new SyncEngine.Shot(
"123456789012/12345678901234567890/20261024123456_1234.png", 30L * 1024 * 1024,
Double.MAX_VALUE);
observed(engine, transfer, gallery, maximum);
reconcile(engine, transfer, gallery, maximum);
assertEquals(maximum.size, engine.items().get(0).file.length());
assertEquals("appOnly", engine.items().get(0).destination);
}
@Test public void symbolicLinksCannotEscapeOriginalsOrReplaceJournal() throws Exception {
File root = temporary.newFolder(), outside = temporary.newFolder();
Files.createSymbolicLink(new File(root, "originals").toPath(), outside.toPath());
fails(() -> new SyncEngine(root, true));
Files.delete(new File(root, "originals").toPath());
File target = new File(outside, "receipts"); Files.write(target.toPath(), new byte[]{1});
Files.createSymbolicLink(new File(root, "receipts.json").toPath(), target.toPath());
fails(() -> new SyncEngine(root, true));
assertArrayEquals(new byte[]{1}, Files.readAllBytes(target.toPath()));
}
@Test public void nestedSymlinkIntroducedDuringDownloadCannotEscapeRoot() throws Exception {
File root = temporary.newFolder(), outside = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
super.download(shot, file);
Files.createDirectory(new File(root, "originals").toPath());
Files.createSymbolicLink(new File(root, "originals/123").toPath(), outside.toPath());
}
};
Gallery gallery = new Gallery(); observed(engine, transfer, gallery, A);
fails(() -> reconcile(engine, transfer, gallery, A));
assertDownloadPending(engine, "appOnly");
assertEquals(0, outside.list().length);
}
@Test public void failedPreferenceWriteRollsBackGalleryAndPendingDisposition() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A); gallery.deny = true;
reconcile(engine, transfer, gallery, A);
File journal = new File(root, "receipts.json"); byte[] saved = Files.readAllBytes(journal.toPath());
Files.delete(journal.toPath()); Files.createDirectory(journal.toPath());
fails(() -> engine.setGallery(false));
assertEquals("pending", engine.items().get(0).destination);
Files.delete(journal.toPath()); Files.write(journal.toPath(), saved);
gallery.deny = false; reconcile(engine, transfer, gallery, A);
assertEquals(1, gallery.saves);
assertEquals("saved", engine.items().get(0).destination);
}
@Test public void failedQueueWriteDoesNotStartDownloadOrChangeMemory() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
File journal = new File(root, "receipts.json"); byte[] saved = Files.readAllBytes(journal.toPath());
Files.delete(journal.toPath()); Files.createDirectory(journal.toPath());
fails(() -> reconcile(engine, transfer, gallery, A));
assertTrue(transfer.downloads.isEmpty());
assertDownloadPending(engine, "appOnly");
Files.delete(journal.toPath()); Files.write(journal.toPath(), saved);
reconcile(engine, transfer, gallery, A);
assertEquals(1, engine.items().size());
}
@Test public void exportedImageWithFailedReceiptWriteRecoversThroughIdempotentExporter() throws Exception {
File root = temporary.newFolder();
File journal = new File(root, "receipts.json");
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer();
AtomicReference<byte[]> pendingJournal = new AtomicReference<>();
Gallery gallery = new Gallery() {
private boolean exists;
@Override public boolean export(SyncEngine.Shot shot, File file,
java.util.function.BooleanSupplier eligible) throws Exception {
attempts++;
if (exists) return true; // Recover the previous external save without another insert.
assertTrue(eligible.getAsBoolean());
exists = true; saves++;
pendingJournal.set(Files.readAllBytes(journal.toPath()));
Files.delete(journal.toPath()); Files.createDirectory(journal.toPath());
return true;
}
};
observed(engine, transfer, gallery, A);
fails(() -> reconcile(engine, transfer, gallery, A));
assertEquals("pending", engine.items().get(0).destination);
Files.delete(journal.toPath()); Files.write(journal.toPath(), pendingJournal.get());
SyncEngine relaunched = new SyncEngine(root, true);
reconcile(relaunched, transfer, gallery);
assertEquals("saved", relaunched.items().get(0).destination);
assertEquals(1, gallery.saves);
assertEquals(2, gallery.attempts);
assertEquals(1, transfer.downloads.size());
}
@Test public void failedBaselineWriteCannotTurnHistoryIntoNewScreenshots() throws Exception {
File root = temporary.newFolder();
File journal = new File(root, "receipts.json");
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
byte[] initialJournal = Files.readAllBytes(journal.toPath());
Files.delete(journal.toPath()); Files.createDirectory(journal.toPath());
fails(() -> reconcile(engine, transfer, gallery, A));
Files.delete(journal.toPath()); Files.write(journal.toPath(), initialJournal);
reconcile(engine, transfer, gallery, A);
reconcile(engine, transfer, gallery, A);
assertTrue(engine.items().isEmpty());
assertTrue(transfer.downloads.isEmpty());
}
@Test public void firstObservedOffStaysAppOnlyWhenEnabledBeforeStabilityIncludingRelaunch() throws Exception {
for (boolean restart : new boolean[]{false, true}) {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
assertDownloadPending(engine, "appOnly");
engine.setGallery(true);
if (restart) engine = new SyncEngine(root, true);
reconcile(engine, transfer, gallery, A);
assertTrue(engine.items().get(0).file.isFile());
assertEquals("appOnly", engine.items().get(0).destination);
assertEquals(0, gallery.attempts);
reconcile(new SyncEngine(root, true), transfer, gallery, A);
assertEquals(0, gallery.attempts);
}
}
@Test public void observingOnOffOnStaysAppOnlyBeforeStabilityIncludingRelaunch() throws Exception {
for (boolean restart : new boolean[]{false, true}) {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer(); Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
assertDownloadPending(engine, "pending");
engine.setGallery(false);
assertDownloadPending(engine, "appOnly");
if (restart) engine = new SyncEngine(root, false);
engine.setGallery(true);
if (restart) engine = new SyncEngine(root, true);
// An intervening metadata change must preserve the first observation's disposition.
SyncEngine.Shot changed = new SyncEngine.Shot(FIRST, 5, A.time + 1);
reconcile(engine, transfer, gallery, changed);
assertDownloadPending(engine, "appOnly");
reconcile(engine, transfer, gallery, changed);
assertTrue(engine.items().get(0).file.isFile());
assertEquals("appOnly", engine.items().get(0).destination);
assertEquals(0, gallery.attempts);
}
}
@Test public void permanentGalleryFailureAllowsLaterOriginalAndExportInSamePoll() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer();
IOException unavailable = new IOException("permanent Gallery failure");
Gallery gallery = new Gallery() {
@Override public boolean export(SyncEngine.Shot shot, File file,
java.util.function.BooleanSupplier eligible) throws Exception {
if (shot.id.equals(FIRST)) throw unavailable;
return super.export(shot, file, eligible);
}
};
observed(engine, transfer, gallery, A);
assertSame(unavailable, fails(() -> reconcile(engine, transfer, gallery, A)).getCause());
fails(() -> reconcile(engine, transfer, gallery, A, B));
assertFalse(item(engine, SECOND).file.exists()); // B has only one observation.
assertSame(unavailable, fails(() -> reconcile(engine, transfer, gallery, A, B)).getCause());
assertEquals(Arrays.asList(FIRST, SECOND), transfer.downloads);
assertTrue(item(engine, SECOND).file.isFile());
assertEquals("saved", item(engine, SECOND).destination);
assertEquals("pending", item(engine, FIRST).destination);
fails(() -> reconcile(engine, transfer, gallery, A, B));
assertEquals(2, transfer.downloads.size());
assertEquals(1, gallery.saves);
SyncEngine relaunched = new SyncEngine(root, true);
assertEquals("pending", item(relaunched, FIRST).destination);
assertEquals("saved", item(relaunched, SECOND).destination);
}
@Test public void permanentTransferFailureAllowsLaterOriginalInSamePollAndStaysQueued() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
IOException unavailable = new IOException("permanent transfer failure");
Transfer transfer = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
if (shot.id.equals(FIRST)) { downloads.add(shot.id); throw unavailable; }
super.download(shot, file);
}
};
Gallery gallery = new Gallery();
observed(engine, transfer, gallery, A);
fails(() -> reconcile(engine, transfer, gallery, A));
fails(() -> reconcile(engine, transfer, gallery, A, B));
assertSame(unavailable, fails(() -> reconcile(engine, transfer, gallery, A, B)).getCause());
assertFalse(item(engine, FIRST).file.exists());
assertTrue(item(engine, SECOND).file.isFile());
assertEquals("appOnly", item(engine, SECOND).destination);
assertEquals(Arrays.asList(FIRST, FIRST, FIRST, SECOND), transfer.downloads);
SyncEngine relaunched = new SyncEngine(root, false);
fails(() -> reconcile(relaunched, transfer, gallery, A, B));
assertEquals(1, Collections.frequency(transfer.downloads, SECOND));
assertFalse(item(relaunched, FIRST).file.exists());
assertTrue(item(relaunched, SECOND).file.isFile());
}
@Test public void permanentTruncatedInvalidOrUndecodableImageDoesNotStarveLaterOriginal() throws Exception {
for (String mode : Arrays.asList("truncated", "invalid", "decoder")) {
SyncEngine engine = new SyncEngine(temporary.newFolder(), false);
Transfer transfer = new Transfer() {
private boolean first;
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
super.download(shot, file);
first = shot.id.equals(FIRST);
if (first && mode.equals("truncated")) {
try (RandomAccessFile contents = new RandomAccessFile(file, "rw")) {
contents.setLength(shot.size - 1);
}
}
}
@Override public boolean validImage(File file) throws Exception {
if (first && mode.equals("decoder")) throw new IOException("decoder failure");
return !first;
}
};
Gallery gallery = new Gallery();
reconcile(engine, transfer, gallery);
reconcile(engine, transfer, gallery, A, B);
IOException error = fails(() -> reconcile(engine, transfer, gallery, A, B));
assertTrue(error.getMessage().contains(FIRST));
assertEquals(Arrays.asList(FIRST, SECOND), transfer.downloads);
assertFalse(item(engine, FIRST).file.exists());
assertTrue(item(engine, SECOND).file.isFile());
fails(() -> reconcile(engine, transfer, gallery, A, B));
assertEquals(1, Collections.frequency(transfer.downloads, SECOND));
assertEquals(2, Collections.frequency(transfer.downloads, FIRST));
}
}
@Test public void fatalJournalCommitAfterRetryableFailureHaltsLaterDownloadsUntilRecovered() throws Exception {
File root = temporary.newFolder();
File journal = new File(root, "receipts.json");
SyncEngine engine = new SyncEngine(root, true);
Transfer transfer = new Transfer();
SyncEngine.Shot third = shot("123/456/20261024120002_3.jpg");
IOException unavailable = new IOException("permanent Gallery failure");
AtomicReference<byte[]> pendingJournal = new AtomicReference<>();
Gallery gallery = new Gallery() {
@Override public boolean export(SyncEngine.Shot shot, File file,
java.util.function.BooleanSupplier eligible) throws Exception {
if (shot.id.equals(FIRST)) throw unavailable;
if (shot.id.equals(SECOND) && pendingJournal.get() == null) {
assertTrue(eligible.getAsBoolean());
pendingJournal.set(Files.readAllBytes(journal.toPath()));
Files.delete(journal.toPath()); Files.createDirectory(journal.toPath());
}
return true;
}
};
reconcile(engine, transfer, gallery);
reconcile(engine, transfer, gallery, A, B, third);
IOException fatal = fails(() -> reconcile(engine, transfer, gallery, A, B, third));
assertNotSame(unavailable, fatal.getCause()); // Journal failure supersedes deferred Gallery failure.
assertTrue(journal.isDirectory());
assertEquals(Arrays.asList(FIRST, SECOND), transfer.downloads);
assertEquals("pending", item(engine, SECOND).destination);
assertFalse(item(engine, third.id).file.exists());
Files.delete(journal.toPath()); Files.write(journal.toPath(), pendingJournal.get());
SyncEngine relaunched = new SyncEngine(root, true);
assertSame(unavailable, fails(() -> reconcile(relaunched, transfer, gallery, A, B, third)).getCause());
assertEquals(Arrays.asList(FIRST, SECOND, third.id), transfer.downloads);
assertEquals("saved", item(relaunched, SECOND).destination);
assertEquals("saved", item(relaunched, third.id).destination);
assertEquals("pending", item(relaunched, FIRST).destination);
}
@Test public void fatalOriginalInstallationFailureHaltsLaterDownloads() throws Exception {
File root = temporary.newFolder();
SyncEngine engine = new SyncEngine(root, false);
Transfer transfer = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
super.download(shot, file);
if (shot.id.equals(FIRST)) Files.write(new File(root, "originals").toPath(), new byte[]{1});
}
};
Gallery gallery = new Gallery();
reconcile(engine, transfer, gallery);
reconcile(engine, transfer, gallery, A, B);
IOException fatal = fails(() -> reconcile(engine, transfer, gallery, A, B));
assertEquals("Invalid originals directory", fatal.getMessage());
assertEquals(Collections.singletonList(FIRST), transfer.downloads);
assertFalse(item(engine, FIRST).file.exists());
assertFalse(item(engine, SECOND).file.exists());
}
@Test public void lateRetryableFailureAfterStopCannotChangeFreshJournalOrSurfaceAnError() throws Exception {
File root = temporary.newFolder();
File journal = new File(root, "receipts.json");
SyncEngine engine = new SyncEngine(root, false);
Transfer fresh = new Transfer(); Gallery gallery = new Gallery();
CountDownLatch entered = new CountDownLatch(1), release = new CountDownLatch(1);
Transfer slow = new Transfer() {
@Override public void download(SyncEngine.Shot shot, File file) throws Exception {
entered.countDown(); await(release);
throw new IOException("late transfer failure");
}
};
reconcile(engine, fresh, gallery);
reconcile(engine, fresh, gallery, A, B);
try (Async async = new Async(() -> reconcile(engine, slow, gallery, A, B))) {
byte[] saved;
try {
await(entered); engine.stop();
reconcile(engine, fresh, gallery, A, B);
saved = Files.readAllBytes(journal.toPath());
} finally { release.countDown(); }
async.finish();
assertArrayEquals(saved, Files.readAllBytes(journal.toPath()));
}
assertEquals(Arrays.asList(FIRST, SECOND), fresh.downloads);
assertTrue(item(engine, FIRST).file.isFile());
assertTrue(item(engine, SECOND).file.isFile());
}
}
+1
View File
@@ -0,0 +1 @@
plugins { id 'com.android.application' version '9.1.0' apply false }
+2
View File
@@ -0,0 +1,2 @@
org.gradle.jvmargs=-Xmx2g -Dfile.encoding=UTF-8
android.useAndroidX=true
Binary file not shown.
+8
View File
@@ -0,0 +1,8 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.4.1-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionSha256Sum=2ab2958f2a1e51120c326cad6f385153bb11ee93b3c216c5fccebfdfbb7ec6cb
Vendored Executable
+248
View File
@@ -0,0 +1,248 @@
#!/bin/sh
#
# Copyright © 2015 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/2d6327017519d23b96af35865dc997fcb544fb40/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"
+93
View File
@@ -0,0 +1,93 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env python3
"""Use the same reproducible headset bundle as the iPhone app."""
import pathlib
import subprocess
import sys
root = pathlib.Path(__file__).resolve().parents[2]
out = pathlib.Path(sys.argv[1]).resolve()
out.mkdir(parents=True, exist_ok=True)
version = subprocess.check_output(
[sys.executable, str(root / 'ios/scripts/make_frame_bundle.py'),
str(out / 'frame-bundle.tar.gz')], cwd=root, text=True).strip().splitlines()[-1]
(out / 'frame-bundle.version').write_text(version + '\n')
+4
View File
@@ -0,0 +1,4 @@
pluginManagement { repositories { google(); mavenCentral(); gradlePluginPortal() } }
dependencyResolutionManagement { repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS); repositories { google(); mavenCentral() } }
rootProject.name = 'FrameControl'
include ':app'
-4
View File
@@ -13,10 +13,6 @@ const path = require("path");
const REQUIRED = [
"ui/server.py",
"ui/frame_power.py",
"ui/frame_status.py",
"scripts/frame_power_sample.py",
"scripts/frame_power_report.py",
"ui/frame_android.py",
"frame/android/lepton-app.sh",
"frame/openxr-compat/XrApiLayer_FRAME_compat.json",
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "frame-control",
"version": "0.4.18",
"version": "0.4.13",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frame-control",
"version": "0.4.18",
"version": "0.4.13",
"license": "MIT",
"devDependencies": {
"electron": "^44.4.5",
+2 -4
View File
@@ -1,7 +1,7 @@
{
"name": "frame-control",
"productName": "Frame Control",
"version": "0.4.18",
"version": "0.4.13",
"description": "Desktop app for managing a Valve Steam Frame over SSH",
"private": true,
"main": "main.js",
@@ -68,9 +68,7 @@
"from": "../scripts",
"to": "scripts",
"filter": [
"*.sh",
"frame_power_sample.py",
"frame_power_report.py"
"*.sh"
]
},
{
+85
View File
@@ -0,0 +1,85 @@
# Frame Control for Android phones
The native Android companion lives in `android/`. It requires Android 10 or
later and a Steam Frame with Developer Mode and SSH enabled. This is a new
implementation, separate from the headset's `frame/android` Lepton tools.
## Pair and sync
1. Enter the headset address, SSH port and username (normally `steamos`).
2. Choose **Pair headset**, compare the displayed SSH fingerprint with the
headset's identity, and accept it. Enter the Developer Mode password once.
The app installs this phone's SSH public key and verifies key login before
storing the pairing. The private key stays in Android Keystore; the
password is not saved. A changed headset host key is refused.
3. Choose **Start syncing** and allow notifications. The persistent notification
shows the active connected-device session and offers **Stop syncing**.
4. Take a new screenshot. Its original downloads into Frame Control. **Save
screenshots to Gallery** is enabled by default and adds it to
`Pictures/Frame Control`. Android 10+ lets an app create its own Gallery
items without broad photo-reading permission.
The first successful listing establishes a history baseline. Existing screenshots
are not imported automatically. The app waits for two equal size/time listings,
checks the original's byte count and image decode, and saves it atomically.
Identity includes the paired headset, Steam account, game and filename.
Turning Gallery off keeps automatic downloads inside Frame Control. Pending
exports become app-only; enabling it again does not export those earlier items.
Previously saved Gallery items stay there. Original files remain available from
**Open / share original** when disconnected. The app keeps each headset's
library separately.
Gallery export uses a deterministic app-owned MediaStore row. A pending row is
rewritten after interruption, and a completed row is recognized before inserting
another. A failed export leaves the app original available for retry. Preferences
and the final Gallery commit are serialized so a disabled setting is respected.
**Headset controls** opens the shared Frame Control web interface through a
loopback-only SSH tunnel. The server has a fresh session key and listens only on
the headset's loopback interface. The tunnel and server end with the phone's
SSH session. This initial Android shell does not implement all iPhone-specific
web bridges; native screenshot originals use the downloads screen.
## Background behavior and release boundary
Sync is an explicit, user-started Android `connectedDevice` foreground service.
It can continue after leaving the app while its notification is present and
reconnects when the headset returns. It is not a Firebase push receiver.
Android battery-saving/Doze, loss of network, force-stop or process termination
can delay or end the session. The app does not request battery-optimization
exemption, use unrelated background modes, or restart itself after force-stop.
Restart sync from the app if the session ends.
The full cross-platform request still requires actual locked-phone background
proof. iOS APNs delivery is a separate unimplemented dependency. An emulator
test is not evidence that either physical platform meets that requirement.
Do not publish this as a verified always-on screenshot feature.
## Build and verify
Use JDK 17 and an Android SDK containing platform 36/build-tools 36.0.0:
```sh
export JAVA_HOME=/path/to/jdk17
export ANDROID_HOME=/path/to/android-sdk
./android/gradlew -p android assembleDebug testDebugUnitTest lintDebug
```
The Gradle wrapper pins version 9.4.1 and its distribution checksum. The build
generates the same reproducible headset bundle as iOS, including the existing
SHA-256-checked KDE Connect package fetch. Generated assets and build outputs
are ignored. No signing key, headset password or machine-specific SDK path is
committed. `assembleDebug` produces `android/app/build/outputs/apk/debug/app-debug.apk`.
It uses a development signing identity, not a configured Play release key.
`assembleDebugAndroidTest` builds instrumentation for real MediaStore round-trip,
retry recovery and app-only behavior. Optional real-headset hooks exist only in
the separate test APK (`ScreenshotDeviceTest`), not in the product app. They
provision a public test key and permit original/Gallery hash readback without
adding a product authentication bypass. Remove the temporary headset key and
fixtures after live tests.
No Play app creation, store upload, invitation or public release is part of a
debug build. Preserve a dedicated release signing identity and verify physical
devices before a production release.
+31 -64
View File
@@ -12,41 +12,15 @@ with no sign-in files or keys in the archives. Restoring one game's saves on top
of identical files wrote nothing and left nothing behind. The session check
reported Game Mode, so **Return to Game Mode** changed nothing. Not tested on a
headset: a restore after a real factory reset, a restore that replaces files,
Android data restore. Unit tests and the fake Frame cover those (see
[Verification](#verification)).
On 2026-10-24, the real-device reproduction in [issue #142](https://github.com/saphid/frame-control/issues/142)
confirmed full Plasma Desktop Mode on the same build: the Plasma session target
was active and the Game Mode target inactive. Desktop Mode also ran gamescope
for `steamvr-plasma.service`, so the old process check incorrectly reported
Game Mode. SteamOS's `steamosctl switch-to-game-mode` restored Game Mode and
reversed the target states; battery stayed at 99%, and the rollback timer and
shared lock were cleaned up. After #143 passed CI, the corrected code was
tested in another full Plasma session under a three-minute rollback timer.
`ui/frame_backup.py session` reported Desktop Mode despite gamescope, and
`ui/frame_backup.py game-mode` invoked the same guarded Return function used by
the button. Game Mode became active and a second call made no switch. Battery
remained at 99%; the timer and lock were removed. That test used the production
CLI rather than clicking the desktop button.
Later on 2026-10-24, the published **v0.4.17 Linux x86_64 AppImage** passed the
literal packaged-button check on the same Frame/build. With the shared lock and
a verified three-minute rollback timer, the headset entered full Plasma Desktop
Mode. In the packaged Electron window, **Tools → Backup & recovery → Check**
reported Desktop Mode despite gamescope. A trusted automated pointer click on
the visible **Return to Game Mode** button, accepting its confirmation, returned
`switched: true` and `steamosctl switch-to-game-mode`. The systemd targets then
confirmed Game Mode. A second confirmed button click returned `switched: false`;
the Game Mode service invocation and SteamVR PID stayed unchanged. **Check**
then reported Game Mode and disabled Return. The test app/server and virtual
display were stopped; the timer and shared lock were removed. Steam and SteamVR remained running
at 99% battery. [Commands, responses, cleanup and limits](evidence/return-game-mode-packaged-2026-10-24.json),
[before](evidence/return-game-mode-packaged-2026-10-24-before.png) and
[after](evidence/return-game-mode-packaged-2026-10-24-after.png).
This used the unchanged extracted AppImage payload with its bundled Python on
Xvfb (`--no-sandbox --disable-gpu`), following the packaged Export test in #156.
FUSE mounting, desktop integration, hardware rendering and the view inside a
worn headset were not checked. Older SteamOS builds remain fixture-tested.
Android data restore, and the switch out of a real Desktop Mode session. Unit
tests and the fake Frame cover those (see [Verification](#verification)).
On 2026-10-24 a read-only session check on the same build again reported Game
Mode (`gamescope=true`, `plasma=false`, `nested_desktop=false`) and the
`steamos-session-select` command was present. Opening the nested desktop for a
clipboard test did not enter full Desktop Mode. A full session switch was not
attempted because it restarts the headset session and a reliable return path
was not established; the **Return to Game Mode** branch for a real full
Desktop Mode session remains unverified.
## Back up
@@ -144,28 +118,29 @@ normally runs *inside* it as one panel (`steamos-nested-desktop`, see
[how-the-frame-works.md](how-the-frame-works.md#the-layer-cake)). If that panel
is what you see, you're already in Game Mode: close it from the Steam dashboard.
Full Desktop Mode is a different session. **Check** reads the main user's
systemd session targets over SSH: `gamescope-session.target` and
`plasma-workspace-wayland.target`. Game Mode requires the former active and the
latter inactive; Desktop Mode requires the reverse. This takes precedence over
process names: full Desktop Mode can run gamescope for its VR overlay, and
Game Mode can run a nested Plasma panel. Check changes nothing.
Full Desktop Mode is a different session. SteamOS picks the session through
SDDM's autologin file, `/etc/sddm.conf.d/zz-steamos-autologin.conf` (or the
default `/etc/sddm.conf.d/steamos.conf`), and switches with
`steamos-session-select` (`holo-session-select` on newer images). This is what
Valve's own devkit tools on the Frame read and run
([`steamos-get-status`](../frame/devkit-utils/steamos-get-status),
[`steamos-delete`](../frame/devkit-utils/steamos-delete)).
Conflicting, transitional, failed or unreadable targets produce **unknown**.
On older images where both targets are explicitly absent, the fallback accepts
only gamescope without Plasma/nested desktop as Game Mode, or Plasma without
gamescope/nested desktop as Desktop Mode. Mixed processes remain unknown.
The SDDM autologin file is still reported as a login preference; it cannot
establish which session is running.
**Check** reads that file and whether gamescope and Plasma are running; it
changes nothing. On the Frame gamescope's process is `gamescope-wl`, started by
`start-gamescope-session`, and either counts as Game Mode. Only when Plasma is
running without gamescope (Desktop Mode) does **Return to Game Mode** run
exactly `steamos-session-select gamescope` (or `holo-session-select
gamescope`) over SSH, after asking. It closes the desktop and anything open in
it. Frame Control never edits the SDDM files itself, and runs nothing when the
headset is in Game Mode, when it can't tell which mode it's in, or when the
image has neither command.
After confirmation, **Return to Game Mode** runs exactly
`steamosctl switch-to-game-mode` when available, or the older
`holo-session-select gamescope` / `steamos-session-select gamescope` command.
It closes the desktop and anything open in it. Frame Control never edits the
SDDM files itself, and runs no switch when the headset is in Game Mode, when
its mode is unknown, or when no supported switch command is installed. A
refused command reports an error; a request being sent does not verify that
the new session has started.
**Uncertain:** whether the Frame offers a full Desktop Mode session at all
(only the nested desktop has been seen on BUILD_ID 20260922.6101926 and
20260925.6191901), and whether `steamos-session-select` there works without
the Developer Mode password. If the switch is refused, Frame Control says so and
leaves the headset as it was.
### Manual fallback
@@ -176,9 +151,7 @@ If the button can't help, or the headset isn't reachable:
Desktop Mode session, so a restart brings Game Mode back. That the Frame
behaves the same is **inferred**, not tested.
2. **Over SSH** (Developer Mode on), run Valve's switch yourself:
`ssh frame steamosctl switch-to-game-mode` (verified on SteamOS 0.5.6).
Older images may instead provide `steamos-session-select gamescope` or
`holo-session-select gamescope`.
`ssh frame steamos-session-select gamescope`.
3. **Boot menu:** hold Power 10 seconds, then power on while holding the AUX
button on top of Power. Choose `Current`, or `Previous` to start the other
SteamOS slot; both keep your data. Details and the further options
@@ -199,9 +172,3 @@ restore replaces only its own files and keeps the old ones, unsafe archives
names that try to leave the backups folder, and the exact Game Mode command.
`tests/e2e/test_recovery_kit.py` backs up a Proton save from the fake Frame through
the server, deletes it and restores it. Neither involves a real headset.
Isolated session tests in the same suite reproduce full Plasma plus gamescope,
Game Mode with and without the nested desktop, conflicting/transitional targets,
missing targets on older builds, failed/timed-out systemd reads, selector
compatibility, and the confirmed Return action with SSH mocked. Run them with
`python3 -m unittest discover -s tests -p test_backup.py -v`.
@@ -0,0 +1,168 @@
# Android phone screenshot sync evidence — 2026-10-24
Work: [#108](https://github.com/saphid/frame-control/issues/108). The user
authorized a new Android companion after the earlier APK could not be located.
This is a development build, not a production or Play release.
## Implemented
Native Android 10+ companion in `android/`, with pinned SSH transport and a
non-exportable Android Keystore phone key. Pairing checks the host fingerprint
before accepting a password and verifies key login before saving the pairing.
The shared headset server runs through a session-bound loopback tunnel.
All retained libraries receive Gallery opt-out, including previously paired
headsets. Paging keeps older offline originals reachable beyond the first 100.
A failed image or Gallery export retries independently without starving later
downloads; local persistence errors still halt immediately.
The user starts a connected-device foreground service with an ongoing Stop
notification. New screenshot originals download automatically. Gallery is ON
by default; turning it off latches pending and newly discovered items to
app-only, including across relaunch and later re-enabling. Historical captures
are baselined instead of imported. Receipts, original files, and MediaStore
recovery support interrupted downloads/exports without duplicate Gallery rows.
See [usage and background limits](../android-phone.md).
## Build and automated checks
- JDK 17, Gradle 9.4.1, AGP 9.1.0; compile/target API 36, minimum API 29.
- `assembleDebug testDebugUnitTest assembleDebugAndroidTest lintDebug` exited
zero after review fixes: `/tmp/frame-108-android-build-final-r5.log`.
- 34 JVM sync-engine tests, zero failures/errors/skips. Tests cover baseline,
stability, identity, metadata limits, destination latching, retries, atomic
receipts, corrupt state, unsafe paths, late cancellation callbacks, per-item
failure isolation and fatal persistence errors.
- API 36 arm64 emulator MediaStore tests passed: original byte round-trip,
app-only across relaunch, interrupted pending row recovery and retry dedup.
The final rebuilt APK passed **five selected native checks, no skips**:
three MediaStore/multiple-library cases plus the actual headset fixture
Gallery hash (`/tmp/frame-108-android-final-device-r4.log`, four tests), then
app-only original/no-Gallery readback
(`/tmp/frame-108-android-final-apponly-device-r4.log`, one test).
- Existing server suite: 1,333 tests, 11 skips, exit zero,
`/tmp/frame-108-android-server-tests.log`. Website: 29 passed, zero skips,
`/tmp/frame-108-android-site-tests.log`.
- Lint has no errors. Warnings include dependency/target freshness, English UI
literals, backup-version compatibility, and the shared controls' JavaScript.
## Actual headset → Android emulator → Gallery
Target: task-owned headless API 36 emulator `emulator-5588`, with the actual
Steam Frame reached over SSH. Initial baseline contained nine historical
captures and imported none. An emulator-generated public test key was installed
temporarily through the existing trusted SSH alias; no password was exposed.
The unauthenticated native host-key probe matched the trusted headset key.
Synthetic fixture IDs use a separate test account/game namespace; this verifies
new-file delivery from the real headset, not the hardware capture shortcut.
| Check | Result |
| --- | --- |
| Gallery default ON | `999999999990/108/20261024100101_1.png` automatically downloaded and receipt became `saved` |
| Source / app / Gallery SHA-256 | `c9411d718c8a3477b7c7a6eed5ab827570605fce977bdeef5aaf11d29ba88316` |
| MediaStore dedup | Exactly one matching Gallery row; native test reads its original bytes |
| Shared headset controls | Actual authenticated WebView loaded the headset app library |
| Gallery OFF, app on Home | Second fixture automatically downloaded in 4.293 seconds (one synthetic trial) |
| App-only source / app SHA-256 | `6a1513b2dc3459b457f7dd86adcde7298d99dccc31c73904d7c6698703edb961` |
| App-only independent readback | One selected native test passed: correct app original and no matching Gallery row |
| Re-enable and restart | App-only receipt and absent Gallery row preserved; first image remains saved, no duplicate |
| Open/share original | Native chooser opened for the app-only original and was canceled; nothing sent |
| Final r5 APK, Gallery ON while Home | Third new fixture `999999999990/108/20261024101413_1.png` reached Gallery in 5.288 seconds (one synthetic trial); same source/app/Gallery hash as the first fixture, one matching row |
Background readback: `/tmp/frame-108-android-background-device-r1.log`.
Home was foreground with the app service active; the screen stayed on. This
is not a locked/Doze or long-duration physical-phone test.
The final r5 background Gallery readback passed separately (one selected native
test, no skip): `/tmp/frame-108-android-final-background-gallery-r5.log`.
Sanitized [machine readback](android-phone-screenshot-sync-2026-10-24/live-proof.json)
is committed; local full readback: `/tmp/frame-108-android-live-proof.json`. Fixture source
metadata: `/tmp/frame-108-android-fixtures.json`. The optional provisioning test
passed separately. Earlier full-class instrumentation runs included two
optional skipped hooks; those are not counted as four executed scenarios.
![Native saved receipt with Gallery enabled](android-phone-screenshot-sync-2026-10-24/gallery-on.png)
![Gallery re-enabled without backfilling app-only downloads](android-phone-screenshot-sync-2026-10-24/reenabled.png)
![Native connection and Gallery default](android-phone-screenshot-sync-2026-10-24/setup.png)
![Shared controls loaded from the real headset](android-phone-screenshot-sync-2026-10-24/controls.png)
## Verification boundaries
No physical Android phone, Doze reliability, hardware screenshot latency,
password-pairing UI, Play signing identity or production upload has been
verified. Android sync uses an explicit foreground session, not Firebase.
The iOS foreground implementation has separate
[evidence](phone-screenshot-sync-2026-10-24.md); APNs, its delivery service,
notification extension and locked-phone Photos export remain unimplemented or
unverified. The user's full-feature tested-and-working release condition is
not satisfied.
T3's device tools incorrectly inferred the Android SDK from the platform-tools
Cask and could not open this emulator. The Astra worker used the app-owned
agent-device launcher with a task-local session/config and the correct SDK.
No global SDK/T3 settings or physical devices were changed.
UI verification: provider `codex`, model `gpt-6-astra`, completed task
`node:delegated-task:command%3Amcp%3Ad5d74833-0742-471e-af70-a23afbe44dab%3Adelegate-task%3Aframe-108-android-ui-astra-20261024-r1`.
Report: `/tmp/frame-108-android-ui-report.md`.
## Review and final artifacts
The APK is `android/app/build/outputs/apk/debug/app-debug.apk`, signed with the
development identity. No store upload or release has been performed.
Final r6 APK SHA-256: `e6a99a35154c21293975ea165eb967d8cc7ad84565f063255c3848ada1169f6e`.
Preferred different-model review (`claudeAgent` / `claude-opus-5-5`) failed
before work with `Claude API rate limit reached. Try again later.` Task:
`node:delegated-task:command%3Amcp%3Ad5d74833-0742-471e-af70-a23afbe44dab%3Adelegate-task%3Aframe-108-android-review-opus-20261024-r1`.
Implementation and independent fresh fallback reviews used Sol; they are not
different-model evidence.
Provider `codex_secondary`, model `gpt-6.1-sol`, high, completed round 1:
`node:delegated-task:command%3Amcp%3Ad5d74833-0742-471e-af70-a23afbe44dab%3Adelegate-task%3Aframe-108-android-review-sol-20261024-r1`.
All four findings were accepted and fixed: failed-item starvation, API-29
pending-row recovery query, opt-out for other retained headset libraries, and
inaccessible originals beyond the latest 100. Report:
`/tmp/frame-108-android-review-sol-r1.md`. No CODING_STANDARDS.md was found in
the task or primary worktree; CONTRIBUTING.md was included in review context.
Provider `codex_secondary`, model `gpt-6.1-sol`, high, completed round 2:
`node:delegated-task:command%3Amcp%3Ad5d74833-0742-471e-af70-a23afbe44dab%3Adelegate-task%3Aframe-108-android-review-sol-20261024-r2`.
It cleared all four prior fixes and independently reran 34 JVM tests, then found
a library-switch cache identity issue: matching receipt fields could retain a
Share action bound to the previous headset. The refresh signature now includes
the absolute original path, including the headset library root.
The final r6 build (`assembleDebug assembleDebugAndroidTest lintDebug`) exited
zero: `/tmp/frame-108-android-build-r6.log`. A new native regression launches
the real activity, switches between two libraries with matching receipt fields
but different original bytes, and verifies that Share now selects B’s bytes.
The chooser is intercepted; nothing is sent externally. This regression and
the three existing MediaStore/multiple-library cases passed together: **four
selected tests, no skips**, `/tmp/frame-108-android-final-device-r6.log`. Core
code is unchanged from the 34 independently passing JVM tests. The r6 change
does not alter the r5 transfer/storage path exercised against the real headset.
Provider `codex_secondary`, model `gpt-6.1-sol`, high, completed round 3:
`node:delegated-task:command%3Amcp%3Ad5d74833-0742-471e-af70-a23afbe44dab%3Adelegate-task%3Aframe-108-android-review-sol-20261024-r3`.
**No actionable findings** in the frozen 25-file executable target (`android/`
and `.github/workflows/android.yml`), with the remaining share-binding finding
resolved. All executable fingerprints stayed unchanged during review. Report:
`/tmp/frame-108-android-review-sol-r3.md`. The review inspected the passing r6
native/build logs and unchanged core covered by round 2’s independent 34-test
run; it did not claim new device runs or release validation.
Cleanup removed the one exact temporary public key and all three exact synthetic
headset fixture paths, verified absent. Other authorized keys and real captures
were preserved. Readback: `/tmp/frame-108-android-cleanup.json`. The task-owned
`emulator-5588` was shut down after the live tests, then restarted only for
the r6 share-binding regression and shut down again;
no other device was stopped. Build artifacts, the AVD and local test logs remain
available for follow-up verification.
@@ -0,0 +1 @@
{"removedTestKeys": 1, "removedFixtures": ["999999999990/108/20261024100101_1.png", "999999999990/108/20261024100102_1.png", "999999999990/108/20261024101413_1.png"], "testKeyAbsent": true, "allFixturesAbsent": true}
Binary file not shown.

After

Width:  |  Height:  |  Size: 391 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 185 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 177 KiB

@@ -0,0 +1,57 @@
{
"gallery": {
"id": "999999999990/108/20261024100101_1.png",
"sha256": "c9411d718c8a3477b7c7a6eed5ab827570605fce977bdeef5aaf11d29ba88316",
"bytes": 3620,
"createdAt": 1792836168.6100879,
"receipt": {
"destination": "saved",
"observations": 2,
"shot": {
"id": "999999999990/108/20261024100101_1.png",
"size": 3620,
"time": 1792836168.6217859
},
"state": "downloaded"
},
"readbackAt": 1792836209.819923
},
"appOnly": {
"id": "999999999990/108/20261024100102_1.png",
"sha256": "6a1513b2dc3459b457f7dd86adcde7298d99dccc31c73904d7c6698703edb961",
"bytes": 3623,
"createdAt": 1792836368.616906,
"background": "Home foreground; connectedDevice service active",
"receipt": {
"destination": "appOnly",
"observations": 2,
"shot": {
"id": "999999999990/108/20261024100102_1.png",
"size": 3623,
"time": 1792836368.6332257
},
"state": "downloaded"
},
"readbackAt": 1792836372.909552,
"elapsedSeconds": 4.293
},
"finalBackgroundGallery": {
"id": "999999999990/108/20261024101413_1.png",
"sha256": "c9411d718c8a3477b7c7a6eed5ab827570605fce977bdeef5aaf11d29ba88316",
"bytes": 3620,
"createdAt": 1792836871.5926661,
"background": "Home foreground; connectedDevice service active; final r5 APK",
"receipt": {
"destination": "saved",
"observations": 2,
"shot": {
"id": "999999999990/108/20261024101413_1.png",
"size": 3620,
"time": 1792836871.6072419
},
"state": "downloaded"
},
"readbackAt": 1792836876.8808918,
"elapsedSeconds": 5.288
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 238 KiB

@@ -1,38 +0,0 @@
{
"date_utc": "2026-10-24T07:54:37Z",
"frame_control_tag": "v0.4.16",
"steamos_version": "0.5.6",
"steamos_build": "20261015.6141207",
"host": "lxso2",
"interface": "source UI served by a temporary private Frame Control server, opened in T3 Code's desktop browser",
"action": "Clicked the visible Export button for Steam's retained real saved clip in Capture & Live",
"clip_id": "10230640/clips/clip_13009719723623972864_20261024_072115",
"api_observed": {
"post_clips_export_http": 200,
"get_job_http": 200,
"get_clips_http": 200
},
"ui_result": "Saved clip_13009719723623972864_20261024_072115.mp4 to ~/Videos/SteamFrame; button changed to Export again",
"output": {
"path": "~/Videos/SteamFrame/clip_13009719723623972864_20261024_072115.mp4",
"bytes": 159829,
"sha256": "f8a476eefb2bde7ba5cb464632ec9b952eea3ab1610b5bed2098bf1f11d62cf1",
"matches_previously_fully_decoded_export": true,
"duration_seconds": 10.288532,
"video": "H.264 1920x1080",
"audio": "AAC stereo"
},
"cleanup": {
"temporary_server_stopped": true,
"shared_device_lock_released": true,
"battery_percent_after": 99,
"desktop_mode_active_after": false,
"test_title_created": false,
"steam_source_clip_retained": true
},
"limits": [
"The test used the source UI, not a packaged desktop installer.",
"The retained source clip was nearly black; visible gameplay capture remains unverified.",
"Perceptual audio/video sync, long clips, trims and automatic copying were not tested."
]
}
-29
View File
@@ -1,29 +0,0 @@
{
"date_utc": "2026-10-24",
"version": "v0.4.17",
"steam_os_version": "0.5.6",
"steam_os_build": "20261015.6141207",
"result": "PASS: the real Steam-saved clip exported beyond 60 seconds",
"method": "Steam manual recording of a reserved test title, then Frame Control's production frame_clips.export_clip path; no packaged GUI button click",
"source": {
"kind": "saved Steam clip",
"duration_seconds": 100.801,
"chunks": 68,
"gaps": 0
},
"export": {
"format": "MP4",
"bytes": 1377849,
"sha256": "82ef31f65e6933d122bc326d664beb2eb2e3ba6969f26c0032552e80fbe8d823",
"duration_seconds_ffprobe": 100.833335,
"video": "H.264, 1920x1080, 6049 frames",
"audio": "AAC, 48000 Hz, stereo",
"full_ffmpeg_decode": "passed without errors"
},
"cleanup": "Restored Steam recording mode to its prior off setting; stopped and removed only the reserved title and its processes; no recording or running Steam apps remained; Game Mode active, battery 100%, shared test lock released",
"limits": [
"This exercised the production export code path, not a packaged GUI Export click.",
"Visible gameplay quality, perceptual audio/video sync, trims and longer recordings were not checked.",
"The newly saved Steam source clip was retained; no user recording was deleted."
]
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

@@ -1,94 +0,0 @@
{
"date_utc": "2026-10-24T09:08:35.523Z",
"result": "PASS: packaged Electron visible Export click saved the retained real Steam clip beyond 60 seconds",
"frame_control_tag": "v0.4.17",
"documentation_base_commit": "4e456480aefbe541e334dbd247328537f535bd94",
"host": "lxso2",
"steamos_version": "0.5.6",
"steamos_build": "20261015.6141207",
"package": {
"asset": "Frame-Control-linux-x86_64.AppImage",
"url": "https://github.com/saphid/frame-control/releases/download/v0.4.17/Frame-Control-linux-x86_64.AppImage",
"sha256": "b1b8eb82bf73a227b2cd36e8f0ef3a2ebdf7967c5853efb08a9634f38227015d",
"launch": "Verified release asset; --appimage-extract, then unmodified squashfs-root/AppRun. Electron used the bundled server and Python, not checkout source.",
"display": "Xvfb :137, 1440x1000x24; Electron window 1400x950 including native chrome",
"flags": [
"--no-sandbox",
"--disable-gpu",
"--remote-debugging-port=49371",
"--remote-debugging-address=127.0.0.1",
"--user-data-dir=/tmp/frame-packaged-export-20261024/electron-profile"
],
"isolation": "Throwaway XDG_CONFIG_HOME, FRAME_CONTROL_DATA_DIR and Electron profile; telemetry and update checks disabled; existing SSH identity and normal home Videos destination used."
},
"ui": {
"navigation": "Dismissed optional email prompt with No thanks; clicked Tools, then Capture & Live; scrolled Steam clips into view.",
"row": "Steam recording \u00b7 Clip; Oct 24, 8:53 AM \u00b7 1:40 \u00b7 2 MB",
"action": "Playwright connected to this Electron instance over local CDP; locator.click() clicked its visible button named exactly Export. The test did not directly invoke an export function or API POST.",
"button": "button.small[data-clip-export=\"0\"]",
"click_is_trusted": true,
"button_bounds_css_pixels": {
"x": 1300.390625,
"y": 701.15625,
"width": 61.609375,
"height": 28
},
"post_clips_export_http": 200,
"job": "0ab20b1788adce63",
"job_poll_http": 200,
"saved_message": "Saved clip_15658413054201167872_20261024_085343.mp4 to ~/Videos/SteamFrame",
"final_row": "ON PC; Export again",
"screenshots": [
"clip-packaged-export-2026-10-24-before.png",
"clip-packaged-export-2026-10-24-after.png"
],
"screenshot_scope": "Cropped to Steam clip rows and saved notification; unrelated screenshot thumbnails and host connection identity excluded."
},
"source": {
"clip_id": "10230640/clips/clip_15658413054201167872_20261024_085343",
"duration_seconds": 100.801,
"duration_provenance": "Retained real Steam clip from clip-long-2026-10-24.json; current packaged UI displays 1:40."
},
"output": {
"path": "~/Videos/SteamFrame/clip_15658413054201167872_20261024_085343.mp4",
"bytes": 1377849,
"sha256": "82ef31f65e6933d122bc326d664beb2eb2e3ba6969f26c0032552e80fbe8d823",
"matches_prior_production_export_size_and_sha256": true,
"duration_seconds_ffprobe": 100.833335,
"duration_difference_from_steam_seconds": 0.032335,
"video": "H.264, 1920x1080, 6049 decoded frames",
"audio": "AAC, 48000 Hz, stereo, 4724 decoded audio frames",
"validation": [
"ffprobe -v error -count_frames -show_entries format=duration,size:stream=codec_name,codec_type,width,height,sample_rate,channels,nb_read_frames -of json <output>: exit 0",
"sha256sum <output>: exact match to prior production export",
"ffmpeg -v error -xerror -i <output> -f null -: exit 0, no errors"
]
},
"cleanup": {
"shared_lock": "Acquired with ssh frame mkdir /tmp/frame-test.lock after read-only battery/build/lock checks; released with rmdir after app shutdown; absence read back.",
"battery_percent_before": 99,
"battery_percent_after": 99,
"battery_status": "Not charging; charger Full",
"packaged_app_and_server_exited": true,
"test_xvfb_stopped": true,
"frame_export_cache_directories_remaining": 0,
"ffmpeg_processes_after": 0,
"game_mode_and_steamvr_running_after": true,
"steam_source_clip_retained": true,
"user_recordings_deleted": false,
"headset_install_or_settings_changes": false,
"local_export_retained": true
},
"limits": [
"The unchanged AppImage payload ran after extraction on a virtual X desktop, with Chromium sandbox and GPU disabled. FUSE mounting, desktop integration and hardware rendering were not tested.",
"The click was automated trusted pointer input in a visible Electron window on Xvfb, not a human click on a physical monitor. T3 collaborative preview opened successfully but drives separate browser tabs; Electron-native automation was used for this window.",
"No new game or Steam recording was started. Visible gameplay quality, audible content and perceptual audio/video sync were not assessed.",
"Clips longer than this retained 100-second clip, trims and automatic copying remain unverified.",
"No release, tag or application code was changed."
],
"validation_local": {
"clip_regression_suite": "python3 -m unittest discover -s tests -p test_clips.py -v: 34 tests passed in 20.173s, including real ffmpeg long-recording export",
"json": "python3 -m json.tool: passed",
"diff": "git diff --check: passed"
}
}
@@ -1,95 +0,0 @@
{
"date": "2026-10-24",
"command": "scripts/frame-power-sample.sh --count 2 --interval 1",
"exitCode": 0,
"stopped": null,
"lockPolicy": "Existing sampler checked local and remote /tmp/frame-test.lock before reading; no lock acquired, bypassed or removed. Read-only run only.",
"samples": [
{
"time": 1792828506.4317346,
"osBuild": "20261015.6141207",
"elapsedSeconds": 0.057539,
"battery": {
"battery.percent": {
"value": 99,
"unit": "%",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.status": {
"value": "Not charging",
"unit": "state",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.watts": {
"value": -0.06,
"unit": "W",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.tempC": {
"value": 24.0,
"unit": "C",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
}
}
},
{
"time": 1792828507.4357972,
"osBuild": "20261015.6141207",
"elapsedSeconds": 1.061612,
"battery": {
"battery.percent": {
"value": 99,
"unit": "%",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.status": {
"value": "Not charging",
"unit": "state",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.watts": {
"value": -0.05,
"unit": "W",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
},
"battery.tempC": {
"value": 24.0,
"unit": "C",
"source": "/sys/class/power_supply/max1720x_bat_7-36",
"state": "available"
}
}
}
],
"limitations": [
"No real low-charge or thermal condition induced.",
"No native notification delivery verified.",
"No worn-headset or native package tests.",
"Read-only parser/transport check; no background monitor or power-saving claim."
],
"browserCheck": {
"page": "Production ui/index.html served statically; API calls intentionally unavailable",
"viewport": {
"width": 1280,
"height": 800
},
"bridge": "Synthetic desktop notification bridge; no native OS notification sent",
"result": "Initially unchecked and visible only after desktop host/bridge setup. Checkbox enabled through browser. Synthetic 12%, Discharging, -4 W samples at 0/30/60 seconds produced one bridge request, an in-page status and a toast. No horizontal overflow.",
"apiLimit": "Static server returned expected 404/501 API errors; not an end-to-end live desktop connection test."
},
"offlineValidation": [
"test_desktop_battery_comfort.py: 2 tests, production JS with monotonic fake clock and notification bridge",
"test_comfort*.py: 25 tests",
"test_battery.py: 7 tests",
"test_power*.py: 36 tests (loopback HTTP test run outside socket-restricted sandbox)",
"test_ui_workspace.py: 1 Python wrapper / Node workspace suite",
"test_ui_review.py: 1 Python wrapper / 9 Node regression tests, including switch reset and retained cooldown"
]
}
@@ -1,206 +0,0 @@
{
"date_utc": "2026-10-24",
"base_commit": "de931bc",
"issue": 73,
"result": "Packaged Desktop Capture passed for a mapped nested KDE compositor on the real Frame; wearer-visible overlay condition was not established. Do not close #73 from this test.",
"package": {
"tag": "v0.4.17",
"asset": "Frame-Control-linux-x86_64.AppImage",
"sha256": "b1b8eb82bf73a227b2cd36e8f0ef3a2ebdf7967c5853efb08a9634f38227015d",
"release_digest_matched": true,
"panel_helper_matches_base": true,
"unmodified_payload": true,
"host": "lxso2",
"display": "Xvfb :139",
"flags": [
"--no-sandbox",
"--disable-gpu"
],
"private_profile_and_data": true,
"telemetry_and_update_checks_disabled": true
},
"preflight": {
"steamos_version": "0.5.6",
"build_id": "20261015.6141207",
"battery_percent": 99,
"battery_status": "Not charging",
"game_mode": true,
"full_plasma_session": false,
"nested_desktop_running": false,
"nested_runtime_present": false,
"lock_present": false,
"steam_pid": 297826,
"vrserver_pid": 297506,
"vrcompositor_pid": 297620,
"gamescope_service_invocation": "e8c1085b3bbd4e79a323b3e4a9bc6355",
"focusable_windows": [],
"focused_windows": []
},
"session": {
"started_utc": "2026-10-24T09:43:48Z",
"lock_acquired": true,
"launch": [
"systemd-run",
"--user",
"--unit=fc-issue73-desktop-20261024",
"--collect",
"--property=RuntimeMaxSec=180s",
"--property=TimeoutStopSec=15s",
"--property=KillMode=control-group",
"--setenv=DISPLAY=:0",
"--setenv=XDG_RUNTIME_DIR=/run/user/1000",
"--setenv=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus",
"--setenv=XDG_SESSION_TYPE=x11",
"--setenv=XDG_CURRENT_DESKTOP=gamescope",
"/usr/bin/steamos-nested-desktop"
],
"response": "Running as unit: fc-issue73-desktop-20261024.service; invocation ID: 92cff1e57c794212997d46c0a68877ec\n"
},
"discovery": {
"focus": 62914560,
"focus_display": ":0",
"panels": [
{
"window": 62914560,
"display": ":0",
"name": "KDE Wayland Compositor - Press right control key to grab input",
"x": 0,
"y": 0,
"width": 1920,
"height": 1080,
"app": 0,
"focused": true
}
]
},
"window_evidence": {
"xwininfo_map_state": "IsViewable",
"width": 1920,
"height": 1080,
"focused_after_show_request": true,
"steamvr_overlay_key": "valve.steam.desktopgame.0",
"steamvr_overlay_visibility_after_show_request": "not_visible"
},
"capture": {
"repeat_utc": "2026-10-24T09:44:47.323Z",
"route": "/api/screenshot?view=panel&window=62914560&display=%3A0",
"http": 200,
"content_type": "image/png",
"content_length": 1209980,
"source_header": "panel",
"decoded": {
"width": 1920,
"height": 1080,
"source": "panel",
"selected": {
"window": 62914560,
"display": ":0",
"name": "KDE Wayland Compositor - Press right control key to grab input",
"x": 0,
"y": 0,
"width": 1920,
"height": 1080,
"app": 0,
"focused": true
},
"stamp": "9:44:48 AM",
"badge": "Desktop \u00b7 KDE Wayland Compositor - Press right control key to grab input"
},
"canvas_png_bytes": 2011278,
"first_capture_pixel_statistics": {
"width": 1920,
"height": 1080,
"min": 2.3333333333333335,
"max": 255,
"hidden": false
},
"first_capture_click_event": {
"isTrusted": true,
"utc": "2026-10-24T09:44:12.738Z"
},
"actions": "Two Playwright locator.click() actions on the visible #shotBtn in the actual packaged Electron window; first click recorded isTrusted=true. No direct screenshot API invocation by harness."
},
"device_cleanup": {
"utc": "2026-10-24T09:45:33Z",
"before_stop": {
"exit": 0,
"out": "MainPID=300739\nControlGroup=/user.slice/user-1000.slice/user@1000.service/app.slice/fc-issue73-desktop-20261024.service\nActiveState=active\n",
"err": ""
},
"hide_dashboard": {
"exit": 0,
"out": "HideDashboard",
"err": ""
},
"stop_owned_unit": {
"exit": 0,
"out": "",
"err": ""
},
"after_stop": {
"exit": 0,
"out": "MainPID=0\nLoadState=not-found\nActiveState=inactive\n",
"err": ""
},
"desktop_processes": {
"exit": 1,
"out": "",
"err": ""
},
"nested_runtime_exists": false,
"session": {
"exit": 0,
"out": "Id=gamescope-session.target\nActiveState=active\nInvocationID=a2fee8ebf55c42c7b546390d14cd985c\n\nId=plasma-workspace-wayland.target\nActiveState=inactive\nInvocationID=\n\nId=gamescope-session.service\nActiveState=active\nInvocationID=e8c1085b3bbd4e79a323b3e4a9bc6355\n",
"err": ""
},
"steam": {
"exit": 0,
"out": "297826 /home/steamos/.local/share/Steam/steamrtarm64/steam -cef-enable-debugging -deckard -gamepadui -steamdeck -steamos3 -vrgamepadui\n",
"err": ""
},
"steamvr": {
"exit": 0,
"out": "297506 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-297420\n297620 /opt/steamvr/bin/linuxarm64/vrcompositor\n",
"err": ""
},
"xprops": {
"exit": 0,
"out": "GAMESCOPE_FOCUSABLE_WINDOWS(CARDINAL) = \nGAMESCOPE_FOCUSED_WINDOW(CARDINAL) = \n",
"err": ""
},
"battery": {
"capacity": "99",
"status": "Not charging"
},
"owned_lock_released": true
},
"host_cleanup": {
"electron": {
"pid": 1578431,
"exists_after": false
},
"xvfb": {
"pid": 1578428,
"exists_after": false
},
"x_socket_absent": true
},
"delayed_final_readback": "2026-10-24T09:46:39Z\nshared-lock-absent\nLoadState=not-found\nActiveState=inactive\nId=gamescope-session.target\nActiveState=active\n\nId=plasma-workspace-wayland.target\nActiveState=inactive\nInvocationID=e8c1085b3bbd4e79a323b3e4a9bc6355\n297826 /home/steamos/.local/share/Steam/steamrtarm64/steam -cef-enable-debugging -deckard -gamepadui -steamdeck -steamos3 -vrgamepadui\n297506 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-297420\n297620 /opt/steamvr/bin/linuxarm64/vrcompositor\nnested-runtime-absent\n100\nFull\nGAMESCOPE_FOCUSABLE_WINDOWS(CARDINAL) = \nGAMESCOPE_FOCUSED_WINDOW(CARDINAL) = \n",
"validation": {
"focused_panel_suite": "42 tests passed in 14.805s (test_touch.py), including production-page Node behavior tests",
"website": "2 tests passed (node --test site/test/*.test.mjs)",
"node_panels": "Passed independently (node tests/page/desk_panels.mjs)",
"metadata_and_links": "JSON parsed; result/cleanup assertions and relative evidence/streaming/testing links passed",
"diff": "git diff --check passed"
},
"limits": [
"SteamVR continued reporting not_visible after ShowDashboard, despite mapped X window and confirmed focus. No worn-headset visual assessment; does not establish the reporter's visible-overlay condition.",
"SteamOS 0.5.6 only; reporter used 0.4.5 with unknown Frame Control version and no reproduction steps.",
"Nested desktop launched with SteamOS steamos-nested-desktop from a bounded transient user unit using Steam display/session variables; not launched through a wearer's Steam UI.",
"Extracted unchanged AppImage on a virtual X display with Chromium sandbox and GPU disabled; no FUSE, physical monitor or hardware-rendering test.",
"No Live video, Control input or full Desktop Mode test.",
"Nested-desktop journal included gamescope WSI/Vulkan-hook warnings. No warning was dismissed; captured compositor pixels were nonblank.",
"Initial CDP response.body() yielded zero bytes despite successful image decode. Repeat verified Content-Length, fresh decoded image, panel selection and PNG export from canvas; the initial raw-body size is not treated as capture failure or proof.",
"No private desktop pixels, issue screenshot or raw desktop journal committed."
]
}
@@ -1,100 +0,0 @@
# Packaged Desktop Capture, 2026-10-24
Base: `de931bc`. Package: Linux x86_64 **v0.4.17**. Device: real Frame,
SteamOS **0.5.6**, BUILD_ID **20261015.6141207**. Host: lxso2.
[Sanitized state and response evidence](desktop-capture-packaged-2026-10-24.json).
## Result and issue boundary
The packaged app discovered the nested KDE compositor and both automated
Desktop **Capture** button clicks succeeded. The repeat returned HTTP 200,
`Content-Type: image/png`, `X-Capture-Source: panel`, and Content-Length
1,209,980. The app decoded a fresh, nonblank 1920×1080 image and displayed
the Desktop panel badge. No further product failure was reproduced or fixed.
This does **not** establish the full visible-desktop condition in
[#73](https://github.com/saphid/frame-control/issues/73). `xwininfo` reported
the desktop window `IsViewable`, and gamescope confirmed focus after
`vrcmd --showdashboard valve.steam.desktopgame.0`. However, SteamVR still
reported that overlay `not_visible`. There was no worn-headset visual check.
The observation proves capture of a mapped nested compositor, not its visible
presentation to a wearer. Do not close #73 from this evidence.
The issue screenshot shows a desktop panel in the headset view while Desktop
Capture logs “no app panel is open in the headset.” The reporter used SteamOS
0.4.5, gave no Frame Control version or reproduction steps, and has not replied
to the maintainer's 0.4.3 update request. The duplicate page function was already
fixed; [PR #152](https://github.com/saphid/frame-control/pull/152) subsequently
improved unknown-state diagnostics. Its [earlier investigation](panel-discovery-2026-10-24.md)
was blocked from starting a desktop by another task's lock. This run adds a
current-package capture observation, without reproducing the older failure.
## Procedure
1. Read-only preflight found Game Mode active, full Plasma inactive, Steam and
SteamVR running, battery 99%, no nested desktop/runtime and no test lock.
2. Compared the retained AppImage's SHA-256 with the live v0.4.17 release asset
digest: `b1b8eb82bf73a227b2cd36e8f0ef3a2ebdf7967c5853efb08a9634f38227015d`.
Extracted an independent copy; `cmp` confirmed its `frame_touch.py` matches
the base checkout. No payload was edited.
3. Started the package's own AppRun, bundled server and Python on Xvfb `:139`
with private config/cache/data/profile, telemetry and update checks disabled,
and `--no-sandbox --disable-gpu`. Used the existing authorized `ssh frame`
identity. T3 preview opened the package server page but does not attach to
the existing Electron window. The literal Electron button test used the
same Playwright-over-loopback-CDP workflow as the earlier packaged tests.
4. Acquired `/tmp/frame-test.lock` with atomic `mkdir` and wrote an owner marker.
Rechecked no nested runtime or Plasma was present. Started SteamOS's
`/usr/bin/steamos-nested-desktop` in the owned transient user unit
`fc-issue73-desktop-20261024.service`, using Steam's display/session variables.
`RuntimeMaxSec=180s`, `TimeoutStopSec=15s` and `KillMode=control-group` bounded
the session. This launches a desktop panel inside Game Mode; no full Desktop
Mode switch was sent.
5. In the actual packaged Electron window, navigated Tools → Capture & Live →
Desktop and clicked Capture with `locator.click()`. The first click event
recorded `isTrusted=true`. The page selected the KDE compositor on `:0`;
the production screenshot path used window `62914560`. Requested the desktop
dashboard, read back focus/visibility, and clicked Capture again. Recorded
response metadata, fresh decoded image dimensions and canvas pixel range.
No direct screenshot API call was substituted for these button clicks.
6. Ran the packaged `frame_touch.py panels` read-only over SSH to retain its
exact discovery result: one focused KDE compositor panel, app ID 0, 1920×1080,
display `:0`, with no diagnostic `why` field.
## Cleanup and limits
The owned desktop session ran **09:43:48–09:45:33 UTC** (105 seconds). Hid the
dashboard to restore its initial hidden state, stopped only the owned unit,
and verified it was unloaded/inactive with MainPID 0. No Plasma/KWin desktop
process or nested runtime remained; gamescope's focusable/focused lists were
empty again. Released only this task's lock, after checking its owner marker.
Delayed readback at **09:46:39 UTC** confirmed the lock absent, Game Mode active,
full Plasma inactive and the original Steam, vrserver and vrcompositor PIDs
still running. The Game Mode service invocation was unchanged. Battery was
99% at cleanup and 100%/Full on the delayed check. Closed both owned preview
tabs, Electron and Xvfb; their processes and X socket were gone. No test server
remained. No reboot, installation, security/global setting change or full
Desktop Mode transition was performed.
The nested desktop journal included gamescope WSI/Vulkan-hook warnings. No
warning was dismissed. Capture produced nonblank compositor pixels, but this
run does not assess their usability to a wearer. Private image files stayed
outside the repository; only state and response metadata are published.
The initial CDP `response.body()` read yielded zero bytes despite successful
decoding in the page. The repeat verified the PNG Content-Length, fresh image
dimensions/source/selection and a 2,011,278-byte PNG exported from the canvas.
The initial raw-body measurement is a harness limitation, not evidence of a
product capture failure. No Live video, Control input, physical monitor, FUSE
mounting or hardware rendering was tested. Older SteamOS and other packages
remain outside this observation.
## Offline verification
`python3 -m unittest discover -s tests -p test_touch.py -v`: **42 tests passed**,
including the production page's Node panel behavior tests, duplicate-function
guard, absent versus empty panel diagnostics, alternate displays, capture/input
target validation and discovery deadlines. JSON parsing and `git diff --check`
were also checked. No new regression test was added because this is an
evidence-only change with no reproduced product defect.
@@ -1,114 +0,0 @@
# Linux PC host feasibility on lxso2, 2026-10-24
**Blocked on this host's current desktop environment. No feature enabled and
no feature PR opened.** Investigation for [issue #23](https://github.com/saphid/frame-control/issues/23),
starting at `4e45648` (main after #154). This concerns flat desktop panels,
not Linux PC VR streaming.
## Decision
The smallest useful candidate is one explicitly selected **whole screen**,
captured through the ScreenCast portal's restricted PipeWire connection,
encoded locally with GStreamer/x264, and sent to the existing viewer through
the shared SSH controller. Input, individual windows and virtual displays
would remain unsupported until separately verified. This candidate was **not
implemented**: lxso2 currently cannot exercise the real capture permission path.
Development packages could unblock compilation, but would not create a desktop,
portal backend or consented screen. Synthetic encoding, an Xvfb screen grab or
using the Frame as its own host would not verify Linux PC desktop capture.
No host packages or services were changed.
## Evidence
Host: `lxso2`, Linux x86-64, kernel `6.8.0-139-generic`.
- `DISPLAY`, `WAYLAND_DISPLAY` and `XDG_SESSION_TYPE` were unset. The user bus
was reachable at `/run/user/1000/bus`.
- `/run/user/1000/pipewire-0` and `/run/user/1000/wayland-0` did not exist.
No processes matched Xorg, Xwayland, Xvfb, GNOME Shell, KWin Wayland, Weston,
PipeWire, WirePlumber or xdg-desktop-portal at the process check.
- D-Bus `ListNames` returned D-Bus, systemd and anonymous connections, with no
`org.freedesktop.portal.Desktop`. `ListActivatableNames` also had no desktop
portal. These calls queried D-Bus itself without activating capture services.
- `dpkg-query` found no `xdg-desktop-portal*` or `pipewire*` packages.
`libgstreamer1.0-0:amd64` was installed at `1.24.2-1ubuntu0.1`, but
`gstreamer1.0-tools` was uninstalled. `gst-launch-1.0` and `gst-inspect-1.0`
were absent from PATH. Xvfb was installed, but supplies no portal by itself.
- `pkg-config --modversion` failed for `gstreamer-1.0`,
`gstreamer-app-1.0`, `gstreamer-video-1.0` and `libpipewire-0.3`.
`gio-unix-2.0` reported `2.80.0`.
- `desktop/bundle/pc-host.so` did not exist.
`python3 desktop/build.py --with-x264 --out /tmp/frame-linux-pc-feasibility`
exited **1** before compilation: required GStreamer pkg-config modules were
missing. No native build or encoder runtime success is claimed.
Reproduce the non-activating portal checks in the intended desktop user's
session, using that session's actual runtime directory and bus:
```sh
gdbus call --session --dest org.freedesktop.DBus \
--object-path /org/freedesktop/DBus --method org.freedesktop.DBus.ListNames
gdbus call --session --dest org.freedesktop.DBus \
--object-path /org/freedesktop/DBus --method org.freedesktop.DBus.ListActivatableNames
pkg-config --modversion gstreamer-1.0 gstreamer-app-1.0 gstreamer-video-1.0 gio-unix-2.0
```
## Existing implementation reviewed
- `ui/frame_pcview.py` selects PCView only on Windows and rejects Linux even
when enabled. Linux installers do not carry its native bundle.
- `desktop/portal.c` creates a **RemoteDesktop** session, requests pointer and
keyboard devices, and selects monitors and windows (`types=3`). It is not
ScreenCast-only or screen-only. Its restricted fd/node and fresh-connection
handling are reusable.
- `ui/frame_pc_agent.py` calls the selection a shared window or screen and
picks VA-API when the element is present, falling back to x264. A bounded
x264 slice needs explicit encoder policy and unsupported-input behavior;
exposing the existing Linux path alone would not meet that scope.
- `app/main.js` starts Python with the inherited environment; it does not
supply a compositor, portal backend or capture permission.
- `.github/workflows/pc-host.yml` installs native dependencies and tests
synthetic x264/protocol behavior with fake capture consent/input. This does
not establish interactive Linux desktop capture.
- Issue #23 and its six existing comments distinguish prior synthetic
helper/viewer results from untested Linux desktop capture and consent.
## Verification and limits
`python3 -m unittest discover -s tests -p 'test_pc*.py' -v` exited **0**:
27 tests ran successfully, with one additional native-class setup skip
(`native PC libraries not built on this host`). Coverage includes the compiled
C controller trace, Python adapters, grants and default-off/Windows-only gates.
It does not prove native GStreamer capture or local-client streaming.
No fake/local native stream, consent prompt, real desktop capture, Frame viewer
run, benchmark, input or window mode was tested in this investigation. No SSH
connection to the Frame was needed; no lock was acquired and no headset install,
launch, stop or settings change was attempted. Issue #113 was left alone.
No issue comments or other public messages were sent.
## Concrete unblock and next slice
1. Supply an interactive Linux desktop, physical or VM, with a working
ScreenCast portal/backend, PipeWire and accessible consent dialog. Installing
the portal frontend alone is insufficient. A VM can establish software
capture/x264 behavior while physical-host behavior remains unverified.
2. Install the native dependencies listed in `pc-host.yml` in that environment
and build a local `--with-x264` bundle. Resolve x264 distribution
licensing/source obligations before shipping it: the current build labels
that option test-only and the Windows installer omits it.
3. Implement a ScreenCast-only session with monitor-only selection, no
RemoteDesktop/device request, disabled input in protocol and viewer, hidden
window/virtual-display modes, and explicit x264 selection. Preserve the
default-off gate and existing Mac/Windows paths.
4. Verify consent acceptance/refusal/cancellation, Stop/reopen, connection loss
and portal closure with a local client decoding actual captured content.
Run the shared benchmark and record results without inferring hardware or
physical laser behavior.
5. Only after local capture works, acquire the shared Frame lock once and
follow [the headset procedure](../testing.md#headset-smoke-test) for a brief
viewer test. If occupied, leave the lock alone and defer device testing.
These are prerequisites and proposed scope, not a ready feature. The Linux
host request in #23 remains open.
-142
View File
@@ -1,142 +0,0 @@
# Media surround hardening: 2026-10-24
PR [#51](https://github.com/saphid/frame-control/pull/51), branch
`theatre-media-surround-hardening`, merged main `2ded4c5` without rebasing.
Both fixes remain necessary on that main: an optional-surround error escapes
`play()`, and a non-`RuntimeError` during overlay teardown skips the remaining
handles and OpenVR shutdown.
## Offline checks
- Focused media suite: **23 tests passed** on Linux / Python 3.12.3.
- Final full suite: **1,381 tests, OK (15 skipped)** in 145.384 s with
`PYTHONFAULTHANDLER=1 python3 -m unittest discover -s tests -v`.
The service-regression module passed **33 tests**. Player/server compilation
and `git diff --check` passed. Current macOS/Windows validation is left to CI.
- The startup/upload, creation/setup, deferred-surround refusal and teardown
regression tests were also run with main's player implementation: they
failed as expected (six errors across subtests). The screen-refusal test
passed, confirming that actual screen failures still propagate.
- The historical macOS failure was job
[109239733717](https://github.com/saphid/frame-control/actions/runs/36516462623/job/109239733717),
on `dac9dbc`, Python 3.12: 619 tests, three failures in artwork resolver
tests (`test_resolver_slot_released_when_thread_cannot_start`,
`test_timed_out_lookups_are_capped`,
`test_url_fetch_refuses_private_hosts_and_honours_deadline`). Media tests
passed. Late DNS workers retained semaphore slots; the first failed
assertion could also leak acquired slots into later tests.
- Main already contains `8051687`'s wait-for-idle and unconditional slot
cleanup, preserved by `d546568`. The current artwork module passed all
**40 tests** locally. No additional artwork changes were needed.
- The first full run completed **1,380 tests, 15 skipped**, with one failure:
`test_a_second_server_is_refused_until_the_first_exits` observed SIGSEGV
(`-11`) instead of exit 1. All 41 server tests passed on a focused rerun.
Main started daemon index/TLS workers before refusing a second server;
its existing shutdown notes document a race with OpenSSL interpreter exit.
This crash mechanism is inferred; the first run did not retain a native
fault trace. The subprocess assertion now retains stderr for diagnosis.
A deterministic regression now checks that rejection starts no listener,
cleanup sweep or background workers. It failed before moving the ownership
check ahead of startup work and passed afterwards.
The surround retry now logs a refusal before discarding that cosmetic work.
Creation, setup and first-upload failures also allow the screen to play;
standby remains retryable, and screen errors remain fatal. Teardown tests
exercise OpenVR errors, unexpected exceptions and a late `InterruptedError`,
checking that all handles are attempted and shutdown runs.
## Earlier blocked real Frame attempt
Read-only SSH preflight: SteamOS **0.5.6**, BUILD_ID **20261015.6141207**;
SteamVR server PID **297506**, compositor PID **297620**; existing media
service `inactive/dead`, MainPID 0; battery **99%**, fuel gauge `Not charging`,
charger `Full`. The shared lock was absent at preflight.
The subsequent atomic `mkdir /tmp/frame-test.lock` failed with
`FileExistsError(17, 'File exists')`: another session acquired it in between.
The probe stopped before uploading files, launching media, creating overlays,
or changing any state. The foreign lock was neither modified nor released.
All generated fixtures remained on the computer and were then removed; no
headset cleanup was necessary. Steam/SteamVR and the prior player state were
left alone.
## Follow-up real Frame smoke at the exact PR head
**Passed, 2026-10-24 10:35:02–10:35:23 UTC**, with byte-identical media modules
from **`5d551a81658d9ff14f49285c28609d506cbc20aa`**. This later run resolves
the earlier lack of fresh playback evidence. No production code changed.
The headset was unworn. SteamOS **0.5.6**, BUILD_ID **20261015.6141207**,
SteamVR **2.18.2**; battery **100%, Full** before and after.
Read-only preflight found Game Mode (`XDG_CURRENT_DESKTOP=gamescope`,
`start-gamescope` running, no `plasmashell`), Steam/SteamVR running, no media
overlays, and the installed media service inactive/not loaded. The test then
acquired `/tmp/frame-test.lock` with atomic `mkdir`; it did not take over a
foreign lock. Under the lock, those safety conditions were checked again.
Only generated media and the PR's four unchanged modules were uploaded, under
the test-owned, uid-1000 directory
`/tmp/pr51-5d551a8-4c0976a52564-utfbk_m0/`. The clip was a **12 s, 30 fps,
960×540 half-SBS H.264** MP4 with solid red/cyan eye images and **silent AAC**.
It was generated with FFmpeg's `color`, `hstack` and `anullsrc` sources; no
personal or external media was used.
The exact `frame_media_remote.run()` Play/status/Stop path was exercised by
importing its unmodified module and changing only its in-memory `ROOT`,
`RUNTIME`, `STATUS` and `UNIT` constants to the isolated test locations and
`frame-control-media-pr51-5d551a8-4c0976a52564.service`. The real
`frame_media_player.py` ran unchanged with `--layout auto --theatre`, using
`h264_v4l2m2m` and Pulse output. This checks the player and its systemd lifecycle;
it does **not** reverify the HTTP/UI upload or runtime installer, nor the
literal production unit name. The installed media runtime, log, status and
library were preserved.
- First Play: `starting` → `playing` → **`ended`**, **360 frames**, **160 dropped**,
**12.0397 s**, `running=false`. The drops were the normal unworn/standby
`SetOverlayRaw` retry path; no fault was forced. At 60 frames, dropped was 0.
`vrcmd --overlays` listed `framecontrol.media.surround` as **1×1 visible**
and `framecontrol.media.screen` as **visible**. These are SteamVR metadata,
not a wearer or eye-capture verification.
- After EOF: unit **inactive/dead, not-found, MainPID 0**, no test player or
decoder, and neither owned overlay remained. Stop after EOF succeeded and
retained `ended`.
- Second Play reached `playing`, 60 frames, 0 dropped. Explicit Stop returned
**`stopped`, 72 frames, 0 dropped, `running=false`**. Again the unit was
inactive/dead/not-found with MainPID 0, no player/decoder and no owned overlays.
- The test unit's `ExecStart` and cgroup identified the unique temporary
runtime. First-run player/decoder PIDs were **309650/309655**; second-run
PIDs were **309805/309810**. Stop controlled only this dedicated test unit.
Its player log was empty; no surround refusal or teardown error was logged.
### Uploaded SHA-256 values
```text
generated_SBS.mp4 1b39cac1d3b092b89657250c817019b68e87a474976ccd44aeb38e8eee44ffca
frame_media.py 001e3158f4c3931794846b21f1e4cb553349d9326b4db1454a01976d446ad97a
frame_media_player.py 86b0a140ebff857c2819c74d3ac45d1897735c585d44c7360ad6f18c2ecf2645
frame_media_remote.py b60386eb192d32594358246a4e665474c543600f2c564c2efd554ef38d7a631d
frame_splat.py f35750926f1149c41b95617a302012406d355c0dfdd4530438b0c42ee6d55daa
```
### Cleanup and limits
The temporary directory, including its fixture, code, bytecode, status and log,
was removed only after verifying that its player and decoder had exited. The
transient unit was collected. The test released only its own lock after cleanup.
All installed media runtime file hashes, including `player.log` and
`status.json`, were unchanged. The installed `frame-control-media.service`
remained inactive/not loaded. Steam **297826**, vrserver **297506**,
vrcompositor **297620**, vrdashboard **297721**, and start-gamescope **296997**
kept the same PIDs. The pre-existing overlay key/dimension/visibility/type
inventory matched the final inventory, and all dashboard main panels remained
`not_visible`. No dashboard command, wake request, global setting, reboot or
Steam/SteamVR stop was issued.
Cosmetic surround refusal and exceptional teardown remain **fake-tested**;
no real refusal or teardown fault was induced. This short unworn-headset
smoke does not verify appearance, eye isolation, comfort, audible sound/lip
sync (the AAC was silent), sustained display performance, or the issue #28
report of laggy 1080p video. It does not cover new H.265, photo or splat cases.
Historical checks remain in the earlier PR #51 description and
[media playback notes](../vr-video.md). The PR remains draft and unmerged
for parent review and CI.
-178
View File
@@ -1,178 +0,0 @@
# Keyboard and trackpad through gamescope, 2026-10-24
PR: [#66](https://github.com/saphid/frame-control/pull/66), `pad-gamescope`.
Parent/main: v0.4.18 commit `0da1fe5`. Merge commit: `1737366`, with parents
`cba23fb` (the existing PR) and `0da1fe5` (main). No history was rewritten.
## Merge and implementation
Both `/api/input` (keyboard/trackpad) and `/api/touch` (live-view Control)
start and poll the gamescope helper. KDE Connect starts only when Unicode
typing needs it. The existing ordered, bounded queue and switch cancellation
remain in place; cross-service typing order is an estimate, not an application
acknowledgement.
The merge keeps main's intended-device validation and work reservation before
a GET starts an input helper, immutable SSH routes, launch generations and
serialized writes. Switching headsets also stops the power report and resets
both input agents and queued typing before publishing the new route. The page
keeps main's Tools sections, clipboard controls and single instances of the
keyboard/trackpad and VR tools, with the PR's gamescope help text.
The start/status regression tests now exercise the gamescope helper for both
endpoints. An explicit `/api/input` test checks that start and poll use that
helper without starting KDE Connect. Existing stale/missing-headset and
route-reservation tests remain active.
## Real Frame: focused click and ASCII keys received
Host: lxso2, `ssh frame`. SteamOS **0.5.6**, BUILD_ID **20261015.6141207**,
SteamVR **2.18.2**. At **11:05 UTC**, Game Mode was active, full Plasma was
inactive, Steam and SteamVR were running, battery was **99%**, and gamescope's
focusable/focused window lists were empty. The shared test lock was absent.
Existing X11 tools and libX11 were available; Tk could not load its library.
At **11:09:23 UTC**, an initial read-only preflight saw no focused window
but activity level **0** (idle), rather than the required **3** (standby), and
aborted before lock acquisition or input. At **11:09:57 UTC**, another task
held the shared lock with a focused panel; that lock and panel were left alone.
At **11:20:22 UTC**, read-only checks found the lock absent, no focused or
focusable windows, activity **3** and battery **98%**. The bounded probe ran at
**11:20:37 UTC** against implementation commit `5641d3c`:
1. Rechecked empty focus and standby, acquired `/tmp/frame-test.lock` with
atomic `mkdir`, wrote a unique owner marker and rechecked both conditions.
2. Created only a disposable 640×400 X11 event-recorder window on `:0`, tagged
with test app ID `2000999066`. It had a 45-second alarm and closed on stdin
EOF. No existing application, full desktop or dashboard was started.
3. Waited for gamescope to report that exact window (`62914561`) focused.
Started the production server's `TouchAgent` over SSH; it reported ready.
4. Called production `server.remote_touch` for positioning, then
`server.remote_input` with a left click, `Frame66` and the Enter key button.
Both responses reported `sent: true`. More importantly, the **window itself
recorded ButtonPress/Release (button 1), the ASCII key events in order and
Return**. `XLookupString` decoded exactly `Frame66\r`. The events were
delivered through gamescope's EIS socket, not fabricated with XSendEvent.
5. Closed the helper by EOF and waited for exit, closed the recorder, checked
empty focus and standby, then removed only the owner-matching test lock.
[State and received events](pad-gamescope-2026-10-24/readback.json) and the
[probe](pad-gamescope-2026-10-24/probe.py)/[X11 recorder](pad-gamescope-2026-10-24/recorder.py)
are retained. The archived scripts use checkout-relative source paths; the
recorded run used the same harness from `/tmp`. Motion records omit an unused
field that only has meaning for key/button events.
Delayed readback at **11:21:46 UTC** confirmed no test lock or recorder process,
empty focused/focusable lists, Game Mode active, full Plasma inactive, activity
3 and battery 98%. The original Steam and vrserver PIDs were unchanged. No
install, security/global setting change, dashboard/session switch, reset or
reboot was performed. No headset helper or owned lock remained.
This establishes **focused-panel click and ASCII key delivery on `:0`**.
It does not establish wearer-visible presentation, Unicode delivery or GUI
interaction. The absolute-position setup reached the recorder's edge
(639,399), rather than its centre; pixel-accurate absolute pointing is **not**
proved by this small-window probe. The trackpad click and typed keys still
arrived in the owned focused window. The original PR's earlier acceptance-only
observations remain historical; this observation adds actual window events.
## Offline checks and remaining limits
- `python3 -m unittest discover -s tests -p test_input.py -v`: **58 passed**.
- `python3 -m unittest discover -s tests -p test_touch.py -v`: **43 passed**.
- `python3 -m unittest discover -s tests -p test_service_regressions.py -v`:
**34 passed**.
- `python3 -m unittest discover -s tests -v`: **1,403 ran, OK, 15 skipped**
(Linux x86_64, Python 3.12; 150.825 seconds).
- Full fake-Frame e2e in task-specific Docker project `frame-control-pr66`:
**47 passed** after the lock fix (192.270 seconds). Both task containers,
their network and all three volumes were removed afterward. The run emitted
an unclosed test-socket ResourceWarning; the suite returned OK.
- Node page suites: `ctrl_gestures.mjs`, `pad_queue.mjs`, `desk_panels.mjs`
passed, using the production page functions.
- `node --test app/test/*.test.js`: **3 test files passed**.
- `node --test site/test/*.test.mjs`: **2 test files passed**.
- Python compilation, desktop/site JavaScript syntax and `git diff --check`
passed. Fake-Frame e2e and exact-head CI results are recorded in the PR after completion.
The first full regression run executed 1,401 tests and found two tests still
expecting `/api/input` to start the old KDE Connect helper. They were updated
to the intended gamescope route. The final full run includes the lazy-KDE
regression above and the lock-order regression below. The initial restricted-sandbox input run could not open
loopback sockets; the focused rerun with managed network access passed.
Review of the merged code then found a lock inversion: the Unicode worker
held the pad-send lock while `InputAgent.start()` acquired the route lock,
whereas a headset switch held the route lock before resetting the pad. A
bounded subprocess regression reproduced the deadlock with the real
`InputAgent.send/start` methods and an offline launch stub. Unicode delivery
and resets now take the route lock before the pad lock; the route lock is
reentrant so an agent can capture its route inside that reservation. The new
regression passed after the fix, as did all **58 focused input tests** and
the **1,403-test full suite** (15 skipped). The local fake-Frame suite also
passed **47 tests after this fix**. Exact-head CI results are recorded in the PR.
Remaining device work requires an idle, available Frame and the shared lock:
verify Unicode in a real text field, ordering under real typing load and
precise absolute pointing for small panels. Worn-headset presentation,
packaged Electron pointer lock and Windows GUI execution remain separate
checks. Automated Windows/macOS CI does not establish those interactions.
## Review round 2: queue saturation and failure recovery
The coordinator identified silent eviction in both `_queue_for_kde()` and the
worker's retry path. Both sliced the oldest accepted events off at 200 entries,
even though the page had already received `sent: true` and discarded its copy.
Admission now serializes concurrent requests, checks capacity before any pointer
or keyboard send, and returns `sent: false, state: waiting` if the whole batch
will not fit. Text stays in the bounded queue while in flight, and is removed
only after the helper accepts it. Rejected mixed batches send no clicks; retrying
them cannot duplicate clicks already sent by this server. The page polls until
it can retry. Pointer input stays usable while accepted text is pending.
The old worker also left its worker marker and one-hour timing sentinel behind
if `send()` raised. Failure/timeout handling now retains unsent text, clears the
worker/timing state and exposes an error through input status. The page polls
accepted text without requiring another keypress. **Try again** explicitly
restarts the retained queue. Thread-start failures follow the same retention
rule. A stale worker's exception cannot change the new route's state.
The proposed send-to-enqueue headset-switch race was not reachable through the
normal HTTP switch path: `Handler.do_POST()` reserves `working(meant)` across
both operations; `devices_post()` shares `_work_lock` and refuses a moving
operation while work is active. Connector configuration/reconnect paths defer
changed headsets while work runs. A production-handler regression attempts a
switch at both send and enqueue and passes even on the previous implementation.
No separate connector change was needed. Atomic queue admission now uses the
existing route-before-pad lock order as well, because starting either input
agent can capture its route synchronously.
Three deterministic saturation regressions fail on `215db7b` and pass with the
fix: capacity during the typing delay, capacity during an unsuccessful-send
retry, and two concurrent producers competing for the last slots. They check
exact accepted event sequences, reject a mixed click/text batch before its
pointer effects, and verify delivery exactly once after retry. Additional tests
cover explicit recovery after timeout, helper errors, send exceptions and
thread-start failure, stale-worker isolation, HTTP status/start behavior, and
the production page's polling/error behavior.
No new active headset test was performed: this change affects host queue
admission/recovery and status handling, not Frame event translation or injection.
The earlier real click/ASCII evidence remains scoped to its recorded commit.
Real Unicode, packaged Electron pointer lock, Windows GUI, wearer-visible
presentation and precise small-window absolute positioning remain unverified.
`sent: true` means accepted by this server, not application delivery. The queue
is in memory; switching headsets cancels it and server exit loses it. A transport
failure after a partial write can still make delivery ambiguous; this is not an
end-to-end exactly-once protocol.
Round 2 local verification: **141 focused input/touch/service tests passed**
(62 input, 43 touch, 36 service; 20.061 seconds), and **1,409 server tests ran, OK, 15 skipped**
(Linux x86_64, Python 3.12, 153.198 seconds). All three production-page Node
suites passed; app tests passed (3 files), site tests passed (2 files), Python
compilation and `git diff --check` passed. The existing skipped tests require
zsh, macOS/Swift or built native PC libraries. Current-head CI is tracked in
PR #66; the previous 47-test local fake-Frame run above is historical.
@@ -1,129 +0,0 @@
"""Active real-Frame evidence probe: aborts unless idle, then owns the shared lock.
Run manually on an authorised host: python3 docs/evidence/pad-gamescope-2026-10-24/probe.py
The recorded run used /tmp paths; archived paths are relative to this checkout.
"""
import base64
import datetime
import json
import os
from pathlib import Path
import queue
import shlex
import subprocess
import sys
import tempfile
import threading
import time
import uuid
ROOT = Path(__file__).resolve().parents[3]
data = tempfile.TemporaryDirectory(prefix='pr66-live-')
os.environ.update(FRAME_CONTROL_DATA_DIR=data.name, FRAME_CONTROL_TELEMETRY='0', FRAME_PRIVATE_SSH='1')
sys.path.insert(0, str(ROOT / 'ui'))
import server
SSH = ['ssh', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=5', 'frame']
owner = 'pr66-' + uuid.uuid4().hex
result = {'time': datetime.datetime.now(datetime.timezone.utc).isoformat(), 'owner': owner,
'base': '0da1fe5', 'route': 'production server.remote_input -> TouchAgent SSH -> frame_touch.py -> gamescope EIS',
'events': []}
recorder = None
locked = False
rows = queue.Queue()
def ssh(cmd):
return subprocess.run([*SSH, cmd], capture_output=True, text=True, timeout=15, check=True).stdout
def focus():
return json.loads(server.ssh('python3 - focus', stdin=(ROOT/'ui/frame_touch.py').read_text(), timeout=15))
def activity():
return json.loads(ssh('/opt/steamvr/bin/linuxarm64/vrcmd --stats'))[0]['activity_level']
def read_rows():
for raw in recorder.stdout:
rows.put(json.loads(raw))
def collect(seconds):
end = time.monotonic() + seconds
while time.monotonic() < end:
try:
result['events'].append(rows.get(timeout=max(0.01, end-time.monotonic())))
except queue.Empty:
break
try:
result['initial_focus'] = focus()
result['initial_activity'] = activity()
assert not result['initial_focus'].get('window'), 'existing focused panel; do not interrupt'
assert result['initial_activity'] == 3, 'headset is not in standby'
ssh('mkdir /tmp/frame-test.lock && printf %s ' + shlex.quote(owner) + ' > /tmp/frame-test.lock/owner')
locked = True
result['lock_acquired'] = True
assert not focus().get('window') and activity() == 3, 'session changed during lock acquisition'
code = base64.b64encode(Path(__file__).with_name('recorder.py').read_bytes()).decode()
command = 'python3 -u -c ' + shlex.quote("import base64;exec(compile(base64.b64decode('" + code + "'),'pr66-recorder','exec'))")
recorder = subprocess.Popen([*SSH, command], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, text=True)
threading.Thread(target=read_rows, daemon=True).start()
initial = rows.get(timeout=8)
result['recorder'] = initial
assert initial.get('ready'), initial
window = initial['window']
for _ in range(20):
current = focus()
if current.get('window') == window:
break
time.sleep(0.2)
result['focused_panel'] = current
assert current.get('window') == window and current.get('display') == ':0', 'owned panel did not obtain focus; no input sent'
server.remote_input({})
for _ in range(50):
if server._touch.status.get('state') in ('ready', 'error'):
break
time.sleep(0.1)
result['agent_status'] = dict(server._touch.status)
assert result['agent_status'].get('state') == 'ready', result['agent_status']
target = {'window': window, 'display': ':0'}
assert focus().get('window') == window and activity() == 3
result['position_response'] = server.remote_touch({'events': [dict(target, fx=0.5, fy=0.5)]})
time.sleep(0.2)
assert focus().get('window') == window and activity() == 3
result['pad_response'] = server.remote_input({'events': [{'singleclick': True}, {'key': 'Frame66'}, {'specialKey': 12}]})
collect(2)
received = result['events']
result['received_text'] = ''.join(e.get('text', '') for e in received if e.get('type') == 2)
result['click_received'] = any(e.get('type') == 4 and e.get('code') == 1 and e.get('window') == window for e in received)
result['release_received'] = any(e.get('type') == 5 and e.get('code') == 1 and e.get('window') == window for e in received)
result['passed'] = result['click_received'] and result['release_received'] and result['received_text'] == 'Frame66\r'
assert result['passed'], 'event recorder did not observe expected click and keys'
except Exception as e:
result['error'] = str(e)
finally:
proc = server._touch.proc
server.reset_pad_route()
if proc:
try:
proc.wait(timeout=8)
result['input_helper_exited'] = True
except subprocess.TimeoutExpired:
proc.terminate()
proc.wait(timeout=5)
result['input_helper_terminated'] = True
if recorder:
recorder.stdin.close()
recorder.wait(timeout=8)
collect(0.2)
result['recorder_exit'] = recorder.returncode
result['recorder_stderr'] = recorder.stderr.read()[-1000:]
if locked:
result['final_focus'] = focus()
result['final_activity'] = activity()
ssh('test "$(cat /tmp/frame-test.lock/owner)" = ' + shlex.quote(owner)
+ ' && rm /tmp/frame-test.lock/owner && rmdir /tmp/frame-test.lock')
result['owned_lock_released'] = True
Path('/tmp/pr66-live.json').write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps(result, indent=2))
data.cleanup()
sys.exit(0 if result.get('passed') else 1)
@@ -1,280 +0,0 @@
{
"time": "2026-10-24T11:20:37.392607+00:00",
"owner": "pr66-7c2a8e8851f449bcb0bb6a040be94d99",
"base": "0da1fe5",
"route": "production server.remote_input -> TouchAgent SSH -> frame_touch.py -> gamescope EIS",
"events": [
{
"type": 6,
"window": 62914561,
"x": 320,
"y": 200,
"synthetic": false
},
{
"type": 6,
"window": 62914561,
"x": 639,
"y": 399,
"synthetic": false
},
{
"type": 4,
"window": 62914561,
"x": 639,
"y": 399,
"code": 1,
"synthetic": false
},
{
"type": 5,
"window": 62914561,
"x": 639,
"y": 399,
"code": 1,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 50,
"synthetic": false,
"text": "",
"keysym": 65505
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 41,
"synthetic": false,
"text": "F",
"keysym": 70
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 41,
"synthetic": false
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 50,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 27,
"synthetic": false,
"text": "r",
"keysym": 114
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 27,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 38,
"synthetic": false,
"text": "a",
"keysym": 97
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 38,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 58,
"synthetic": false,
"text": "m",
"keysym": 109
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 58,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 26,
"synthetic": false,
"text": "e",
"keysym": 101
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 26,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 15,
"synthetic": false,
"text": "6",
"keysym": 54
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 15,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 15,
"synthetic": false,
"text": "6",
"keysym": 54
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 15,
"synthetic": false
},
{
"type": 2,
"window": 62914561,
"x": 639,
"y": 399,
"code": 36,
"synthetic": false,
"text": "\r",
"keysym": 65293
},
{
"type": 3,
"window": 62914561,
"x": 639,
"y": 399,
"code": 36,
"synthetic": false
},
{
"cleaned": true
}
],
"initial_focus": {
"window": null
},
"initial_activity": 3,
"lock_acquired": true,
"recorder": {
"ready": true,
"window": 62914561,
"pid": 317210,
"display": ":0",
"appid": 2000999066
},
"focused_panel": {
"window": 62914561,
"display": ":0",
"name": "Frame Control PR66 disposable input probe",
"x": 0,
"y": 0,
"width": 640,
"height": 400,
"root": [
1920,
1080
],
"app": 2000999066
},
"agent_status": {
"state": "ready",
"focus": 62914561
},
"position_response": {
"state": "ready",
"focus": 62914561,
"sent": true
},
"pad_response": {
"state": "ready",
"focus": 62914561,
"sent": true
},
"received_text": "Frame66\r",
"click_received": true,
"release_received": true,
"passed": true,
"input_helper_exited": true,
"recorder_exit": 0,
"recorder_stderr": "",
"final_focus": {
"window": null
},
"final_activity": 3,
"owned_lock_released": true,
"implementation_commit": "5641d3c316dfe225321f7233840cc83ff8c3f527",
"device": {
"steamos": "0.5.6",
"build_id": "20261015.6141207",
"steamvr": "2.18.2",
"battery_before": 98,
"battery_after": 98
},
"delayed_cleanup": {
"time": "2026-10-24T11:21:46Z",
"shared_lock_absent": true,
"owned_recorder_absent": true,
"game_mode": "active",
"full_plasma": "inactive",
"steam_pid": 297826,
"vrserver_pid": 297506,
"focusable_windows": [],
"focused_windows": [],
"activity_level": 3
},
"limits": [
"No wearer or packaged Electron/GUI interaction",
"ASCII only; Unicode and cross-service ordering not tested on device",
"The positioning setup reached the recorder edge (639,399); pixel-accurate absolute targeting is not established by this probe"
]
}
@@ -1,82 +0,0 @@
"""Disposable X11 event recorder used for the PR #66 focused-input observation."""
import ctypes as C
import json
import os
import select
import signal
import time
X = C.CDLL('libX11.so.6')
P, U = C.c_void_p, C.c_ulong
for name, restype, args in [
('XOpenDisplay', P, [C.c_char_p]), ('XDefaultRootWindow', U, [P]),
('XCreateSimpleWindow', U, [P, U, C.c_int, C.c_int, C.c_uint, C.c_uint, C.c_uint, U, U]),
('XInternAtom', U, [P, C.c_char_p, C.c_int]),
('XChangeProperty', C.c_int, [P, U, U, U, C.c_int, C.c_int, P, C.c_int]),
('XStoreName', C.c_int, [P, U, C.c_char_p]),
('XSelectInput', C.c_int, [P, U, C.c_long]), ('XMapWindow', C.c_int, [P, U]),
('XFlush', C.c_int, [P]), ('XPending', C.c_int, [P]),
('XConnectionNumber', C.c_int, [P]), ('XNextEvent', C.c_int, [P, P]),
('XLookupString', C.c_int, [P, P, C.c_int, P, P]),
('XDestroyWindow', C.c_int, [P, U]), ('XCloseDisplay', C.c_int, [P]),
]:
fn = getattr(X, name)
fn.restype, fn.argtypes = restype, args
class InputEvent(C.Structure):
_fields_ = [('type', C.c_int), ('serial', U), ('send_event', C.c_int),
('display', P), ('window', U), ('root', U), ('subwindow', U),
('time', U), ('x', C.c_int), ('y', C.c_int), ('x_root', C.c_int),
('y_root', C.c_int), ('state', C.c_uint), ('code', C.c_uint),
('same_screen', C.c_int)]
class Event(C.Union):
_fields_ = [('type', C.c_int), ('input', InputEvent), ('pad', C.c_long * 24)]
def say(**row):
print(json.dumps(row), flush=True)
def end(*_):
raise TimeoutError('probe ended')
signal.signal(signal.SIGTERM, end)
signal.signal(signal.SIGALRM, end)
signal.alarm(45)
d = X.XOpenDisplay(b':0')
assert d, 'X display unavailable'
w = 0
try:
w = X.XCreateSimpleWindow(d, X.XDefaultRootWindow(d), 0, 0, 640, 400, 0, 0, 0x224466)
for name, value in [(b'STEAM_GAME', 2000999066), (b'_NET_WM_PID', os.getpid())]:
val = U(value)
X.XChangeProperty(d, w, X.XInternAtom(d, name, 0), 6, 32, 0, C.byref(val), 1)
X.XStoreName(d, w, b'Frame Control PR66 disposable input probe')
X.XSelectInput(d, w, 1 | 2 | 4 | 8 | 64 | 32768 | 2097152)
X.XMapWindow(d, w)
X.XFlush(d)
say(ready=True, window=w, pid=os.getpid(), display=':0', appid=2000999066)
while True:
while X.XPending(d):
event = Event()
X.XNextEvent(d, C.byref(event))
if event.type in (2, 3, 4, 5, 6):
e = event.input
row = dict(type=event.type, window=e.window, x=e.x, y=e.y,
synthetic=bool(e.send_event))
if event.type != 6:
row['code'] = e.code
if event.type == 2:
buf, keysym = C.create_string_buffer(32), U()
n = X.XLookupString(C.byref(e), buf, 32, C.byref(keysym), None)
row.update(text=buf.raw[:n].decode('utf-8', 'replace'), keysym=keysym.value)
say(**row)
ready, _, _ = select.select([0, X.XConnectionNumber(d)], [], [], 0.2)
if 0 in ready and not os.read(0, 1024):
break
except TimeoutError:
pass
finally:
if w:
X.XDestroyWindow(d, w)
X.XCloseDisplay(d)
say(cleaned=True)
@@ -1,65 +0,0 @@
# Panel discovery investigation, 2026-10-24
Base: v0.4.16 (`040581a`). Scope: issue #73, Desktop Capture and Control.
## Evidence and change
Issue #73 reports SteamOS 0.4.5 with its desktop visibly running, without a
Frame Control version or steps. The maintainer's 2026-10-12 comment identifies
the duplicate `loadPanels` UI function fixed in 0.4.3 and requests the exact
remaining message and SteamOS build if it persists. There is no reporter
follow-up. At this base, the functions are distinct and the existing Node
regression checks their uniqueness and the Desktop GET request.
Read-only source inspection found a remaining diagnostic ambiguity:
`read_props()` returns `{}` for failed/timed-out X queries and missing root
properties. `panels()` treated that like a confirmed empty focusable-window
list. With no detected Plasma process it claimed no panel was open; with Plasma
it claimed SteamOS was not listing the desktop. Neither claim follows from an
unreadable list.
The change checks for the presence of `GAMESCOPE_FOCUSABLE_WINDOWS` before
those claims. An absent list now reports unknown panel state, suggests checking
SteamVR and retrying/Headset view, and asks for the exact message and SteamOS
version if persistent. Missing-tool and whole-run deadline explanations retain
precedence. No new probes, retries, timeouts, input or capture routes are added.
## Real-device boundary
A preliminary read-only SSH inspection returned SteamOS 0.5.6, BUILD_ID
`20261015.6141207`, battery 99%, discharging, and X0/X1 socket names. It made
no headset changes. The shared lock was absent at that moment.
The subsequent atomic `mkdir /tmp/frame-test.lock` failed with `File exists`.
The guarded command exited with status 3 before the session, X-property or
panel-helper probes ran. The other task's lock was left alone. No application
was installed, started, stopped or focused; no headset settings changed.
That implementation probe was blocked by the occupied shared device. The
coordinator later supplied a separate narrow read-only observation on SteamOS
0.5.6, BUILD_ID `20261015.6141207`, at battery 99%, with the shared lock absent
and both `xprop` and `xwininfo` installed:
- In **Game Mode**, `DISPLAY=:0 xprop -root GAMESCOPE_FOCUSABLE_WINDOWS
GAMESCOPE_FOCUSED_WINDOW` returned both properties **present but empty**.
- On `DISPLAY=:1`, both properties were reported **not found**.
This observation distinguishes a confirmed empty list on `:0` from absent
properties on `:1`. It does **not** reproduce #73's Desktop Mode report on
SteamOS 0.4.5, and it does **not** test this PR's code on the headset. It is a
coordinator-supplied observation, separate from the blocked implementation probe
above; no new headset probe was run for this follow-up.
Remaining limit: there is no reproduction establishing a further
capture/discovery fix for the reported Desktop Mode/build combination. This PR
improves a demonstrated diagnostic error; it must not close #73 as a verified
new capture fix.
## Offline verification
Focused Python regressions distinguish absent, focus-only, failed/timed-out X
queries, confirmed empty and desktop-running lists. Existing tests cover
missing tools, alternate displays, successful discovery and the 12-second
deadline. The production page's Node behavior test verifies that both Capture
and Control preserve the unreadable-list advice. These use fake X/HTTP/DOM
boundaries and do not prove real SteamOS 0.4.5 capture or physical input.
-168
View File
@@ -1,168 +0,0 @@
# Saved panel layouts: feasibility recheck, 2026-10-24
Issue: [#31](https://github.com/saphid/frame-control/issues/31). Base: main
`5b47cd3`. This investigation covers spatial save/restore for existing
Steam/gamescope panels without another overlay application.
## Decision and user behavior
Keep saved layouts unavailable. The current Frame exposes some geometry,
including a standing-space coordinate transform, but this investigation did
not establish a complete dock-state snapshot or a controlled restore path for
these panels. The existing companion/headset switcher still lists panels and
requests focus; users arrange them with SteamVR's dashboard controls. No layout
file, new control, overlay application or background service is added.
This is a limit of the investigated route, not proof that every native
implementation is impossible. In particular, failure of
`GetOverlayTransformAbsolute` does **not** mean all pose readback is denied.
## Device and read-only results
SteamOS **0.5.6**, BUILD_ID **20261015.6141207**; `vrcmd --version`:
**2.18.2**. `gamescope-session.service` was active and
`plasma-workspace.target` inactive: Game Mode. gamescope used the OpenVR backend
and `--virtual-connector-strategy PerAppId`. Battery was 99%, Not charging,
before the probe and 100%, Full, afterwards.
The [exact probe](panel-layout-2026-10-24/probe.py) loads only
`FnTable:IVROverlay_028` from the shipped
`/opt/steamvr/bin/linuxarm64/libopenvr_api.so`. It uses application type 3
(`VRApplication_Background`, avoiding runtime startup) and shuts down its own
OpenVR connection in `finally`. Initialization and interface acquisition both
returned 0. It calls getters only, with no image capture or input injection.
[Raw result](panel-layout-2026-10-24/readback.json):
| Getter | `valve.steam.desktopgame.0` | `valve.steam.gamepadui.main` |
|---|---|---|
| FindOverlay | 0, Success | 0, Success |
| Rendering process | `gamescope-wl` | `gamescope-wl` |
| Width | 0; 2.6700000763 m | 0; 2.6670000553 m |
| Curvature | 0; 0 | 0; 0 |
| Transform type | 0; 5, DashboardTab | 0; 5, DashboardTab |
| Absolute transform | 18, WrongTransformType | 18, WrongTransformType |
| Tracked-device-relative transform | 18, WrongTransformType | 18, WrongTransformType |
| Coordinate transform, center, standing origin | 0; identity matrix | 0; scaled/translated matrix |
| Coordinate transform, center, seated origin | 23, RequestFailed | 23, RequestFailed |
| Coordinate transform, center, raw origin | 23, RequestFailed | 23, RequestFailed |
The dashboard and both panels were reported hidden; neither panel was active.
The standing-space matrix is the transform for the overlay point `(0.5, 0.5)`;
it does not identify World/Dashboard/Theater/controller docking or recreate a
dashboard frame. The Steam panel's basis has scale about 0.369. Treat neither
that scaled matrix nor the hidden Gamescope panel's identity as a validated
physical placement to feed to an absolute-transform setter. Visible placement,
matrix freshness and correspondence to dashboard size were not checked by a
wearer.
## API and dashboard boundary
The ABI and result names were checked against Valve's
[OpenVR v2.12.14 C header](https://github.com/ValveSoftware/openvr/blob/v2.12.14/headers/openvr_capi.h)
and [C++ interface comments](https://github.com/ValveSoftware/openvr/blob/v2.12.14/headers/openvr.h).
That header declares `IVROverlay_028`; its SHA-256 is
`d58a2b149e434b2b8ee25cb4253ca6157feed8092f621e78ecfed6846d35d3e4`.
The absolute getter is explicitly limited to absolute transforms. The
coordinate getter returns the 3D transform for a 2D overlay point. Public
`IVROverlay_028` has no getter/setter pair for dashboard dock location or frame
scene-graph state. The probe fails closed if that exact interface is absent.
Read-only inspection of the shipped dashboard
`/opt/steamvr/resources/webinterface/dashboard/chunk~8012d0c89.js`
(SHA-256 `937ec62c67a481f030a416099ceb98b0fe2a702aea5a0705601c6c40d4969f6d`)
found:
- `dockLocation` reads an internal `_getDockLocation(frameID)` store;
`SetDockLocation` updates that store or sends a VRLink request.
- `m_mapLastRelativeTransformForDockLocation` keeps per-location transforms.
World initialization requests a dashboard scene-graph transform through
`requestSGTransform`, then updates the component's `xfTransform`.
- The `vrcmd_dock_overlay` handler accepts `dashboard`, `lefthand`, `righthand`,
`theater` and `world`. It selects the frame associated with the supplied key,
**falls back to the active frame if no match exists**, sets dock location and
may switch frame. This is not an exact pose restore or a readback response.
Those are observations of shipped internal code, not proof that Frame Control
can safely access its state through a supported external API. No dashboard
JavaScript was injected, private store edited or VRLink command sent. The
active-frame fallback is an additional reason not to experiment with docking
commands on a stale key.
The [2026-09-28 probe](../panels.md#saved-spatial-layouts-blocked-on-the-current-panel-route)
on SteamOS 0.4.1 / SteamVR 2.18.1 recorded an absolute setter returning
12 (`PermissionDenied`) on a temporary, test-owned gamescope panel. That setter
result is **historical**, not a new test on 2.18.2. The newer read-only results
above neither overturn nor reverify it.
## Commands, safety and cleanup
Read-only inspection used `ssh frame` with `BatchMode=yes` and
`ConnectTimeout=10`, `cat /etc/os-release`, battery sysfs reads,
`systemctl --user is-active gamescope-session.service plasma-workspace.target`,
process inspection, and `vrcmd --version`, `--help`, `--overlays`. An initial
`--listoverlays` attempt returned `Unknown command`; `--overlays` is the actual
enumeration command. Dashboard sources were read/copied to local `/tmp` for
inspection. No files were installed on the Frame.
Run the retained probe from the repository root, following
[the shared-device rules](../testing.md#headset-smoke-test). Stop if the lock is
occupied or battery is at/below 20%; never remove another task's lock:
```sh
ssh -o BatchMode=yes -o ConnectTimeout=10 frame \
'set -eu; mkdir /tmp/frame-test.lock || exit 3; trap "rmdir /tmp/frame-test.lock" EXIT; timeout 25 python3 -' \
< docs/evidence/panel-layout-2026-10-24/probe.py
```
This command acquired the lock, completed with exit 0 and released it. A
subsequent check confirmed the lock absent, Game Mode active, and Steam,
`vrserver` and `vrcompositor` running. The probe's only headset filesystem
change was creating/removing that shared lock. Its Python source ran from
stdin. There were no test panels, installs, profiles or services to remove.
No mutation test was essential: without verified dock/relative-frame readback,
there was no exact original layout and bounded rollback to justify changing a
panel. Existing user panels were never focused, shown, hidden, resized, moved
or docked. No global settings, security, passwords, runtime processes or modes
were changed; no reboot or reset occurred.
## Validation and remaining gaps
Existing `tests/test_panels.py` covers main-panel enumeration, hidden panels,
focus revalidation, unavailable-runtime errors and HTTP authentication/Host/
Origin guards. `tests/test_page_scripts.py` checks the production page's shared
JavaScript scope. These offline checks do not prove spatial placement. The
retained probe is evidence tooling, not a new product API; no fake layout
backend or layout tests claiming device support were added.
Validation passed:
```sh
python3 -m unittest discover -s tests -p test_panels.py -v # 8 passed
python3 -m unittest discover -s tests -p test_page_scripts.py -v # 5 passed
python3 -m py_compile docs/evidence/panel-layout-2026-10-24/probe.py
python3 -m json.tool docs/evidence/panel-layout-2026-10-24/readback.json
git diff --check
```
The initial panel-suite attempt hit sandbox socket restrictions in four HTTP
tests; rerunning with loopback sockets permitted passed all eight. No real
headset connection is used by those tests. The full suite and fake Frame were
not run for this evidence-only change.
Before shipping layouts, a native route still needs proof of dock state and
relative-frame readback, authorized restore on an owned test panel with exact
rollback, and correspondence between saved size/pose and visible placement.
It also needs stable source identity and a fresh capture-permission/ticket
flow when reopening viewers. Reconnect, panel recreation, reboot recovery,
Steam-launched gameplay, controller docking and worn-headset input were not
tested here. The existing Frame Control-owned media renderer is not proof that
Steam/gamescope panels can be restored. No third-party overlay app was used.
## Announcement
We rechecked saved panel layouts on SteamOS 0.5.6. SteamVR exposes some panel
geometry, but we have not verified safe restoration of Steam's panel positions
and docking. Saved layouts remain unavailable; use SteamVR's dashboard to
arrange panels. The panel switcher continues to list and show open panels.
@@ -1,89 +0,0 @@
"""Read-only issue #31 probe; pipe to `ssh frame 'timeout 25 python3 -'`.
Acquire /tmp/frame-test.lock first per docs/testing.md. No overlay is created,
focused, docked or changed. Background init (3) does not start SteamVR.
ABI: ValveSoftware/openvr v2.12.14 headers/openvr_capi.h, IVROverlay_028.
"""
import ctypes as C
import json
from pathlib import Path
H = C.c_uint64
Matrix = C.c_float * 12
class Vector2(C.Structure):
_fields_ = [('v', C.c_float * 2)]
def probe():
vr = C.CDLL('/opt/steamvr/bin/linuxarm64/libopenvr_api.so')
vr.VR_InitInternal2.argtypes = [C.POINTER(C.c_int), C.c_int, C.c_char_p]
vr.VR_InitInternal2.restype = C.c_uint32
vr.VR_GetGenericInterface.argtypes = [C.c_char_p, C.POINTER(C.c_int)]
vr.VR_GetGenericInterface.restype = C.c_void_p
vr.VR_ShutdownInternal.argtypes = []
vr.VR_ShutdownInternal.restype = None
err = C.c_int()
vr.VR_InitInternal2(C.byref(err), 3, None)
report = {'init_error': err.value, 'interface': 'IVROverlay_028', 'panels': []}
if err.value:
return report
try:
ptr = vr.VR_GetGenericInterface(b'FnTable:IVROverlay_028', C.byref(err))
report['interface_error'] = err.value
if err.value or not ptr:
return report
table = C.cast(ptr, C.POINTER(C.c_void_p))
def function(slot, result, *args):
return C.CFUNCTYPE(result, *args)(table[slot])
def read(slot, handle, output_type):
output = output_type()
rc = function(slot, C.c_int, H, C.POINTER(output_type))(handle, C.byref(output))
return {'rc': rc, 'value': output.value if rc == 0 else None}
report['dashboard_visible'] = function(68, C.c_bool)()
for key in ('valve.steam.desktopgame.0', 'valve.steam.gamepadui.main'):
row = {'key': key}
report['panels'].append(row)
handle = H()
row['find_rc'] = function(0, C.c_int, C.c_char_p, C.POINTER(H))(
key.encode(), C.byref(handle))
if row['find_rc']:
continue
pid = function(10, C.c_uint32, H)(handle)
row['rendering_pid'] = pid
try:
row['rendering_comm'] = Path('/proc/%d/comm' % pid).read_text().strip()
except OSError:
row['rendering_comm'] = None
row['visible'] = function(45, C.c_bool, H)(handle)
row['active_dashboard'] = function(69, C.c_bool, H)(handle)
row['width_m'] = read(23, handle, C.c_float)
row['curvature'] = read(25, handle, C.c_float)
row['transform_type'] = read(32, handle, C.c_int)
for name, slot, scalar in (('absolute', 34, C.c_int),
('tracked_device_relative', 36, C.c_uint32)):
origin = scalar()
matrix = Matrix()
rc = function(slot, C.c_int, H, C.POINTER(scalar), C.POINTER(Matrix))(
handle, C.byref(origin), C.byref(matrix))
row[name] = {'rc': rc, 'origin_or_device': origin.value if rc == 0 else None,
'matrix': list(matrix) if rc == 0 else None}
row['coordinate_transforms'] = []
for origin in (0, 1, 2): # seated, standing, raw/uncalibrated
matrix = Matrix()
rc = function(46, C.c_int, H, C.c_int, Vector2, C.POINTER(Matrix))(
handle, origin, Vector2((.5, .5)), C.byref(matrix))
row['coordinate_transforms'].append({
'tracking_origin': origin, 'coordinates': [.5, .5], 'rc': rc,
'matrix': list(matrix) if rc == 0 else None})
return report
finally:
vr.VR_ShutdownInternal()
if __name__ == '__main__':
print(json.dumps(probe(), indent=2, allow_nan=False))
@@ -1,152 +0,0 @@
{
"init_error": 0,
"interface": "IVROverlay_028",
"panels": [
{
"key": "valve.steam.desktopgame.0",
"find_rc": 0,
"rendering_pid": 297037,
"rendering_comm": "gamescope-wl",
"visible": false,
"active_dashboard": false,
"width_m": {
"rc": 0,
"value": 2.6700000762939453
},
"curvature": {
"rc": 0,
"value": 0.0
},
"transform_type": {
"rc": 0,
"value": 5
},
"absolute": {
"rc": 18,
"origin_or_device": null,
"matrix": null
},
"tracked_device_relative": {
"rc": 18,
"origin_or_device": null,
"matrix": null
},
"coordinate_transforms": [
{
"tracking_origin": 0,
"coordinates": [
0.5,
0.5
],
"rc": 23,
"matrix": null
},
{
"tracking_origin": 1,
"coordinates": [
0.5,
0.5
],
"rc": 0,
"matrix": [
1.0,
0.0,
0.0,
0.0,
0.0,
1.0,
0.0,
0.0,
0.0,
0.0,
1.0,
0.0
]
},
{
"tracking_origin": 2,
"coordinates": [
0.5,
0.5
],
"rc": 23,
"matrix": null
}
]
},
{
"key": "valve.steam.gamepadui.main",
"find_rc": 0,
"rendering_pid": 297037,
"rendering_comm": "gamescope-wl",
"visible": false,
"active_dashboard": false,
"width_m": {
"rc": 0,
"value": 2.6670000553131104
},
"curvature": {
"rc": 0,
"value": 0.0
},
"transform_type": {
"rc": 0,
"value": 5
},
"absolute": {
"rc": 18,
"origin_or_device": null,
"matrix": null
},
"tracked_device_relative": {
"rc": 18,
"origin_or_device": null,
"matrix": null
},
"coordinate_transforms": [
{
"tracking_origin": 0,
"coordinates": [
0.5,
0.5
],
"rc": 23,
"matrix": null
},
{
"tracking_origin": 1,
"coordinates": [
0.5,
0.5
],
"rc": 0,
"matrix": [
0.3689989447593689,
0.0,
0.0008894503116607666,
-0.5048462152481079,
0.0,
0.36899998784065247,
0.0,
1.2510287761688232,
-0.0008894503116607666,
0.0,
0.3689989447593689,
-1.1726053953170776
]
},
{
"tracking_origin": 2,
"coordinates": [
0.5,
0.5
],
"rc": 23,
"matrix": null
}
]
}
],
"interface_error": 0,
"dashboard_visible": false
}
@@ -0,0 +1,119 @@
# iOS foreground screenshot downloads — 2026-10-24
This is implementation and proof for the foreground iOS portion of
[#108](https://github.com/saphid/frame-control/issues/108). It is not the full
requested background/cross-platform feature and is not released.
## Source and merge
Dedicated worktree: `/Users/saphid/.t3/worktrees/steam-frame/feat-phone-screenshot-autosync-108`.
Branch: `feat/phone-screenshot-autosync-108`.
First merged fetched main
[1d4c57d6](https://github.com/saphid/frame-control/commit/1d4c57d6)
into the existing Mac-space parent
[3fdc6c46](https://github.com/saphid/frame-control/commit/3fdc6c46).
Merge commit: [52772323](https://github.com/saphid/frame-control/commit/52772323).
The dirty primary checkout was left untouched. Review found and fixed the PC
helper's missing public route for the extracted `mac-stream.js` dependency.
Independent merge re-review ran 45 tests successfully, one skipped, and found
no remaining actionable findings in its seven-file target.
## Implemented behavior
- Native polling every four seconds while active, independent of the visible web tab.
- First successful listing becomes the durable baseline; only new captures queue.
- Two equal metadata observations, exact byte count and complete ImageIO decode
precede an atomic original write outside purgeable caches.
- Receipts and originals distinguish pinned headset identity, Steam account,
app ID and filename. Reconnect/relaunch resumes pending work.
- Photos is enabled by default. Add-only permission is requested in foreground.
Denied/failed exports keep originals; app-only captures never automatically
export after re-enabling the setting.
- Native downloads sheet exposes retained files through ShareLink, including
during connection failure/relaunch. Ambiguous interrupted Photos commits need
explicit recovery rather than risking an automatic duplicate.
- Backgrounding stops polling; no unrelated background mode is used.
## Automated verification
Normal signed Simulator build and full native test suite: **20 tests passed,
zero failures**, including eight screenshot tests. Result bundle:
`/tmp/frame-108-ios-all-r3.xcresult`; log: `/tmp/frame-108-ios-all-r3.log`.
Both command exit status and xcresult test count were checked. Private
DerivedData cleanup completed. Test target was the independently created
Simulator `F250F91E-2E7D-4833-B029-A275F11FDC7D` on iOS 26.5.
Screenshot tests cover baseline/stability, truncated data, restart dedup,
Photos failure/retry, changes during download/consent, cancellation,
account/app identity, corrupt-ledger failure, uncertain export handling,
late success and failure across journal replacement, default preference and
offline app-only originals. Download/export dependencies in these tests are
injected; the live test below exercises actual transport and PhotoKit.
Python iOS packaging/Foundation contracts: 5 passed. Page scripts: 5 passed.
`swiftc -typecheck` for the journal and scoped `git diff --check` passed.
## Actual headset → Simulator → Photos proof
Ran the production signed app against the real headset over its pinned SSH
connection and the actual deployed `/api/shots` and `/api/shots/image` endpoints.
Created two temporary valid PNGs under a unique synthetic Steam account/app
directory after the app had baselined the existing nine screenshots.
| Check | Observed result |
|---|---|
| Default Photos ON | Original downloaded and receipt marked `saved`; Photos contained one byte-identical original |
| Source / app copy / Photos SHA-256 | `a36eb2d497e6ecce17c767189a70b3b7e732a22309d2421cdc715cf13a7668ac` |
| Photos OFF | Second original downloaded with `appOnly`; no matching Photos original |
| Re-enable + relaunch | Second receipt stayed `appOnly`; first still had exactly one Photos original |
Machine readback: `/tmp/frame-108-live-proof.json`. Photos readback hashed files
in this dedicated Simulator's own Media/DCIM directory, independently of the
app receipt. No personal phone Photos database was accessed. This is a
synthetic capture-file test, not proof of the headset screenshot-button path,
capture latency, physical-device behavior, or background delivery.
The OFF preference was applied through Simulator defaults on restart, not a
UI tap. The production setting setter and persisted opt-out were exercised in
the native tests. Astra inspected the real native sheet in both states and
found no major clipping, overlap or actionable visual defect.
![Photos enabled and app-only setting](phone-screenshot-sync-2026-10-24/settings-proof.png)
An initial unsigned test build could not retain a stable Keychain identity;
the real transport test used normal Simulator signing. App launches with a
console attached were kept alive for the live checks.
## Review provenance
Preferred Opus routes failed before work with API rate limits. Implementation
used the current GPT-6.1 Sol session; reviews are disclosed as fresh same-model
checks, not different-model evidence.
Provider `codex`, model `gpt-6.1-sol`, high, completed:
- r1: `node:delegated-task:command%3Amcp%3Ac03980f6-1e71-4cc4-af1a-1d0961b9aaad%3Adelegate-task%3Aframe-108-ios-sync-review-20261024-r1`.
Found stale export callback persistence and missing offline access. Both fixed.
- r2: `node:delegated-task:command%3Amcp%3Ac03980f6-1e71-4cc4-af1a-1d0961b9aaad%3Adelegate-task%3Aframe-108-ios-sync-review-20261024-r2`.
No actionable findings in the complete executable diff against `52772323`
plus the four new Swift files. Report: `/tmp/frame-108-ios-sync-review-r2.md`.
Visual inspection used provider `codex`, `gpt-6-astra`, completed task
`frame-108-downloads-proof-inspection-20261024-r3`; report:
`/tmp/frame-108-downloads-proof-inspection.md`.
## Remaining release requirements
- Actual APNs delivery, notification extension/shared state, persistent headset
event source and authenticated delivery service are still unimplemented.
Main-app push capability is not configured; no usable provider credential
location was found in the bounded checks. Existing App Store Connect signing
access is not an APNs provider key.
- Physical locked/background iPhone download and Photos export remain unproven.
- A new native Android companion was subsequently authorized and implemented.
See [Android phone](../android-phone.md) for its explicit foreground-service
behavior and [Android evidence](android-phone-screenshot-sync-2026-10-24.md)
for the build and emulator checks. This does not supply physical-device proof.
- No TestFlight upload, store submission or release was performed. The user's
release condition requires the full behavior to be tested and working.
Binary file not shown.

After

Width:  |  Height:  |  Size: 403 KiB

@@ -1,36 +0,0 @@
{
"date": "2026-10-24",
"base": "c7489a8 (v0.4.14)",
"feature": "Desktop Tools power and heat observations",
"automated": {
"targeted": "python3 -m unittest discover -s tests -p 'test_power*.py': 35 passed",
"uiAndResourceGuard": "node --test tests/power_desktop.cjs: 7 passed",
"desktopApp": "node --test app/test/*.test.js: 37 passed",
"packaging": "Materialized ui/scripts resources selected by app/package.json; isolated Python -I imported frame_power, constructed remote payload including frame_status, and read idle status. Missing resource detection tested. Shared phone bundle import passed.",
"syntax": "Python compilation, sampler shell syntax and git diff --check passed",
"fullSuite": "python3 -m unittest discover -s tests -v: 1363 tests, OK (skipped=15), 153.274 seconds",
"transientFailure": "An earlier full run had one nested sandbox-test failure: the existing OneServer second-server child exited -11 (SIGSEGV), expected 1. Isolated sandbox rerun passed (6 tests), then the complete suite passed. No unrelated server-lifecycle code was changed."
},
"browser": {
"environment": "T3 collaborative Chromium preview, Linux, local Python Handler server; synthetic observation subprocess at accelerated one-second intervals; normal headset SSH disabled during start/stop/download verification",
"observations": [
"Opened #powerReport and verified labeled duration selector, disabled Stop/downloads before start and accessible sample progress.",
"Started a synthetic observation and observed 11 of 12 samples in progress; completion allowed another start.",
"Started again, observed progress, stopped, and saw 7 of 12 samples retained with explicit incomplete state and original frame alias.",
"Clicked HTML download and saw the UI acknowledge the request; T3's download list did not record the blob download, so filesystem save was not verified.",
"Fetched all three authenticated export endpoints in the browser: HTML 200 (7599 characters), JSON 200 (2618 characters, 7 samples), CSV 200 (977 characters); each retained the stop reason."
]
},
"realFrame": {
"lock": "ssh frame 'mkdir /tmp/frame-test.lock' succeeded before real-device probe; this task removed only that owned empty lock with rmdir after verification",
"probe": "Ran the production desktop Observation using strict, batch SSH to frame. The first attempt exposed a missing structured reason on early remote lock refusal; fixed and reran successfully.",
"result": "0 of 12 samples; running=false; stopped='Smoke session active (/tmp/frame-test.lock); no sensors read'. Exported HTML included the same incomplete reason.",
"scope": "Lock refusal only. No successful real sensor session, installation, app launch/stop, power control or thermal control was performed for this feature."
},
"limitations": [
"Native Electron installers were not built or launched; resource staging and build-hook contracts were tested.",
"One latest report is retained only in server memory. Starting again, restarting the server or quitting discards it; the UI warns users to download first.",
"Stopping ends only the owned local SSH process. The bounded remote reader discovers closure on its next write; immediate remote cancellation is not guaranteed.",
"Phone-hosted observation starts are unsupported; the shared server bundle still imports without desktop sampler resources."
]
}
@@ -1,34 +0,0 @@
{
"date": "2026-10-24",
"scope": "Read-only three-sample HTML export and browser download check; no installs, app launches, or remote process-control commands",
"preflight": {
"smokeLock": "absent",
"version": "0.5.6",
"build": "20261015.6141207"
},
"command": "scripts/frame-power-sample.sh --count 3 --interval 1 --html /tmp/frame-power-export-check.html",
"exitCode": 0,
"sampleCount": 3,
"metricCount": 91,
"unavailableGroups": [
"proximity"
],
"browser": {
"engine": "T3 collaborative Chromium preview",
"widths": [
1280,
390
],
"jsonDownloadMatches": true,
"csvDownloadMatches": true,
"csvDataRows": 276,
"pageScripts": 0,
"horizontalOverflow": "confined to focusable table region"
},
"limits": [
"No real-device Ctrl+C cancellation test; local child termination and retained samples verified with subprocess fixtures",
"No screen-reader, Windows or macOS verification",
"No controlled baseline, load or worn-headset testing",
"Sensor accuracy not established"
]
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

@@ -1,226 +0,0 @@
{
"date_utc": "2026-10-24T09:27:25.354Z",
"result": "PASS: literal packaged desktop Return to Game Mode button switched the real Frame from full Plasma; a repeat visible click made no second switch",
"documentation_base_commit": "71288d8980040c14f777f0e417f41f7105eaaac4",
"host": "lxso2",
"steamos_version": "0.5.6",
"steamos_build": "20261015.6141207",
"prior_evidence": [
"https://github.com/saphid/frame-control/issues/142",
"https://github.com/saphid/frame-control/pull/143",
"https://github.com/saphid/frame-control/pull/156"
],
"package": {
"tag": "v0.4.17",
"asset": "Frame-Control-linux-x86_64.AppImage",
"url": "https://github.com/saphid/frame-control/releases/download/v0.4.17/Frame-Control-linux-x86_64.AppImage",
"sha256": "b1b8eb82bf73a227b2cd36e8f0ef3a2ebdf7967c5853efb08a9634f38227015d",
"verification": "sha256sum matched the live release asset digest. Independently extracted the retained asset into a new private test directory. cmp confirmed bundled frame_backup.py and frame_backup_remote.py equal the checkout versions. No source or payload was changed.",
"launch": "--appimage-extract, then squashfs-root/AppRun --no-sandbox --disable-gpu --remote-debugging-port=49372 --remote-debugging-address=127.0.0.1 --user-data-dir=/tmp/frame-return-button-20261024/electron-profile --window-size=1400,950",
"display": "Xvfb :138 -screen 0 1440x1000x24 -nolisten tcp; Electron content screenshot 1400x923",
"isolation": "Private XDG_CONFIG_HOME, XDG_CACHE_HOME, FRAME_CONTROL_DATA_DIR and Electron profile; FRAME_CONTROL_TELEMETRY=0 and FRAME_CONTROL_NO_UPDATE_CHECK=1. Existing authorized ssh frame identity retained; bundled server and Python used."
},
"preflight": {
"read_only_first": true,
"initial_mode": "gamescope-session.target=active; plasma-workspace-wayland.target=inactive",
"initial_battery_percent": 99,
"initial_battery_status": "Not charging; charger Full",
"steam_and_steamvr_running": true,
"running_steam_apps": [],
"default_login_mode": "game",
"lock": "mkdir /tmp/frame-test.lock succeeded atomically; no foreign lock was removed. Initial harmless probe attempt released its own lock, then a fresh atomic acquisition covered both full-session attempts."
},
"rollback": {
"unit": "fc-return-button-rollback-20261024",
"probe": "systemd-run --user --unit=fc-return-button-probe-20261024 --on-active=2s --timer-property=AccuracySec=1s --property=Type=oneshot --property=RemainAfterExit=yes /usr/bin/true",
"probe_execution": "Result=success\nExecMainStartTimestamp=Sat 2026-10-24 20:27:07 AEDT\nExecMainExitTimestamp=Sat 2026-10-24 20:27:07 AEDT\nExecMainCode=1\nExecMainStatus=0\n",
"arm": "systemd-run --user --unit=fc-return-button-rollback-20261024 --on-active=180s --timer-property=AccuracySec=1s /bin/sh -c <guarded command>",
"guarded_command": "if [ \"$(systemctl --user is-active gamescope-session.target)\" != active ]; then exec /usr/bin/steamosctl switch-to-game-mode; fi",
"verified_before_desktop": "TimersMonotonic={ OnActiveUSec=3min ; next_elapse=1d 21h 53min 49.130174s }\nNextElapseUSecMonotonic=1d 21h 53min 49.130174s\nBindsTo=\nPartOf=\nActiveState=active\nExecStart={ path=/bin/sh ; argv[]=/bin/sh -c if [ \"$(systemctl --user is-active gamescope-session.target)\" != active ]; then exec /usr/bin/steamosctl switch-to-game-mode; fi ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }\nBindsTo=\nPartOf=\n",
"user_manager_linger": "yes",
"desktop_entry": "ssh frame steamosctl switch-to-desktop-mode",
"first_click_utc": "2026-10-24T09:27:27.037Z",
"cancelled_before_deadline": true,
"fired": false,
"limit": "The benign transient timer was observed executing. The actual rollback timer configuration was read back, then cancelled after the UI restored Game Mode; its recovery action was not deliberately allowed to fire."
},
"ui": {
"navigation": "Clicked nav Tools, Backup & recovery, and Check in the actual packaged Electron window. Initial Game Mode disabled Return. After full Plasma entry, Check enabled it.",
"automation": "Playwright connected to the packaged Electron instance over local CDP and used locator.click() on the visible button; accepted the native JavaScript confirm dialog. No Return API/CLI or production handler was directly invoked by the test. T3 preview was inspected separately and closed; it cannot control this Electron window.",
"clicks": [
{
"isTrusted": true,
"utc": "2026-10-24T09:27:27.037Z"
},
{
"isTrusted": true,
"utc": "2026-10-24T09:27:32.942Z"
}
],
"button_bounds_css_pixels": {
"x": 47,
"y": 709.59375,
"width": 156.203125,
"height": 44
},
"confirmation_dialogs": [
{
"type": "confirm",
"message": "Return the headset to Game Mode?\n\nThis closes the desktop and any apps open in it, then starts Steam's Game Mode.",
"utc": "2026-10-24T09:27:27.042Z"
},
{
"type": "confirm",
"message": "Return the headset to Game Mode?\n\nThis closes the desktop and any apps open in it, then starts Steam's Game Mode.",
"utc": "2026-10-24T09:27:32.943Z"
}
],
"desktop_check": {
"http": 200,
"body": {
"mode": "desktop",
"gamescope": true,
"plasma": true,
"nested_desktop": false,
"configured": "gamescope-wayland.desktop",
"config_file": "/etc/sddm.conf.d/steamos.conf",
"selector": "steamosctl",
"session_targets": {
"gamescope-session.target": "inactive",
"plasma-workspace-wayland.target": "active"
},
"mode_source": "session-targets"
}
},
"return_responses": [
{
"http": 200,
"body": {
"message": "Asked the headset to return to Game Mode; it restarts its session in a few seconds",
"state": {
"mode": "desktop",
"gamescope": true,
"plasma": true,
"nested_desktop": false,
"configured": "gamescope-wayland.desktop",
"config_file": "/etc/sddm.conf.d/steamos.conf",
"selector": "steamosctl",
"session_targets": {
"gamescope-session.target": "inactive",
"plasma-workspace-wayland.target": "active"
},
"mode_source": "session-targets"
},
"switched": true,
"command": "steamosctl switch-to-game-mode"
},
"utc": "2026-10-24T09:27:27.316Z"
},
{
"http": 200,
"body": {
"message": "The headset is already in Game Mode. The desktop you see is a panel: close it from the Steam dashboard.",
"state": {
"mode": "game",
"gamescope": true,
"plasma": false,
"nested_desktop": false,
"configured": "gamescope-wayland.desktop",
"config_file": "/etc/sddm.conf.d/steamos.conf",
"selector": "steamosctl",
"session_targets": {
"gamescope-session.target": "active",
"plasma-workspace-wayland.target": "inactive"
},
"mode_source": "session-targets"
},
"switched": false
},
"utc": "2026-10-24T09:27:33.126Z"
}
],
"final_check": {
"http": 200,
"body": {
"mode": "game",
"gamescope": true,
"plasma": false,
"nested_desktop": false,
"configured": "gamescope-wayland.desktop",
"config_file": "/etc/sddm.conf.d/steamos.conf",
"selector": "steamosctl",
"session_targets": {
"gamescope-session.target": "active",
"plasma-workspace-wayland.target": "inactive"
},
"mode_source": "session-targets"
}
},
"final_return_disabled": true,
"final_text": "Game Mode: the headset is running normally. A desktop you see is a panel; close it from the Steam dashboard.",
"screenshots": [
"return-game-mode-packaged-2026-10-24-before.png",
"return-game-mode-packaged-2026-10-24-after.png"
],
"screenshot_scope": "Cropped to the session controls and repeat-action notification; host identity, connection address and backup folder excluded."
},
"systemd_verification": {
"command": "ssh frame systemctl --user show gamescope-session.target plasma-workspace-wayland.target -p Id -p ActiveState -p SubState; ssh frame systemctl --user show gamescope-session.service -p InvocationID",
"before": "Id=gamescope-session.target\nActiveState=inactive\nSubState=dead\n\nId=plasma-workspace-wayland.target\nActiveState=active\nSubState=active\nInvocationID=892b843dbe1a4d2ba91b93057bd38463\n295589 /home/steamos/.local/share/Steam/steamrtarm64/steam -silent -vrdisable -deckard\n295066 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-294978\n",
"after_first_click": "Id=gamescope-session.target\nActiveState=active\nSubState=active\n\nId=plasma-workspace-wayland.target\nActiveState=inactive\nSubState=dead\nInvocationID=e8c1085b3bbd4e79a323b3e4a9bc6355\n297506 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-297420\n",
"after_repeat_click": "Id=gamescope-session.target\nActiveState=active\nSubState=active\n\nId=plasma-workspace-wayland.target\nActiveState=inactive\nSubState=dead\nInvocationID=e8c1085b3bbd4e79a323b3e4a9bc6355\n297826 /home/steamos/.local/share/Steam/steamrtarm64/steam -cef-enable-debugging -deckard -gamepadui -steamdeck -steamos3 -vrgamepadui\n297942 /home/steamos/.local/share/Steam/steamrtarm64/steam -child-update-ui -child-update-ui-socket 8 -cef-enable-debugging -deckard -gamepadui -steamdeck -steamos3 -vrgamepadui\n297506 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-297420\n",
"repeat_did_not_restart_game_mode": true,
"unchanged_game_service_invocation": "e8c1085b3bbd4e79a323b3e4a9bc6355",
"unchanged_steamvr_pid": 297506,
"delayed_final_readback_utc": "2026-10-24T09:28:05Z"
},
"cleanup": {
"complete": true,
"final_readback": "2026-10-24T09:28:05Z\nId=gamescope-session.target\nActiveState=active\n\nId=plasma-workspace-wayland.target\nActiveState=inactive\nInvocationID=e8c1085b3bbd4e79a323b3e4a9bc6355\nNEXT LEFT LAST PASSED UNIT ACTIVATES\n\n0 timers listed.\nId=fc-return-button-rollback-20261024.timer\nLoadState=not-found\nActiveState=inactive\n\nId=fc-return-button-rollback-20261024.service\nLoadState=not-found\nActiveState=inactive\n\nId=fc-return-button-probe-20261024.timer\nLoadState=not-found\nActiveState=inactive\n\nId=fc-return-button-probe-20261024.service\nLoadState=not-found\nActiveState=inactive\n297826 /home/steamos/.local/share/Steam/steamrtarm64/steam -cef-enable-debugging -deckard -gamepadui -steamdeck -steamos3 -vrgamepadui\n297506 /opt/steamvr/bin/linuxarm64/vrserver -waitformonitor -ipccontrolfilepath /tmp/SteamVR-IPCControlFile-297420\n99\nCharging\ngame\n",
"owned_lock_release_exit": 0,
"owned_lock_absent": true,
"host_processes": {
"xvfb": {
"pid": 1562020,
"exists": false,
"cmd": null,
"state": null
},
"electron": {
"pid": 1562023,
"exists": false,
"cmd": null,
"state": null
},
"x_socket_absent": true
},
"remaining_test_servers": [],
"battery_final_percent": 99,
"battery_final_status": "Charging",
"battery_range_observed_percent": [
99,
100
],
"game_mode_steam_steamvr_running": true,
"default_login_mode_after": "game",
"headset_installs_or_global_settings_changed": false,
"rebooted": false,
"notes": "Stopping the timer succeeded. systemctl stop also named its never-started, unloaded service and returned 5; explicit readback confirms both rollback and probe timer/service units are not-found/inactive. No further switch was sent."
},
"attempts_and_harness_limits": [
"Initial harmless timer probe was garbage-collected before its exit metadata could be read. No Desktop Mode switch was attempted. Repeated with RemainAfterExit=yes and verified successful execution.",
"First full Plasma attempt clicked the visible packaged Return button and got HTTP 200 / switched=true. The harness then treated absent pgrep matches during the expected session restart as a fatal error. Independent readback confirmed Game Mode and SteamVR had returned. Rollback remained armed until that readback, then was cancelled. This incomplete attempt was not counted as the final pass.",
"Final full Plasma attempt tolerated transiently absent Steam/SteamVR while polling systemd targets, passed both trusted visible clicks, and completed the final Check. Stopping the unused rollback service returned unit-not-loaded; explicit subsequent readback verified all four owned probe/rollback units not-found/inactive and zero timers. Cleanup was completed before releasing our lock."
],
"limits": [
"Unchanged AppImage payload ran after extraction on a virtual X desktop with Chromium sandbox and GPU disabled. FUSE mounting, desktop integration and hardware rendering were not tested.",
"Automated trusted pointer clicks in a visible Electron window; no physical-monitor human click or worn-headset visual/interaction assessment.",
"Verified only SteamOS 0.5.6 build 20261015.6141207 and the Linux x86_64 v0.4.17 package; older selectors/builds and other desktop platforms remain covered only by their existing tests.",
"No release, tag, application code, passwords or security/global settings changed."
],
"validation": {
"focused_suite": "python3 -m unittest discover -s tests -p test_backup.py -v: 28 tests passed in 2.479s",
"json": "python3 -m json.tool: passed",
"diff": "git diff --check: passed"
}
}
-39
View File
@@ -6,45 +6,6 @@ This is Frame Control code using Python, Steam and SteamVR already on the Frame.
![Family and comfort controls in the desktop app](img/comfort-desktop.png)
## Desktop low-battery alert without a session
In the desktop app, open **Tools → Family & comfort** and check **Alert on this
computer at 15% or less**. This separate option does not start a session, install
a worker or change the headset. It starts off whenever the page opens and turns
off when you switch headsets. The choice is only for the current open page.
It consumes the battery parser already used by normal status refreshes (also
shared by the read-only power sampler). It adds no SSH calls, timers, sensor
reads, third-party push or heat thresholds. Normal automatic status refreshes
pause while the page is hidden, so this is **not a background or safety monitor**.
Keep the app visible and connected. Use **Enable / test notifications** to check
system delivery; operating-system settings may block notifications.
Two fresh readings, at least 20 seconds and at most 90 seconds apart, must show
15% or less, driver state `Discharging`, and signed battery power below −0.5 W.
This avoids alerting on plugged-in idle readings, which can report discharging.
Missing, invalid, stale or interrupted readings break confirmation. Disconnect,
a failed status request, hiding the page or toggling the option also breaks
confirmation. Unknown readings never mean zero charge.
An alert appears in the page and requests a native desktop notification once
per low-charge episode. Two readings at 20% or more rearm it, and at least ten
minutes must pass between alerts (including across headset switches). A single
recovery spike cannot rearm it. Toggling off/on does not clear the episode latch.
Notification failure stays visible in the page and does not retry on every
refresh. While an active family session has battery alerts enabled, that session
handles battery alerts instead. Multiple app windows have independent state;
this does not coordinate notifications between computers or windows.
A read-only check on 2026-10-24, build **20261015.6141207**, received two samples
at 99%, `Not charging`, approximately −0.06 W and 24 °C. No low-battery condition
was induced, no notification was sent to the real headset, and no headset state
was changed. The sampler's local and remote smoke-lock checks allowed the run;
no lock was acquired or removed. Deterministic tests exercise low charge,
recovery, missing readings, idle drain, timing and notification denial. See
[recorded evidence](evidence/desktop-battery-comfort-2026-10-24.json). Native OS
notification display, worn use and macOS/Windows packaging were not verified.
## Sessions
Set a limit of 1–240 minutes, optional break and check-in intervals, then
+5 -61
View File
@@ -58,44 +58,6 @@ local SSH setup is unavailable. Panel verification currently means a connected
run normally exits 1. Automation should read the labelled checks rather than
treating a warning as proof the headset is unusable.
## Read-only repair plan and before/after reports
```sh
scripts/frame-doctor.sh --plan --save-report before.json
# Review the plan with the headset user. Any manual action is separate.
scripts/frame-doctor.sh --plan --compare-before before.json --save-report after.json
```
`--plan` prints advice; it never executes repair commands or prompts for approval.
There is no automatic repair mode. Advice about Wi-Fi, display timeouts, low
storage, a missing Steam install marker or keep-awake requires the exact
recognized section, severity and evidence. Other warnings request local review;
unknown evidence never justifies a repair. A signature or missing marker alone
does not establish a cause. Restart risk, rising counters or unknown counters
block all section-specific plans and direct you to the recovery runbook.
Nothing changes passwords, headset security, pairing, services or power settings.
Reports are optional **local** JSON files containing only fixed diagnostic
summaries, a format version and an opaque SSH-target configuration fingerprint.
The fingerprint binds comparisons to the same alias, identity/routing
configuration and fallback addresses; it does not prove physical headset
identity or that the headset was not replaced behind that configuration. Keep
using trusted SSH host keys. Report creation is exclusive (existing files and
symlinks are refused), with owner-only permissions on POSIX. Protect reports
using your local account permissions on Windows. Choose a fresh output filename
for each run. Doctor never writes a report on the headset.
`--compare-before` validates a bounded baseline before connecting and refuses
changed target configuration. It prints each section's before/after status and
prints both fixed evidence messages when they differ, even if the status
stayed the same. Changes are observations, not proof a repair worked or caused them;
recent journal windows can simply age out. There is no intervening repair by
doctor, freshness guarantee, or authenticated report signature. If either run
skipped deeper checks, comparison is blocked and no improvement is claimed.
Unreachable or invalid probes produce no new baseline or comparison. Malformed
baseline/target mismatch/report-write errors exit **2**; otherwise normal
diagnostic exit codes apply. No saved report is needed for ordinary diagnostics.
## First-time setup and pairing
- **No SSH response:** wake the headset with its power button. Check the cable
@@ -127,16 +89,15 @@ is the source for these section numbers and observed signatures (September
2026 builds). Its section 4 describes freezing and verifying health-check
trackers before prolonged recovery during a live loop. That changes headset
state, so this read-only implementation stops and directs you there instead.
The optional plan and baseline comparison are a guarded next step for issue
#34; automatic fixes and full recovery automation remain unimplemented. Review
build-specific recovery instructions with the person using the headset; a repair can remove data.
It does not implement issue #34's automatic fixes, before/after repair runs,
or full recovery automation. Review build-specific recovery instructions with
the person using the headset; a repair can remove data.
Never reload `ath12k`, suspend remotely, kill gamescope to diagnose a loop,
reset or force power off during extraction/repair, or write passwords to disk.
The doctor never restarts services, signals processes, repairs Steam, changes
WoWLAN/standby settings, takes keep-awake ownership, installs anything or writes
a remote report. Only `--save-report` creates a local file. Nothing needs
restoring on the headset after its checks. Manual/device tests must
a report. Nothing needs restoring after its checks. Manual/device tests must
follow the [shared-device lock and battery procedure](testing.md#headset-smoke-test).
## Privacy and verification
@@ -146,9 +107,7 @@ arguments, environment variables, SSH errors, account IDs, network addresses,
SSIDs, keys and tokens are not printed. No telemetry or HTTP request is sent;
the only pairing-service probe is a TCP connection. SSH produces normal access
logs. Review summaries before sharing them; feature availability can still be
personal information. Redirecting stdout or using `--save-report` saves a report
only if you choose it.
Treat the target fingerprint as private metadata; it is a hash, not encryption.
personal information. Redirecting stdout saves a report only if you choose it.
Unit tests use mocked boundaries. `scripts/e2e.sh test_doctor` covers asleep,
disk-full, SSH/devkit-off and missing-key faults against the fake Frame. Its
@@ -160,18 +119,3 @@ correctly skipped all eleven deeper sections and exited 1. No raw logs,
addresses or keys were printed, and no repair or restart was attempted.
This verifies the safety stop on that device; the deeper checks are not
claimed as headset-tested.
Follow-up verification on 2026-10-24: `--plan --save-report` and then
`--plan --compare-before` ran read-only on the same real Frame. Both reported
nonempty health-check trackers, skipped deeper checks and exited 1. The plan
was blocked and the comparison reported `risk -> risk` without claiming
improvement. No install, launch, stop or repair was performed; this validates
the guarded report path, not the deeper checks or any manual repair.
The follow-up's 17 focused offline tests passed. All five doctor fake-Frame
tests passed, including a disk-full baseline followed by fault removal and a
fresh comparison. The final evidence-output change also passed that comparison
test again. Because zsh is unavailable on this host, the repository's images
were built and its Compose harness run directly under the isolated project
`frame-doctor-plan-34`; its containers and volumes were removed afterwards.
These storage changes were fake fault controls, not doctor repairs.
Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+171 -2
View File
@@ -57,6 +57,25 @@ makes.
| Sleep, restart, shut down ask in a terminal | The page asks for the Developer Mode password |
| Compatibility reports kept on the computer | Kept on the Frame (`~/.local/share/Frame Control`) |
## Automatic screenshot downloads (in development)
The screenshot-download implementation on this branch copies new Steam screenshots
while Frame Control is open and connected. It starts with the first successful
screenshot listing as its baseline; older captures still use the existing Save
action. Originals remain inside the app, and **Save screenshots to Photos** is
enabled by default, subject to add-only Photos permission.
Open **Downloads & Photos** on the Screenshots page, or **Screenshot downloads**
on the native connection screen. Turn the Photos setting off to keep new downloads
only in Frame Control. Re-enabling it does not export those app-only captures.
The local list and its Share buttons work without a headset connection. Failed
Photos exports retain the original and retry while the app is open; an interrupted
Photos transaction needs a manual check before saving again to avoid duplicates.
Background notification delivery and Android support are not implemented by this
change, and it has not been released. See the [verification record](evidence/phone-screenshot-sync-2026-10-24.md)
and [remaining work](https://github.com/saphid/frame-control/issues/108).
## Building
```sh
@@ -69,6 +88,88 @@ The build packs the Frame bundle from the checkout, so the phone always runs
the page and server from the same commit. Running on a phone needs your own
signing team in Xcode (Signing & Capabilities).
## TestFlight
Join the [Frame Control beta](https://testflight.apple.com/join/9xjW91g6),
install Apple's TestFlight app, then tap **Install** or **Update** next to Frame
Control. Internal account testers can also use their invitation. The first
working phone build was version 0.1.0, build 202610140516, uploaded on
2026-10-14. On 2026-10-16, build **202610160824** was uploaded and approved for
internal and external testing. App Store Connect confirmed both builds were in
the external **Public Beta** group, with its public link enabled and a limit of
50 testers. The new build declares the encryption-compliance answer in the app
and preserves the previous build's exact server bundle.
### Upload a build
You need Xcode, XcodeGen, a paid Apple Developer Program team, an App Store
Connect app record for `com.saphid.framecontrol`, and an App Store Connect team
API key with the signing access your release setup needs. This Mac relies on
Apple's cloud-managed distribution certificates: an **App Manager** key was
rejected with `Cloud signing permission error`, so this setup needs **Admin**
access. A setup with a local distribution signing identity may be able to use
a narrower role. Create the key under **Users and Access →
Integrations → App Store Connect API**. Download it directly to
`~/.appstoreconnect/private_keys/AuthKey_<KEY_ID>.p8`, outside the repository,
and restrict it to its owner (`chmod 600`). Apple allows one download. Keep
the private key out of chat, screenshots, logs and Git.
From the repository root:
```sh
ASC_KEY_ID=YOUR_KEY_ID \
ASC_ISSUER_ID=YOUR_ISSUER_UUID \
ASC_KEY_PATH="$HOME/.appstoreconnect/private_keys/AuthKey_YOUR_KEY_ID.p8" \
TEAM_ID=YOUR_TEAM_ID \
ios/scripts/testflight.sh
```
The script regenerates the Xcode project, archives with automatic signing,
checks the archived build number, and uploads to App Store Connect. `TEAM_ID`
defaults to this project's team, `NJZUMEA4BN`. `BUILD_NUMBER` defaults to UTC
`YYYYMMDDHHMM`; wait for a new minute between uploads, or supply a unique,
higher build number. The archive, export options and export output are under
`ios/build/testflight/<BUILD_NUMBER>/` (ignored by Git).
Build from a reviewed, clean checkout. The build packs the server, web pages
and helpers from the current checkout, including uncommitted changes; it does
not limit the bundle to the last commit.
An upload succeeding means Apple received the build. Wait for it to finish
processing in **App Store Connect → Frame Control → TestFlight** and check
that it is ready for testing. `ios/project.yml` declares
`ITSAppUsesNonExemptEncryption: false`, so each generated app declares that it
uses no non-exempt encryption, matching the previous builds' export-compliance
answers. Reassess that declaration if the app's cryptography changes.
To replace a key, create and securely download the replacement, verify that it
can access this app, then revoke the old key in Users and Access. Update the
upload environment to use the replacement. Revocation does not remove an
already uploaded TestFlight build from testers' phones.
### Add testers
**Internal:** add an existing App Store Connect user with an eligible role to
an internal testing group, and add the build to that group (or enable automatic
distribution). New account users must first accept their account invitation.
Internal testing grants App Store Connect access; use external testing for
people who only need to try the app.
**External:** create an external testing group, complete Beta App Information
and review contact details, add a build and its **What to Test** notes, then
submit it for TestFlight App Review when required. Use
<https://framecontrol.fun/privacy/> for the privacy policy and
<https://framecontrol.fun/feedback/> for feedback. Review notes must explain
that the app requires a Steam Frame with Developer Mode enabled and a user
password; do not provide your own headset credentials to reviewers. Apple may
need a way to evaluate features that require the hardware.
After Apple approves the build for external testing, enable the group's public
link and choose a tester limit. Share that link to let testers join without an
App Store Connect account. Creating a group alone does not make a build public.
Do not describe the link as ready until the approved build is in the group and
the link is enabled.
## Verified
<img src="img/iphone-tabs.jpg" alt="The four tabs in the iPhone app, connected to a Frame" width="900">
@@ -106,8 +207,76 @@ Sending text to the Frame's clipboard, or copying the Frame's to the phone
(**Type on the headset → Copy the headset's clipboard**), needs the desktop
panel open in the headset, as on the desktop app.
Not yet exercised: Android display changes through podman (no Android app was
running), a real sleep/restart/shut down on the Frame, and a physical iPhone.
On a physical iPhone, 2026-10-14: Alex reported build 202610140516 was working
after pairing and trying the password visibility button. A server process
started by the phone and its SSH key on the Frame confirmed the connection.
This was a setup/connection smoke check; it did not verify every app feature.
On a physical iPhone 16 Pro running iOS 27.2, 2026-10-16: the same build launched
and automatically reconnected to the Frame. A device screenshot showed the
green connection indicator, 100% headset battery and the loaded Android
catalog. This confirms connection and rendering, not status refresh or actions
inside the app. A separate signed automation runner was installed and passed
its health check. Initial interaction attempts were blocked by the phone's
lock; testing resumed once Alex unlocked it and kept the screen awake.
<img src="img/iphone-physical-20261016.jpg" alt="Frame Control connected to the Frame on a physical iPhone, showing battery and the Android catalog" width="360">
In that continuation, still on build **202610140516**, the status header
refreshed, the Games tab loaded 14 installed titles with sizes and Play
controls, and a uniquely named 114-byte text file was selected through the
iPhone's Files picker and sent to `~/Downloads` on the Frame. An SSH readback
matched the file's contents exactly; this verifies phone-to-Frame upload,
not a download back to the phone. Both the iCloud source fixture and the
uploaded test file were removed and checked absent.
Capture returned a blank headset frame with an asleep/unworn explanation.
Save opened the native share sheet and reached the Photos permission prompt;
no completed Photos save was verified. A final Settings check showed
**Add Photos Only** enabled, without photo-library read access. A test string
sent through the clipboard control was refused with the expected prerequisite:
`plasmashell` was not running, so the desktop panel must be opened in the
headset. The phone's existing clipboard was not read. Live mode started and
reported **30 fps**, but its visible frames remained blank, so meaningful
video playback is still unverified. Android listed 10 installed apps; a
separate SSH check found zero running containers and zero Android ADB
listeners, so display-setting changes were not exercised.
<img src="img/iphone-games-physical-20261016.jpg" alt="Frame Control's Games tab on the physical iPhone, showing the installed library and Play controls" width="360">
<img src="img/iphone-live-physical-20261016.jpg" alt="Live mode reporting 30 fps on the physical iPhone with blank headset content and the page horizontally clipped" width="360">
Testing also exposed an enlarged, horizontally clipped page after text entry
and navigation. Its cause has not yet been isolated. Restarting Frame Control
restored the normal layout and automatically reconnected to the headset.
### Mac follow-up
On 2026-10-17, the installed Mac app **0.4.6** was tested with the headset
unworn. Connection/status worked, and its connection test reported
`Answered in 90 ms · SSH works`. Still capture completed but both eyes were
blank. Live video failed with `Not a video capture device` when opening
`/dev/video99`, then fell back to blank still captures. No visible scene or
video playback was verified. The desktop was inactive; no headset services
or settings were changed. Live was stopped and the existing app left on Home.
These are Mac results; they do not verify the iPhone build or establish the
cause of its earlier blank output.
<img src="img/mac-live-error-20261017.png" alt="Mac Frame Control activity log reporting Live video failed: Not a video capture device" width="900">
### Physical-phone follow-up checklist
- With the headset awake and worn, capture visible headset content, save it to
Photos, then verify live video.
- Send test text to the Frame's clipboard with its desktop panel open.
- With an Android app running, change and restore its display settings.
- Reproduce and fix the page enlargement and horizontal clipping after text
entry and navigation.
- After saving headset work, deliberately test sleep, restart and shut down.
These remaining checks are pending on a physical phone. Android display
changes through podman and real sleep/restart/shut down also remain unverified
by the Simulator checks above.
Debug builds have Simulator test hooks (`FRAME_TEST_HOST`, `FRAME_TEST_PAGE`,
`FRAME_TEST_JS`, and the tunnel URL in the app's Caches folder); release builds
+112
View File
@@ -0,0 +1,112 @@
# Mac space: the Mac's desktop and its windows, placed in the room
Design notes, started 2026-09-28. The goal is Apple Vision Pro's Mac Virtual
Display, plus something Vision Pro can't do: pulling any Mac window out
to float on its own. Labels as in [ssh.md](ssh.md): **verified**,
**documented**, **reported**, **inferred**.
## What to match (Vision Pro's Mac Virtual Display)
Researched 2026-09-28. Sources: Apple Support
[118521](https://support.apple.com/en-us/118521) and the
[Vision Pro guide](https://support.apple.com/guide/apple-vision-pro/use-mac-virtual-display-tan357ede966/visionos);
[9to5Mac](https://9to5mac.com/2024/01/30/heres-what-its-like-using-vision-pro-as-an-external-mac-display/),
[UploadVR](https://www.uploadvr.com/visionos-2-2-ultrawide-mac-virtual-display/)
and [AppleInsider](https://appleinsider.com/inside/apple-vision-pro/tips/how-to-use-mac-virtual-display-for-apple-vision-pro-with-desktop-macs-and-macbooks).
- **Connecting.** Look at the Mac and a **Connect** button floats above it.
There's also a Control Center route. The Mac's own screen goes dark
(**documented**/**reported**).
- **Placement.** The display appears where you were looking, near the Mac,
at a large-monitor size. After that it stays put in the room: moving the
laptop doesn't move it. Holding the Digital Crown recenters everything
(**documented**).
- **Handling.**
- A **window bar** under the display moves it, with pinch and drag, and
also moves it nearer or farther.
- Its bottom **corners** resize it.
- **Close** sits to the left of the bar (**documented**).
- **Modes.** Standard (16:9), Wide (21:9) and Ultrawide (32:9, about 10K
pixels across). All are curved since visionOS 2.2 (**documented**).
- **Input.**
- The Mac's keyboard and trackpad drive it.
- The pointer moves to whichever window you look at (Universal Control).
- Mac audio plays in the headset (**documented**).
- **What it can't do.** You can't pull a single Mac window out of the
virtual display (**reported**, several sources). That's our
differentiator.
## What the Frame allows
- **Moving SteamVR's panels from our own process: no** (**verified**
2026-09-28). `FindOverlay("valve.steam.desktopgame.<id>")` works from a
separate OpenVR app, but `SetOverlayTransformAbsolute` returns 12
(`VROverlayError_PermissionDenied`). The panel reported transform type 5
(dashboard tab). Only SteamVR's dashboard, which runs in `vrwebhelper`,
moves them, through its privileged `VRHTML.VROverlay`, and it has no
DevTools port. See `/tmp/fmv-space/ovprobe.py` for the probe, which stays
outside the repo.
- **WebXR in Chromium XR: yes** (**verified** 2026-09-26/27,
[webxr-chromium.md](webxr-chromium.md)).
- `immersive-vr` runs at 72–75 fps.
- Passthrough shows wherever the page leaves pixels transparent.
- Controllers are `tracked-pointer`s with `xr-standard` gamepads, and
squeeze events arrive.
- Hands are tracked with 25 joints.
- **Our own OpenVR overlays: possible, but most work.** We would own their
transforms and get laser input as mouse events for free. But the Frame
would need native H.264 decoding and texture upload.
## Decision: a WebXR "Mac space" page first
The WebXR page (`ui/mac-space.html`) holds everything, with passthrough
behind it:
- **The desktop.** A large curved panel shows the Mac's main display,
placed above the laptop. The first time, you place it yourself; after
that it's remembered in the room.
- **Windows.** Each separated window gets its own panel.
- **Grabbing.** A bar under each panel moves it (squeeze, or pinch with a
hand). Its corners resize it, and there's a close button to the bar's
left, as on Vision Pro.
- **Pulling a window out.** Grab a window's title bar on the desktop panel
and drag it off the edge. The page starts a `separate:<window id>`
stream, the window moves to its own display on the Mac, and its new
panel follows your hand until you let go. Dropping a panel back onto the
desktop panel stops the stream, and the window goes back.
- **Input.** The laser and trigger click and drag, the thumbstick scrolls,
and the Mac's keyboard types, as today.
Trade-offs:
- It's the headset's active app while you use it. Other VR apps pause, but
SteamVR's own overlays still draw on top.
- Text sharpness depends on how the page renders. WebXR quad layers (the
`layers` feature) would let SteamVR's compositor sample each panel
directly, so they're worth checking first.
## Building blocks
- **One module for the protocol.** `ui/mac-stream.js` holds the stream
protocol (connect, clock sync, decoding, acks and reports, input), shared
by `mac-view.html` (the 2D panel) and `mac-space.html`.
- **A space key from the agent.** Frame Control gets a ticket with
`src=space`, and the page trades it once for a **space key**, kept in
memory. That key lists the windows and displays (with their positions,
for picking the one under the pointer) and opens streams for any of
them. Stop All revokes it.
- **Anchoring.**
- v1: manual placement, remembered in the room (to be checked: whether
`local-floor` stays put between sessions on the Frame).
- v2: the Mac shows a marker on its screen for a moment, and the headset
finds it. That needs camera access, which is **unknown** on the Frame:
the passthrough cameras are monochrome, and `IVRTrackedCamera` hasn't
been tried.
## Open questions
- Frame rate with several software-decoded streams at once.
- Whether `local-floor` placements survive restarts.
- Whether WebXR quad layers work in Chromium XR's OpenXR path.
- Whether Chromium grants `requestSession` without a click in the page
(needed for a one-click "Open Mac space").
+1 -18
View File
@@ -190,24 +190,7 @@ HTTP focus is not evidence of worn-headset laser input. Follow the
## Saved spatial layouts: blocked on the current panel route
**Rechecked 2026-10-24**, SteamOS 0.5.6, BUILD_ID `20261015.6141207`,
SteamVR 2.18.2: a read-only probe of the existing Gamescope and Steam main
panels still reports type 5 (`DashboardTab`), and both the absolute and
tracked-device-relative getters return 18 (`WrongTransformType`). Width and
curvature read successfully. **Some pose readback is available:**
`GetTransformForOverlayCoordinates` at the panel center succeeds in standing
space; seated and raw origins return 23 (`RequestFailed`). Both panels and the
dashboard were hidden. That matrix snapshot does not identify docking, prove
visible placement, or establish a restore operation. The shipped dashboard
stores dock location and relative transforms internally; no supported external
snapshot/restore pair was established. No setters or docking commands were run
in this recheck. See the [exact probe, results, cleanup and remaining
gaps](evidence/panel-layout-2026-10-24.md). Saved layouts remain unavailable.
### Earlier mutation evidence (2026-09-28)
**Verified 2026-09-28**, SteamOS 0.4.1, BUILD_ID `20260925.6191901`,
SteamVR 2.18.1, using a temporary xterm panel with
**Verified 2026-09-28**, same build, using a temporary xterm panel with
`STEAM_GAME=2000999031` and `FnTable:IVROverlay_028` from
`/opt/steamvr/bin/linuxarm64/libopenvr_api.so`:
-6
View File
@@ -107,12 +107,6 @@ The Linux adapter from #38 (desktop portal, PipeWire, VA-API) is in the tree,
because it shares `capture.c` and the controller, and CI still builds and tests
it. The app never offers it and the Linux installers don't include it.
The [2026-10-24 lxso2 feasibility check](evidence/linux-pc-host-lxso2-2026-10-24.md)
found no running or activatable desktop portal, no PipeWire socket and missing
native build dependencies. Linux support remains disabled. The evidence records
the build failure and prerequisites for a screen-only, view-only x264 slice;
Linux capture, input and window modes have not been verified there.
## Measuring
The Mac benchmark accepts the PC host:
+2 -104
View File
@@ -24,89 +24,6 @@ The first sample has no CPU utilization: two valid `/proc/stat` observations
are required. CPU utilization then covers the interval between observations,
averaged across the machine, rather than the utilization of one process.
## Start, stop and open an accessible report
From a source checkout with Python 3 and your already trusted `ssh frame`
connection, start a local observation:
```sh
scripts/frame-power-sample.sh --count 12 --interval 5 --html power-report.html
```
Progress is printed on your computer as each sample arrives. Wait for completion,
or press **Ctrl+C once** to stop local collection and save the samples already
received. Open `power-report.html` in your browser using its Open File command.
The file is self-contained and works offline: no server, scripts, external
resources, account or upload. It includes a table with named metrics, source
paths, units, minimum/mean/maximum/last values, and available/unknown counts.
Keyboard users can focus the table region to scroll through its columns.
Charging states and entirely unavailable sensor families remain explicit.
Use **Download JSON samples and summary** or **Download CSV samples** to save
raw measurements. The HTML also embeds those measurements; review it before
sharing. No automatic browser launch occurs.
HTML sessions are limited to `count × interval <= 600` seconds, with a 30-second
transport allowance and at most twelve seconds of local child/pipe cleanup. The default
12 samples at five-second intervals take about 55 seconds. Existing output files
are never overwritten; choose a new filename for each run. `--html` cannot be
combined with `--csv`. The desktop flow below uses the same sampler and renderer.
Stop, timeout, SSH failure or a smoke-test lock marks the report incomplete;
earlier received samples remain downloadable. Exit status is 75 for incomplete
reports, 0 for completion. A connection that cannot start may instead return 1
without an HTML file. Empty reports clearly show zero samples, not a successful
measurement. If saving the file fails, received JSON is printed on stdout as a
recovery path. Do not treat a partial report as a full baseline.
Stopping terminates only this command's local SSH child. It sends no remote
process-control command. The bounded remote Python reader exits when it next
writes to the closed connection (normally at the next sample, up to one interval
later); it can still read sensors before discovering the disconnect. Stop is
not an immediate headset-side cancellation guarantee. The reader still checks
the smoke-test lock before every sample. No lock is acquired or removed by this
flow, and no power, thermal, service, password or security controls are exposed.
## Desktop Tools → Power & heat report
In the packaged desktop app, connect to an already paired Frame, open **Tools →
Power & heat report**, choose **1 minute** (12 samples) or **5 minutes** (60 samples),
and select **Start observation**. Samples arrive every five seconds, starting at
zero (about 55 or 295 seconds total). Progress shows received samples and the
original headset alias. There is at most one observation per local app server;
viewing progress never starts another SSH connection. No terminal, checkout,
extra dependency or password is needed.
Use **Stop and keep samples** to end local collection. Disconnect, timeout,
app shutdown or switching to another headset also ends collection. Switching
network addresses for the same headset does not retarget an existing observation:
it keeps its captured SSH route and may finish incomplete if that route fails.
As with the CLI, only the owned local SSH child is terminated. The remote reader
is bounded and discovers a closed connection on its next write. Desktop sessions
have a 30-second transport allowance and bounded child cleanup. Stream lines are
limited to 262,144 characters, sample events to the requested count, and the receive queue
to count + 2 events; desktop SSH stderr is discarded rather than stored unbounded.
The existing smoke lock is checked locally before start and remotely before each
sample; this feature never creates, bypasses or removes that lock.
**Download HTML report**, **Download JSON** and **Download CSV** save local
snapshots through the browser/app download mechanism. Open the downloaded HTML
in your browser to view the offline table and its embedded JSON/CSV downloads.
Snapshots taken before completion explicitly say incomplete. Desktop CSV includes
`collection.state`, `collection.requestedSamples` and `collection.receivedSamples`
metadata rows with the original headset alias in the source column, even when no
sensor sample arrived. CSV text fields, including the alias, are escaped if they
begin with a spreadsheet formula prefix. JSON records the alias and device ID;
HTML names the alias.
Only the latest observation is retained in server memory, including partial
samples after stop or disconnect. **Download before starting another observation,
restarting the server or quitting the app**: those discard the previous report.
Page reloads retain server state. Download requests include an observation ID, so
an old page cannot silently download or stop a replacement observation. Reports
are not uploaded or automatically opened, and may contain device identifiers,
OS builds and sensor paths. Review before sharing. The phone-hosted UI does not
support starting observations.
## What it reads
- Battery percentage, driver charging state, signed battery watts and battery
@@ -192,33 +109,14 @@ Quiet mode, compositor changes, CPU/fan/charging changes and remote-wake tests
are separate work. No controls are offered here. Follow the shared-device
procedure in [testing.md](testing.md#headset-smoke-test) for future experiments.
The accessible HTML export was checked on the same build on **2026-10-24**:
three streamed samples, 91 metric summaries and 276 CSV data rows. Chromium
preview checks at 1280 px and 390 px confirmed the table stays in its scroll
region; both browser downloads matched the original samples. See
[export evidence](evidence/power-export-2026-10-24.json). Ctrl+C retention and
local process cleanup were tested with local subprocess fixtures, not on the
real headset. Screen-reader, Windows and macOS checks were not performed.
Offline validation:
```sh
python3 -m unittest discover -s tests -p 'test_power*.py' -v
python3 -m unittest discover -s tests -p test_power_sample.py -v
bash -n scripts/frame-power-sample.sh
python3 -m py_compile scripts/frame_power_sample.py scripts/frame_power_report.py
python3 -m py_compile scripts/frame_power_sample.py
```
Fixtures cover units, signed and zero values, missing/malformed/permission-denied
readings, named/discovered channels, unknown charging state, CPU warmup,
summary counts, CSV quoting, strict SSH transport and smoke-lock interruption.
Export fixtures also cover HTML escaping, download round trips, zero versus unknown,
stream completion/failure, smoke-lock stop, timeout, Ctrl+C retention, local child
cleanup, duration bounds and refusing to overwrite reports.
Desktop validation on **2026-10-24** covered API start/stop, a synthetic browser
observation with partial exports, shared-module resource staging under isolated
Python and missing-resource build guards. A real Frame probe under this task's
owned smoke lock correctly returned zero samples with an explicit incomplete
reason; no successful live sensor baseline or native installer run was performed.
See [desktop evidence and limitations](evidence/power-desktop-2026-10-24.json).
+16 -62
View File
@@ -8,13 +8,13 @@ Both are on **Tools → Capture & Live**. Everything is saved to
Nothing needs installing on the computer or on the Frame: the work is done by
the `ffmpeg` that SteamOS ships.
**Recording and Steam clip export have each been tested on a headset.** On a Frame
**Recording is tested on a headset; Steam clip export isn't yet.** On a Frame
running SteamOS 0.5.6 (build 20261015.6141207), Record and Stop saved an
8-second 1080p H.264 recording with AAC sound to the computer, Stop took about a
second, and nothing was left running or stored on the Frame. A later test on
that build created a saved clip with Steam's own recorder and exported it with
Frame Control's production export function. The parts we're least sure of are
listed under [What hasn't been checked](#what-hasnt-been-checked).
second, and nothing was left running or stored on the Frame. That Frame had no
Steam recordings, so the clip list came back empty and export has only been
tested with files in Steam's format made by `ffmpeg`. The parts we're least sure
of are listed under [What hasn't been checked](#what-hasnt-been-checked).
## Record
@@ -147,65 +147,19 @@ stand-in `ffmpeg`, so Stop, the time limit and the clean-up are tested as they
run on the Frame. Where `ffmpeg` is installed, they also make a 65-second DASH
recording with Steam's file names and export it with the real `ffmpeg`.
## Real Steam clip verification and remaining checks
## What hasn't been checked
On 2026-10-24, v0.4.15 on build `20261015.6141207` recorded a ten-second
reserved test session through Steam's own game recorder, saved it as a clip,
then exported it with `frame_clips.export_clip`, the same implementation used
by the desktop API. The 159,829-byte MP4 decoded completely: 10.29 seconds,
1920×1080 H.264 video and stereo AAC audio (SHA-256
`f8a476eefb2bde7ba5cb464632ec9b952eea3ab1610b5bed2098bf1f11d62cf1`).
All 616 decoded video frames and the decoded audio matched Steam's source
segments. Recording mode was restored
to off; the reserved title and its processes were removed, the shared test lock
was released, and the battery remained at 99%. The Steam source clip was kept.
The 2026-10-24 read-only clip list on build `20261015.6141207` again returned
zero clips: its `gamerecordings` directory held only `gamerecording.pb`. No
Steam recording was created during that check, so a real Steam clip export is
still unverified. The recording and Stop path were verified earlier on that
build, as described above. Remaining checks:
The inspected frame is nearly black in both Steam's source and the export, so
this establishes file creation and faithful export, **not visible gameplay
capture**. On v0.4.16, the literal desktop **Export** button was also clicked
against that retained real Steam clip in the source UI on lxso2. The job
completed and saved `~/Videos/SteamFrame/clip_13009719723623972864_20261024_072115.mp4`;
its 159,829 bytes and SHA-256 matched the previously decoded export exactly.
The temporary server stopped, the shared device lock was released, and the
headset remained in Game Mode at 99% battery. See
[`docs/evidence/clip-gui-export-2026-10-24.json`](evidence/clip-gui-export-2026-10-24.json)
for the narrow proof. This did not exercise the packaged desktop installer.
On 2026-10-24, the exact v0.4.17 tag exported a second real Steam-saved clip
through `frame_clips.export_clip`, the same production path used by **Export**.
Steam listed the clip at **100.801 seconds**, with 68 chunks and no gaps. The
copied MP4 was **100.833 seconds** by `ffprobe` and decoded completely with
`ffmpeg`; it was not cut at 60 seconds. The test restored Steam's recording
mode, removed its reserved title and processes, and left Game Mode active at
100% battery. See [the long-clip evidence](evidence/clip-long-2026-10-24.json).
That code-path test did not click **Export** in the packaged desktop app or
verify visible gameplay quality or perceptual audio/video sync.
Later that day, the published v0.4.17 x86_64 Linux AppImage also passed the
packaged desktop **Export** check against the retained 100.801-second clip.
The asset SHA-256 was verified before launching its unchanged extracted payload
in an isolated Electron profile on a virtual X desktop. An automated pointer
click on the visible **Tools → Capture & Live → Steam clips → Export** button
saved the MP4 and changed the row to **ON PC / Export again**. The saved file
was 100.833335 seconds, decoded completely, and matched the prior production
export by size and SHA-256. The app and test display were closed,
the Frame export cache was empty, the source clip was retained, and the shared
lock was released at 99% battery with Game Mode and SteamVR still running.
See [the packaged export evidence](evidence/clip-packaged-export-2026-10-24.json)
and its cropped before/after screenshots. This verifies the packaged Electron
button and bundled export path; AppImage FUSE mounting, desktop integration,
physical display/GPU rendering, gameplay quality and perceptual sync remain
unverified. The virtual-display launch used `--no-sandbox --disable-gpu`.
Remaining checks include:
- Perceptual audio/video sync and audible content. Frame Control's separate
**Record** path stamps camera and audio inputs with the wall clock because
they otherwise count from different clocks.
- Clips longer than the 100-second test and trims starting partway through a
Steam recording. Two real saved clips' directory layouts were checked; the
parser remains deliberately loose
for other layouts.
- Audio and video sync. Both inputs are stamped with the wall clock, because
the camera device and the audio server otherwise count from different clocks.
- The exact layout of `gamerecordings` on the Frame's SteamOS build, and
whether a clip's segments start mid-recording. The parser is deliberately
loose about the layout.
- How much x264 at 1080p30 costs the Frame while a game is running.
- Automatic copying: that 30 seconds without a write is a safe sign Steam has
finished saving a clip, and how a once-a-minute listing feels on the Frame.
+1 -4
View File
@@ -14,10 +14,7 @@ count. So a build reaches people only when you publish it, after testing it.
`.github/workflows/release.yml` builds macOS, Windows and Linux, and
attaches everything to a **draft** release for that tag. Nobody is
offered a draft. Editing a draft while installers are uploading can make
GitHub temporarily show its tag as `untagged-…`. The workflow finds that
draft by its exact version title, restores the tag association and retries
the affected upload; it refuses ambiguous matches.
offered a draft.
2. Download the draft's installers and test them. An installed copy of the
previous version won't offer the draft, so install it directly.
+4 -39
View File
@@ -7,9 +7,6 @@ This covers three directions, plus input:
- **C. iPhone → Frame**: mirror the phone inside the headset.
- **PC VR from Linux**: [feasibility and options](linux-vr-streaming.md),
including Valve's streaming and USB support. No Linux host tested yet.
- **Linux desktop → Frame panels**: [PC-host feasibility on lxso2](evidence/linux-pc-host-lxso2-2026-10-24.md).
Blocked on missing desktop portal/PipeWire and native build dependencies;
Linux desktop capture remains disabled and unverified.
- **Input**: type and point in the Frame from the Mac or iPhone.
- **Live view and Control**: watch a panel flat and tap on it to use it.
@@ -154,31 +151,10 @@ two fingers to scroll, two-finger tap to right-click) plus a text field that
types on the Frame. On a computer, clicking the pad passes your mouse and
keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac).
Since Control (below), the pad goes through **gamescope's own input socket**
([`ui/frame_touch.py`](../ui/frame_touch.py)), so it reaches whichever panel has
focus, Steam's own menus included, and needs nothing installed on the Frame.
Only characters that aren't on a US keyboard (accents, emoji) go through
**KDE Connect** as below; it starts the first time you type one. (Before
Control, everything went through KDE Connect, which is why the notes below
describe it in detail.)
The server keyboard queue holds at most 200 events, including text being sent.
A batch with an accent sends its whole keyboard part through KDE Connect, and
later keys follow it there while typing is pending. If the batch will not fit,
the server rejects it before sending any clicks or keys; the page keeps it and
retries after polling. Accepted events are never evicted to make room. A timeout
or helper failure retains unsent text and shows **Try again**; that button resumes
the retained queue. The queue lives in memory and is lost if the server exits.
Switching headsets cancels queued text and stops both input helpers before the
new route is published. Both controls validate the intended headset before they
start. Ordering between the two input services uses a typing-time estimate;
neither service acknowledges when characters reach the application.
[PR #66 verification on 2026-10-24](evidence/pad-gamescope-2026-10-24.md)
records an owned focused X11 panel receiving a click and `Frame66` plus Enter
on a real Frame with SteamOS 0.5.6, after the shared headset became available.
The helper, window and owner-marked lock were cleaned up. This is ASCII input
proof, not wearer-visible, Unicode or packaged Electron GUI verification.
It goes through **KDE Connect**, the first-party route (KDE makes the Frame's
desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py)
on the Frame, which talks KDE Connect's own LAN protocol to the Frame's
`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking.
**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901):
@@ -309,17 +285,6 @@ How (**verified 2026-09-29**, SteamOS 0.4.1, build 20260925.6191901):
[#73](https://github.com/saphid/frame-control/issues/73) (SteamOS 0.4.5, desktop
open but not found). The likely cause there was the page itself: a second
`loadPanels` (the panel switcher's) replaced the Desktop view's, fixed in #84.
If the `GAMESCOPE_FOCUSABLE_WINDOWS` property cannot be read, discovery now
reports that the panel list is unreadable and whether a panel is open is
unknown. A failed X connection or missing property is not evidence that the
desktop is closed. Check that SteamVR is running, retry or use Headset view;
if it persists, report the exact message and SteamOS version. The confirmed
empty-list advice is retained when the property is present but empty.
[2026-10-24 investigation and verification limits](evidence/panel-discovery-2026-10-24.md).
A later [packaged v0.4.17 test on the real Frame](evidence/desktop-capture-packaged-2026-10-24.md)
captured a mapped, focused nested KDE compositor on SteamOS 0.5.6. SteamVR
still reported its overlay hidden, so that test does not establish the
wearer's visible-desktop condition in #73 or justify closing it.
- Tested from the iPhone app (Simulator): a tap on the Desktop view focused a
text box in the panel and the text field typed into it; a trackpad move went
exactly (+40, +25).
+20 -3
View File
@@ -1,11 +1,28 @@
# Tailscale: use the Frame from anywhere
# Tailscale: advanced remote-access setup
With Tailscale on the Frame, the `frame` SSH alias works off the home LAN, and
so does everything built on it: Frame Control, the scripts and the Mac app.
Tailscale is intended to make the `frame` SSH alias available off the home LAN,
including to Frame Control and the scripts. Off-LAN access has not yet been
verified; the checks below cover the direct LAN path only. Read the exposure
section before enabling it: services normally restricted to loopback become
reachable by other devices on your tailnet.
When the Mac and the Frame are on the same network, Tailscale connects them
directly, so there's no relay in the way (`tailscale ping frame` → `via
192.168.1.50:41641`, 20 ms).
These are source-checkout commands for macOS, not packaged-app menu actions
or Windows PowerShell commands. The connection script currently uses macOS
netcat options; this route is not supported on Linux. Install Git, OpenSSH,
zsh, Python 3 and curl on your Mac; the desktop app's bundled Python is not
available to these scripts. Then obtain the scripts:
```sh
git clone https://github.com/saphid/frame-control.git
cd frame-control
```
Enable Developer Mode on the awake headset and establish working SSH access
first using the [connection guide](frame-control.md). Run from the checkout:
```sh
scripts/tailscale-on-frame.sh # install or update, then approve the login URL
scripts/connect.sh frame.<tailnet>.ts.net # point the alias at Tailscale (the script prints this)
+3 -28
View File
@@ -12,16 +12,6 @@ A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/ste
## Unit tests
The gamescope keyboard/trackpad route has focused suites in `test_input.py`,
`test_touch.py` and `test_service_regressions.py`. They cover pad translation,
ASCII/modifier keys, Unicode queuing/retries, stale panel targets, release
handling, route cancellation, and intended-headset start/status guards.
`tests/page/pad_queue.mjs` and `tests/page/ctrl_gestures.mjs` execute the page's
real queue and gesture functions in Node. These are offline tests; their
acknowledgements do not prove that a real application received input.
See [the 2026-10-24 PR #66 evidence](evidence/pad-gamescope-2026-10-24.md)
for the real focused-panel click/key observation, cleanup and remaining limits.
The read-only power/heat sampler has fixture and transport tests in
`tests/test_power_sample.py`. Run
`python3 -m unittest discover -s tests -p test_power_sample.py -v`.
@@ -70,11 +60,8 @@ scripts/e2e.sh test_doctor
They cover privacy filtering, host-key preservation, restart safety and missing
journal evidence, plus asleep, full-title-storage, SSH/devkit-off and missing-key
faults. The plan/report tests cover exact-evidence advice, blocked plans,
private exclusive baselines, malformed input, target mismatches and comparisons
without repair claims. The fake-Frame storage test records a full-disk baseline,
clears only the fake fault and compares a fresh run. The fake's doctor-only
`systemctl show` responses assume stable restart counters and inactive keep-awake; unsupported queries fail. These tests do not
faults. The fake's doctor-only `systemctl show` responses assume stable restart
counters and inactive keep-awake; unsupported queries fail. These tests do not
exercise a real SteamOS user manager or touch a headset.
`tests/fakeframe/` builds a container that stands in for the headset, and a
@@ -175,14 +162,6 @@ Record the build, actual interaction results, cleanup and any unworn-headset
limits alongside screenshots or logs. These are caller responsibilities; the
smoke script below does not acquire this shared lock itself.
[Packaged Desktop Capture evidence, 2026-10-24](evidence/desktop-capture-packaged-2026-10-24.md)
records v0.4.17 GUI button captures of a real mapped nested desktop on SteamOS
0.5.6, a bounded owned user unit, and cleanup back to the original Game Mode
state. Check X window mapping/focus and SteamVR overlay visibility separately:
successful capture of a mapped window does not prove the desktop is visible
to a wearer. That run's overlay stayed `not_visible`; #73 remains unverified
for its reported visible-desktop/0.4.5 condition. Only state/log metadata is public.
```sh
scripts/frame-smoke.sh # needs `ssh frame` to work without a password
scripts/frame-smoke.sh --pair # also pairs a throwaway key: approve it in the headset
@@ -392,11 +371,7 @@ trees on the fake Frame. None of it has run on a headset.
`tests/test_media.py` covers layout evidence and overrides, OU eye ordering,
hardware-decoder command construction, malformed splats, stereo parallax and
fake-Frame library/process ownership. It also covers optional-surround refusal
at creation, setup, first upload and retry after standby, fatal screen refusal,
and best-effort overlay teardown with shutdown.
Run `python3 -m unittest discover -s tests -p test_media.py -v`.
`tests/e2e/test_media_transfer.py` checks the real
fake-Frame library/process ownership. `tests/e2e/test_media_transfer.py` checks the real
HTTP/SSH upload and library listing without pretending the fake renders VR.
Real decode timings and captured stereo output are recorded in
[vr-video.md](vr-video.md). Generated media only; no external player required.
-8
View File
@@ -101,14 +101,6 @@ Stills and the theatre surround are re-sent after waking. Five minutes
without an accepted frame is reported as an error. Headset-view captures
taken during standby show a flat dark frame, not our screen.
**Optional surround:** if SteamVR refuses the theatre dark surround at startup
or on a retry after standby, playback continues without it and the player log
records `Theatre surround unavailable`. Screen errors still fail playback.
On exit, the player attempts to destroy every owned overlay and calls OpenVR
shutdown even if an overlay's teardown raises an exception. Refusal and
teardown errors are covered by fake-OpenVR tests; a real surround refusal has
not been induced safely. See the [current hardening checks](evidence/media-surround-2026-10-24.md).
![Media panel in Frame Control while a photo plays on the Frame](img/media-ui-panel.png)
**Verified failed route:** GStreamer 1.24.2's `playbin` selected
+24 -8
View File
@@ -7,13 +7,13 @@
objects = {
/* Begin PBXBuildFile section */
C1234567890ABCDEF0123456 /* NativeSessionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = D1234567890ABCDEF0123456 /* NativeSessionTests.swift */; };
A1234567890ABCDEF0123456 /* NativeSessionState.swift in Sources */ = {isa = PBXBuildFile; fileRef = B1234567890ABCDEF0123456 /* NativeSessionState.swift */; };
0CF0B365CE5974753E8C3C2B /* ScreenshotJournal.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0D48FCBB02F0F2212C7DE3C1 /* ScreenshotJournal.swift */; };
0DEE50BD563B1D8C328C4C0A /* HeadsetServer.swift in Sources */ = {isa = PBXBuildFile; fileRef = EDC7BA8014DBC302D08FD397 /* HeadsetServer.swift */; };
12B21D3319BAF5AE79948560 /* FrameLink.swift in Sources */ = {isa = PBXBuildFile; fileRef = 16644E7FDA7ADD5B232EB700 /* FrameLink.swift */; };
1A07EC692B0FF723907EA77B /* WebShell.swift in Sources */ = {isa = PBXBuildFile; fileRef = D6C4E6C28315CA8729FCAAEA /* WebShell.swift */; };
41A697B9924018DA48F24A1F /* Keys.swift in Sources */ = {isa = PBXBuildFile; fileRef = 237D9AF04EEA257AB382F60E /* Keys.swift */; };
4622FE0F0D6499CD642C29A2 /* InstallLink.swift in Sources */ = {isa = PBXBuildFile; fileRef = BF0FCA7117DA3ABA449B4EE0 /* InstallLink.swift */; };
46EF7595BE5B36664A11AEB5 /* NativeSessionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8F1CAE943C2556A960890079 /* NativeSessionTests.swift */; };
476D8858DC2C9E6616B084BC /* PortForwarder.swift in Sources */ = {isa = PBXBuildFile; fileRef = A7F6ED116569D0ABABF6ED65 /* PortForwarder.swift */; };
765661DBC0E6798A27CC60DB /* RootView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9B24E1BCD4F69A24C7DEF02F /* RootView.swift */; };
78427FC66780623F31E7501E /* FrameControlApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = 93C8E0D7C3F4F628941B3D5A /* FrameControlApp.swift */; };
@@ -21,8 +21,12 @@
9657F7BC23E3352E5AB30777 /* SetupView.swift in Sources */ = {isa = PBXBuildFile; fileRef = DB544223FC60A59CC3E8EF5F /* SetupView.swift */; };
A0C5B00E257230A38DBD9E54 /* ComfortNotificationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E81218B75FEEE47B8D8BAE20 /* ComfortNotificationTests.swift */; };
A8C7AED25A6280682FCE45DC /* Citadel in Frameworks */ = {isa = PBXBuildFile; productRef = 6BA549B6CC0A0CB847126456 /* Citadel */; };
C046F310BA857BCEC98BE00E /* ScreenshotDownloadsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5EBD10B73DF1E95F67A3B074 /* ScreenshotDownloadsView.swift */; };
DC043FB74BE2D23F3A5826BF /* FrameControlTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */; };
DFD89170E502260F005EEC73 /* ScreenshotDownloads.swift in Sources */ = {isa = PBXBuildFile; fileRef = 07617FD9B214626526FB1F03 /* ScreenshotDownloads.swift */; };
E2FD56BB5BC1D9AF5CDBF210 /* ScreenshotSyncTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = AB8610920C7D797E5963CF88 /* ScreenshotSyncTests.swift */; };
E6898C714A92D3979F73B6E1 /* FrameFinder.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */; };
F0ECE06D5DD29F19CC41FDFB /* NativeSessionState.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9E8BE9A8C421AD40A0F4105B /* NativeSessionState.swift */; };
F94D0252F8CC5854314B84B2 /* AppModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A11F3431826A26B247C0695 /* AppModel.swift */; };
/* End PBXBuildFile section */
@@ -37,19 +41,23 @@
/* End PBXContainerItemProxy section */
/* Begin PBXFileReference section */
D1234567890ABCDEF0123456 /* NativeSessionTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = NativeSessionTests.swift; sourceTree = "<group>"; };
B1234567890ABCDEF0123456 /* NativeSessionState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = NativeSessionState.swift; sourceTree = "<group>"; };
07617FD9B214626526FB1F03 /* ScreenshotDownloads.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ScreenshotDownloads.swift; sourceTree = "<group>"; };
0D48FCBB02F0F2212C7DE3C1 /* ScreenshotJournal.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ScreenshotJournal.swift; sourceTree = "<group>"; };
16644E7FDA7ADD5B232EB700 /* FrameLink.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameLink.swift; sourceTree = "<group>"; };
1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameControlTests.swift; sourceTree = "<group>"; };
237D9AF04EEA257AB382F60E /* Keys.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Keys.swift; sourceTree = "<group>"; };
2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameFinder.swift; sourceTree = "<group>"; };
5EBD10B73DF1E95F67A3B074 /* ScreenshotDownloadsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ScreenshotDownloadsView.swift; sourceTree = "<group>"; };
6B5B6718C5EA77FA67F6B14C /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist; path = Info.plist; sourceTree = "<group>"; };
6FBC8D0B5ED7BF1C06F99892 /* FrameControlTests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = FrameControlTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
8A11F3431826A26B247C0695 /* AppModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppModel.swift; sourceTree = "<group>"; };
8F1CAE943C2556A960890079 /* NativeSessionTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = NativeSessionTests.swift; sourceTree = "<group>"; };
8F2CB550FC81C01E6BDD5A71 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = "<group>"; };
93C8E0D7C3F4F628941B3D5A /* FrameControlApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameControlApp.swift; sourceTree = "<group>"; };
9B24E1BCD4F69A24C7DEF02F /* RootView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RootView.swift; sourceTree = "<group>"; };
9E8BE9A8C421AD40A0F4105B /* NativeSessionState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = NativeSessionState.swift; sourceTree = "<group>"; };
A7F6ED116569D0ABABF6ED65 /* PortForwarder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PortForwarder.swift; sourceTree = "<group>"; };
AB8610920C7D797E5963CF88 /* ScreenshotSyncTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ScreenshotSyncTests.swift; sourceTree = "<group>"; };
BF0FCA7117DA3ABA449B4EE0 /* InstallLink.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InstallLink.swift; sourceTree = "<group>"; };
D6C4E6C28315CA8729FCAAEA /* WebShell.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WebShell.swift; sourceTree = "<group>"; };
DB544223FC60A59CC3E8EF5F /* SetupView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SetupView.swift; sourceTree = "<group>"; };
@@ -104,9 +112,11 @@
isa = PBXGroup;
children = (
8A11F3431826A26B247C0695 /* AppModel.swift */,
B1234567890ABCDEF0123456 /* NativeSessionState.swift */,
93C8E0D7C3F4F628941B3D5A /* FrameControlApp.swift */,
BF0FCA7117DA3ABA449B4EE0 /* InstallLink.swift */,
9E8BE9A8C421AD40A0F4105B /* NativeSessionState.swift */,
07617FD9B214626526FB1F03 /* ScreenshotDownloads.swift */,
0D48FCBB02F0F2212C7DE3C1 /* ScreenshotJournal.swift */,
);
path = App;
sourceTree = "<group>";
@@ -116,7 +126,8 @@
children = (
E81218B75FEEE47B8D8BAE20 /* ComfortNotificationTests.swift */,
1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */,
D1234567890ABCDEF0123456 /* NativeSessionTests.swift */,
8F1CAE943C2556A960890079 /* NativeSessionTests.swift */,
AB8610920C7D797E5963CF88 /* ScreenshotSyncTests.swift */,
);
path = FrameControlTests;
sourceTree = "<group>";
@@ -125,6 +136,7 @@
isa = PBXGroup;
children = (
9B24E1BCD4F69A24C7DEF02F /* RootView.swift */,
5EBD10B73DF1E95F67A3B074 /* ScreenshotDownloadsView.swift */,
DB544223FC60A59CC3E8EF5F /* SetupView.swift */,
);
path = Views;
@@ -266,15 +278,18 @@
buildActionMask = 2147483647;
files = (
F94D0252F8CC5854314B84B2 /* AppModel.swift in Sources */,
A1234567890ABCDEF0123456 /* NativeSessionState.swift in Sources */,
78427FC66780623F31E7501E /* FrameControlApp.swift in Sources */,
E6898C714A92D3979F73B6E1 /* FrameFinder.swift in Sources */,
12B21D3319BAF5AE79948560 /* FrameLink.swift in Sources */,
0DEE50BD563B1D8C328C4C0A /* HeadsetServer.swift in Sources */,
4622FE0F0D6499CD642C29A2 /* InstallLink.swift in Sources */,
41A697B9924018DA48F24A1F /* Keys.swift in Sources */,
F0ECE06D5DD29F19CC41FDFB /* NativeSessionState.swift in Sources */,
476D8858DC2C9E6616B084BC /* PortForwarder.swift in Sources */,
765661DBC0E6798A27CC60DB /* RootView.swift in Sources */,
DFD89170E502260F005EEC73 /* ScreenshotDownloads.swift in Sources */,
C046F310BA857BCEC98BE00E /* ScreenshotDownloadsView.swift in Sources */,
0CF0B365CE5974753E8C3C2B /* ScreenshotJournal.swift in Sources */,
9657F7BC23E3352E5AB30777 /* SetupView.swift in Sources */,
1A07EC692B0FF723907EA77B /* WebShell.swift in Sources */,
);
@@ -286,7 +301,8 @@
files = (
A0C5B00E257230A38DBD9E54 /* ComfortNotificationTests.swift in Sources */,
DC043FB74BE2D23F3A5826BF /* FrameControlTests.swift in Sources */,
C1234567890ABCDEF0123456 /* NativeSessionTests.swift in Sources */,
46EF7595BE5B36664A11AEB5 /* NativeSessionTests.swift in Sources */,
E2FD56BB5BC1D9AF5CDBF210 /* ScreenshotSyncTests.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
+14
View File
@@ -22,6 +22,8 @@ final class AppModel: ObservableObject {
/// Native recovery remains reachable even when WebKit cannot draw a page.
@Published private(set) var webRecoveryMessage: String?
@Published private(set) var webReloadID = 0
let screenshots = ScreenshotDownloads()
@Published var showScreenshotDownloads = false
func webContentFailed(at url: URL, message: String) {
guard phase == .ready(url) else { return }
@@ -45,6 +47,7 @@ final class AppModel: ObservableObject {
let saved = UserDefaults.standard.data(forKey: Self.settingsKey).flatMap { try? JSONDecoder().decode(FrameSettings.self, from: $0) }
settings = saved
phase = saved == nil ? .setup : .connecting("Connecting")
if let identity = UserDefaults.standard.string(forKey: Self.hostKeyKey) { screenshots.load(identity: identity) }
}
var deviceName: String { UIDevice.current.userInterfaceIdiom == .pad ? "iPad" : "iPhone" }
@@ -146,6 +149,7 @@ final class AppModel: ObservableObject {
/// Every connection attempt has a number; anything that finishes after a newer
/// attempt started (or the user went back to setup) closes what it made and stops.
private func invalidate() {
screenshots.stop()
attempt += 1
retrying = false
webRecoveryMessage = nil
@@ -200,6 +204,7 @@ final class AppModel: ObservableObject {
}
#endif
phase = .ready(URL(string: page)!)
startScreenshots()
// Runs at once if it stopped in the moment since the check above.
server.whenExited { [weak self] tail in
Task { @MainActor in self?.lost(mine, "Frame Control on the headset stopped. \(tail.suffix(200))") }
@@ -254,6 +259,7 @@ final class AppModel: ObservableObject {
/// Called when the app comes back to the foreground: iOS may have dropped the
/// connection, or left it looking open, while it was in the background.
func resume() {
startScreenshots()
switch phase {
case .ready:
let mine = attempt
@@ -284,6 +290,7 @@ final class AppModel: ObservableObject {
}
private func teardown() async {
screenshots.stop()
forwarder?.stop()
forwarder = nil
server = nil
@@ -298,4 +305,11 @@ final class AppModel: ObservableObject {
_ = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes)
return bytes.map { String(format: "%02x", $0) }.joined()
}
func suspend() { screenshots.stop() }
private func startScreenshots() {
guard case .ready(let url) = phase, let hostKey else { return }
screenshots.start(url: url, identity: hostKey)
}
}
@@ -38,6 +38,7 @@ struct FrameControlApp: App {
}
.onChange(of: scenePhase) { _, phase in
if phase == .active { model.resume() }
else if phase == .background { model.suspend() }
}
}
}
@@ -0,0 +1,157 @@
import CryptoKit
import ImageIO
import Photos
import SwiftUI
import UIKit
@MainActor
final class ScreenshotDownloads: ObservableObject {
struct Item: Identifiable {
let id: String
let file: URL
let date: Date
let status: String
}
@Published private(set) var items: [Item] = []
@Published private(set) var message = "New screenshots download while Frame Control is open."
@Published private(set) var saveToPhotos: Bool
private let defaults: UserDefaults
private let storageRoot: URL?
private static let preference = "frame.screenshots.saveToPhotos"
private var engine: ScreenshotSyncEngine?
private var task: Task<Void, Never>?
private var session: URL?
private var identity: String?
init(defaults: UserDefaults = .standard, storageRoot: URL? = nil) {
self.defaults = defaults
self.storageRoot = storageRoot
saveToPhotos = defaults.object(forKey: Self.preference) == nil || defaults.bool(forKey: Self.preference)
}
func setPhotos(_ enabled: Bool) {
do {
try engine?.setPhotos(enabled)
defaults.set(enabled, forKey: Self.preference)
saveToPhotos = enabled
refresh()
if enabled { Task { _ = await Self.photoPermission() } }
} catch { message = "Couldn't save the screenshot setting: \(error.localizedDescription)" }
}
/// Local files and settings remain available without a live SSH session.
func load(identity: String) {
guard self.identity != identity || engine == nil else { return }
do {
let digest = SHA256.hash(data: Data(identity.utf8)).map { String(format: "%02x", $0) }.joined()
let base = try storageRoot ?? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask,
appropriateFor: nil, create: true)
self.engine = try ScreenshotSyncEngine(root: base.appendingPathComponent("Screenshots/\(digest)", isDirectory: true),
photos: saveToPhotos)
self.identity = identity
refresh()
} catch {
engine = nil
self.identity = nil
items = []
message = "Screenshot downloads need attention: \(error.localizedDescription)"
}
}
func start(url: URL, identity: String) {
if session == url, task != nil { return }
stop()
load(identity: identity)
if let engine {
session = url
task = Task { [weak self] in
while !Task.isCancelled {
guard let self, self.session == url else { return }
if UIApplication.shared.applicationState == .active {
await self.poll(url: url, engine: engine)
}
do { try await Task.sleep(nanoseconds: 4_000_000_000) }
catch { return }
}
}
}
}
func stop() {
task?.cancel()
task = nil
engine?.stop()
session = nil
}
private func poll(url: URL, engine: ScreenshotSyncEngine) async {
do {
let data = try await Self.get("/api/shots", from: url)
struct Listing: Decodable { let shots: [HeadsetScreenshot] }
let shots = try JSONDecoder().decode(Listing.self, from: data).shots
guard !Task.isCancelled, session == url else { return }
try await engine.reconcile(shots, download: { shot in
try await Self.get("/api/shots/image", id: shot.id, from: url)
}, validate: { data in
guard let image = CGImageSourceCreateWithData(data as CFData, nil),
CGImageSourceGetStatus(image) == .statusComplete else { return false }
return CGImageSourceCreateImageAtIndex(image, 0, nil) != nil
}, export: { file, allowed in
guard !Task.isCancelled, UIApplication.shared.applicationState == .active else { throw CancellationError() }
guard await Self.photoPermission() else {
throw NSError(domain: "FrameScreenshots", code: 1,
userInfo: [NSLocalizedDescriptionKey: "Originals are kept in Frame Control. Allow Photos access in iOS Settings to save them to Photos."])
}
// Cancellation or a toggle change while the permission prompt was
// visible must not start a Photos transaction.
guard !Task.isCancelled, allowed() else { throw CancellationError() }
try await PHPhotoLibrary.shared().performChanges {
PHAssetCreationRequest.forAsset().addResource(with: .photo, fileURL: file, options: nil)
}
})
guard !Task.isCancelled, session == url else { return }
message = "New screenshots download while Frame Control is open."
} catch {
guard !Task.isCancelled, session == url else { return }
message = "\(error.localizedDescription) Downloads retry while the app is open."
}
refresh()
}
private static func get(_ path: String, id: String? = nil, from page: URL) async throws -> Data {
let original = URLComponents(url: page, resolvingAgainstBaseURL: false)
let key = original?.queryItems?.first(where: { $0.name == "key" })?.value
var parts = original!
parts.path = path
parts.queryItems = id.map { [URLQueryItem(name: "id", value: $0)] }
parts.fragment = nil
var request = URLRequest(url: parts.url!, cachePolicy: .reloadIgnoringLocalCacheData, timeoutInterval: 30)
request.setValue(key, forHTTPHeaderField: "X-Frame-UI")
let (data, response) = try await URLSession.shared.data(for: request)
guard let http = response as? HTTPURLResponse, http.statusCode == 200 else { throw URLError(.badServerResponse) }
return data
}
private static func photoPermission() async -> Bool {
var status = PHPhotoLibrary.authorizationStatus(for: .addOnly)
if status == .notDetermined, UIApplication.shared.applicationState == .active {
status = await PHPhotoLibrary.requestAuthorization(for: .addOnly)
}
return status == .authorized || status == .limited
}
private func refresh() {
guard let engine else { return }
items = engine.journal.receipts.values.filter(\.downloaded).map { receipt in
let status: String
switch receipt.destination {
case .saved: status = "Saved to Photos"
case .appOnly: status = "In Frame Control"
case .pending: status = "In Frame Control · waiting for Photos"
case .committing: status = "In Frame Control · Photos save interrupted; check Photos before saving again"
}
return Item(id: receipt.shot.id, file: engine.file(for: receipt.shot.id),
date: Date(timeIntervalSince1970: receipt.shot.time), status: status)
}.sorted { $0.date > $1.date }
}
}
@@ -0,0 +1,151 @@
import Foundation
struct HeadsetScreenshot: Codable, Equatable {
let id: String
let size: Int
let time: Double
var valid: Bool {
id.range(of: #"^[0-9]{1,12}/[0-9]{1,20}/[0-9]{14}_[0-9]{1,4}\.(jpg|png)$"#, options: .regularExpression) != nil
&& size > 0 && size <= 30 * 1024 * 1024 && time.isFinite
}
}
/// All receipt state lives beside the originals, outside purgeable caches.
/// An interrupted Photos transaction is deliberately not retried automatically:
/// add-only access cannot check whether its asset was committed before a crash.
struct ScreenshotJournal: Codable {
enum Export: String, Codable { case pending, appOnly, committing, saved }
struct Receipt: Codable {
var shot: HeadsetScreenshot
var downloaded = false
var destination: Export
}
var baseline: Set<String>?
var receipts: [String: Receipt] = [:]
mutating func observe(_ shots: [HeadsetScreenshot], photos: Bool) {
let shots = shots.filter(\.valid)
guard let baseline else {
self.baseline = Set(shots.map(\.id))
return
}
for shot in shots where !baseline.contains(shot.id) {
if var receipt = receipts[shot.id] {
if !receipt.downloaded { receipt.shot = shot; receipts[shot.id] = receipt }
} else {
receipts[shot.id] = Receipt(shot: shot, destination: photos ? .pending : .appOnly)
}
}
}
mutating func keepPendingInApp() {
for id in receipts.keys where receipts[id]?.destination == .pending || receipts[id]?.destination == .committing {
receipts[id]?.destination = .appOnly
}
}
}
/// Serialized reconciler shared by foreground polling and future event delivery.
/// Network and Photos operations are injected so failures and late callbacks can
/// be exercised against the actual persistence and state transitions.
@MainActor
final class ScreenshotSyncEngine {
let root: URL
private(set) var journal: ScreenshotJournal
private var previous: [String: HeadsetScreenshot] = [:]
private var generation = UUID()
private var running = false
private(set) var saveToPhotos: Bool
init(root: URL, photos: Bool) throws {
self.root = root
saveToPhotos = photos
let ledger = root.appendingPathComponent("receipts.json")
if FileManager.default.fileExists(atPath: ledger.path) {
// Corrupt state must never silently become a fresh receipt ledger.
journal = try JSONDecoder().decode(ScreenshotJournal.self, from: Data(contentsOf: ledger))
} else { journal = ScreenshotJournal() }
if !photos { journal.keepPendingInApp() }
try persist()
}
func stop() { generation = UUID() }
func setPhotos(_ enabled: Bool) throws {
let oldJournal = journal
let oldSetting = saveToPhotos
saveToPhotos = enabled
if !enabled { journal.keepPendingInApp() }
do { try persist() }
catch { journal = oldJournal; saveToPhotos = oldSetting; throw error }
}
func file(for id: String) -> URL { root.appendingPathComponent(id) }
private func persist() throws {
try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true)
try JSONEncoder().encode(journal).write(to: root.appendingPathComponent("receipts.json"), options: .atomic)
}
func reconcile(_ shots: [HeadsetScreenshot],
download: (HeadsetScreenshot) async throws -> Data,
validate: (Data) -> Bool,
export: (URL, () -> Bool) async throws -> Void) async throws {
guard !running else { return }
running = true
defer { running = false }
let mine = generation
func current() -> Bool { mine == generation && !Task.isCancelled }
guard current() else { return }
let valid = shots.filter(\.valid)
let observed = Dictionary(valid.map { ($0.id, $0) }, uniquingKeysWith: { _, last in last })
let stable = previous
previous = observed
journal.observe(valid, photos: saveToPhotos)
try persist() // Queue before touching an original or the photo library.
var firstError: Error?
for id in journal.receipts.keys.sorted() {
guard current(), var receipt = journal.receipts[id], receipt.shot.valid else { continue }
do {
let file = file(for: id)
if !receipt.downloaded {
// Steam may still be writing. Require two equal observations.
guard observed[id] == receipt.shot, stable[id] == receipt.shot else { continue }
let data = try await download(receipt.shot)
guard current() else { return }
guard data.count == receipt.shot.size, validate(data) else {
throw CocoaError(.fileReadCorruptFile)
}
try FileManager.default.createDirectory(at: file.deletingLastPathComponent(), withIntermediateDirectories: true)
try data.write(to: file, options: .atomic)
// Re-read destination: the user can change it during download.
receipt = journal.receipts[id]!
receipt.downloaded = true
journal.receipts[id] = receipt
try persist()
}
if receipt.destination == .pending, saveToPhotos {
journal.receipts[id]?.destination = .committing
try persist()
do {
try await export(file, { current() && self.saveToPhotos && self.journal.receipts[id]?.destination == .committing })
} catch {
guard current() else { return }
// A reported transaction failure is safe to retry. A crash
// leaves committing, which needs explicit user recovery.
if journal.receipts[id]?.destination == .committing {
journal.receipts[id]?.destination = saveToPhotos ? .pending : .appOnly
}
try persist()
throw error
}
guard current() else { return }
journal.receipts[id]?.destination = .saved
try persist()
}
} catch { if firstError == nil { firstError = error } }
}
if let firstError { throw firstError }
}
}
+3 -1
View File
@@ -31,6 +31,8 @@
</array>
<key>CFBundleVersion</key>
<string>1</string>
<key>ITSAppUsesNonExemptEncryption</key>
<false/>
<key>LSApplicationQueriesSchemes</key>
<array>
<string>ssh</string>
@@ -50,7 +52,7 @@
<key>NSLocalNetworkUsageDescription</key>
<string>Frame Control finds your Steam Frame on your network and connects to it.</string>
<key>NSPhotoLibraryAddUsageDescription</key>
<string>Frame Control saves headset captures and screenshots to your photo library when you ask it to.</string>
<string>Frame Control automatically saves new headset screenshots to Photos when Save screenshots to Photos is enabled.</string>
<key>UILaunchScreen</key>
<dict>
<key>UIColorName</key>
+9
View File
@@ -39,6 +39,15 @@ struct RootView: View {
}
.preferredColorScheme(.dark)
.tint(.frameBlue)
.safeAreaInset(edge: .bottom) {
if case .ready = model.phase {} else {
Button("Screenshot downloads") { model.showScreenshotDownloads = true }
.padding().frame(maxWidth: .infinity).background(Color.frameBackground)
}
}
.sheet(isPresented: $model.showScreenshotDownloads) {
ScreenshotDownloadsView(downloads: model.screenshots)
}
}
}
@@ -0,0 +1,40 @@
import SwiftUI
struct ScreenshotDownloadsView: View {
@ObservedObject var downloads: ScreenshotDownloads
@Environment(\.dismiss) private var dismiss
var body: some View {
NavigationStack {
List {
Section {
Toggle("Save screenshots to Photos", isOn: Binding(get: { downloads.saveToPhotos }, set: downloads.setPhotos))
} footer: {
Text("Enabled by default. Turn off to keep new downloads only in Frame Control. Existing Photos are kept. App-only downloads stay app-only when this is turned on again.")
}
Section {
Text(downloads.message)
Text("Automatic downloads begin with new screenshots after the first connection. Keep the app open and connected; background delivery is not available yet.")
.foregroundStyle(.secondary)
}
Section("Downloaded from this headset") {
if downloads.items.isEmpty { Text("No new screenshots downloaded yet.").foregroundStyle(.secondary) }
ForEach(downloads.items) { item in
HStack {
VStack(alignment: .leading, spacing: 4) {
Text(item.date, format: .dateTime.month().day().hour().minute())
Text(item.status).font(.caption).foregroundStyle(.secondary)
}
Spacer()
ShareLink(item: item.file) { Image(systemName: "square.and.arrow.up") }
.accessibilityLabel("Share screenshot")
}
}
}
}
.navigationTitle("Screenshot downloads")
.navigationBarTitleDisplayMode(.inline)
.toolbar { ToolbarItem(placement: .confirmationAction) { Button("Done") { dismiss() } } }
}
}
}
+26 -3
View File
@@ -7,6 +7,7 @@ struct SetupView: View {
@ObservedObject var model: AppModel
@StateObject private var finder = FrameFinder()
@State private var password = ""
@State private var showPassword = false
@State private var manualHost = ""
@State private var user = "steamos"
@State private var showOther = false
@@ -92,13 +93,35 @@ struct SetupView: View {
// MARK: step 2
@ViewBuilder private var passwordStep: some View {
SecureField("Developer Mode password", text: $password)
HStack(spacing: 8) {
Group {
if showPassword {
TextField("Developer Mode password", text: $password)
} else {
SecureField("Developer Mode password", text: $password)
}
}
.textContentType(.password)
.textInputAutocapitalization(.never)
.autocorrectionDisabled()
.submitLabel(.go)
.focused($passwordFocused)
.onSubmit(connect)
.padding(12)
.background(Color.black.opacity(0.28), in: RoundedRectangle(cornerRadius: 10))
.padding(.vertical, 12)
Button {
showPassword.toggle()
passwordFocused = true // keep typing after the field swaps
} label: {
Image(systemName: showPassword ? "eye.slash" : "eye").foregroundStyle(Color.frameMuted)
.frame(width: 44, height: 44)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
.accessibilityLabel(showPassword ? "Hide password" : "Show password")
}
.padding(.leading, 12)
.padding(.trailing, 2)
.background(Color.black.opacity(0.28), in: RoundedRectangle(cornerRadius: 10))
Text("Haven't set one? On the Frame: Steam Settings → Developer → Set User Password. It's only used now, to let this \(model.deviceName) in; it isn't saved.")
.font(.footnote).foregroundStyle(Color.frameMuted)
Button(action: connect) {
+4
View File
@@ -59,6 +59,7 @@ struct WebShell: UIViewRepresentable {
setUpConnection: () => call("setUpConnection"),
open: (what) => call("open", what),
saveImages: (images) => call("saveImages", images),
screenshotDownloads: () => call("screenshotDownloads"),
onInstallLink: (cb) => { installCb = cb; return call("installLinkReady", window.__frameDocumentID); },
};
window.__frameInstallLink = (req) => { if (!installCb) return false; installCb(req); return true; };
@@ -162,6 +163,9 @@ struct WebShell: UIViewRepresentable {
guard !images.isEmpty else { return replyHandler(nil, "No images to save") }
share(images)
replyHandler(["message": "Choose Save Image to keep \(images.count == 1 ? "it" : "them") in Photos"], nil)
case "screenshotDownloads":
model.showScreenshotDownloads = true
replyHandler(true, nil)
case "installLinkReady":
guard let id = arg as? String, let web,
let data = try? JSONSerialization.data(withJSONObject: id, options: .fragmentsAllowed),
@@ -0,0 +1,198 @@
import Foundation
import CryptoKit
import XCTest
@testable import Frame_Control
@MainActor
final class ScreenshotSyncTests: XCTestCase {
private let old = HeadsetScreenshot(id: "1/10/20261024010000_1.jpg", size: 3, time: 1)
private let new = HeadsetScreenshot(id: "1/10/20261024010001_1.jpg", size: 3, time: 2)
private func temporary() throws -> URL {
let root = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true)
addTeardownBlock { try? FileManager.default.removeItem(at: root) }
return root
}
func testBaselineStabilityAtomicOriginalAndRestartDedup() async throws {
let root = try temporary()
let engine = try ScreenshotSyncEngine(root: root, photos: true)
var downloads = 0, exports = 0
let fetch: (HeadsetScreenshot) async throws -> Data = { _ in downloads += 1; return Data([1, 2, 3]) }
let save: (URL, () -> Bool) async throws -> Void = { file, allowed in
XCTAssertTrue(allowed())
XCTAssertEqual(try Data(contentsOf: file), Data([1, 2, 3]))
exports += 1
}
try await engine.reconcile([old], download: fetch, validate: { _ in true }, export: save)
try await engine.reconcile([old, new], download: fetch, validate: { _ in true }, export: save)
XCTAssertEqual(downloads, 0, "Do not import history or read a file still being written")
try await engine.reconcile([old, new, new], download: fetch, validate: { _ in true }, export: save)
XCTAssertEqual(downloads, 1)
XCTAssertEqual(exports, 1)
let restarted = try ScreenshotSyncEngine(root: root, photos: true)
try await restarted.reconcile([old, new], download: fetch, validate: { _ in true }, export: save)
XCTAssertEqual(downloads, 1)
XCTAssertEqual(exports, 1)
XCTAssertEqual(restarted.journal.receipts[new.id]?.destination, .saved)
}
func testInterruptedDownloadRetriesAndAppOnlyNeverExportsLater() async throws {
let root = try temporary()
let engine = try ScreenshotSyncEngine(root: root, photos: false)
let fetch: (HeadsetScreenshot) async throws -> Data = { _ in Data([1, 2, 3]) }
let noExport: (URL, () -> Bool) async throws -> Void = { _, _ in XCTFail("App-only item exported") }
try await engine.reconcile([], download: fetch, validate: { _ in true }, export: noExport)
try await engine.reconcile([new], download: fetch, validate: { _ in true }, export: noExport)
do {
try await engine.reconcile([new], download: { _ in Data([1]) }, validate: { _ in true }, export: noExport)
XCTFail("Truncated original accepted")
} catch {}
XCTAssertFalse(FileManager.default.fileExists(atPath: engine.file(for: new.id).path))
let restarted = try ScreenshotSyncEngine(root: root, photos: true)
try await restarted.reconcile([new], download: fetch, validate: { _ in true }, export: noExport)
try await restarted.reconcile([new], download: fetch, validate: { _ in true }, export: noExport)
XCTAssertEqual(restarted.journal.receipts[new.id]?.destination, .appOnly)
XCTAssertTrue(restarted.journal.receipts[new.id]!.downloaded)
}
func testDestinationChangeDuringDownloadAndPermissionPrompt() async throws {
let engine = try ScreenshotSyncEngine(root: temporary(), photos: true)
let noExport: (URL, () -> Bool) async throws -> Void = { _, _ in XCTFail("Destination changed while downloading") }
try await engine.reconcile([], download: { _ in Data() }, validate: { _ in true }, export: noExport)
try await engine.reconcile([new], download: { _ in Data() }, validate: { _ in true }, export: noExport)
try await engine.reconcile([new], download: { _ in
try engine.setPhotos(false)
try engine.setPhotos(true)
return Data([1, 2, 3])
}, validate: { _ in true }, export: noExport)
XCTAssertEqual(engine.journal.receipts[new.id]?.destination, .appOnly)
let other = HeadsetScreenshot(id: "2/10/20261024010001_1.jpg", size: 3, time: 2)
try await engine.reconcile([new, other], download: { _ in Data() }, validate: { _ in true }, export: noExport)
do {
try await engine.reconcile([new, other], download: { _ in Data([1, 2, 3]) }, validate: { _ in true }, export: { _, allowed in
try engine.setPhotos(false)
try engine.setPhotos(true)
XCTAssertFalse(allowed(), "Re-enabling must not revive an export cancelled during consent")
throw CancellationError()
})
} catch {}
XCTAssertEqual(engine.journal.receipts[other.id]?.destination, .appOnly)
}
func testCancellationRejectsLateOriginalAndIdentityIncludesAccountAndApp() async throws {
let engine = try ScreenshotSyncEngine(root: temporary(), photos: false)
let variants = [new, HeadsetScreenshot(id: "2/10/20261024010001_1.jpg", size: 3, time: 2),
HeadsetScreenshot(id: "1/20/20261024010001_1.jpg", size: 3, time: 2)]
let save: (URL, () -> Bool) async throws -> Void = { _, _ in XCTFail("Unexpected export") }
try await engine.reconcile([], download: { _ in Data() }, validate: { _ in true }, export: save)
try await engine.reconcile(variants, download: { _ in Data() }, validate: { _ in true }, export: save)
try await engine.reconcile(variants, download: { _ in engine.stop(); return Data([1, 2, 3]) }, validate: { _ in true }, export: save)
XCTAssertEqual(engine.journal.receipts.count, 3)
XCTAssertTrue(engine.journal.receipts.values.allSatisfy { !$0.downloaded })
XCTAssertNotEqual(engine.file(for: variants[0].id), engine.file(for: variants[1].id))
XCTAssertNotEqual(engine.file(for: variants[0].id), engine.file(for: variants[2].id))
}
func testUncertainPhotosCommitIsNotDuplicatedAndCorruptLedgerFailsClosed() async throws {
let root = try temporary()
var journal = ScreenshotJournal()
journal.observe([], photos: true)
journal.observe([new], photos: true)
journal.receipts[new.id]?.downloaded = true
journal.receipts[new.id]?.destination = .committing
try JSONEncoder().encode(journal).write(to: root.appendingPathComponent("receipts.json"))
let engine = try ScreenshotSyncEngine(root: root, photos: true)
try await engine.reconcile([new], download: { _ in XCTFail(); return Data() }, validate: { _ in true }, export: { _, _ in XCTFail("Possible duplicate Photos asset") })
XCTAssertEqual(engine.journal.receipts[new.id]?.destination, .committing)
try Data("broken".utf8).write(to: root.appendingPathComponent("receipts.json"))
XCTAssertThrowsError(try ScreenshotSyncEngine(root: root, photos: true))
XCTAssertFalse(HeadsetScreenshot(id: "../../private.jpg", size: 3, time: 2).valid)
}
func testPhotosFailureRetainsOriginalAndRetriesOnlyExport() async throws {
let root = try temporary()
let engine = try ScreenshotSyncEngine(root: root, photos: true)
var downloads = 0, exports = 0
let fetch: (HeadsetScreenshot) async throws -> Data = { _ in downloads += 1; return Data([1, 2, 3]) }
let denied: (URL, () -> Bool) async throws -> Void = { _, _ in throw CocoaError(.fileWriteNoPermission) }
try await engine.reconcile([], download: fetch, validate: { _ in true }, export: denied)
try await engine.reconcile([new], download: fetch, validate: { _ in true }, export: denied)
do {
try await engine.reconcile([new], download: fetch, validate: { _ in true }, export: denied)
XCTFail("Denied export reported success")
} catch {}
XCTAssertEqual(downloads, 1)
XCTAssertEqual(engine.journal.receipts[new.id]?.destination, .pending)
XCTAssertEqual(try Data(contentsOf: engine.file(for: new.id)), Data([1, 2, 3]))
let restarted = try ScreenshotSyncEngine(root: root, photos: true)
try await restarted.reconcile([], download: fetch, validate: { _ in true }, export: { _, allowed in
XCTAssertTrue(allowed()); exports += 1
})
XCTAssertEqual(downloads, 1)
XCTAssertEqual(exports, 1)
XCTAssertEqual(restarted.journal.receipts[new.id]?.destination, .saved)
}
func testLateExportSuccessAndFailureCannotOverwriteReplacementJournal() async throws {
for succeeds in [true, false] {
let root = try temporary()
let oldEngine = try ScreenshotSyncEngine(root: root, photos: true)
let fetch: (HeadsetScreenshot) async throws -> Data = { _ in Data([1, 2, 3]) }
let noop: (URL, () -> Bool) async throws -> Void = { _, _ in }
try await oldEngine.reconcile([], download: fetch, validate: { _ in true }, export: noop)
try await oldEngine.reconcile([new], download: fetch, validate: { _ in true }, export: noop)
var suspended: CheckedContinuation<Void, Never>?
let task = Task {
try await oldEngine.reconcile([new], download: fetch, validate: { _ in true }, export: { _, allowed in
await withCheckedContinuation { suspended = $0 }
XCTAssertFalse(allowed(), "Stopped export must lose eligibility")
if !succeeds { throw CocoaError(.fileWriteNoPermission) }
})
}
while suspended == nil { await Task.yield() }
oldEngine.stop()
let replacement = try ScreenshotSyncEngine(root: root, photos: false)
let other = HeadsetScreenshot(id: "2/10/20261024010001_1.jpg", size: 3, time: 3)
try await replacement.reconcile([new, other], download: fetch, validate: { _ in true }, export: noop)
try await replacement.reconcile([new, other], download: fetch, validate: { _ in true }, export: noop)
suspended!.resume()
try await task.value
let reloaded = try ScreenshotSyncEngine(root: root, photos: true)
XCTAssertEqual(reloaded.journal.receipts.count, 2)
XCTAssertEqual(reloaded.journal.receipts[other.id]?.destination, .appOnly)
XCTAssertTrue(reloaded.journal.receipts[other.id]!.downloaded)
XCTAssertEqual(reloaded.journal.receipts[new.id]?.destination, .appOnly)
}
}
func testDefaultPreferenceAndOfflineAppOnlyLibraryAfterRelaunch() throws {
let root = try temporary()
let suite = "frame-screenshot-test-\(UUID().uuidString)"
let defaults = try XCTUnwrap(UserDefaults(suiteName: suite))
defer { defaults.removePersistentDomain(forName: suite) }
let downloads = ScreenshotDownloads(defaults: defaults, storageRoot: root)
XCTAssertTrue(downloads.saveToPhotos)
downloads.setPhotos(false)
let identity = "test-headset-pinned-key"
let digest = SHA256.hash(data: Data(identity.utf8)).map { String(format: "%02x", $0) }.joined()
let folder = root.appendingPathComponent("Screenshots/\(digest)")
try FileManager.default.createDirectory(at: folder, withIntermediateDirectories: true)
var ledger = ScreenshotJournal()
ledger.observe([], photos: false)
ledger.observe([new], photos: false)
ledger.receipts[new.id]?.downloaded = true
try JSONEncoder().encode(ledger).write(to: folder.appendingPathComponent("receipts.json"))
let file = folder.appendingPathComponent(new.id)
try FileManager.default.createDirectory(at: file.deletingLastPathComponent(), withIntermediateDirectories: true)
try Data([1, 2, 3]).write(to: file)
let restarted = ScreenshotDownloads(defaults: defaults, storageRoot: root)
XCTAssertFalse(restarted.saveToPhotos)
restarted.load(identity: identity) // No URL, connection or network task.
XCTAssertEqual(restarted.items.count, 1)
XCTAssertEqual(restarted.items[0].status, "In Frame Control")
XCTAssertEqual(try Data(contentsOf: restarted.items[0].file), Data([1, 2, 3]))
}
}
+2 -1
View File
@@ -34,6 +34,7 @@ targets:
path: FrameControl/Info.plist
properties:
CFBundleDisplayName: Frame Control
ITSAppUsesNonExemptEncryption: false
UILaunchScreen:
UIColorName: ""
UISupportedInterfaceOrientations: [UIInterfaceOrientationPortrait, UIInterfaceOrientationLandscapeLeft, UIInterfaceOrientationLandscapeRight]
@@ -41,7 +42,7 @@ targets:
UIUserInterfaceStyle: Dark
NSLocalNetworkUsageDescription: Frame Control finds your Steam Frame on your network and connects to it.
NSBonjourServices: [_steamos-devkit._tcp]
NSPhotoLibraryAddUsageDescription: Frame Control saves headset captures and screenshots to your photo library when you ask it to.
NSPhotoLibraryAddUsageDescription: Frame Control automatically saves new headset screenshots to Photos when Save screenshots to Photos is enabled.
NSAppTransportSecurity:
NSAllowsLocalNetworking: true
LSApplicationQueriesSchemes: [ssh, sftp, steamlink, rdp]
+60
View File
@@ -0,0 +1,60 @@
#!/bin/zsh
# Archive Frame Control and upload it to TestFlight.
#
# Needs a paid Apple Developer Program team and an App Store Connect API key
# (Users and Access → Integrations → App Store Connect API). This Mac needs
# Admin access for Apple's cloud-managed distribution signing; App Manager
# alone was rejected with "Cloud signing permission error". See docs/iphone.md.
#
# ASC_KEY_ID=ABC123 ASC_ISSUER_ID=uuid ASC_KEY_PATH=~/.appstoreconnect/AuthKey_ABC123.p8 \
# ios/scripts/testflight.sh
#
# TEAM_ID defaults to the team in this Mac's signing certificate. The build
# number is the current UTC time, so every upload is higher than the last.
set -euo pipefail
: "${ASC_KEY_ID:?set ASC_KEY_ID}" "${ASC_ISSUER_ID:?set ASC_ISSUER_ID}" "${ASC_KEY_PATH:?set ASC_KEY_PATH}"
TEAM_ID="${TEAM_ID:-NJZUMEA4BN}"
BUILD_NUMBER="${BUILD_NUMBER:-$(date -u +%Y%m%d%H%M)}"
# Resolve the key before changing directory; expand a quoted leading ~/.
key_path="${ASC_KEY_PATH/#\~\//$HOME/}"
key_path="${key_path:A}"
[[ -f "$key_path" ]] || { echo "no API key at $key_path" >&2; exit 1; }
cd "${0:A:h}/.."
out="build/testflight/$BUILD_NUMBER"
mkdir -p "$out"
auth=(-allowProvisioningUpdates
-authenticationKeyPath "$key_path"
-authenticationKeyID "$ASC_KEY_ID"
-authenticationKeyIssuerID "$ASC_ISSUER_ID")
xcodegen generate
xcodebuild archive -project FrameControl.xcodeproj -scheme FrameControl \
-configuration Release -destination generic/platform=iOS \
-archivePath "$out/FrameControl.xcarchive" \
DEVELOPMENT_TEAM="$TEAM_ID" CODE_SIGN_STYLE=Automatic \
CURRENT_PROJECT_VERSION="$BUILD_NUMBER" "${auth[@]}"
# Check the archived build number, not just the argument passed.
archived=$(plutil -extract CFBundleVersion raw \
"$out/FrameControl.xcarchive/Products/Applications/Frame Control.app/Info.plist")
[[ "$archived" == "$BUILD_NUMBER" ]] || { echo "archived build $archived, expected $BUILD_NUMBER" >&2; exit 1; }
cat > "$out/ExportOptions.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>upload</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>automatic</string>
<key>manageAppVersionAndBuildNumber</key><false/>
</dict></plist>
EOF
xcodebuild -exportArchive -archivePath "$out/FrameControl.xcarchive" \
-exportOptionsPlist "$out/ExportOptions.plist" -exportPath "$out/export" "${auth[@]}"
echo "Uploaded build $BUILD_NUMBER. It shows in TestFlight once App Store Connect finishes processing."
Loaded 100 of 154 files, more files were not shown because too many files have changed in this diff. Show more