* Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:
- Desktop: the app panel in use in the headset, streamed from its own window
(x11grab of gamescope's redirected window), so it doesn't move as the
wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
on a computer the mouse, wheel and keyboard work directly. On the headset
view the view is a trackpad. A text field and key row type from a phone.
Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
the view loses focus; keys for the Frame no longer trigger Frame Control's
own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
presses, keys, text and scrolls name their panel (display and window: ids
repeat across :0 and :1, told apart by pid), and the Frame drops them if
focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
packs ":1" into the first value), so a window id repeated across :0 and :1
can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
input too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
"Connecting…"; text goes in 100-character pieces so releases don't wait
behind a long paste; a cancelled mouse gesture releases what's held.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
arrived while paused were dropped), and waits for the device once per
batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Trackpad and key row go through gamescope's input; KDE Connect only for accents and emoji
The pad reaches whichever panel has focus (Steam's menus too), needs nothing installed, and no longer clamps to 1280x720. Characters that aren't on a US keyboard start KDE Connect the first time they're typed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Touch: a bare release no longer clears the stale-focus notice before the page has seen it
The page leaves the panel off releases, so a release says nothing about focus.
Found in review of the stale-latch fix.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: keep typing in order and acknowledge it once, with accents through one KDE Connect queue
Review found accent-only input repeated forever (sent was false), text split
across two transports out of order, accents stuck until the next keypress,
duplicated on retry, and racy. A batch with an accent now sends its whole
keyboard through KDE Connect; later keys follow while any are queued; the
server owns one ordered, bounded queue drained by a single thread; sent means
the whole batch is taken.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: drain the KDE Connect queue without losing, stranding or reordering input
Second review found the queue could be trimmed under an in-flight send (deleting
unsent events), the worker could retire as a request enqueued, and ASCII could
overtake an accent still being typed (or the reverse). The worker now takes its
batch out while sending and puts it back on failure, retires under the lock, and
each way waits out the other's estimated typing time.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: count gamescope's real typing time, with the settle margin added once
Review found the estimate used 12 ms a character where the agent takes 16 ms
(32 shifted), and added the margin on every request so it piled up.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Document gamescope pad verification and shared-device limits
* Prevent Unicode input startup from deadlocking a headset switch
* Record real focused-panel click and ASCII key delivery
* Retain accepted Unicode input under queue saturation and failure
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
* Never end playback over the cosmetic surround; always shut down OpenVR
Review (SWE-2 Max): a non-standby error re-sending the theatre surround
could end a healthy still, and a non-RuntimeError during DestroyOverlay
skipped VR_ShutdownInternal. drain() now drops a surround that fails for
other reasons, and close() shuts down in finally. Fake overlay handles no
longer depend on call order.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Play without the theatre surround if SteamVR refuses it
Review (SWE-2 Max): a non-standby error creating the cosmetic surround
at startup still ended playback before the screen appeared. Theatre now
logs it and plays without the surround. close() treats any per-overlay
teardown error as best effort (signals are already ignored there).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Verify media surround fallback and guard server startup ownership
* Record exact-head real Frame theatre playback smoke
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
* Add bounded desktop power and heat observations with offline downloads
* Fix desktop power tests on Python 3.9
---------
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
ui/index.html: main added the Back up tool link beside the Mac one;
both links and both host checks are kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
host_view now picks the PC host on Windows only; Linux and macOS keep
the Mac view as on main, so #38's Linux adapter is never offered. The PC
host starts nothing, loads no native library and lists no windows until
the person ticks "Turn on PC in the headset" on the card (action
"enable" on /api/macview). Turning it off ends streams, viewers and the
helper. The choice is a client preference, like mvSeparate.
Tools lists "PC in the headset" on Windows only, beside the macOS-only
Mac link, with an Experimental tag and a short note on what hasn't been
tried. The Mac card's wording and Linux-portal tweaks from #38 go back
to main's, apart from the shared "your computer" hint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #38's native PC host. The only conflict was ui/index.html: main
moved the macview card under Tools, so this keeps main's layout; #38's
loadMacView changes applied cleanly. Scoping to Windows and the
Experimental opt-in follow in separate commits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings → Copy to this computer, off by default, with a switch each for
screenshots and Steam clips. While a headset is connected, a worker in the
server looks about once a minute and copies what's new with save_shots and
frame_clips.export_clip: only what's newer than turning it on, never what it
copied before (remembered per headset in auto-sync.json, so deleted files stay
deleted), never Steam's background recording or a clip written to in the last
30 s. It waits for other work on the headset, backs off to 15 minutes while
the headset is unreachable, and pauses when this computer is out of space.
Copy everything now copies the rest once.
The Frame-side clip listing now reports each recording's idle time, by the
Frame's clock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tools → Backup & recovery copies game saves (Proton prefixes' AppData,
Documents and Saved Games, Unity and Godot saves), Steam controller layouts,
desktop settings and Android app data (the existing backup-data helper) to
~/Documents/Frame Control Backups/<time>/ with a manifest of every file's size
and SHA-256 and the headset's BUILD_ID. Backing up only reads the headset;
sign-in files, keys and keyrings are refused by name wherever they are.
Restore checks each part against the headset first and skips, rather than
fails, games not installed yet, prefixes Proton hasn't made, a missing Steam
account and closed or missing Android apps. It asks before replacing anything,
writes only inside each part's folders and keeps every replaced file under
~/.local/share/frame-control/restore-previous/.
Return to Game Mode reads the SDDM session config and runs the switch Valve's
devkit tools use (steamos-session-select / holo-session-select gamescope),
only when the headset isn't already in Game Mode and only after confirmation.
Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running now, at the top of Games, lists what's running on the Frame (Steam
games, sideloaded titles, non-Steam shortcuts, Android apps), with Force quit
on each, on a game's library tile and in a sideloaded title's menu. The
Control Center gets a Force quit a game card. Each asks first: unsaved
progress will be lost. Nothing running says so.
ui/frame_running.py finds games by Steam's `reaper SteamLaunch AppId=<id>`
process and its tree, never by name. It asks Steam to quit
(SteamClient.Apps.TerminateApp), then sends SIGTERM and SIGKILL to the tree,
skipping init, the SSH session, other users and Steam, gamescope, SteamVR,
sshd and the session. Android apps also get the Apps tab's container stop.
The fake Frame runs games under a stand-in reaper and lists them in
SteamUIStore.RunningApps, so tests/e2e/test_running.py force quits real
process trees. Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record on Capture & Live runs ffmpeg on the Frame (x264 from /dev/video99, the
default output's monitor through PipeWire's PulseAudio server), stops it when the
SSH channel closes, remuxes to .mp4 and copies it to ~/Videos/SteamFrame. If the
audio can't be read, it records video only and says so. 20-minute limit, free
space checked on both ends.
Steam clips lists Steam's game recordings (DASH sessions under
userdata/<account>/gamerecordings) and Export joins each stream's init segment
and chunks, then has the Frame's ffmpeg copy them into one .mp4, so clips longer
than Steam's own 60 s export come out whole.
The logic is in ui/frame_clips.py for reuse. Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Home, Tools and the Control Center now call the keyboard "Type on the
headset", and T opens it from anywhere. The clipboard moves from Send to
Frame to sit beside the keyboard, with a new Copy the headset's clipboard
button: the server reads Klipper's getClipboardContents on the headset
desktop's bus (base64 back, 1 MB limit, empty and image clipboards say so),
and the desktop app, the iPhone app or frame_host.set_clipboard_text puts
it on the clipboard here. Clipboard text is never logged.
Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's Import Android game panel joins Quest and Meta under Apps > Browse,
and its Import button (androidBackupOpen, which lived in the old Android
store header and would have thrown at startup here) is now an Add / Install
choice. docs/android-backups.md points to the new places.
Home's shortcuts (Type on the Frame and the rest) now show their tool before
focusing it, so typing works on the first tap and phones open the keyboard
(Sol review P3). A tool panel already in its workspace isn't moved again, which
also dropped focus on the follow-up hashchange.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps the four primary destinations and ports main's newer work into them:
the Home "Right now" panel (volume, cast, quick actions) replaces the old
device block, whose readings now live in the header's headset popover; the
install queue sits above the Apps views; Import from my Quest and My Meta
library sit with Browse; the pad, panel switcher, Mac view and media panels
take main's markup. Scrolling follows main's #scroll container. Settings
stays a tab, so main's header gear is not added. The old .home-status
toggle is gone (main's startup test caught it touching a missing node).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Primary tabs become Apps (Installed / Browse / Linux apps), Games (Installed /
Owned / Store), Tools and Settings, with shortcuts 1-4 and one Add / Install
chooser for catalogs, Steam, Linux apps, files and install links. Installed
rows show identity, one state and Open/Stop; the rest moves to app details.
Review fixes (#106): input, touch, approvals, artwork repair, keyboard helper
delivery and catalogue jobs stay bound to the headset they were meant for;
report preview follows the latest consent; failed Store reads are not an empty
library; keyboard access to device details and the Store sheet; the iOS
bundle ships the artwork renderer and OpenXR resources; typed addresses and
custom SSH ports survive setup; WebKit crashes recover; native preferences
survive relaunch; APK update rollback and repair stay retryable.
Refs #101, #104, #106
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Installs from the Android store, Import from my Quest and My Meta library
now go into one server-side queue and run one after another, instead of
being refused while another runs. The page shows the queue with Remove
for installs that haven't started; each queued install runs only on the
headset it was queued for.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ui/store.html, scripts/store-on-frame.sh and the /store route (built in
another thread and left uncommitted in the main checkout) join this branch,
rebuilt on /api/search: the same sources, categories, phone-only rule and
artwork as the Store, as a 1280x800 laser panel. A category rail with
Installed, eight art cards a page, a featured VR app on Home, a detail
sheet with screenshots, install progress and game data, the thumbstick
turning pages and the on-screen keyboard.
- Artwork: the proxy allowed four name lookups at once and failed the rest
immediately, so a page of cards lost most of its images until the retry.
Public answers are now kept for five minutes and a lookup waits for a slot
within its own budget.
- Phones: one-line shelf headings, a shorter store header, wider shelf
cards, and a two-column library.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The store browsed F-Droid alphabetically, so it opened on proxies, port
checkers and root tools. It now groups apps by what a headset is good for
(VR, Games, Emulators, Watch & listen, Read & learn, Create, Work, Web &
chat, Tools) from F-Droid's own categories, hides phone-only apps unless
asked, and browses each source's best-illustrated, newest apps first.
- Store: a category sidebar (chips on narrow windows), one shelf per
category on the home page with no repeats, a featured VR app, and cards
that show a banner, phone screenshots side by side, or the icon over its
own blur instead of a letter, plus whether it fits the Frame.
- Library: its own view with art cards, Play, and a menu for Test, Report
and Remove; filters for VR, flat and running apps; display settings and
reports below. Artwork comes from the saved source indexes.
- Images: listings register three screenshots per app instead of twelve,
handles are indexed by URL, and the bound is 16,384, so a large browse no
longer forgets its own images before the page loads them.
- F-Droid cache version 3 keeps each app's categories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for the sign-in window:
- Each sign-in has a random attempt id. Closing, timing out or signing out
stops cookie intake first, then tells the server (new cancel-signin action)
not to save a connect from that attempt, even one still waiting on Meta
(frame_meta.connect/cancel_signin; 409 from the server). Disconnect signs
out of the window before deleting the token.
- will-frame-navigate holds every frame to the allow-list; a frame going
elsewhere is stopped, never opened outside.
- The window is destroyed, not closed, and will-prevent-unload is overridden,
so beforeunload can't keep it alive; sign-out clears the session only after
the window is gone.
- Permission checks refused as well as requests; no developer tools.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, in ui/frame_android.py where main added COPY_MIN_RATE (size-scaled
copy timeouts) beside the install lock this branch made reentrant: both kept,
the lock stays an RLock. ui/index.html and ui/server.py merged on their own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Frame error during a test (an ssh failure with empty stderr raises its
stdout, which can be the game's log) becomes the fixed "couldn't read the
game's log from the Frame" message, with no original text and no chained
cause. run_test wraps every step; the /api/meta test job also maps any
other exception to a fixed text, and the CLI and Install and test go
through run_test. Close failures no longer echo their error text.
- Log scans are linear: signatures look at the first 4 KB of each line and
every repetition followed by more required text is bounded; excerpts
split the redacted line once and fullmatch only words of 64 characters or
less; the generic OC/FRL token pattern matches each run once (no
lookahead) and keeps it unless it has a digit. Timing tests: 100 KB
adversarial lines through classify, redact and every excerpt each finish
well under 200 ms (the old library and token patterns took 19 s and 9 s).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opt-in, off until you connect: paste your own Meta access token, which is
checked with a read-only "who am I" query and kept owner-only in
data_dir/meta-account.json. Lists the Quest apps the account owns, downloads
an owned app's APK and OBB from Meta's download server into the install
cache (size limits, declared-size and optional SHA-256 checks, cleaned up
after every install), and installs it through Import from my Quest's checks,
install and two-step data copy. Install and test runs the game, classifies
the outcome from its log (runs, runs with issues, quits, crashes, hangs) with
a plain-English cause, and records it through the existing compatibility
report path with source meta-store and the version code, shared only per the
Privacy settings.
The token goes only to graph.oculus.com and securecdn.oculus.com over
HTTPS, is dropped on redirects elsewhere, and is redacted (any case,
percent-encoded, before truncation) from every message. A disconnect wins
over a connect in flight and stops downloads. No licence-check bypass.
Protocol facts come from the public QAVS, OculusGraphQLApiLib, DownQuest and
BSManager sources (docs/meta-library.md); the code is our own.
UI: Android -> My Meta library. CLI: ui/frame_meta.py. API: /api/meta.
Tested offline with a fake Meta and fake Frame (tests/test_meta.py).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- An older Quest copy no longer replaces a newer install on the Frame: installs
now record versionCode, and a downgrade is refused unless allowed
(--allow-downgrade, "allowDowngrade", or the panel's checkbox).
- The game-data copy only targets this package's own install record and
instance at the Quest's exact version code, and checks Android's own
dumpsys in that instance before copying.
- After the (possibly long) pulls the target is checked again, and the final
checks, the transfer and closing the game hold the shared install lock,
now reentrant so install_obb and install can be called under it.
- Closing the game is confirmed; a failed or incomplete close is reported
instead of claiming the game was closed.
- Split members are read from their raw bytes: stored members must be exactly
their declared size, deflated streams must end exactly there, then the
checksum; forged sizes with prefix checksums are refused.
- A split's resource table counts as empty only if no type chunk defines any
entry (bags included) and every chunk is understood.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New ui/frame_quest.py (CLI: devices | list | import), GET/POST /api/quest and
an Import from my Quest panel on the Android page. It finds a Quest on this
computer's adb (never Frame Control's own tunnelled serials), lists third-party
apps with split, OBB and Android/data details, pulls the APKs into a per-user
cache and checks them before copying anything else: Meta's Platform SDK
licence check (libovrplatformloader.so) and 32-bit-only builds are refused,
VrApi-only builds flagged. It then installs through the normal APK path, adds
OBBs with the existing helper and, if asked, Android/data files, and always
removes the cache. No licence-check bypass is bundled, called or linked.
Tests use a fake adb (tests/fakeadb.py). Docs: docs/quest-import.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F-Droid repositories can now be added with HTTP basic auth, either as
https://user:pass@host/fdroid/repo (as F-Droid's client accepts) or with
separate username/password fields (store form, server API, and the CLI's
--username/--password-stdin). The login is stripped from the saved URL and
kept in a new shared module, apk_sources/_credentials.py, which writes
source-credentials.json in the data folder with mode 0600.
A credential is only ever added per request to its exact origin (scheme,
host, port): redirects elsewhere get nothing and have Authorization
removed. HTTPS only. 401 (or 403 with a login) on add gives a "check the
username and password" error and saves nothing; signature, pin, TOFU,
hash and rollback checks are unchanged. Removing a repository deletes its
login.
GitHub gets a user-settable token (FRAME_GITHUB_TOKEN still wins), sent as
Bearer only to api.github.com. If GitHub answers 401, the request is
retried once without it and the rejection is reported in source settings.
The source list shows only whether a token is set, never its value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every call (one per 5 s poll) sent ~35 KB of frame_comfort.py, frame_status.py
and frame_steam.py over ssh, though they are already cached on the headset in a
content-addressed folder. Now the first ssh only checks that folder and runs
the cached copy; if any file is missing it answers {"comfort": "missing"} and
a second call writes the sources as before (umask 077, temp file then rename,
same version hash).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page polled POST /api/comfort status every 5 s whatever the connection,
and every POST Failure >= 500 went to diagnostics. ssh already marks link
failures, but those are still sent once per session per category, so each
session with a sleeping, away or re-keyed headset sent up to three reports
from this poll (most of 0.4.2's error reports). GET /api/status never reports
Failures at all.
- server: a polled POST read (POLLS: comfort status) that couldn't reach the
headset isn't a diagnostic, as with GET /api/status. Real comfort errors,
and link failures from starting or cancelling a session, still are.
- page: the comfort poll starts after the first status check and skips its
ssh while the Frame is offline (online === false); status checks bring
it back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frame_connected now carries headset_id, a one-way hash of the headset's
/etc/machine-id worked out on the headset (frame_status.py), and
headsets_saved, how many headsets this copy of the app has set up. It is
sent once per build for each headset. The hash goes to usage statistics
only: the status endpoint and MCP clients don't get it. docs/privacy.md
says what is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
