* Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:
- Desktop: the app panel in use in the headset, streamed from its own window
(x11grab of gamescope's redirected window), so it doesn't move as the
wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
on a computer the mouse, wheel and keyboard work directly. On the headset
view the view is a trackpad. A text field and key row type from a phone.
Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
the view loses focus; keys for the Frame no longer trigger Frame Control's
own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
presses, keys, text and scrolls name their panel (display and window: ids
repeat across :0 and :1, told apart by pid), and the Frame drops them if
focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
packs ":1" into the first value), so a window id repeated across :0 and :1
can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
input too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
"Connecting…"; text goes in 100-character pieces so releases don't wait
behind a long paste; a cancelled mouse gesture releases what's held.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
arrived while paused were dropped), and waits for the device once per
batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Trackpad and key row go through gamescope's input; KDE Connect only for accents and emoji
The pad reaches whichever panel has focus (Steam's menus too), needs nothing installed, and no longer clamps to 1280x720. Characters that aren't on a US keyboard start KDE Connect the first time they're typed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Touch: a bare release no longer clears the stale-focus notice before the page has seen it
The page leaves the panel off releases, so a release says nothing about focus.
Found in review of the stale-latch fix.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: keep typing in order and acknowledge it once, with accents through one KDE Connect queue
Review found accent-only input repeated forever (sent was false), text split
across two transports out of order, accents stuck until the next keypress,
duplicated on retry, and racy. A batch with an accent now sends its whole
keyboard through KDE Connect; later keys follow while any are queued; the
server owns one ordered, bounded queue drained by a single thread; sent means
the whole batch is taken.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: drain the KDE Connect queue without losing, stranding or reordering input
Second review found the queue could be trimmed under an in-flight send (deleting
unsent events), the worker could retire as a request enqueued, and ASCII could
overtake an accent still being typed (or the reverse). The worker now takes its
batch out while sending and puts it back on failure, retires under the lock, and
each way waits out the other's estimated typing time.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Pad: count gamescope's real typing time, with the settle margin added once
Review found the estimate used 12 ms a character where the agent takes 16 ms
(32 shifted), and added the margin on every request so it piled up.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Document gamescope pad verification and shared-device limits
* Prevent Unicode input startup from deadlocking a headset switch
* Record real focused-panel click and ASCII key delivery
* Retain accepted Unicode input under queue saturation and failure
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
* Never end playback over the cosmetic surround; always shut down OpenVR
Review (SWE-2 Max): a non-standby error re-sending the theatre surround
could end a healthy still, and a non-RuntimeError during DestroyOverlay
skipped VR_ShutdownInternal. drain() now drops a surround that fails for
other reasons, and close() shuts down in finally. Fake overlay handles no
longer depend on call order.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Play without the theatre surround if SteamVR refuses it
Review (SWE-2 Max): a non-standby error creating the cosmetic surround
at startup still ended playback before the screen appeared. Theatre now
logs it and plays without the surround. close() treats any per-overlay
teardown error as best effort (signals are already ignored there).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Verify media surround fallback and guard server startup ownership
* Record exact-head real Frame theatre playback smoke
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
* Add bounded desktop power and heat observations with offline downloads
* Fix desktop power tests on Python 3.9
---------
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
ui/index.html: main added the Back up tool link beside the Mac one;
both links and both host checks are kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
host_view now picks the PC host on Windows only; Linux and macOS keep
the Mac view as on main, so #38's Linux adapter is never offered. The PC
host starts nothing, loads no native library and lists no windows until
the person ticks "Turn on PC in the headset" on the card (action
"enable" on /api/macview). Turning it off ends streams, viewers and the
helper. The choice is a client preference, like mvSeparate.
Tools lists "PC in the headset" on Windows only, beside the macOS-only
Mac link, with an Experimental tag and a short note on what hasn't been
tried. The Mac card's wording and Linux-portal tweaks from #38 go back
to main's, apart from the shared "your computer" hint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#38 swapped the Mac agent's Swift RateController for a binding to
desktop/controller.c. That is a change to a shipped, headset-verified
feature that this Windows-only PR can't test on a Mac, so the Mac agent,
its build script and the staleness check go back to main. The C port
stays for the Windows host and is still checked against the frozen trace
of the Swift original.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #38's native PC host. The only conflict was ui/index.html: main
moved the macview card under Tools, so this keeps main's layout; #38's
loadMacView changes applied cleanly. Scoping to Windows and the
Experimental opt-in follow in separate commits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a Frame (build 20261015.6141207) gamescope runs as gamescope-wl, so the session
check said unknown, and the button would have restarted a headset that was in
Game Mode. Game Mode now also counts gamescope-wl and start-gamescope-session, and
the switch runs only when Plasma is running without gamescope.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a Frame (SteamOS 0.5.6, build 20261015.6141207) Stop left ffmpeg recording:
ffmpeg n7.0 there catches SIGTERM and SIGINT and carries on. The recording shell
now gives ffmpeg a FIFO as stdin, types q into it on Stop or a dropped channel, and
SIGKILLs it if it's still there 10 s later. Tested end to end on that Frame: an
8 s 1080p H.264 + AAC .mp4 saved 1.3 s after Stop, nothing left on the Frame.
Also records the read-only charge-limit checks run on the same Frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings → Copy to this computer, off by default, with a switch each for
screenshots and Steam clips. While a headset is connected, a worker in the
server looks about once a minute and copies what's new with save_shots and
frame_clips.export_clip: only what's newer than turning it on, never what it
copied before (remembered per headset in auto-sync.json, so deleted files stay
deleted), never Steam's background recording or a clip written to in the last
30 s. It waits for other work on the headset, backs off to 15 minutes while
the headset is unreachable, and pauses when this computer is out of space.
Copy everything now copies the rest once.
The Frame-side clip listing now reports each recording's idle time, by the
Frame's clock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tools → Backup & recovery copies game saves (Proton prefixes' AppData,
Documents and Saved Games, Unity and Godot saves), Steam controller layouts,
desktop settings and Android app data (the existing backup-data helper) to
~/Documents/Frame Control Backups/<time>/ with a manifest of every file's size
and SHA-256 and the headset's BUILD_ID. Backing up only reads the headset;
sign-in files, keys and keyrings are refused by name wherever they are.
Restore checks each part against the headset first and skips, rather than
fails, games not installed yet, prefixes Proton hasn't made, a missing Steam
account and closed or missing Android apps. It asks before replacing anything,
writes only inside each part's folders and keeps every replaced file under
~/.local/share/frame-control/restore-previous/.
Return to Game Mode reads the SDDM session config and runs the switch Valve's
devkit tools use (steamos-session-select / holo-session-select gamescope),
only when the headset isn't already in Game Mode and only after confirmation.
Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running now, at the top of Games, lists what's running on the Frame (Steam
games, sideloaded titles, non-Steam shortcuts, Android apps), with Force quit
on each, on a game's library tile and in a sideloaded title's menu. The
Control Center gets a Force quit a game card. Each asks first: unsaved
progress will be lost. Nothing running says so.
ui/frame_running.py finds games by Steam's `reaper SteamLaunch AppId=<id>`
process and its tree, never by name. It asks Steam to quit
(SteamClient.Apps.TerminateApp), then sends SIGTERM and SIGKILL to the tree,
skipping init, the SSH session, other users and Steam, gamescope, SteamVR,
sshd and the session. Android apps also get the Apps tab's container stop.
The fake Frame runs games under a stand-in reaper and lists them in
SteamUIStore.RunningApps, so tests/e2e/test_running.py force quits real
process trees. Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record on Capture & Live runs ffmpeg on the Frame (x264 from /dev/video99, the
default output's monitor through PipeWire's PulseAudio server), stops it when the
SSH channel closes, remuxes to .mp4 and copies it to ~/Videos/SteamFrame. If the
audio can't be read, it records video only and says so. 20-minute limit, free
space checked on both ends.
Steam clips lists Steam's game recordings (DASH sessions under
userdata/<account>/gamerecordings) and Export joins each stream's init segment
and chunks, then has the Frame's ffmpeg copy them into one .mp4, so clips longer
than Steam's own 60 s export come out whole.
The logic is in ui/frame_clips.py for reuse. Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Home, Tools and the Control Center now call the keyboard "Type on the
headset", and T opens it from anywhere. The clipboard moves from Send to
Frame to sit beside the keyboard, with a new Copy the headset's clipboard
button: the server reads Klipper's getClipboardContents on the headset
desktop's bus (base64 back, 1 MB limit, empty and image clipboards say so),
and the desktop app, the iPhone app or frame_host.set_clipboard_text puts
it on the clipboard here. Clipboard text is never logged.
Not tested on a headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's Import Android game panel joins Quest and Meta under Apps > Browse,
and its Import button (androidBackupOpen, which lived in the old Android
store header and would have thrown at startup here) is now an Add / Install
choice. docs/android-backups.md points to the new places.
Home's shortcuts (Type on the Frame and the rest) now show their tool before
focusing it, so typing works on the first tap and phones open the keyboard
(Sol review P3). A tool panel already in its workspace isn't moved again, which
also dropped focus on the follow-up hashchange.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps the four primary destinations and ports main's newer work into them:
the Home "Right now" panel (volume, cast, quick actions) replaces the old
device block, whose readings now live in the header's headset popover; the
install queue sits above the Apps views; Import from my Quest and My Meta
library sit with Browse; the pad, panel switcher, Mac view and media panels
take main's markup. Scrolling follows main's #scroll container. Settings
stays a tab, so main's header gear is not added. The old .home-status
toggle is gone (main's startup test caught it touching a missing node).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Section headings, the tabs, the brand and the activity bar drop small capitals
for sentence case at one heading size. On a phone with a headset picker, the
connection pill keeps only its status dot so the name shows once. Sideloaded
titles get Play and a ⋯ menu (Add artwork, Remove) like installed apps. Manage
sources moves under Source & compatibility. Library artwork is a Settings
page, so Tools no longer lists it.
Refs #101, #106
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
