Compare commits

..
Author SHA1 Message Date
SAPHIDandClaude Opus 5.5 e179616ad0 Record the headset with its sound, and export Steam clips of any length
Record on Capture & Live runs ffmpeg on the Frame (x264 from /dev/video99, the
default output's monitor through PipeWire's PulseAudio server), stops it when the
SSH channel closes, remuxes to .mp4 and copies it to ~/Videos/SteamFrame. If the
audio can't be read, it records video only and says so. 20-minute limit, free
space checked on both ends.

Steam clips lists Steam's game recordings (DASH sessions under
userdata/<account>/gamerecordings) and Export joins each stream's init segment
and chunks, then has the Frame's ffmpeg copy them into one .mp4, so clips longer
than Steam's own 60 s export come out whole.

The logic is in ui/frame_clips.py for reuse. Not tested on a headset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:13:16 +00:00
23 changed files with 1537 additions and 604 deletions

No files matched your search

+6 -6
View File
@@ -63,17 +63,17 @@ About 4,500 F-Droid apps rated for the Frame. One click installs each as its own
<tr>
<td valign="top">
**📁 Files and games**<br>
Drag files onto the window to send them. Drop a game's .zip, folder or .exe to add it to the Steam library, with Proton or the Linux runtime picked for you.
**📁 Files, games and clipboard**<br>
Drag files onto the window to send them. Drop a game's .zip, folder or .exe to add it to the Steam library, with Proton or the Linux runtime picked for you. Send text or your clipboard straight to the headset's desktop.
</td>
<td valign="top">
**📸 Screenshots**<br>
Browse the shots you take in the headset and save them to your Pictures folder.
**📸 Screenshots and recordings**<br>
Browse the shots you take in the headset and save them to your Pictures folder. Record what the headset shows, with its sound, or export Steam's clips at any length, as .mp4 files in your Videos folder ([not tested on a headset yet](docs/recording.md)).
**⌨️ Type on the headset**<br>
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Copy text to the headset's clipboard and back again. Nothing else to install, anywhere. Press T to open it.
**⌨️ Keyboard and trackpad**<br>
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Nothing else to install, anywhere.
</td>
</tr>
-6
View File
@@ -282,12 +282,6 @@ ipcMain.handle("server:restart", (e) => {
if (win && e.sender === win.webContents && e.senderFrame && e.senderFrame.url.startsWith("data:")) restartServer();
});
ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : "");
// Text from the headset's clipboard onto this computer's.
ipcMain.handle("clipboard:write", (e, text) => {
if (!fromUi(e) || typeof text !== "string") return false;
clipboard.writeText(text);
return true;
});
ipcMain.handle("preferences:read", (e) => fromUi(e) ? preferences.read() : {});
ipcMain.handle("preferences:write", (e, key, value) => fromUi(e) ? preferences.write(key, value) : false);
// A PNG or JPEG (a screenshot) onto the clipboard as an image.
+2 -3
View File
@@ -1,5 +1,5 @@
// Lets the page read and write this computer's clipboard through Electron, so sending
// it to the Frame, and copying the Frame's back, needs no pbpaste, PowerShell, xclip or wl-clipboard. Also tells
// Lets the page read this computer's clipboard through Electron, so sending it
// to the Frame needs no pbpaste, PowerShell, xclip or wl-clipboard. Also tells
// the page where a dropped file or folder lives, so a folder can be sideloaded
// as a title without zipping it (the local server reads it from there).
// It can put a screenshot on the clipboard as an image, open Set Up Connection when
@@ -17,7 +17,6 @@ contextBridge.exposeInMainWorld("frameApp", {
},
notify: (message, request) => ipcRenderer.invoke("comfort:notify", message, request),
readClipboard: () => ipcRenderer.invoke("clipboard:read"),
writeClipboard: (text) => ipcRenderer.invoke("clipboard:write", text),
writeImage: (bytes) => ipcRenderer.invoke("clipboard:writeImage", bytes),
setUpConnection: () => ipcRenderer.invoke("connection:setup"),
restartServer: () => ipcRenderer.invoke("server:restart"), // the "couldn't start" page's Try Again
-25
View File
@@ -36,28 +36,3 @@ desktop has to be running in the headset. It's text only, and pastes over about
A simpler fallback: `ssh frame 'cat > ~/clip.txt'` < file, then open it in the
headset.
### In Frame Control
**Tools → Type on the headset** (also on Home, and press `T` anywhere) has the
clipboard next to the keyboard and trackpad, both ways:
- **Send text** or **Send this computer's clipboard** puts text on the headset's
clipboard, as above.
- **Copy the headset's clipboard** brings it back: copy something in a desktop
app on the Frame, then press it, and paste on your computer. It calls
Klipper's `getClipboardContents` on the same D-Bus bus, and sends the text
back as base64, so accents, emoji and line breaks arrive unchanged. The
desktop app and the iPhone app put it on their own clipboard. With the server
in a browser, the server sets the clipboard of the computer it runs on:
`pbcopy` on macOS, PowerShell's `Set-Clipboard` (or `clip.exe`) on Windows,
`wl-copy`, `xclip` or `xsel` on Linux.
It's text only, and copying back takes up to 1 MB; a bigger clipboard is
refused with a message, so send big things as a file. An empty clipboard, or
one holding an image, says so. Neither the server nor its log ever records what
was on the clipboard.
**Not tested on a headset yet.** Reading uses the same Klipper interface as
sending, which was verified on 2026-09-25, but `getClipboardContents` itself
hasn't been tried on a Frame. The fake Frame (`tests/fakeframe`) answers it.
+9 -11
View File
@@ -25,7 +25,7 @@ settings, reports), **Tools** (in the headset: your computer's windows, the medi
player and the panel switcher; then sending files and text, Flatpaks, remote and
power), **Devices** (your headsets and their addresses), and **Control** (the
searchable Control Center, metrics, supported settings and headset guidance).
Keys 1–6 switch between them, and `T` opens Type on the headset. The gear in the header (key 7) opens **Settings**:
Keys 1–6 switch between them. The gear in the header (key 7) opens **Settings**:
privacy and updates, library artwork and the assistant. The battery chip opens
the headset's details: storage, memory, temperature, Wi-Fi, uptime and SteamOS
build. Files can be dropped anywhere in the window. A
@@ -49,6 +49,10 @@ counts them while they run.
gamescope's flat layer instead.
- **Screenshots** you take in the headset with Steam's shortcut: browse them and
save them to `~/Pictures/SteamFrame`.
- **Record** what the headset shows, with its sound, to an `.mp4` in
`~/Videos/SteamFrame`, and **Steam clips**: export clips and background
recordings from Steam's game recording at any length, without re-encoding.
See [recording.md](recording.md). Not tested on a headset yet.
- **Battery** with charging state: charge rate in watts, time to full or empty,
charger type and wattage (for example USB-C PD 20 W), and battery temperature.
- **Status**: storage, memory, temperature, Wi-Fi, uptime, and whether SteamVR,
@@ -80,14 +84,8 @@ counts them while they run.
their own Android app. A game's `.zip`, folder or `.exe` becomes a title in
the Steam library (Valve's Devkit Game path, with Proton or the Steam Linux
Runtime picked from the program's header), listed under **Sideloaded titles**
with Launch and Remove; see [sideloading.md](sideloading.md).
- **Type on the headset** (on Home, in Tools, in the Control Center, or press
`T`): use your computer's keyboard and mouse, or a phone's touchscreen, in the
Frame's apps through KDE Connect, and move clipboard text both ways. Send
typed text or your computer's clipboard to the headset, or copy the headset's
clipboard back: copy in a desktop app on the Frame, then **Copy the headset's
clipboard**. The clipboard needs the desktop panel open in the headset; it's
text only, up to 1 MB. See [file-transfer.md](file-transfer.md#clipboard).
with Launch and Remove; see [sideloading.md](sideloading.md). Send typed text, or your computer's clipboard, to the
Frame clipboard.
- **Mac in the headset** (macOS): show any Mac window, or a whole screen, as
its own panel in the headset. Place it with the SteamVR dashboard, click and
scroll with the laser, and type on the Mac. Streams hardware H.264 through
@@ -226,8 +224,8 @@ Google publishes no arm64 Linux platform-tools. Set Up Connection runs
`ui/frame_connect.py` in your terminal emulator (GNOME Terminal, Konsole, xterm
and others). The log is at
`~/.config/Frame Control/logs/server.log`. Running `ui/server.py` in a browser
instead of the app, sending the clipboard, or copying the headset's to this
computer, needs `wl-clipboard` (Wayland), `xclip` or `xsel` (X11).
instead of the app, sending the clipboard needs `wl-clipboard` (Wayland) or
`xclip` (X11).
## Building
+2 -3
View File
@@ -102,9 +102,8 @@ app installed opens its App Store page.
Things iOS asks the first time: **Local Network** (tap Allow, or the app can't
see the Frame), and **Paste** when you send the iPhone's clipboard (tap Allow
Paste, or set Settings → Apps → Frame Control → Paste from Other Apps → Allow).
Sending text to the Frame's clipboard, or copying the Frame's to the phone
(**Type on the headset → Copy the headset's clipboard**), needs the desktop
panel open in the headset, as on the desktop app.
Sending text to the Frame's clipboard needs the desktop panel open in the
headset, as on the desktop app.
Not yet exercised: Android display changes through podman (no Android app was
running), a real sleep/restart/shut down on the Frame, and a physical iPhone.
+120
View File
@@ -0,0 +1,120 @@
# Recording and Steam clips
Frame Control can record what the headset shows, with its sound, and save it as
an `.mp4` on your computer. It can also export the clips and background
recordings from Steam's own game recording, at any length, as one `.mp4` each.
Both are on **Tools → Capture & Live**. Everything is saved to
`~/Videos/SteamFrame`, next to the screenshots in `~/Pictures/SteamFrame`.
Nothing needs installing on the computer or on the Frame: the work is done by
the `ffmpeg` that SteamOS ships.
**Not tested on a headset yet.** This is built from what the code already does
on the Frame (the live view, the media player) and from how Steam is reported
to store its recordings. The parts we're least sure of are listed under
[What hasn't been checked](#what-hasnt-been-checked). Please report what you see.
## Record
Press **Record** next to **Live**. The button turns into **Stop**, and a line
under the viewer counts the time. Press **Stop**: the recording is finished on
the Frame, converted to `.mp4` there (without re-encoding) and copied to
`~/Videos/SteamFrame/rec-<date>-<time>.mp4`. The line says when it's saved.
- **What's in it**: the same picture as **Live**, SteamVR's copy of the headset
view (one undistorted 1920×1080 eye, from `/dev/video99`), at 30 fps and
8 Mbit/s, encoded with x264 (the Frame's hardware encoder crashes `ffmpeg`).
The sound is what the headset plays: the monitor of PipeWire's default
output, read through its PulseAudio server (`-f pulse -i @DEFAULT_MONITOR@`).
The microphone isn't recorded.
- **Without sound**: before each recording Frame Control checks for a fraction
of a second that `ffmpeg` can read that audio. If it can't, the recording goes
ahead as video only, and the page and the saved message say so.
- **Limits**: a recording stops by itself after 20 minutes and is saved as if
you'd pressed Stop. It needs about 2.9 GB free on the Frame to start (room for
a full-length recording twice, while it's converted), and checks the space on
your computer before copying. One recording runs at a time.
- **Where it waits on the Frame**: `~/.cache/frame-control/video/`, not `/tmp`,
which is a RAM disk on SteamOS. It's deleted once copied.
- **If the connection drops or the app quits** while recording, the Frame
stops `ffmpeg` cleanly when the SSH channel closes, as the live view does. A
recording that was never copied is deleted when the next one starts.
Recording and **Live** can run together; both read the same device.
## Steam clips
**Steam clips** lists what Steam's game recording has kept on the Frame: saved
clips and background recordings, newest first, with the game, length and size.
**Export** joins one into a single `.mp4` in `~/Videos/SteamFrame`, named after
its folder (for example `clip_620_20261020_202134.mp4`), however long it is.
Steam's own export is reported to stop at about 60 seconds. Exports run in the
background; ones already on your computer are marked.
How it works: Steam keeps each recording as MPEG-DASH segments, one folder per
recording session.
```
~/.local/share/Steam/userdata/<account>/gamerecordings/
video/bg_<appid>_<YYYYMMDD>_<HHMMSS>/ background recording
clips/clip_<appid>_<YYYYMMDD>_<HHMMSS>/video/bg_<...>/ a saved clip
session.mpd
init-stream0.m4s chunk-stream0-00001.m4s chunk-stream0-00002.m4s ...
init-stream1.m4s chunk-stream1-00001.m4s ...
```
Each stream's `init-stream` segment followed by its chunks, in number order, is
a playable fragmented MP4 of that stream. On the Frame, `ui/frame_clips.py`
builds those files and has `ffmpeg` put the video and audio into one `.mp4`
with `-c copy`, so nothing is re-encoded and it takes seconds. The result is
copied to the computer and deleted from the Frame. The export needs free space
on the Frame of about twice the recording's size while it runs.
The listing doesn't rely on the folder names: any folder under `clips/` or
`video/` (up to four levels down) holding `init-stream*.m4s` counts. A stream
missing its init segment or its chunks is left out. Missing chunk numbers are
shown as "pieces missing"; the rest still exports.
## For developers
`ui/frame_clips.py` holds the logic, so other features (such as syncing clips
automatically) can reuse it:
- `list_clips(run_frame)` and `export_clip(id, run_frame, run, fetch)` are the
computer's side; the server passes how to reach the Frame.
- `list_recordings(userdata)` and `export(userdata, id)` are the Frame's side,
run as `python3 - list` / `python3 - export ID` with the file on stdin.
- `Recorder` runs one recording from Start to the copy.
- `parse_id` checks a clip id (`<account>/<clips|video>/<folder>...`, plain
names only, no `.` or `..`) before it reaches a path or a shell.
The API, with the same guards as every other request:
| Request | Does |
|---|---|
| `GET /api/record` | The recording's state: `idle`, `starting`, `recording` (with `elapsed`, `audio`, `limit`), `stopping`, `saving`, `saved` (with `file`) or `failed` (with `error`) |
| `POST /api/record {"action": "start"\|"stop"}` | Start or stop |
| `GET /api/clips` | Steam's recordings on the Frame, each with `saved` |
| `POST /api/clips/export {"id"}` | Starts an export; poll `/api/job` |
| `POST /api/open {"what": "videos"}` | Opens `~/Videos/SteamFrame` |
They aren't available when the server runs on the Frame itself (the iPhone
app), since they save to a computer.
Tests: `tests/test_clips.py`. They run the recording's shell under bash with a
stand-in `ffmpeg`, so Stop, the time limit and the clean-up are tested as they
run on the Frame. Where `ffmpeg` is installed, they also make a 65-second DASH
recording with Steam's file names and export it with the real `ffmpeg`.
## What hasn't been checked
None of this has run on a Steam Frame yet. In particular:
- That SteamOS's `ffmpeg` can read `@DEFAULT_MONITOR@`. It has the PulseAudio
output device (the media player plays through it), so the input is expected,
but if it fails the recording is video only, as above.
- Audio and video sync. Both inputs are stamped with the wall clock, because
the camera device and the audio server otherwise count from different clocks.
- The exact layout of `gamerecordings` on the Frame's SteamOS build, and
whether a clip's segments start mid-recording. The parser is deliberately
loose about the layout.
- How much x264 at 1080p30 costs the Frame while a game is running.
+1 -1
View File
@@ -144,7 +144,7 @@ capture it.
## Input: type and point in the Frame from the Mac or iPhone
**Built: Home → Type on the headset** (or press `T`), in every version of Frame Control
**Built: Home → Keyboard and trackpad**, in every version of Frame Control
(Mac, Windows, Linux, iPhone and iPad), with nothing to install on the device
you're holding. On a phone the panel is a trackpad (drag to move, tap to click,
two fingers to scroll, two-finger tap to right-click) plus a text field that
-5
View File
@@ -55,7 +55,6 @@ struct WebShell: UIViewRepresentable {
},
notify: (message, request) => call("notify", { message, request }),
readClipboard: () => call("readClipboard"),
writeClipboard: (text) => call("writeClipboard", text),
setUpConnection: () => call("setUpConnection"),
open: (what) => call("open", what),
saveImages: (images) => call("saveImages", images),
@@ -144,10 +143,6 @@ struct WebShell: UIViewRepresentable {
replyHandler(true, nil)
case "readClipboard":
replyHandler(UIPasteboard.general.string ?? "", nil)
case "writeClipboard":
guard let text = arg as? String else { return replyHandler(nil, "Expected text") }
UIPasteboard.general.string = text
replyHandler(true, nil)
case "setUpConnection":
model.showSetup()
replyHandler(nil, nil)
+1 -1
View File
@@ -64,7 +64,7 @@ function sharedPage({productionApi=false} = {}) {
const request=deferred();request.path=path;request.opts=opts;requests.push(request);return request.promise;
};
Object.assign(context,{renderPill(){},renderConnDlg(){},renderBanner(){},viaText(){},netLabel(){},
padReset(){},ctrlReset(){},resetShots(){},closeDevPop(){},loadDevices(){},renderDevices(){},renderTests(){},reloadAll(){},checkContactPrompt(){}});
padReset(){},ctrlReset(){},resetShots(){},resetClips(){},closeDevPop(){},loadDevices(){},renderDevices(){},renderTests(){},reloadAll(){},checkContactPrompt(){}});
vm.runInContext(`const UI_KEY = 'test-only';
const link = {s:null,device:'frame-a',phase:'connected',rev:0,net:null};
const disp = {}, gm = {seq:0}, cat = {}, wi = {}, sourceState = {jobs:new Map(),detail:null,detailRequest:0};
-7
View File
@@ -45,13 +45,6 @@ class Device(harness.FrameTestCase):
self.assertRegex(out['message'], r'^copied via Klipper \(\d+ chars\)$')
self.assertEqual(state()['clipboard'], [text])
def test_clipboard_comes_back_from_klipper(self):
text = 'from the headset: 世界 🎧\n\nwith trailing newlines\n\n'
ok('POST', '/api/clipboard', {'text': text})
out = ok('POST', '/api/clipboard', {'read': True})
self.assertEqual(out['text'], text)
self.assertRegex(out['message'], r"^Copied \d+ characters from the headset's clipboard$")
def test_flatpak_install_and_remove(self):
# Installs run as background jobs (server.start_job); uninstall answers at once.
job = finished(ok('POST', '/api/flatpak', {'id': 'org.videolan.VLC', 'action': 'install'}))
+1 -8
View File
@@ -1,9 +1,7 @@
#!/usr/bin/env python3
"""Stub for qdbus6: records Klipper setClipboardContents calls, the way
Frame Control sends text to the headset desktop's clipboard (server.PASTE,
verified 2026-09-25), and answers getClipboardContents with the last one
(server.COPY). Printing the string and a newline is how qdbus prints a
returned QString; getClipboardContents itself is a guess, not seen on a headset."""
verified 2026-09-25)."""
import os
import sys
@@ -17,10 +15,5 @@ if args[:3] == ['org.kde.klipper', '/klipper', 'org.kde.klipper.klipper.setClipb
sys.exit('Could not connect to D-Bus server')
with fs.update() as s:
s['clipboard'].append(args[3])
elif args == ['org.kde.klipper', '/klipper', 'org.kde.klipper.klipper.getClipboardContents']:
if not os.environ.get('DBUS_SESSION_BUS_ADDRESS'):
sys.exit('Could not connect to D-Bus server')
clips = fs.read()['clipboard']
sys.stdout.buffer.write((clips[-1] if clips else '').encode() + b'\n')
else:
sys.exit(f'qdbus6 stub: unsupported {args}')
-241
View File
@@ -1,241 +0,0 @@
"""The clipboard both ways: the headset's read script, the server's answers, and the
writer for this computer's clipboard on each platform. No headset: SSH and every
clipboard program are mocked, and the read script runs against stub pgrep and qdbus6."""
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import base64
import os
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "ui"))
import frame_host # noqa: E402
import server # noqa: E402
TEXT = "héllo, 世界 🎧\nsecond line\n\n" # accents, CJK, an emoji outside the BMP, trailing newlines
def b64(text):
return base64.b64encode(text.encode()).decode()
class ReadCommand(unittest.TestCase):
def test_reads_klipper_on_the_desktop_bus(self):
self.assertIn("org.kde.klipper.klipper.getClipboardContents", server.COPY)
self.assertIn("DBUS_SESSION_BUS_ADDRESS=$bus qdbus6 org.kde.klipper /klipper", server.COPY)
self.assertIn(server.DESKTOP_CLOSED, server.COPY)
self.assertIn(server.CLIPBOARD_TOO_BIG, server.COPY)
self.assertIn(f"-gt {server.CLIPBOARD_LIMIT + 1} ]", server.COPY) # the limit, plus qdbus6's newline
self.assertIn("base64 -w 0", server.COPY)
def test_command_carries_the_script_intact(self):
encoded = server.COPY_CMD.split("echo ", 1)[1].split(" |", 1)[0]
self.assertEqual(base64.b64decode(encoded).decode(), server.COPY)
self.assertNotIn("'", server.COPY_CMD)
def test_send_still_uses_the_same_bus(self):
self.assertIn(server.DESKTOP_BUS, server.PASTE)
self.assertIn(server.DESKTOP_BUS, server.COPY)
self.assertIn("setClipboardContents", server.PASTE)
@unittest.skipUnless(sys.platform.startswith("linux") and shutil.which("bash") and shutil.which("base64"),
"the script reads /proc, as on the Frame")
class ReadScript(unittest.TestCase):
"""server.COPY for real, with plasmashell's pid and Klipper stubbed."""
def setUp(self):
self.dir = Path(tempfile.mkdtemp(prefix="frame-clip-"))
self.addCleanup(shutil.rmtree, self.dir, True)
self.clip = self.dir / "clip"
# A process whose environment holds the desktop's bus address, standing in for plasmashell.
self.plasma = subprocess.Popen(["sleep", "60"], env={**os.environ, "DBUS_SESSION_BUS_ADDRESS": "unix:path=/fake"})
self.addCleanup(self.plasma.wait)
self.addCleanup(self.plasma.kill)
(self.dir / "pgrep").write_text(f'#!/bin/sh\n[ -e "{self.dir}/closed" ] || echo {self.plasma.pid}\n')
(self.dir / "qdbus6").write_text(
'#!/bin/sh\n[ "$DBUS_SESSION_BUS_ADDRESS" = unix:path=/fake ] || exit 1\n'
f'[ "$3" = org.kde.klipper.klipper.getClipboardContents ] || exit 1\ncat "{self.clip}"; echo\n')
for name in ("pgrep", "qdbus6"):
(self.dir / name).chmod(0o755)
def run_script(self):
def local_ssh(remote, **kw):
r = subprocess.run(["bash", "-c", server.COPY], capture_output=True, text=True,
env={**os.environ, "PATH": f"{self.dir}:{os.environ['PATH']}"})
if r.returncode:
raise server.Failure(r.stderr.strip())
return r.stdout
with mock.patch.object(server, "ssh", local_ssh):
return server.headset_clipboard()
def test_unicode_and_newlines_come_back_exactly(self):
self.clip.write_bytes(TEXT.encode())
self.assertEqual(self.run_script(), TEXT)
def test_empty(self):
self.clip.write_bytes(b"")
with self.assertRaises(server.Failure) as cm:
self.run_script()
self.assertEqual(cm.exception.status, 400)
self.assertIn("empty", str(cm.exception))
def test_too_big_is_refused_on_the_headset(self):
self.clip.write_bytes(b"x" * (server.CLIPBOARD_LIMIT + 1))
with self.assertRaises(server.Failure) as cm:
self.run_script()
self.assertEqual(cm.exception.status, 413)
self.assertIn("1 MB", str(cm.exception))
self.clip.write_bytes(b"x" * server.CLIPBOARD_LIMIT)
self.assertEqual(len(self.run_script()), server.CLIPBOARD_LIMIT)
def test_desktop_closed(self):
(self.dir / "closed").touch()
with self.assertRaises(server.Failure) as cm:
self.run_script()
self.assertEqual(cm.exception.status, 409)
class HeadsetClipboard(unittest.TestCase):
def read(self, out=None, error=None):
def fake_ssh(remote, **kw):
self.assertEqual(remote, server.COPY_CMD)
self.assertNotIn("stdin", kw)
if error:
raise server.Failure(error)
return out
with mock.patch.object(server, "ssh", fake_ssh):
return server.headset_clipboard()
def test_drops_only_qdbus6s_newline(self):
self.assertEqual(self.read(b64(TEXT + "\n") + "\n"), TEXT)
self.assertEqual(self.read(b64("one line\n")), "one line")
def test_empty_and_images_say_so(self):
for out, words in ((b64("\n"), "empty"), ("", "empty"), (b64("▨ 640x480 32bpp\n"), "image")):
with self.assertRaises(server.Failure) as cm:
self.read(out)
self.assertEqual(cm.exception.status, 400)
self.assertIn(words, str(cm.exception))
def test_unreadable_output_is_a_fault(self):
for out in ("not base64!", base64.b64encode(b"\xff\xfe\n").decode()):
with self.assertRaises(server.Failure) as cm:
self.read(out)
self.assertEqual(cm.exception.status, 502)
def test_failures(self):
for error, status in (("The headset's desktop isn't open (plasmashell is not running).", 409),
("The headset's clipboard holds too much to copy (over 1 MB).", 413),
("Klipper call failed (bus: none)", 502)):
with self.assertRaises(server.Failure) as cm:
self.read(error=error)
self.assertEqual(cm.exception.status, status, error)
class ClipboardRequests(unittest.TestCase):
def test_read_returns_the_text_for_the_page(self):
with mock.patch.object(server, "headset_clipboard", return_value=TEXT):
out = server.clipboard({"read": True})
self.assertEqual(out["text"], TEXT)
self.assertNotIn(TEXT, out["message"])
def test_to_computer_sets_this_computers_clipboard(self):
with mock.patch.object(server, "headset_clipboard", return_value=TEXT), \
mock.patch.object(frame_host, "set_clipboard_text") as put:
out = server.clipboard({"toComputer": True})
put.assert_called_once_with(TEXT)
self.assertNotIn("text", out)
self.assertEqual(out["message"], f"Copied {len(TEXT)} characters from the headset to this computer's clipboard")
def test_to_computer_without_a_clipboard_program(self):
with mock.patch.object(server, "headset_clipboard", return_value=TEXT), \
mock.patch.object(frame_host, "set_clipboard_text", side_effect=frame_host.HostError("Can't set the clipboard")):
with self.assertRaises(server.Failure) as cm:
server.clipboard({"toComputer": True})
self.assertEqual(cm.exception.status, 500)
def test_to_computer_refused_when_the_server_runs_on_the_headset(self):
with mock.patch.object(server, "LOCAL", True), mock.patch.object(server, "headset_clipboard") as read:
with self.assertRaises(server.Failure) as cm:
server.clipboard({"toComputer": True})
self.assertEqual(cm.exception.status, 400)
read.assert_not_called()
def test_flags_must_be_true(self):
# Anything else is a send, and an empty one is refused before SSH.
with mock.patch.object(server, "ssh") as ssh:
for body in ({"read": "yes"}, {"toComputer": 1}):
with self.assertRaises(server.Failure):
server.clipboard(body)
ssh.assert_not_called()
class SetClipboardText(unittest.TestCase):
def put(self, platform, have, text=TEXT, fail=()):
runs = []
def run(cmd, **kw):
runs.append((cmd, kw))
if cmd[0] in fail:
if fail[cmd[0]] == "timeout":
raise subprocess.TimeoutExpired(cmd, 10)
return subprocess.CompletedProcess(cmd, 1)
return subprocess.CompletedProcess(cmd, 0)
flags = {"MAC": platform == "mac", "WINDOWS": platform == "windows", "LINUX": platform == "linux"}
with mock.patch.multiple(frame_host, **flags), \
mock.patch.object(frame_host.shutil, "which", lambda name: f"/bin/{name}" if name in have else None), \
mock.patch.object(frame_host.subprocess, "run", run):
frame_host.set_clipboard_text(text)
for _, kw in runs:
# Nothing captured or logged, and no pipes left for a clipboard daemon to hold open.
self.assertEqual((kw["stdout"], kw["stderr"]), (subprocess.DEVNULL, subprocess.DEVNULL))
return runs
def test_mac_pbcopy_in_utf8(self):
[(cmd, kw)] = self.put("mac", {"pbcopy"})
self.assertEqual(cmd, ["pbcopy"])
self.assertEqual(kw["input"], TEXT.encode("utf-8"))
self.assertEqual(kw["env"]["LC_CTYPE"], "UTF-8")
def test_windows_powershell_gets_base64_utf8(self):
[(cmd, kw)] = self.put("windows", {"powershell.exe", "clip.exe"})
self.assertEqual(cmd[0], "powershell.exe")
self.assertIn("Set-Clipboard", cmd[-1])
self.assertIn("FromBase64String", cmd[-1])
self.assertEqual(base64.b64decode(kw["input"]).decode("utf-8"), TEXT)
def test_windows_clip_fallback_gets_utf16_with_a_bom(self):
runs = self.put("windows", {"powershell.exe", "clip.exe"}, fail={"powershell.exe": 1})
self.assertEqual([c[0] for c, _ in runs], ["powershell.exe", "clip.exe"])
data = runs[1][1]["input"]
self.assertEqual(data[:2], b"\xff\xfe")
self.assertEqual(data[2:].decode("utf-16-le"), TEXT)
def test_linux_tries_wayland_then_x11(self):
[(cmd, kw)] = self.put("linux", {"wl-copy", "xclip", "xsel"})
self.assertEqual(cmd, ["wl-copy"])
self.assertEqual(kw["input"], TEXT.encode())
runs = self.put("linux", {"wl-copy", "xclip", "xsel"}, fail={"wl-copy": 1, "xclip": "timeout"})
self.assertEqual([c for c, _ in runs], [["wl-copy"], ["xclip", "-selection", "clipboard", "-in"],
["xsel", "--clipboard", "--input"]])
[(cmd, _)] = self.put("linux", {"xsel"})
self.assertEqual(cmd[0], "xsel")
def test_nothing_to_set_it_with(self):
with self.assertRaises(frame_host.HostError) as cm:
self.put("linux", set())
self.assertIn("wl-clipboard", str(cm.exception))
with self.assertRaises(frame_host.HostError):
self.put("linux", {"wl-copy"}, fail={"wl-copy": 1})
with self.assertRaises(frame_host.HostError):
self.put("mac", set())
if __name__ == "__main__":
unittest.main()
+559
View File
@@ -0,0 +1,559 @@
"""Recordings of the headset view and Steam clip exports (ui/frame_clips.py), without a headset.
Steam's recordings are fixture trees in the layout frame_clips.py describes. The
recording's shell runs for real under bash with a stand-in ffmpeg, so Stop, the time
limit and the clean-up are the real script's behaviour. With ffmpeg installed here, a
65-second DASH recording is made in Steam's file naming and exported for real.
"""
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import json
import os
import shlex
import shutil
import subprocess
import sys
import tempfile
import threading
import time
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "ui"))
import frame_clips as clips # noqa: E402
POSIX = os.name != "nt" and shutil.which("bash")
def write(path, data=b"x"):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
return path
def session(folder, streams=(0, 1), chunks=3, mpd=None, first=1, pad=5):
"""A session folder as Steam writes it: init + numbered chunks per stream."""
for n in streams:
write(folder / f"init-stream{n}.m4s", f"init{n};".encode())
for c in range(first, first + chunks):
write(folder / f"chunk-stream{n}-{c:0{pad}d}.m4s", f"s{n}c{c};".encode())
if mpd is not None:
write(folder / "session.mpd", mpd.encode())
return folder
class Ids(unittest.TestCase):
def test_good_ids(self):
for cid in ("123/video/bg_620_20261020_201950",
"123/clips/clip_620_20261020_202134/video/bg_620_20261020_201950",
"9/clips/a/b/c/d"):
self.assertEqual(clips.parse_id(cid)[0], cid.split("/")[0])
self.assertEqual(clips.export_name("123/clips/clip_620_20261020_202134/video/bg_620_1"),
"clip_620_20261020_202134.mp4")
def test_no_traversal_or_shell_from_ids(self):
for cid in ("", None, 5, "123", "123/video", "123/other/bg", "abc/video/bg", "/123/video/bg",
"123/video/..", "123/video/.", "123/video/bg/../../../../.ssh", "123/video/.hidden",
"123/video/bg;rm -rf ~", "123/video/bg$(id)", "123/video/bg name", "123/video/bg\n",
"123/video/%2e%2e", "123\\video\\bg", "123/video//bg", "123/clips/a/b/c/d/e",
"123/video/" + "a" * 101, "1234567890123/video/bg"):
with self.subTest(cid=cid), self.assertRaises(ValueError):
clips.parse_id(cid)
with self.subTest(cid=cid), self.assertRaises(ValueError):
clips.export_name(cid)
def test_export_paths_checked_before_delete(self):
good = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
self.assertEqual(clips.remove_command(good), f'rm -rf "$HOME"/{clips.WORK}/export-0123456789abcdef')
for bad in (None, "", "/etc/passwd", f"{clips.WORK}/export-0123456789abcdef/../../x.mp4",
f"{clips.WORK}/export-xyz/clip.mp4", f"{clips.WORK}/../export-0123456789abcdef/a.mp4",
f"{clips.WORK}/export-0123456789abcdef/a.mp4; rm -rf ~", ".ssh/export-0123456789abcdef/a.mp4"):
with self.subTest(bad=bad), self.assertRaises(ValueError):
clips.remove_command(bad)
def test_recording_names_checked(self):
for fn in (clips.finish_command, clips.cleanup_command, lambda n: clips.record_command(n, True)):
for bad in ("rec-1", "rec-20261020-201950; rm -rf ~", "../rec-20261020-201950", "*"):
with self.subTest(bad=bad), self.assertRaises(ValueError):
fn(bad)
class Sessions(unittest.TestCase):
def setUp(self):
self.dir = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.dir, ignore_errors=True)
def test_chunks_in_number_order_whatever_the_padding(self):
s = self.dir / "s"
write(s / "init-stream0.m4s")
for c in (10, 2, 1, 100, 9):
write(s / f"chunk-stream0-{c}.m4s")
write(s / "chunk-stream0-00003.m4s")
got = clips.read_session(s)
self.assertEqual([os.path.basename(p) for p in got[0]],
["init-stream0.m4s", "chunk-stream0-1.m4s", "chunk-stream0-2.m4s", "chunk-stream0-00003.m4s",
"chunk-stream0-9.m4s", "chunk-stream0-10.m4s", "chunk-stream0-100.m4s"])
self.assertEqual(clips.chunk_gaps(got[0]), 100 - 6)
def test_unplayable_streams_and_strangers_left_out(self):
s = self.dir / "s"
session(s, streams=(0,), chunks=2)
write(s / "init-stream1.m4s") # no chunks
write(s / "chunk-stream2-00001.m4s") # no init
write(s / "chunk-stream0-00001.m4s.part") # not a chunk
write(s / "thumbnail.jpg")
(s / "chunk-stream0-00009.m4s").mkdir() # a folder, not a file
if POSIX:
os.symlink("/etc/passwd", s / "chunk-stream0-00005.m4s") # links aren't followed
got = clips.read_session(s)
self.assertEqual(list(got), [0])
self.assertEqual(len(got[0]), 3)
def test_manifest_duration(self):
for text, want in (('mediaPresentationDuration="PT1M23.4S"', 83.4), ('mediaPresentationDuration="PT2H"', 7200),
('mediaPresentationDuration="PT0H3M0.000S"', 180), ('mediaPresentationDuration="P1DT1S"', 86401),
('mediaPresentationDuration="PT"', None), ('nothing here', None), (None, None)):
self.assertEqual(clips.mpd_duration(text), want, text)
def test_listing_a_fixture_tree(self):
ud = self.dir / "userdata"
gr = ud / "123" / "gamerecordings"
session(gr / "clips" / "clip_620_20261020_202134" / "video" / "bg_620_20261020_201950", chunks=40,
mpd='<MPD mediaPresentationDuration="PT1M20.0S">')
write(gr / "clips" / "clip_620_20261020_202134" / "thumbnail.jpg")
session(gr / "video" / "bg_1172470_20261021_090000", streams=(0,), chunks=2)
session(gr / "video" / "odd-name", chunks=1) # no stamp: time from the files
(gr / "video" / "empty").mkdir()
session(gr / "video" / ".hidden", chunks=1) # not a name an id can carry
session(gr / "timelines" / "x", chunks=1) # not clips/ or video/
session(ud / "notanaccount" / "gamerecordings" / "video" / "bg_1_20260101_000000", chunks=1)
session(gr / "video" / "a" / "b" / "c" / "d" / "e", chunks=1) # deeper than ids go
if POSIX:
os.symlink(gr, gr / "video" / "loop") # links aren't followed
for f in (gr / "video" / "odd-name").iterdir():
os.utime(f, (1e9, 1e9)) # 2001: older than the others
got = clips.list_recordings(ud)
self.assertEqual([c["id"] for c in got], ["123/video/bg_1172470_20261021_090000",
"123/clips/clip_620_20261020_202134/video/bg_620_20261020_201950",
"123/video/odd-name"]) # newest first
self.assertEqual(got[-1]["time"], 1e9)
by = {c["id"]: c for c in got}
clip = by["123/clips/clip_620_20261020_202134/video/bg_620_20261020_201950"]
self.assertEqual((clip["kind"], clip["appid"], clip["streams"], clip["chunks"], clip["gaps"], clip["duration"]),
("clip", "620", 2, 80, 0, 80.0))
self.assertEqual(clip["file"], "clip_620_20261020_202134.mp4")
self.assertEqual(clip["time"], time.mktime((2026, 10, 20, 20, 21, 34, 0, 0, -1)))
self.assertEqual(clip["size"], sum(f.stat().st_size for f in
(gr / "clips" / "clip_620_20261020_202134" / "video" / "bg_620_20261020_201950").iterdir()
if f.suffix == ".m4s"))
bg = by["123/video/bg_1172470_20261021_090000"]
self.assertEqual((bg["kind"], bg["appid"], bg["streams"], bg["duration"]), ("background", "1172470", 1, None))
self.assertIsNone(by["123/video/odd-name"]["appid"])
self.assertEqual(clips.list_recordings(self.dir / "missing"), [])
def test_listing_marks_what_is_already_here(self):
here = self.dir / "here"
write(here / "clip_620_20261020_202134.mp4")
frame = lambda args, timeout: {"clips": [{"id": "1/clips/clip_620_20261020_202134/video/bg"},
{"id": "1/video/bg_620_20261020_201950"}]}
got = clips.list_clips(frame, folder=here)
self.assertEqual([c["saved"] for c in got["clips"]], [True, False])
self.assertEqual(got["folder"], str(here))
class Export(unittest.TestCase):
"""The Frame's half of an export, against a fixture tree, with ffmpeg stood in."""
def setUp(self):
self.dir = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.dir, ignore_errors=True)
self.ud = self.dir / "home" / clips.STEAM_USERDATA
self.home = self.dir / "home"
self.cid = "123/clips/clip_620_20261020_202134/video/bg_620_20261020_201950"
session(self.ud / "123" / "gamerecordings" / "clips" / "clip_620_20261020_202134" / "video" /
"bg_620_20261020_201950", chunks=12)
def fake_ffmpeg(self, seen, rc=0):
def run(cmd, **kw):
ins = [cmd[i + 1] for i, a in enumerate(cmd) if a == "-i"]
seen.append({"cmd": cmd, "inputs": {Path(i).name: Path(i).read_bytes() for i in ins}})
if rc == 0:
Path(cmd[-1]).write_bytes(b"mp4")
return subprocess.CompletedProcess(cmd, rc, "", "Invalid data found" if rc else "")
return run
def test_streams_stitched_in_order_and_copied_not_reencoded(self):
seen = []
with mock.patch.object(clips.subprocess, "run", self.fake_ffmpeg(seen)):
got = clips.export(self.ud, self.cid, home=self.home)
self.assertRegex(got["path"], clips.EXPORT_PATH)
self.assertTrue(got["path"].endswith("/clip_620_20261020_202134.mp4"))
self.assertEqual(got["size"], 3)
cmd = seen[0]["cmd"]
self.assertEqual(cmd[cmd.index("-c") + 1], "copy")
self.assertEqual([cmd[i + 1] for i, a in enumerate(cmd) if a == "-map"], ["0", "1"])
for n in (0, 1):
self.assertEqual(seen[0]["inputs"][f"stream{n}.mp4"],
f"init{n};".encode() + b"".join(f"s{n}c{c};".encode() for c in range(1, 13)))
out = self.home / got["path"]
self.assertEqual(sorted(p.name for p in out.parent.iterdir()), [out.name]) # stream files gone
def test_failed_join_leaves_nothing_behind(self):
with mock.patch.object(clips.subprocess, "run", self.fake_ffmpeg([], rc=1)):
with self.assertRaisesRegex(clips.ClipError, "Invalid data"):
clips.export(self.ud, self.cid, home=self.home)
self.assertEqual(list((self.home / clips.WORK).glob("export-*")), [])
def test_bad_or_missing_ids(self):
with self.assertRaises(ValueError):
clips.export(self.ud, "123/video/../../../../etc", home=self.home)
with self.assertRaises(ClipErrorOrOSError):
clips.export(self.ud, "123/video/nothing_here", home=self.home)
@unittest.skipUnless(POSIX, "symlinks")
def test_a_linked_account_cannot_reach_outside_userdata(self):
outside = session(self.dir / "elsewhere" / "gamerecordings" / "video" / "bg_1_20260101_000000", chunks=1)
os.symlink(self.dir / "elsewhere", self.ud / "456")
self.assertIsNotNone(outside)
with self.assertRaises(ValueError):
clips.export(self.ud, "456/video/bg_1_20260101_000000", home=self.home)
def test_no_space_on_the_frame(self):
full = mock.Mock(free=10)
with mock.patch.object(clips.shutil, "disk_usage", return_value=full):
with self.assertRaisesRegex(clips.ClipError, "Not enough free space on the Frame"):
clips.export(self.ud, self.cid, home=self.home)
def test_old_exports_swept(self):
work = self.home / clips.WORK
old, new = work / "export-aaaaaaaaaaaaaaaa", work / "export-bbbbbbbbbbbbbbbb"
write(old / "x.mp4"), write(new / "y.mp4")
os.utime(old, (time.time() - 7 * 3600,) * 2)
with mock.patch.object(clips.subprocess, "run", self.fake_ffmpeg([])):
clips.export(self.ud, self.cid, home=self.home)
self.assertFalse(old.exists())
self.assertTrue(new.exists())
def test_main_prints_json(self):
with mock.patch.object(clips.Path, "home", return_value=self.home), \
mock.patch("sys.stdout", new_callable=lambda: __import__("io").StringIO()) as out:
self.assertEqual(clips.main(["list"]), 0)
self.assertEqual(json.loads(out.getvalue())["clips"][0]["id"], self.cid)
with mock.patch("sys.stdout", new_callable=lambda: __import__("io").StringIO()) as out:
self.assertEqual(clips.main(["export", "../../x"]), 1)
self.assertEqual(json.loads(out.getvalue()), {"error": "bad clip id"})
ClipErrorOrOSError = (clips.ClipError, OSError)
class ComputerSide(unittest.TestCase):
def setUp(self):
self.dir = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.dir, ignore_errors=True)
def test_export_copies_then_deletes_on_the_frame(self):
path = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
calls = []
frame = lambda args, timeout: calls.append(("frame", args)) or {"path": path, "size": 3}
run = lambda cmd, timeout: calls.append(("run", cmd))
fetch = lambda remote, local: calls.append(("fetch", remote)) or Path(local).write_bytes(b"mp4")
got = clips.export_clip("1/clips/clip_620/video/bg", frame, run, fetch, folder=self.dir)
self.assertEqual(got["message"], "Saved clip_620.mp4 to ~/Videos/SteamFrame")
self.assertEqual((self.dir / "clip_620.mp4").read_bytes(), b"mp4")
self.assertEqual([c[0] for c in calls], ["frame", "fetch", "run"])
self.assertEqual(calls[2][1], clips.remove_command(path))
self.assertEqual([p.name for p in self.dir.iterdir()], ["clip_620.mp4"])
def test_failed_copy_leaves_no_partial_file_and_still_cleans_up(self):
path = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
ran = []
def fetch(remote, local):
Path(local).write_bytes(b"half")
raise RuntimeError("connection lost")
with self.assertRaisesRegex(RuntimeError, "connection lost"):
clips.export_clip("1/clips/clip_620/video/bg", lambda a, t: {"path": path, "size": 4},
lambda cmd, timeout: ran.append(cmd), fetch, folder=self.dir)
self.assertEqual(list(self.dir.iterdir()), [])
self.assertEqual(ran, [clips.remove_command(path)])
def test_bad_id_never_reaches_the_frame(self):
never = mock.Mock(side_effect=AssertionError("reached the Frame"))
with self.assertRaises(ValueError):
clips.export_clip("1/video/../../.ssh", never, never, never, folder=self.dir)
def test_unexpected_path_from_the_frame_refused(self):
never = mock.Mock(side_effect=AssertionError("deleted or copied"))
with self.assertRaises(ValueError):
clips.export_clip("1/video/bg", lambda a, t: {"path": ".ssh/id_ed25519", "size": 1}, never, never,
folder=self.dir)
class Commands(unittest.TestCase):
def test_record_command_with_and_without_sound(self):
with_sound = shlex.split(clips.record_command("rec-20261020-201950", True))
self.assertEqual(with_sound[:2], ["bash", "-c"])
script = with_sound[2]
self.assertIn(f"-f v4l2 -video_size 1920x1080 -i {clips.STREAM_DEVICE}", script)
self.assertIn("-c:v libx264", script)
self.assertNotIn("v4l2m2m", script) # the hardware encoder crashes ffmpeg
self.assertIn(f"-f pulse -i {clips.AUDIO_SOURCE}", script)
self.assertIn("-map 1:a:0", script)
self.assertIn("-c:a aac", script)
self.assertIn(f"-t {clips.MAX_SECONDS} ", script)
self.assertIn('"$d/rec-20261020-201950.mkv"', script)
self.assertEqual(script.count("-use_wallclock_as_timestamps 1"), 2)
quiet = shlex.split(clips.record_command("rec-20261020-201950", False))[2]
self.assertNotIn("pulse", quiet)
self.assertNotIn("-c:a", quiet)
self.assertEqual(quiet.count("-use_wallclock_as_timestamps 1"), 1)
with self.assertRaises(ValueError):
clips.record_command("rec-20261020-201950", True, max_seconds=clips.MAX_SECONDS + 1)
def test_probe_checks_device_space_and_sound(self):
probe = clips.probe_command()
self.assertIn(f"[ -e {clips.STREAM_DEVICE} ]", probe)
self.assertIn(f"-ge {clips.NEED_KB} ]", probe)
self.assertIn(f"timeout 5 ffmpeg -hide_banner -loglevel error -nostdin -f pulse -i {clips.AUDIO_SOURCE}", probe)
self.assertIn("echo audio=1; else echo audio=0", probe)
self.assertIn(f'd="$HOME/{clips.WORK}"', probe)
def test_finish_remuxes_without_reencoding(self):
cmd = clips.finish_command("rec-20261020-201950")
self.assertIn("-i rec-20261020-201950.mkv -map 0 -c copy -movflags +faststart rec-20261020-201950.mp4", cmd)
self.assertNotIn("libx264", cmd)
self.assertEqual(clips.cleanup_command("rec-20261020-201950"), f'rm -f "$HOME/{clips.WORK}/rec-20261020-201950".*')
def test_stitch_command(self):
self.assertEqual(clips.stitch_command(["a.mp4", "b.mp4"], "out.mp4"),
["ffmpeg", "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-i", "a.mp4", "-i", "b.mp4",
"-map", "0", "-map", "1", "-c", "copy", "-movflags", "+faststart", "out.mp4"])
# A stand-in for ffmpeg on the Frame: records how it was run, writes its output file,
# and finishes the file when asked to stop (SIGTERM), as ffmpeg does. FAKE_FFMPEG_RUN
# makes the recording end by itself after that many seconds (the time limit).
FAKE_FFMPEG = r'''#!/usr/bin/env python3
import json, os, signal, sys, time
home = os.environ["HOME"]
with open(os.path.join(home, "ffmpeg-calls.jsonl"), "a") as f:
f.write(json.dumps(sys.argv[1:]) + "\n")
out = sys.argv[-1]
if "-c" in sys.argv and sys.argv[sys.argv.index("-c") + 1] == "copy": # the remux after a recording
src = sys.argv[sys.argv.index("-i") + 1]
open(out, "wb").write(open(src, "rb").read() + b"+mp4")
sys.exit(0)
stop = []
signal.signal(signal.SIGTERM, lambda *a: stop.append("TERM"))
f = open(out, "wb")
f.write(b"v"); f.flush()
with open(os.path.join(home, "ffmpeg.pid.tmp"), "w") as p:
p.write(str(os.getpid()))
os.replace(os.path.join(home, "ffmpeg.pid.tmp"), os.path.join(home, "ffmpeg.pid"))
end = time.time() + float(os.environ.get("FAKE_FFMPEG_RUN", "60"))
while not stop and time.time() < end:
f.write(b"v"); f.flush(); time.sleep(0.05)
f.close()
open(os.path.join(home, "ffmpeg-ended"), "w").write(stop[0] if stop else "limit")
sys.exit(255 if stop else 0)
'''
@unittest.skipUnless(POSIX, "the recording's shell is bash on the Frame")
class Recording(unittest.TestCase):
"""The Recorder with the real record/finish/cleanup shell, run locally under bash."""
def setUp(self):
self.dir = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.dir, ignore_errors=True)
self.frame = self.dir / "frame" # the Frame's home
(self.frame / clips.WORK).mkdir(parents=True)
bin_dir = self.dir / "bin"
write(bin_dir / "ffmpeg", FAKE_FFMPEG.encode()).chmod(0o755)
self.env = {**os.environ, "HOME": str(self.frame), "PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}"}
self.here = self.dir / "Videos"
self.channels, self.released, self.ran = [], [], []
self.audio = "audio=1"
def open_channel(self, command, stderr):
proc = subprocess.Popen(shlex.split(command), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=stderr,
env=self.env)
self.channels.append(proc)
self.addCleanup(lambda: proc.poll() is None and proc.kill())
return proc
def run_cmd(self, command, timeout):
self.ran.append(command)
if command == clips.probe_command():
return self.audio + "\n" # needs /dev/video99, so answered here
r = subprocess.run(["bash", "-c", command], capture_output=True, text=True, env=self.env, timeout=timeout)
if r.returncode:
raise RuntimeError(r.stderr.strip() or f"exit {r.returncode}")
return r.stdout
def fetch(self, remote, local):
shutil.copyfile(self.frame / remote, local)
def recorder(self):
rec = clips.Recorder(self.open_channel, self.run_cmd, self.fetch,
lambda fn: threading.Thread(target=fn, daemon=True).start(),
release=self.released.append, folder=self.here)
start = rec.start
def started():
# "started" means ffmpeg was launched; wait until it's running, as it would be by a person's Stop.
pid = self.frame / "ffmpeg.pid"
for f in (pid, self.frame / "ffmpeg-ended"):
f.unlink(missing_ok=True)
got = start()
for _ in range(200):
if pid.exists() or (self.frame / "ffmpeg-ended").exists():
break
time.sleep(0.02)
return got
rec.start = started
return rec
def wait_for(self, rec, states, timeout=20):
end = time.time() + timeout
while time.time() < end:
s = rec.status()
if s["state"] in states:
return s
time.sleep(0.05)
self.fail(f"still {rec.status()}")
def ffmpeg_calls(self):
return [json.loads(line) for line in (self.frame / "ffmpeg-calls.jsonl").read_text().splitlines()]
def test_stop_finishes_the_file_and_copies_it_here(self):
rec = self.recorder()
started = rec.start()
self.assertEqual((started["state"], started["audio"], started["message"]), ("recording", True, "Recording"))
self.assertIn("pulse", self.ffmpeg_calls()[0])
time.sleep(0.3)
self.assertEqual(rec.stop()["state"], "stopping")
done = self.wait_for(rec, ("saved", "failed"))
self.assertEqual(done["state"], "saved", done)
self.assertEqual((self.frame / "ffmpeg-ended").read_text(), "TERM") # asked to finish, not killed
saved = Path(done["file"])
self.assertEqual(saved.parent, self.here)
self.assertRegex(saved.name, r"^rec-\d{8}-\d{6}\.mp4$")
self.assertTrue(saved.read_bytes().endswith(b"+mp4"))
self.assertEqual([p.name for p in self.here.iterdir()], [saved.name]) # no .incoming- left
self.assertEqual(list((self.frame / clips.WORK).iterdir()), []) # cleaned on the Frame
self.assertIsNotNone(self.channels[0].poll()) # the channel has gone
self.assertEqual(self.released, self.channels)
self.assertEqual(self.ran[-1], clips.cleanup_command(saved.stem))
def test_channel_closing_stops_ffmpeg_on_the_frame(self):
# The computer's ssh going away closes the channel just as Stop does.
rec = self.recorder()
rec.start()
pid = int((self.frame / "ffmpeg.pid").read_text())
self.channels[0].stdin.close()
self.wait_for(rec, ("saved", "failed"))
self.assertEqual((self.frame / "ffmpeg-ended").read_text(), "TERM")
with self.assertRaises(ProcessLookupError):
for _ in range(100):
os.kill(pid, 0)
time.sleep(0.05)
def test_time_limit_saves_on_its_own(self):
self.env["FAKE_FFMPEG_RUN"] = "0.3"
rec = self.recorder()
rec.start()
done = self.wait_for(rec, ("saved", "failed"))
self.assertEqual(done["state"], "saved", done)
self.assertEqual((self.frame / "ffmpeg-ended").read_text(), "limit")
with self.assertRaisesRegex(clips.ClipError, "Not recording"):
rec.stop()
def test_no_sound_records_video_and_says_so(self):
self.audio = "audio=0"
rec = self.recorder()
started = rec.start()
self.assertFalse(started["audio"])
self.assertIn("without sound", started["message"])
self.assertNotIn("pulse", self.ffmpeg_calls()[0])
rec.stop()
done = self.wait_for(rec, ("saved", "failed"))
self.assertIn("(no sound)", done["message"])
def test_one_at_a_time(self):
rec = self.recorder()
with self.assertRaisesRegex(clips.ClipError, "Not recording"):
rec.stop()
rec.start()
with self.assertRaises(clips.ClipError) as e:
rec.start()
self.assertEqual(e.exception.status, 409)
rec.stop()
self.wait_for(rec, ("saved",))
rec.start() # a new one once the last is saved
rec.stop()
self.wait_for(rec, ("saved",))
def test_failed_start_reports_why(self):
rec = clips.Recorder(lambda cmd, err: subprocess.Popen(["bash", "-c", "echo 'no v4l2 here' >&2; exit 1"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=err),
lambda cmd, t: "audio=1", self.fetch, lambda fn: fn(), release=self.released.append,
folder=self.here)
with self.assertRaisesRegex(clips.ClipError, "no v4l2 here"):
rec.start()
self.assertEqual(rec.status()["state"], "failed")
self.assertEqual(len(self.released), 1)
def test_failed_probe_reports_why(self):
def run(cmd, timeout):
raise clips.ClipError("No headset view device (/dev/video99). Is SteamVR running?")
rec = clips.Recorder(mock.Mock(side_effect=AssertionError("recorded anyway")), run, self.fetch,
lambda fn: fn(), folder=self.here)
with self.assertRaisesRegex(clips.ClipError, "Is SteamVR running"):
rec.start()
self.assertEqual(rec.status()["state"], "failed")
def test_empty_recording_is_an_error_not_a_file(self):
self.env["FAKE_FFMPEG_RUN"] = "0"
rec = self.recorder()
with mock.patch.object(clips, "finish_command", lambda name: f"echo 'The recording is empty' >&2; exit 1"):
rec.start()
done = self.wait_for(rec, ("saved", "failed"))
self.assertEqual(done["state"], "failed")
self.assertIn("The recording is empty", done["error"])
self.assertFalse(self.here.exists() and any(self.here.iterdir()))
self.assertEqual(list((self.frame / clips.WORK).iterdir()), [])
@unittest.skipUnless(shutil.which("ffmpeg") and shutil.which("ffprobe"), "needs ffmpeg and ffprobe")
class RealFfmpeg(unittest.TestCase):
"""A 65 s recording in Steam's DASH naming (ffmpeg's own dash muxer names files the same
way), stitched by the real ffmpeg: longer than the 60 s Steam's own export manages."""
def test_export_of_a_long_recording(self):
d = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, d, ignore_errors=True)
ud = d / "home" / clips.STEAM_USERDATA
folder = ud / "123" / "gamerecordings" / "video" / "bg_620_20261020_201950"
folder.mkdir(parents=True)
r = subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc=size=64x64:rate=10",
"-f", "lavfi", "-i", "sine=frequency=440:sample_rate=8000", "-t", "65",
"-c:v", "libx264", "-preset", "ultrafast", "-g", "20", "-c:a", "aac",
"-seg_duration", "2", "-use_template", "1", "-use_timeline", "0",
"-f", "dash", str(folder / "session.mpd")], capture_output=True, text=True)
if r.returncode != 0 or not (folder / "chunk-stream0-00001.m4s").exists():
self.skipTest("this ffmpeg can't write DASH with libx264: " + r.stderr[-300:])
listed = clips.list_recordings(ud)
self.assertEqual(len(listed), 1)
self.assertGreater(listed[0]["chunks"], 30)
got = clips.export(ud, listed[0]["id"], home=d / "home")
out = d / "home" / got["path"]
probe = json.loads(subprocess.run(["ffprobe", "-v", "error", "-show_streams", "-show_format", "-of", "json", str(out)],
capture_output=True, text=True, check=True).stdout)
self.assertEqual(sorted(s["codec_type"] for s in probe["streams"]), ["audio", "video"])
self.assertGreater(float(probe["format"]["duration"]), 64)
if __name__ == "__main__":
unittest.main()
+25
View File
@@ -109,6 +109,10 @@ class ServerGuards(unittest.TestCase):
self.assertEqual(self.request("GET", "/api/stream")[0], 403)
self.assertEqual(self.request("GET", "/api/shots/image?id=1/250820/20260925225208_1.jpg")[0], 403)
self.assertEqual(self.request("POST", "/api/launch", {"appid": "620"})[0], 403)
self.assertEqual(self.request("GET", "/api/record")[0], 403)
self.assertEqual(self.request("GET", "/api/clips")[0], 403)
self.assertEqual(self.request("POST", "/api/record", {"action": "start"})[0], 403)
self.assertEqual(self.request("POST", "/api/clips/export", {"id": "1/video/bg_620_20261020_201950"})[0], 403)
def test_captures_are_not_cacheable(self):
# Headset captures show everything on screen; nothing may cache them.
@@ -134,11 +138,24 @@ class ServerGuards(unittest.TestCase):
("/api/shots/save", {"ids": [1]}),
("/api/shots/save", {"ids": ["1/250820/../../.ssh/id_ed25519"]}),
("/api/shots/save", {"ids": ["1/250820/20260925225208_1.jpg; rm -rf ~"]}),
("/api/record", {}),
("/api/record", {"action": "explode"}),
("/api/clips/export", {}),
("/api/clips/export", {"id": 5}),
("/api/clips/export", {"id": "1/video/../../../../.ssh"}),
("/api/clips/export", {"id": "1/video/bg;rm -rf ~"}),
("/api/clips/export", {"id": "../1/video/bg"}),
]
for path, body in cases:
status, payload = self.post(path, body)
self.assertEqual(status, 400, f"{path} {body} -> {payload}")
def test_recorder_idle_until_started(self):
status, _, payload = self.request("GET", "/api/record", headers={"X-Frame-UI": "1"})
self.assertEqual((status, json.loads(payload)), (200, {"state": "idle"}))
status, payload = self.post("/api/record", {"action": "stop"})
self.assertEqual(status, 409, payload)
def test_showing_a_shot_needs_it_saved_here(self):
status, payload = self.post("/api/open", {"what": "shot", "id": "1/250820/19990101000000_1.jpg"})
self.assertEqual(status, 404, payload)
@@ -530,6 +547,14 @@ class LocalMode(unittest.TestCase):
self.assertEqual(status, 200, body)
self.assertIsInstance(body["titles"], list)
def test_recordings_are_for_a_computer(self):
for method, path, body in (("GET", "/api/record", None), ("GET", "/api/clips", None),
("POST", "/api/record", {"action": "start"}),
("POST", "/api/clips/export", {"id": "1/video/bg_620_20261020_201950"})):
status, payload = self.request(method, path, body)
self.assertEqual(status, 400, (path, payload))
self.assertIn("computer", payload["error"])
def test_open_is_for_the_app_and_power_needs_a_password(self):
self.assertEqual(self.request("POST", "/api/open", {"what": "terminal"})[0], 400)
status, body = self.request("POST", "/api/open", {"what": "reboot"})
-14
View File
@@ -1,14 +0,0 @@
"""Type on the headset: entry points, the T key and the clipboard copy, without a browser or headset."""
import shutil
import subprocess
import unittest
from pathlib import Path
@unittest.skipUnless(shutil.which('node'), 'Node exercises the page code')
class TypingUi(unittest.TestCase):
def test_behavior(self):
root = Path(__file__).resolve().parents[1]
result = subprocess.run(['node', '--test', str(root / 'tests/typing_ui.cjs')],
capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
-114
View File
@@ -1,114 +0,0 @@
// Type on the headset: where people find it, the T key, and copying the headset's clipboard.
// Production page code with only browser and transport boundaries replaced; no headset.
const test=require('node:test');
const assert=require('node:assert/strict');
const cc=require('../ui/control-center.js');
const {workspaceFixture,html}=require('./workspace_fixture.cjs');
const between=(from,to)=>{const start=html.indexOf(from),end=html.indexOf(to,start);assert.ok(start>=0&&end>start,from);return html.slice(start,end);};
const section=id=>between(`id="${id}"`,'</section>');
test('Home, the Right now card and the Tools directory all name it Type on the headset',()=>{
const home=between('data-page="home"','data-page="capture"');
assert.match(home,/<a href="#pad" title="Type on the headset \(T\)"><span>Type on the headset<\/span><small>Keyboard, trackpad and clipboard<\/small>/);
assert.match(home,/<button data-go="pad" title="Type on the headset \(T\)">Type on the headset<\/button>/);
assert.match(between('id="toolDirectory"','<h2>Your Frame</h2>'),/href="#pad"[^>]*><span>Type on the headset<\/span>/);
assert.doesNotMatch(html,/Keyboard &(amp;)? trackpad<\/span>|Type on the Frame</);
});
test('the clipboard, both ways, lives beside the keyboard, not in Send to Frame',()=>{
const pad=section('pad');
for(const id of ['clipText','clipSend','clipMac','clipFrom'])assert.match(pad,new RegExp(`id="${id}"`),id);
assert.match(pad,/Text only, up to 1 MB/);
const transfer=section('transfer');
assert.doesNotMatch(transfer,/id="clip/);
assert.match(transfer,/<a href="#pad">Type on the headset<\/a>/);
});
test('the tool page is titled Type on the headset',()=>{
const f=workspaceFixture();
f.navigate('#pad');
assert.equal(f.activePage,'tools');assert.equal(f.el('pad').hidden,false);
assert.equal(f.el('toolTitle').textContent,'Type on the headset');
assert.equal(f.el('transfer').hidden,true);
});
function keyFixture() {
const f=workspaceFixture(),keys=[];let focused=0,started=0;
Object.assign(f.context,{pad:{state:'off'},padStart(){started++;},ctrl:{on:false},PAGES:['apps','games','tools','settings'],
history:{pushState(_,__,url){f.context.location.hash=url;}}});
f.context.document.addEventListener=(type,fn)=>{if(type==='keydown')keys.push(fn);};
f.context.document.querySelector=()=>null;
f.context.document.activeElement={tagName:'BODY'};
f.el('padType').focus=()=>{focused++;};
f.run(between('function showNow(','$("nowSend").onclick'));
f.run(between('document.addEventListener("keydown", e => {\n if (e.metaKey || e.ctrlKey || e.altKey','// ---- after an APK install'));
const press=(key,extra={})=>{let prevented=false;keys.forEach(fn=>fn({key,preventDefault(){prevented=true;},...extra}));return prevented;};
return {f,press,get focused(){return focused;},get started(){return started;}};
}
test('T opens Type on the headset, turns the keyboard on and puts the cursor in its field',()=>{
const k=keyFixture();
k.f.navigate('#games');
assert.equal(k.press('t'),true,'the T must not also be typed into the field it focuses');
assert.equal(k.f.context.location.hash,'#pad');assert.equal(k.f.activePage,'tools');
assert.equal(k.started,1);assert.equal(k.focused,1);
k.f.context.pad.state='ready';k.press('T');assert.equal(k.started,1,'already on: not started again');
});
test('T stays out of the way of typing, shortcuts and keys going to the Frame',()=>{
const k=keyFixture();
k.f.navigate('#games');
k.press('t',{metaKey:true});k.press('t',{ctrlKey:true});k.press('t',{altKey:true});
k.f.context.document.activeElement={tagName:'INPUT'};k.press('t');
k.f.context.document.activeElement={tagName:'BODY'};k.f.context.document.querySelector=s=>s==='dialog[open]'?{}:null;k.press('t');
k.f.context.document.querySelector=()=>null;
k.f.context.ctrl.on=true;k.f.context.document.activeElement=k.f.el('viewer');k.press('t');
assert.equal(k.f.context.location.hash,'#games');assert.equal(k.started,0);
k.f.context.document.activeElement={tagName:'BODY'};k.press('3');assert.equal(k.f.context.location.hash,'#tools','number keys still switch pages');
});
test('Home’s button goes through the same path, showing the tool once before focusing it',()=>{
const f=workspaceFixture(),buttons=[{dataset:{go:'pad'}},{dataset:{go:'media'}}];let focusedVisible=null;
f.context.document.querySelectorAll=(q=>s=>s==='[data-go]'?buttons:q(s))(f.context.document.querySelectorAll);
Object.assign(f.context,{pad:{state:'error'},padStart(){f.calls.push(['padStart']);},history:{pushState(_,__,url){f.context.location.hash=url;}}});
f.el('padType').focus=()=>{focusedVisible=f.activePage==='tools'&&!f.el('pad').hidden;};
f.navigate('#home');f.run(between('function showNow(','$("nowSend").onclick'));
buttons[0].onclick();
assert.equal(focusedVisible,true);assert.equal(f.calls.filter(c=>c[0]==='padStart').length,1);
buttons[1].onclick();assert.equal(f.context.location.hash,'#media');
});
function copyFixture(frameApp) {
const calls=[],f=workspaceFixture();
Object.assign(f.context,{window:{frameApp},api:async(path,body)=>{calls.push(JSON.parse(JSON.stringify([path,body])));return body.read?{text:'héllo 🎧\n',message:'Copied 8 characters from the headset\'s clipboard'}:{message:'Copied to this computer'};}});
f.run(between('async function copyHeadsetClipboard()','$("clipFrom").onclick'));
return {f,calls};
}
test('in the app, the headset’s clipboard is read and the app writes it, text untouched',async()=>{
const written=[],{f,calls}=copyFixture({writeClipboard:async text=>{written.push(text);return true;}});
const result=await f.run('copyHeadsetClipboard()');
assert.deepEqual(calls,[['/api/clipboard',{read:true}]]);
assert.deepEqual(written,['héllo 🎧\n']);
assert.equal(result.message,'Copied 8 characters from the headset\'s clipboard');
assert.equal(result.text,undefined,'the text is not handed on to be logged or toasted');
});
test('in a browser, the server sets the clipboard of the computer it runs on',async()=>{
for(const frameApp of [undefined,{readClipboard(){}}]){
const {f,calls}=copyFixture(frameApp);
assert.equal((await f.run('copyHeadsetClipboard()')).message,'Copied to this computer');
assert.deepEqual(calls,[['/api/clipboard',{toComputer:true}]]);
}
});
test('a failed read writes nothing',async()=>{
const written=[],f=workspaceFixture();
Object.assign(f.context,{window:{frameApp:{writeClipboard:async t=>written.push(t)}},api:async()=>{throw new Error("The headset's clipboard is empty");}});
f.run(between('async function copyHeadsetClipboard()','$("clipFrom").onclick'));
await assert.rejects(f.run('copyHeadsetClipboard()'),/empty/);
assert.deepEqual(written,[]);
});
test('the Control Center finds Type on the headset by typing, keyboard and clipboard words',()=>{
const input=cc.catalog.find(item=>item.id==='input');
assert.equal(input.title,'Type on the headset');
assert.deepEqual(input.links[0],['#pad','Type on the headset']);
for(const query of ['type','typing','keyboard','clipboard','copy paste','kde connect'])
assert.ok(cc.matches(input,query,'all'),query);
const files=cc.catalog.find(item=>item.id==='files');
assert.ok(files.links.some(([href])=>href==='#pad'),'Send files points clipboard seekers to the right place');
});
+1 -1
View File
@@ -8,7 +8,7 @@ function connectionFixture() {
const f=workspaceFixture(), c=f.context;
Object.assign(c,{URLSearchParams,Date,clearTimeout,link:{s:{phase:'connected'},device:'a',phase:'connected',rev:0},disp:{list:[],seq:0},cat:{apps:null},wi:{},devGen:1,androidSeq:0,androidPending:new Set(),titlesSeq:0,volTimer:null,refreshing:null,lastImg:null,lastShot:null,
sourceState:{jobs:new Map(),detail:null,detailRequest:0},touches:new Map(),gesture:null,padType:{blur(){},value:''},
log(){},renderPill(){},renderConnDlg(){},renderHome(){},resetShots(){},renderBanner(){},loadDevices(){},renderDevices(){},renderTests(){},viaText(){return 'test';},netLabel(){return 'test';},refresh:()=>Promise.resolve(),reloadAll(){},checkContactPrompt(){},ctrlReset(){},padShow:s=>{c.pad.state=s.state;}});
log(){},renderPill(){},renderConnDlg(){},renderHome(){},resetShots(){},resetClips(){},renderBanner(){},loadDevices(){},renderDevices(){},renderTests(){},viaText(){return 'test';},netLabel(){return 'test';},refresh:()=>Promise.resolve(),reloadAll(){},checkContactPrompt(){},ctrlReset(){},padShow:s=>{c.pad.state=s.state;}});
c.window.connDevice='a';
f.run('const UI_KEY="test";'+pick(html,'const HEADSET_FREE =','// What a panel shows'));
f.run(pick(html,'function onConnection(s) {','// Read the event stream'));
+6 -8
View File
@@ -68,9 +68,9 @@
{id:'sideload', category:'apps', icon:'game', title:'Sideloaded games', kind:'flow', label:'Existing controls',
description:'Send a game folder, ZIP or executable. Frame Control selects Proton or the Linux runtime and adds a Steam library entry.',
keywords:'windows exe zip linux devkit runtime shortcuts launch remove', links:[['#sideloaded','Manage titles'],['#transfer','Send a game']]},
{id:'files', category:'apps', icon:'files', title:'Send files', kind:'flow', label:'Existing controls',
description:'Send files to ~/Downloads, or a game to the Steam library. Clipboard text, both ways, is in Type on the headset.',
keywords:'upload transfer downloads sftp storage', links:[['#transfer','Send files'],['#pad','Clipboard']]},
{id:'files', category:'apps', icon:'files', title:'Files & clipboard', kind:'flow', label:'Existing controls',
description:'Send files to ~/Downloads, or send text to the Frame’s clipboard. Clipboard transfer needs the desktop panel open.',
keywords:'upload transfer downloads paste copy text sftp storage', links:[['#transfer','Send files & text']]},
{id:'media', category:'apps', icon:'play', title:'Video & stereo photos', kind:'flow', label:'Preview feature',
description:'Send and play H.264/H.265 video, SBS/OU photos or small splats with the built-in player. Choose a layout and theatre surround.',
keywords:'movie photo sbs ou 3d splat gaussian theatre media player', links:[['#media','Open media player']],
@@ -85,11 +85,9 @@
{id:'remote', category:'network', icon:'display', title:'Remote access', kind:'flow', label:'Existing controls',
description:'Open SSH, SFTP, Steam Link or remote desktop using the host’s existing tools. Developer Mode enables headset SSH and xrdp.',
keywords:'terminal shell remote desktop rdp xrdp streaming steam link sftp', links:[['#power','Open remote tools']]},
{id:'input', category:'network', icon:'keyboard', title:'Type on the headset', kind:'flow', label:'Existing controls',
description:'Type and point in the Frame’s apps with this computer’s keyboard and mouse, and copy text both ways between the clipboards. Press T anywhere to open it.',
keywords:'mouse pointer keyboard trackpad touch click kde connect input type typing text clipboard copy paste send',
links:[['#pad','Type on the headset'],['#view','Panel control']],
guidance:'The keyboard sets up KDE Connect on the Frame the first time. The clipboard needs the desktop panel open in the headset.'},
{id:'input', category:'network', icon:'keyboard', title:'Keyboard & trackpad', kind:'flow', label:'Existing controls',
description:'Type and point in the Frame’s apps with KDE Connect, or use Control on the live panel view. First use sets up the input helper.',
keywords:'mouse pointer keyboard trackpad touch click kde input', links:[['#pad','Open keyboard & trackpad'],['#view','Panel control']]},
{id:'mac', category:'network', icon:'display', title:'Mac in the headset', kind:'flow', label:'macOS only', platform:'mac',
description:'Stream Mac screens or individual windows into headset panels. Choose quality and give each window its own display.',
keywords:'macos mac window screen sharing stream quality sharp balanced light permissions', links:[['#macview','Configure Mac streaming']],
+580
View File
@@ -0,0 +1,580 @@
#!/usr/bin/env python3
"""Video from the headset: recordings of what it shows, with its sound, and Steam's own clips.
Two halves share this file.
On the computer, server.py builds the shell commands below, runs them over SSH and
copies the finished .mp4 to ~/Videos/SteamFrame (Recorder does a recording from Start
to the copy; export_clip does a Steam clip). Nothing here needs ffmpeg on the computer.
On the Frame, this file runs as `python3 - list` or `python3 - export ID` (sent on
stdin, like frame_steam.py, so stdlib only): it lists Steam's game recordings and
stitches one into a single .mp4 with SteamOS's ffmpeg, copying the streams as they are.
It prints one JSON object, or {"error": ...}.
Steam's game recording keeps DASH segments, one folder per recording session:
~/.local/share/Steam/userdata/<account>/gamerecordings/
video/bg_<appid>_<YYYYMMDD>_<HHMMSS>/ background recording
clips/clip_<appid>_<YYYYMMDD>_<HHMMSS>/video/bg_<appid>_<...>/ a saved clip
session.mpd
init-stream0.m4s chunk-stream0-00001.m4s chunk-stream0-00002.m4s ...
init-stream1.m4s chunk-stream1-00001.m4s ...
Each stream's init segment followed by its chunks, in order, is a fragmented MP4 of
that stream. The folder names are as users and Steam's own files describe them; the
listing doesn't depend on them: any folder under clips/ or video/ (up to four levels)
holding init-stream*.m4s counts as a session.
"""
import json
import os
import re
import secrets
import shlex
import shutil
import subprocess
import sys
import tempfile
import threading
import time
from pathlib import Path
# ---- where things go ---------------------------------------------------------
VIDEOS_DIR = Path.home() / "Videos" / "SteamFrame" # on the computer, beside ~/Pictures/SteamFrame
# On the Frame, relative to its home: recordings in progress and stitched exports.
# Not /tmp: it's a RAM disk there, and a recording can be over a gigabyte.
WORK = ".cache/frame-control/video"
# ---- recording what the headset shows ------------------------------------------
#
# The same picture the live view streams: SteamVR's copy of the headset view in the
# v4l2loopback device, encoded with x264 (the hardware encoder crashes ffmpeg). The
# sound is the monitor of PipeWire's default output, read through its PulseAudio
# server: SteamOS's ffmpeg has the pulse device (the media player writes to it, see
# docs/vr-video.md). If it can't be read, the recording goes ahead without sound and
# says so. Both inputs are stamped with the wall clock: the camera device and the
# audio server would otherwise count from different clocks.
STREAM_DEVICE = "/dev/video99"
AUDIO_SOURCE = "@DEFAULT_MONITOR@" # PulseAudio's name for the default output's monitor
MAX_SECONDS = 20 * 60
FPS = 30
VIDEO_RATE = 8 # Mbit/s at 1080p
AUDIO_RATE = 160 # kbit/s AAC
# Room for a full-length recording twice over (the .mkv, then the .mp4 made from it).
NEED_KB = (VIDEO_RATE * 1000 + AUDIO_RATE) * MAX_SECONDS // 8 * 2 + 512 * 1024
REC_NAME = re.compile(r"rec-\d{8}-\d{6}")
class ClipError(Exception):
def __init__(self, message, status=502):
super().__init__(message)
self.status = status
def recording_name(now=None):
return time.strftime("rec-%Y%m%d-%H%M%S", time.localtime(now))
def probe_command():
"""Checks before a recording, run on the Frame: prints audio=1 or audio=0.
One recording runs at a time, so anything left in the folder is from one that was
cut off (the app quit, or the copy failed); it's removed to free the space."""
return (f"set -u; export XDG_RUNTIME_DIR=${{XDG_RUNTIME_DIR:-/run/user/$(id -u)}}; "
f"[ -e {STREAM_DEVICE} ] || {{ echo 'No headset view device ({STREAM_DEVICE}). Is SteamVR running?' >&2; exit 3; }}; "
f"command -v ffmpeg >/dev/null || {{ echo 'ffmpeg is missing on the Frame' >&2; exit 3; }}; "
f"d=\"$HOME/{WORK}\"; mkdir -p \"$d\" && rm -f \"$d\"/rec-*; "
f"free=$(df -Pk \"$d\" | awk 'NR==2 {{print $4}}'); "
f"[ \"${{free:-0}}\" -ge {NEED_KB} ] || {{ echo 'Not enough free space on the Frame to record: "
f"it needs {NEED_KB // 1024 // 1024 + 1} GB' >&2; exit 4; }}; "
f"if timeout 5 ffmpeg -hide_banner -loglevel error -nostdin -f pulse -i {AUDIO_SOURCE} -t 0.2 -f null - "
f">/dev/null 2>&1; then echo audio=1; else echo audio=0; fi")
def record_command(name, audio, max_seconds=MAX_SECONDS):
"""The recording, run on the Frame through `bash -c`.
ffmpeg runs in the background while the shell watches its stdin, which is the SSH
channel: when that closes (Stop closed it, or the computer's ssh went away) the
shell sends ffmpeg SIGTERM, which makes it finish the file properly (as q or
Ctrl-C would; SIGINT itself is ignored by a background job of a script). A recording
that ends by itself (the time limit, an error) ends the wait too. The shell prints
"started" once ffmpeg is running and "exit N" when it has gone."""
if not REC_NAME.fullmatch(name):
raise ValueError(f"bad recording name {name!r}")
if not 1 <= max_seconds <= MAX_SECONDS:
raise ValueError("max_seconds out of range")
clock = "-thread_queue_size 1024 -use_wallclock_as_timestamps 1"
inputs = f"{clock} -f v4l2 -video_size 1920x1080 -i {STREAM_DEVICE}"
maps = "-map 0:v:0"
codecs = (f"-vf fps={FPS},scale=-2:1080,format=yuv420p -c:v libx264 -preset ultrafast "
f"-b:v {VIDEO_RATE}M -maxrate {VIDEO_RATE}M -bufsize {VIDEO_RATE * 2}M -g {FPS * 2}")
if audio:
inputs += f" {clock} -f pulse -i {AUDIO_SOURCE}"
maps += " -map 1:a:0"
codecs += f" -af aresample=async=1 -c:a aac -b:a {AUDIO_RATE}k"
script = (f"export XDG_RUNTIME_DIR=${{XDG_RUNTIME_DIR:-/run/user/$(id -u)}}; d=\"$HOME/{WORK}\"; "
f"ffmpeg -hide_banner -loglevel error -nostdin -y {inputs} {maps} {codecs} -t {max_seconds} "
f"\"$d/{name}.mkv\" </dev/null >/dev/null 2>\"$d/{name}.log\" & p=$!; "
f"echo started; "
f"while kill -0 $p 2>/dev/null; do read -r -t 1 _; r=$?; [ $r -eq 0 ] || [ $r -gt 128 ] || break; done; "
f"kill $p 2>/dev/null; wait $p; echo \"exit $?\"")
return "bash -c " + shlex.quote(script)
def finish_command(name):
"""After the recording: make the .mp4 the computer gets (no re-encode), on the Frame."""
if not REC_NAME.fullmatch(name):
raise ValueError(f"bad recording name {name!r}")
return (f"cd \"$HOME/{WORK}\" || exit 1; "
f"[ -s {name}.mkv ] || {{ tail -c 2000 {name}.log >&2 2>/dev/null; echo 'The recording is empty' >&2; exit 1; }}; "
f"ffmpeg -hide_banner -loglevel error -nostdin -y -i {name}.mkv -map 0 -c copy -movflags +faststart {name}.mp4 "
f"&& rm -f {name}.mkv && wc -c < {name}.mp4")
def cleanup_command(name):
if not REC_NAME.fullmatch(name):
raise ValueError(f"bad recording name {name!r}")
return f"rm -f \"$HOME/{WORK}/{name}\".*"
class Recorder:
"""One recording at a time, from Start to the .mp4 on this computer.
The server supplies how to reach the Frame, so this can be tested without one:
open_channel(command, stderr) -> Popen of `command` on the Frame, stdin and stdout pipes
run(command, timeout) -> its stdout; raises on failure
fetch(remote, local) copies a file from the Frame (a path relative to its home)
start_thread(fn) starts fn in a thread that counts as headset work
release(proc) forgets a channel once it has closed
"""
def __init__(self, open_channel, run, fetch, start_thread, release=lambda proc: None, folder=None):
self.open_channel, self.run, self.fetch = open_channel, run, fetch
self.start_thread, self.release = start_thread, release
self.folder = Path(folder) if folder else VIDEOS_DIR
self._lock = threading.Lock()
self._state = {"state": "idle"}
self._proc = None
def status(self):
with self._lock:
s = dict(self._state)
if s["state"] == "recording":
s["elapsed"] = round(time.time() - s["started"], 1)
return s
def start(self):
with self._lock:
if self._state["state"] in ("starting", "recording", "stopping", "saving"):
raise ClipError("Already recording, or still saving the last recording", 409)
self._state = {"state": "starting"}
try:
out = self.run(probe_command(), 30)
audio = "audio=1" in out
name = recording_name()
errors = tempfile.TemporaryFile()
proc = self.open_channel(record_command(name, audio), errors)
lines = _line_reader(proc.stdout)
first = _next_line(lines, 20)
if first != "started":
_end(proc)
errors.seek(0)
err = errors.read().decode(errors="replace").strip()
errors.close()
self.release(proc)
raise ClipError(err or "The Frame didn't start recording")
except BaseException as e:
with self._lock:
self._state = {"state": "failed", "error": str(e)}
raise
with self._lock:
self._proc = proc
self._state = {"state": "recording", "name": name, "audio": audio, "started": time.time(),
"limit": MAX_SECONDS}
try:
self.start_thread(lambda: self._watch(proc, name, audio, lines, errors))
except BaseException:
_end(proc)
self.release(proc)
with self._lock:
self._state, self._proc = {"state": "failed", "error": "Couldn't follow the recording"}, None
raise
return dict(self.status(), message="Recording" + ("" if audio else
" without sound: the Frame's audio couldn't be read"))
def stop(self):
"""Ends the recording; the copy to this computer carries on in the background."""
with self._lock:
if self._state["state"] != "recording":
raise ClipError("Not recording", 409)
self._state = dict(self._state, state="stopping")
proc = self._proc
_close(proc.stdin) # the shell on the Frame sees EOF and stops ffmpeg
return {"state": "stopping", "message": "Stopping the recording…"}
def _watch(self, proc, name, audio, lines, errors):
# Waits for ffmpeg to finish: after Stop, or at the time limit (a margin
# past it, so a stuck channel can't hold the recording open forever).
deadline = time.time() + MAX_SECONDS + 60
while True:
stopping = self.status()["state"] == "stopping"
line = _next_line(lines, 30 if stopping else 5)
if line is None or line.startswith("exit "):
break # ffmpeg has gone and the file is finished
if line == "" and (stopping or time.time() > deadline):
break # no answer within 30 s of Stop, or well past the limit: close the channel
_end(proc)
errors.close()
self.release(proc)
with self._lock:
self._proc = None
seconds = round(time.time() - self._state["started"])
self._state = dict(self._state, state="saving")
try:
self.save(name)
msg = f"Saved {name}.mp4 to ~/Videos/SteamFrame" + ("" if audio else " (no sound)")
state = {"state": "saved", "file": str(self.folder / f"{name}.mp4"), "audio": audio,
"seconds": seconds, "message": msg}
except Exception as e:
state = {"state": "failed", "error": f"Couldn't save the recording: {e}"}
finally:
try:
self.run(cleanup_command(name), 20)
except Exception:
pass # the next recording's checks remove it
with self._lock:
self._state = state
def save(self, name):
"""Make the .mp4 on the Frame and copy it here, never leaving a partial file in the folder."""
out = self.run(finish_command(name), 600)
size = int(out.strip().splitlines()[-1]) if out.strip() else 0
_make_room(self.folder, size)
incoming = Path(tempfile.mkdtemp(prefix=".incoming-", dir=self.folder))
try:
self.fetch(f"{WORK}/{name}.mp4", incoming / f"{name}.mp4")
os.replace(incoming / f"{name}.mp4", self.folder / f"{name}.mp4")
finally:
shutil.rmtree(incoming, ignore_errors=True)
def _make_room(folder, size):
folder.mkdir(parents=True, exist_ok=True)
free = shutil.disk_usage(folder).free
if size and free < size + 256 * 1024**2:
raise ClipError(f"Not enough free space on this computer: it needs {size // 1024**2 + 256} MB", 507)
def _close(f):
try:
f.close()
except OSError:
pass
def _end(proc):
"""Close our end of the channel and make sure the local ssh has gone."""
_close(proc.stdin)
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
_close(proc.stdout)
def _line_reader(pipe):
"""Lines from a pipe via a thread (select() can't wait on pipes on Windows); None at EOF."""
import queue
lines = queue.Queue()
def pump():
try:
for raw in iter(pipe.readline, b""):
lines.put(raw.decode(errors="replace").strip())
except (OSError, ValueError):
pass
lines.put(None)
threading.Thread(target=pump, daemon=True).start()
return lines
def _next_line(lines, timeout):
"""The next line, None at EOF, or "" after `timeout` seconds of silence."""
import queue
try:
return lines.get(timeout=timeout)
except queue.Empty:
return ""
# ---- Steam's game recordings ---------------------------------------------------
STEAM_USERDATA = ".local/share/Steam/userdata"
KINDS = {"clips": "clip", "video": "background"}
MAX_DEPTH = 4 # folders below clips/ or video/
PART = re.compile(r"[A-Za-z0-9_][A-Za-z0-9_.-]{0,99}") # no leading dot, so never "." or ".."
INIT = re.compile(r"init-stream(\d{1,3})\.m4s")
CHUNK = re.compile(r"chunk-stream(\d{1,3})-(\d{1,9})\.m4s")
STAMP = re.compile(r"(?:^|_)(\d{1,10})_(\d{8})_(\d{6})$")
EXPORT_PATH = re.compile(re.escape(WORK) + r"/export-[0-9a-f]{16}/[A-Za-z0-9_][A-Za-z0-9_.-]{0,99}\.mp4")
def parse_id(clip_id):
"""A recording's id, "<account>/<clips|video>/<folder>[/<folder>...]", as path parts.
Checked before it goes near a path or a shell: plain names only, no "." or "..",
at most MAX_DEPTH folders. Raises ValueError."""
parts = clip_id.split("/") if isinstance(clip_id, str) and len(clip_id) <= 600 else []
if (len(parts) < 3 or len(parts) > 2 + MAX_DEPTH or not re.fullmatch(r"\d{1,12}", parts[0])
or parts[1] not in KINDS or not all(PART.fullmatch(p) for p in parts[2:])):
raise ValueError("bad clip id")
return parts
def export_name(clip_id):
"""The .mp4's name on the computer: the clip's (or background session's) own folder name."""
return parse_id(clip_id)[2] + ".mp4"
def read_session(folder):
"""{stream number: [init, chunk, chunk...]} for one session folder, chunks in number
order. A stream without its init segment or without chunks can't be played, so it's left out."""
inits, chunks = {}, {}
for entry in os.scandir(folder):
if not entry.is_file(follow_symlinks=False):
continue
m = INIT.fullmatch(entry.name)
if m:
inits[int(m[1])] = entry.path
continue
m = CHUNK.fullmatch(entry.name)
if m:
chunks.setdefault(int(m[1]), []).append((int(m[2]), entry.path))
return {n: [inits[n], *(p for _, p in sorted(chunks[n]))] for n in sorted(inits) if chunks.get(n)}
def chunk_gaps(files):
"""How many chunk numbers are missing between the first and last (Steam deleted or lost them)."""
nums = sorted(int(CHUNK.fullmatch(os.path.basename(f))[2]) for f in files[1:])
return nums[-1] - nums[0] + 1 - len(nums) if nums else 0
def mpd_duration(text):
"""Seconds from the manifest's mediaPresentationDuration ("PT1M23.4S"), or None."""
m = re.search(r'mediaPresentationDuration="P(?:(\d+)D)?T?(?:(\d+)H)?(?:(\d+)M)?(?:([\d.]+)S)?"', text or "")
if not m or not any(m.groups()):
return None
d, h, mi, s = m.groups()
try:
return int(d or 0) * 86400 + int(h or 0) * 3600 + int(mi or 0) * 60 + float(s or 0)
except ValueError:
return None
def _sessions(base, depth=0):
"""Folders under base (clips/ or video/) that hold init segments, without following links."""
try:
entries = list(os.scandir(base))
except OSError:
return
if any(e.is_file(follow_symlinks=False) and INIT.fullmatch(e.name) for e in entries):
yield base
return
if depth < MAX_DEPTH:
for e in sorted(entries, key=lambda e: e.name):
if e.is_dir(follow_symlinks=False) and PART.fullmatch(e.name):
yield from _sessions(e.path, depth + 1)
def describe(userdata, clip_id):
"""What the list shows about one recording, or None if it holds nothing playable."""
parts = parse_id(clip_id)
folder = os.path.join(userdata, parts[0], "gamerecordings", *parts[1:])
streams = read_session(folder)
if not streams:
return None
files = [f for s in streams.values() for f in s]
stats = [os.stat(f) for f in files]
duration = None
for mpd in sorted(p for p in os.listdir(folder) if p.endswith(".mpd")):
try:
duration = mpd_duration(Path(folder, mpd).read_text(errors="replace")[:20000])
except OSError:
pass
if duration is not None:
break
appid, when = None, max(s.st_mtime for s in stats)
for p in parts[2:]:
m = STAMP.search(p)
if m:
appid = m[1]
try:
when = time.mktime(time.strptime(m[2] + m[3], "%Y%m%d%H%M%S"))
except ValueError:
pass
break
return {"id": clip_id, "kind": KINDS[parts[1]], "appid": appid, "time": when,
"size": sum(s.st_size for s in stats), "streams": len(streams),
"chunks": sum(len(s) - 1 for s in streams.values()),
"gaps": sum(chunk_gaps(s) for s in streams.values()), "duration": duration,
"file": export_name(clip_id)}
def list_recordings(userdata):
"""Every Steam recording session on this Frame, newest first (runs on the Frame)."""
found = []
try:
accounts = sorted(e.name for e in os.scandir(userdata) if e.is_dir(follow_symlinks=False)
and re.fullmatch(r"\d{1,12}", e.name))
except OSError:
return []
for account in accounts:
root = os.path.join(userdata, account, "gamerecordings")
for kind in KINDS:
for folder in _sessions(os.path.join(root, kind)):
clip_id = "/".join([account, *os.path.relpath(folder, root).split(os.sep)])
try:
info = describe(userdata, clip_id)
except (OSError, ValueError):
continue # changed while we looked, or a name we don't take
if info:
found.append(info)
found.sort(key=lambda c: c["time"], reverse=True)
return found
def stitch_command(inputs, out):
"""ffmpeg that puts each stream file in one .mp4, copying the streams as they are."""
cmd = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-nostdin", "-y"]
for f in inputs:
cmd += ["-i", str(f)]
for i in range(len(inputs)):
cmd += ["-map", str(i)]
return cmd + ["-c", "copy", "-movflags", "+faststart", str(out)]
def export(userdata, clip_id, home=None):
"""Stitch one recording into an .mp4 under WORK (runs on the Frame). -> its path
relative to home and its size. The computer copies it, then deletes it with
remove_command."""
home = Path(home or Path.home())
parts = parse_id(clip_id)
folder = Path(userdata, parts[0], "gamerecordings", *parts[1:])
real = os.path.realpath(folder)
if os.path.commonpath([real, os.path.realpath(userdata)]) != os.path.realpath(userdata):
raise ValueError("bad clip id")
streams = read_session(real)
if not streams:
raise ClipError("That recording has no complete streams on the Frame", 404)
size = sum(os.path.getsize(f) for s in streams.values() for f in s)
work = home / WORK
work.mkdir(parents=True, exist_ok=True)
_sweep(work)
# The stream files and the .mp4 made from them are each about the recording's size.
if shutil.disk_usage(work).free < size * 2 + 64 * 1024**2:
raise ClipError(f"Not enough free space on the Frame to export it: it needs {(size * 2) // 1024**2 + 64} MB", 507)
out_dir = work / f"export-{secrets.token_hex(8)}"
out_dir.mkdir()
try:
inputs = []
for n, files in streams.items():
path = out_dir / f"stream{n}.mp4"
with open(path, "wb") as dst:
for f in files:
with open(f, "rb") as src:
shutil.copyfileobj(src, dst, 1 << 20)
inputs.append(path)
out = out_dir / export_name(clip_id)
r = subprocess.run(stitch_command(inputs, out), capture_output=True, text=True, errors="replace", timeout=1800)
if r.returncode != 0 or not out.exists():
raise ClipError("ffmpeg couldn't join the recording: " + (r.stderr.strip()[-1500:] or f"exit {r.returncode}"))
for f in inputs:
f.unlink()
except BaseException:
shutil.rmtree(out_dir, ignore_errors=True)
raise
return {"path": out.relative_to(home).as_posix(), "size": out.stat().st_size}
def _sweep(work, older=6 * 3600):
"""Exports left behind by a copy that never finished."""
for d in work.glob("export-*"):
try:
if time.time() - d.stat().st_mtime > older:
shutil.rmtree(d, ignore_errors=True)
except OSError:
pass
def remove_command(path):
"""Delete an export on the Frame once it's been copied (path as export returned it)."""
if not EXPORT_PATH.fullmatch(path or ""):
raise ValueError(f"unexpected export path {path!r}")
return f"rm -rf \"$HOME\"/{shlex.quote(path.rsplit('/', 1)[0])}"
# ---- the computer's side of Steam clips -------------------------------------------
#
# run_frame(args, timeout) runs this file on the Frame with `args` and returns its JSON;
# run(command, timeout) and fetch(remote, local) as for Recorder.
def list_clips(run_frame, folder=None):
"""Steam's recordings on the Frame, each marked saved if its .mp4 is already here."""
folder = Path(folder) if folder else VIDEOS_DIR
clips = run_frame(["list"], 60)["clips"]
for c in clips:
c["saved"] = (folder / export_name(c["id"])).exists()
return {"clips": clips, "folder": str(folder)}
def export_clip(clip_id, run_frame, run, fetch, folder=None):
"""Stitch a Steam recording on the Frame and copy the .mp4 here. -> {"message", "file"}."""
folder = Path(folder) if folder else VIDEOS_DIR
name = export_name(clip_id) # checks the id before anything reaches the Frame
made = run_frame(["export", clip_id], 1900)
path = made.get("path")
remove = remove_command(path) # refuses a path that isn't one of our exports
try:
_make_room(folder, int(made.get("size") or 0))
incoming = Path(tempfile.mkdtemp(prefix=".incoming-", dir=folder))
try:
fetch(path, incoming / name)
os.replace(incoming / name, folder / name)
finally:
shutil.rmtree(incoming, ignore_errors=True)
finally:
try:
run(remove, 30)
except Exception:
pass # the next export sweeps it
return {"message": f"Saved {name} to ~/Videos/SteamFrame", "file": str(folder / name)}
# ---- on the Frame -----------------------------------------------------------------
def main(argv):
userdata = Path.home() / STEAM_USERDATA
try:
if argv[:1] == ["list"]:
result = {"clips": list_recordings(userdata)}
elif len(argv) == 2 and argv[0] == "export":
result = export(userdata, argv[1])
else:
result = {"error": "usage: frame_clips.py list | export ID"}
except (ClipError, ValueError, OSError, subprocess.TimeoutExpired) as e:
result = {"error": str(e)}
print(json.dumps(result))
return 1 if "error" in result else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
-33
View File
@@ -5,7 +5,6 @@ Everything here runs on your computer, not the Frame. Python stdlib only.
CLI (used by the Electron app, so terminal handling lives in one place):
python3 ui/frame_host.py terminal -- CMD [ARG...] # open CMD in a terminal window
"""
import base64
import hashlib
import io
import os
@@ -309,38 +308,6 @@ def clipboard_text():
raise HostError("Can't read the clipboard")
def set_clipboard_text(text):
"""Put TEXT on this computer's clipboard."""
data = text.encode("utf-8")
env = None
if MAC:
# pbcopy reads stdin in the locale's encoding, and the app may start with none (MacRoman).
cmds, env = [["pbcopy"]], {**os.environ, "LC_CTYPE": "UTF-8"}
elif WINDOWS:
# Base64 on stdin, so no console code page touches the text. clip.exe takes UTF-16 with a BOM.
script = ("Set-Clipboard -Value ([Text.Encoding]::UTF8.GetString("
"[Convert]::FromBase64String([Console]::In.ReadToEnd())))")
cmds = [["powershell.exe", "-NoProfile", "-NonInteractive", "-Command", script], ["clip.exe"]]
else:
cmds = [["wl-copy"], ["xclip", "-selection", "clipboard", "-in"], ["xsel", "--clipboard", "--input"]]
for cmd in cmds:
if not shutil.which(cmd[0]):
continue
feed = (base64.b64encode(data) if cmd[0] == "powershell.exe"
else b"\xff\xfe" + text.encode("utf-16-le") if cmd[0] == "clip.exe" else data)
# wl-copy and xclip stay behind to serve the clipboard: give them no pipes to hold open.
try:
r = subprocess.run(cmd, input=feed, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
env=env, timeout=10)
except subprocess.TimeoutExpired:
continue
if r.returncode == 0:
return
if LINUX:
raise HostError("Can't set the clipboard: install wl-clipboard (Wayland) or xclip (X11)")
raise HostError("Can't set the clipboard")
# What Windows' OpenSSH says when it refuses ~/.ssh/config (or a key) for its ACL.
BAD_PERMISSIONS = "Bad owner or permissions on "
+128 -39
View File
@@ -1143,7 +1143,7 @@
<div class="workspace-heading"><div><span class="eyebrow">Your Steam Frame</span><h1 id="homeTitle">Let's connect</h1><p id="homeReadiness" role="status">Checking your headset…</p></div><a class="workspace-link" href="#devices">Devices &amp; connection</a></div>
<div class="home-grid"><section class="panel home-activity"><span class="eyebrow">Current activity</span><h2 id="homeActivity">Connect to see what's running</h2><p id="homeActivityNote">Your installed apps and current downloads appear here.</p><div id="homeRunning" class="home-running"></div><div class="home-actions"><a href="#library" class="workspace-link primary-link">Open Games →</a><a href="#discover" class="workspace-link">Find something new</a></div></section>
<section class="panel home-session"><span class="eyebrow">Background tasks</span><h2 id="homeJobs">No active tasks</h2><p id="homeDownload">Install progress stays here while you explore.</p><button class="small" id="homeLog">View activity log</button></section></div>
<div class="quick-routes"><a href="#capture"><span>Capture &amp; Live</span><small>See and share the headset view</small></a><a href="#pad" title="Type on the headset (T)"><span>Type on the headset</span><small>Keyboard, trackpad and clipboard</small></a><a href="#comfort"><span>Family &amp; comfort</span><small>Sessions, breaks and alerts</small></a><a href="#control"><span>Control Center</span><small>Search settings and live metrics</small></a></div>
<div class="quick-routes"><a href="#capture"><span>Capture &amp; Live</span><small>See and share the headset view</small></a><a href="#pad"><span>Keyboard &amp; trackpad</span><small>Control apps on your Frame</small></a><a href="#comfort"><span>Family &amp; comfort</span><small>Sessions, breaks and alerts</small></a><a href="#control"><span>Control Center</span><small>Search settings and live metrics</small></a></div>
<section class="panel now" id="device">
<div class="shelf-head"><h2>Right now</h2></div>
<p class="now-state" id="nowState">Waiting for the Frame…</p>
@@ -1159,7 +1159,7 @@
<button data-go="media">Play a video or photo</button>
<button data-go="macview" id="nowMac" hidden>Show a Mac window</button>
<button id="nowSend">Send files…</button>
<button data-go="pad" title="Type on the headset (T)">Type on the headset</button>
<button data-go="pad">Type on the Frame</button>
</div>
<p class="hint">Casting shows everything the wearer sees, including anything private.</p>
</section>
@@ -1182,6 +1182,7 @@
<span class="view-actions">
<button class="action" id="shotBtn">Capture</button>
<button id="liveBtn" title="Keep updating, as video">Live</button>
<button id="recBtn" title="Record what the headset shows, with its sound, to ~/Videos/SteamFrame on this computer">Record</button>
<button id="ctrlBtn" title="Control the Frame by tapping or clicking on the view (C)">Control</button>
<button id="copyBtn" disabled title="Copy the image to the clipboard (or right-click it)">Copy</button>
<button id="saveBtn" disabled>Save</button>
@@ -1226,6 +1227,7 @@
<button data-ctrl-key="28" title="Enter">Enter</button>
</span>
</div>
<div class="hint" id="recStatus" role="status" hidden></div>
<div class="hint" id="viewHint"></div>
</section>
@@ -1247,6 +1249,14 @@
<div class="shot-grid" id="shotGrid"><div class="sub">Loading…</div></div>
<div class="hint">Screenshots you take in the headset with Steam's screenshot shortcut. New ones appear on their own. Click one to open it in the viewer, Copy puts it on the clipboard, and Save copies it to <code>~/Pictures/SteamFrame</code>. Right-click for more.</div>
</section>
<section id="clips">
<div class="shelf-head"><h2>Steam clips</h2><span class="count" id="clipCount"></span><span class="spacer"></span>
<button class="small" id="clipsRefresh">Refresh</button>
<button class="small" id="videosFolder" title="Open the SteamFrame folder in your Videos">Show folder</button>
</div>
<div class="list" id="clipList"><div class="sub">Loading…</div></div>
<div class="hint">Clips and background recordings from Steam's game recording in the headset. Export joins one into a single <code>.mp4</code> in <code>~/Videos/SteamFrame</code>, however long it is, without re-encoding it. Recordings you make with <b>Record</b> go there too.</div>
</section>
</div>
<div class="page" data-page="games">
@@ -1402,7 +1412,7 @@
<div class="page" data-page="tools">
<div class="workspace-heading"><div><a href="#tools" id="toolBack" class="workspace-link" hidden>← All tools</a><h1 id="toolTitle">Tools</h1><p id="toolSubtitle">Choose a task for your Frame.</p></div></div>
<div id="toolDirectory"><section class="tool-group"><h2>Content & files</h2><div class="quick-routes"><a href="#transfer"><span>Send to Frame</span><small>Add files, APKs or your own games</small></a><a href="#media"><span>Media player</span><small>Play your own videos and media</small></a><a href="#linuxApps"><span>Linux apps</span><small>Install and manage desktop apps</small></a></div></section><section class="tool-group"><h2>Play & control</h2><div class="quick-routes"><a href="#capture"><span>Capture & Live</span><small>See and control the headset view</small></a><a href="#pad" title="Type on the headset (T)"><span>Type on the headset</span><small>Keyboard, trackpad and clipboard</small></a><a href="#panels"><span>Panel switcher</span><small>Bring an open panel into view</small></a><a href="#vrTools"><span>VR tools</span><small>Performance HUD and VR utilities</small></a><a href="#comfort"><span>Family & comfort</span><small>Session reminders and breaks</small></a><a href="#macview" id="macToolLink" hidden><span>Mac in the headset</span><small>Share Mac windows and displays</small></a></div></section><section class="tool-group"><h2>Your Frame</h2><a class="workspace-link" href="#settings">Connection, display &amp; preferences →</a></section><section class="tool-group"><h2>Assistant</h2><a class="workspace-link" href="/assistant">Open assistant &amp; AI agents →</a><p class="sub">Use your own model endpoint or review a proposed MCP action.</p></section></div>
<div id="toolDirectory"><section class="tool-group"><h2>Content & files</h2><div class="quick-routes"><a href="#transfer"><span>Send to Frame</span><small>Add files, APKs or your own games</small></a><a href="#media"><span>Media player</span><small>Play your own videos and media</small></a><a href="#linuxApps"><span>Linux apps</span><small>Install and manage desktop apps</small></a></div></section><section class="tool-group"><h2>Play & control</h2><div class="quick-routes"><a href="#capture"><span>Capture & Live</span><small>See and control the headset view</small></a><a href="#pad"><span>Keyboard & trackpad</span><small>Type and point in desktop apps</small></a><a href="#panels"><span>Panel switcher</span><small>Bring an open panel into view</small></a><a href="#vrTools"><span>VR tools</span><small>Performance HUD and VR utilities</small></a><a href="#comfort"><span>Family & comfort</span><small>Session reminders and breaks</small></a><a href="#macview" id="macToolLink" hidden><span>Mac in the headset</span><small>Share Mac windows and displays</small></a></div></section><section class="tool-group"><h2>Your Frame</h2><a class="workspace-link" href="#settings">Connection, display &amp; preferences →</a></section><section class="tool-group"><h2>Assistant</h2><a class="workspace-link" href="/assistant">Open assistant &amp; AI agents →</a><p class="sub">Use your own model endpoint or review a proposed MCP action.</p></section></div>
<div class="grid-3" id="toolWorkspace">
<section class="panel" id="vrTools">
<div class="shelf-head"><h2>VR comfort and performance</h2><span class="spacer"></span><span class="sub" id="vrUpdated">Waiting for the Frame</span></div>
@@ -1479,15 +1489,6 @@
for the Frame (3.6 MB, 18 MB unpacked) into your home folder there and pairs with it. No internet needed, and nothing
else to install. <a href="#" data-about>Licences</a></div>
</div>
<div class="shelf-head" style="margin-top:18px"><h2>Clipboard</h2></div>
<textarea id="clipText" placeholder="Text to put on the Frame's clipboard…"></textarea>
<div class="row" style="margin-top:8px">
<button class="action small" id="clipSend">Send text</button>
<button class="small" id="clipMac">Send this computer's clipboard</button>
<button class="small" id="clipFrom">Copy the headset's clipboard here</button>
</div>
<div class="hint">Text goes both ways between here and the headset's desktop: copy in an app on the Frame, then
Copy the headset's clipboard. Needs the desktop panel open in the headset. Text only, up to 1 MB.</div>
</section><section id="androidSettings"> <h2 class="lib-section">Display and reports</h2>
<div class="lib-manage">
<section class="panel disp" id="display">
@@ -1550,7 +1551,13 @@
<option value="vr">VR app</option>
</select>
<div class="progress" id="prog"><i></i></div>
<div class="hint">Clipboard text, both ways, is in <a href="#pad">Type on the headset</a>.</div>
<div class="shelf-head" style="margin-top:18px"><h2>Clipboard</h2></div>
<textarea id="clipText" placeholder="Text to put on the Frame's clipboard…"></textarea>
<div class="row" style="margin-top:8px">
<button class="action small" id="clipSend">Send text</button>
<button class="small" id="clipMac">Send this computer's clipboard</button>
</div>
<div class="hint">Clipboard needs the desktop panel open in the headset.</div>
</section>
<section class="panel" id="media">
@@ -2016,12 +2023,12 @@ function applyHostWording(h) {
// The iPhone app: the server runs on the Frame, and saving goes to the phone's Photos.
document.body.classList.toggle("mobile", !!h.mobile);
$("shotsFolder").hidden = !!h.mobile;
$("recBtn").hidden = $("clips").hidden = !!h.mobile; // recordings are saved to a computer
$("quest").hidden = !!h.mobile;
$("metaLib").hidden = !!h.mobile; // the Quest plugs into a computer; the phone app's server runs on the Frame
if (!h.mobile) $("shotsFolder").textContent = h.fileManager === "your file manager" ? "Open folder" : `Show in ${h.fileManager}`;
if (!h.mobile) $("shotsFolder").textContent = $("videosFolder").textContent = h.fileManager === "your file manager" ? "Open folder" : `Show in ${h.fileManager}`;
$("shotsSaveNew").textContent = h.mobile ? `Save all to ${h.computer}` : `Save new to ${h.computer}`;
$("clipMac").textContent = `Send ${h.computer} clipboard`;
$("clipFrom").textContent = `Copy the headset's clipboard to this ${h.computer}`;
$('mobileConnection').hidden = !h.mobile;
$('desktopConnections').hidden = !!h.mobile;
if (h.mobile) {
@@ -2132,7 +2139,7 @@ function setOnline(ok, why) {
reloadAll();
}
}
function reloadAll() { loadTitles(); loadOwned(); loadAndroid(); loadDisplays(); loadShots(); }
function reloadAll() { loadTitles(); loadOwned(); loadAndroid(); loadDisplays(); loadShots(); loadClips(); }
$("offRetry").onclick = () => refresh();
if (window.frameApp && window.frameApp.setUpConnection) {
$("offSetup").hidden = false;
@@ -2316,16 +2323,10 @@ function showNow(to) {
showPage();
}
// Home's shortcuts to the things people come to do next.
// Type on the headset: Home's button and the T key turn the keyboard on and put the cursor in its field.
function typeOnHeadset() {
showNow("pad");
if (pad.state === "off" || pad.state === "error") padStart();
$("padType").focus({ preventScroll: true });
}
document.querySelectorAll("[data-go]").forEach(b => b.onclick = () => {
const to = b.dataset.go;
if (to === "pad") return typeOnHeadset();
showNow(to);
if (to === "pad") { if (pad.state === "off" || pad.state === "error") padStart(); $("padType").focus({ preventScroll: true }); }
});
$("nowSend").onclick = () => $("fileInput").click();
$("castBtn").onclick = () => {
@@ -3367,15 +3368,6 @@ async function sendComputerClipboard() {
return api("/api/clipboard", { text }, device);
}
$("clipMac").onclick = () => act($("clipMac").textContent, sendComputerClipboard, $("clipMac"));
// The other way. The app (or the iPhone app) writes the clipboard itself; in a browser, the
// server does, on the computer it runs on. The text is never logged.
async function copyHeadsetClipboard() {
if (!window.frameApp?.writeClipboard) return api("/api/clipboard", { toComputer: true });
const { text, message } = await api("/api/clipboard", { read: true });
await window.frameApp.writeClipboard(text);
return { message };
}
$("clipFrom").onclick = () => act("Copy the headset's clipboard", copyHeadsetClipboard, $("clipFrom"));
// ---- file drop ----
const drop = $("drop");
@@ -5133,6 +5125,107 @@ $("shotsFolder").onclick = e => act($("shotsFolder").textContent, () => api("/ap
// Watch for new shots: a cheap listing over the shared SSH connection while the
// window is visible and the Frame is reachable, and again on coming back to it.
setInterval(() => { if (!document.hidden && online) loadShots(true); }, SHOTS_POLL_MS);
// ---- Recording the headset view, and Steam's own clips (server.py, frame_clips.py) ----
// A recording runs on the Frame until Stop; the server then copies the .mp4 here
// and the page follows it by polling /api/record.
const rec = { state: "idle", timer: null };
const clock = t => `${Math.floor(t / 60)}:${String(Math.floor(t % 60)).padStart(2, "0")}`;
function showRec(s) {
const was = rec.state;
rec.state = s.state;
const busy = ["starting", "recording", "stopping", "saving"].includes(s.state);
const b = $("recBtn");
b.textContent = s.state === "recording" ? "Stop" : "Record";
b.classList.toggle("on", s.state === "recording");
b.disabled = busy && s.state !== "recording";
const line = $("recStatus");
line.textContent = s.state === "recording" ? `Recording ${clock(s.elapsed || 0)} (stops by itself at ${clock(s.limit)})` +
(s.audio ? "" : ". No sound: the Frame's audio couldn't be read, so this is video only.")
: s.state === "starting" ? "Starting the recording…"
: s.state === "stopping" ? "Stopping the recording…"
: s.state === "saving" ? `Copying the recording to this ${HOST.computer}…`
: s.state === "saved" ? s.message
: s.state === "failed" ? `Recording failed: ${s.error}` : "";
line.hidden = !line.textContent;
if (was !== s.state && was !== "idle" && s.state === "saved") { log(s.message, "ok"); toast(s.message); }
if (was !== s.state && was !== "idle" && was !== "starting" && s.state === "failed") { log(line.textContent, "e"); toast(line.textContent, true); }
clearTimeout(rec.timer);
if (busy) rec.timer = setTimeout(pollRec, 1000);
}
async function pollRec() {
if (HOST.mobile) return;
clearTimeout(rec.timer);
let s;
try { s = await api("/api/record"); }
catch { if (rec.state !== "idle") rec.timer = setTimeout(pollRec, 3000); return; }
showRec(s);
}
$("recBtn").onclick = async e => {
const start = rec.state !== "recording";
const res = await act(start ? "Start recording" : "Stop recording",
() => api("/api/record", { action: start ? "start" : "stop" }), e.currentTarget);
if (res) showRec(res); else pollRec();
};
const clipsState = { list: [], gen: 0, exporting: new Set() };
async function loadClips() {
if (HOST.mobile) return;
const gen = clipsState.gen;
$("clipsRefresh").disabled = true;
let got;
try { got = await api("/api/clips"); }
catch (e) { if (gen === clipsState.gen) failed($("clipList"), e); return; }
finally { $("clipsRefresh").disabled = false; }
if (gen !== clipsState.gen) return;
clipsState.list = got.clips;
drawClips();
}
// Forget the clips of a headset we've switched away from.
function resetClips() {
clipsState.gen++; clipsState.list = [];
$("clipCount").textContent = "";
$("clipList").innerHTML = `<div class="sub">Loading…</div>`;
}
const megabytes = n => n >= 1e9 ? `${(n / 1e9).toFixed(1)} GB` : `${Math.max(1, Math.round(n / 1e6))} MB`;
function drawClips() {
const list = clipsState.list;
$("clipCount").textContent = list.length ? `${list.length} on the Frame` : "";
$("clipList").innerHTML = list.length ? list.map((c, i) => `<div class="item"><div class="grow">
<div class="t">${esc(c.appid ? shotApp(c.appid) : "Steam recording")} · ${c.kind === "clip" ? "Clip" : "Background recording"}</div>
<div class="s" title="${esc(new Date(c.time * 1000).toLocaleString())}">${esc([shotTime(c.time), c.duration ? clock(c.duration) : "",
megabytes(c.size), c.gaps ? `${c.gaps} pieces missing` : ""].filter(Boolean).join(" · "))}</div></div>
${c.saved ? `<span class="tag">On ${HOST.computer}</span>` : ""}
<button class="small" data-clip-export="${i}"${clipsState.exporting.has(c.id) ? " disabled" : ""}>${c.saved ? "Export again" : "Export"}</button>
</div>`).join("")
: `<div class="sub">No Steam clips or background recordings on the Frame.</div>`;
}
async function exportClip(c, btn) {
const label = `Export ${c.file}`;
clipsState.exporting.add(c.id); btn.disabled = true;
log(label + "…"); toast(label + "…");
try {
const { job } = await api("/api/clips/export", { id: c.id });
for (let misses = 0; ;) {
await new Promise(ok => setTimeout(ok, 1500));
let j;
try { j = await api(`/api/job?id=${encodeURIComponent(job)}`); misses = 0; }
catch (e) { if (++misses >= 20 || /no such job/.test(e.message)) throw e; continue; }
if (!j.done) continue;
if (j.error) throw new Error(j.error);
log(j.message, "ok"); toast(j.message);
break;
}
} catch (e) {
log(`${label} failed: ${e.message}`, "e"); toast(`${label} failed: ${e.message}`, true);
} finally { clipsState.exporting.delete(c.id); loadClips(); }
}
$("clipList").onclick = e => {
const b = e.target.closest("[data-clip-export]");
if (b) exportClip(clipsState.list[+b.dataset.clipExport], b);
};
$("clipsRefresh").onclick = () => loadClips();
$("videosFolder").onclick = e => act($("videosFolder").textContent, () => api("/api/open", { what: "videos" }), e.currentTarget);
document.addEventListener("visibilitychange", () => { if (!document.hidden && online) loadShots(true); });
// ---- Panel switcher: our UI over SteamVR's panel API ----
@@ -5271,7 +5364,7 @@ const SECTION_PAGE = {library:"games", discover:"apps", android:"apps", view:"ca
vrTools:"tools", comfort:"tools", assistant:"tools", pad:"tools", sideloaded:"games", getgames:"games", steamStore:"games", androidLibrary:"apps", appStore:"apps", linuxApps:"apps",
display:"settings", androidSettings:"settings", transfer:"tools", media:"tools", artworkSettings:"settings", power:"settings", macview:"tools", privacy:"settings", panels:"tools"};
const TOOL_TITLES = {transfer:"Send to Frame", media:"Media player", artworkSettings:"Library artwork",
pad:"Type on the headset", panels:"Panel switcher", vrTools:"VR tools", comfort:"Family & comfort", androidSettings:"App display & reports",
pad:"Keyboard & trackpad", panels:"Panel switcher", vrTools:"VR tools", comfort:"Family & comfort", androidSettings:"App display & reports",
power:"Remote & power", macview:"Mac in the headset", privacy:"Privacy & updates"};
const workspace = {source:"all", scope:"installed", apps:"installed", scroll:new Map(), key:null, hash:null, y:0};
let page = "apps";
@@ -5703,10 +5796,6 @@ clientPreferences.ready.then(async () => {
showPage();
document.addEventListener("keydown", e => {
if (e.metaKey || e.ctrlKey || e.altKey || /INPUT|TEXTAREA|SELECT/.test(document.activeElement.tagName) || document.querySelector("dialog[open]")) return;
if ((e.key === "t" || e.key === "T") && !(ctrl.on && document.activeElement === $("viewer"))) {
e.preventDefault(); // or the T lands in the field it focuses, and goes to the Frame
return typeOnHeadset();
}
const n = [...PAGES, 'devices', 'control'][+e.key - 1];
if (n) location.hash = n;
});
@@ -6062,7 +6151,7 @@ if (window.frameApp && window.frameApp.onInstallLink) {
}
setView("headset");
refresh().then(() => { loadShots(); pollComfort(); }); // after status, so app names resolve and comfort knows if the Frame answers
refresh().then(() => { loadShots(); loadClips(); pollRec(); pollComfort(); }); // after status, so app names resolve and comfort knows if the Frame answers
document.addEventListener("visibilitychange", () => { if (!document.hidden && online === false) refresh(); });
</script>
<script>
@@ -6353,7 +6442,7 @@ function onConnection(s) {
disp.list = []; disp.port = null;
// The lists behind those panels too, so filters can't bring the old ones back.
gm.storeDraft = ""; gm.owned = null; gm.byId = new Map(); gm.results = []; gm.store = []; gm.storeQ = null; gm.shown = 0; gm.shownBySource = {owned:0, store:0}; gm.seq++;
androidApps = []; androidSeq++; androidPending.clear(); $('installedDetail').close(); resetShots();
androidApps = []; androidSeq++; androidPending.clear(); $('installedDetail').close(); resetShots(); resetClips();
renderHome();
titlesSeq++; disp.seq++; // answers to loads already on their way are ignored
if (cat.apps) filterCatalog(); // "Installed" tags were the other headset's
+96 -78
View File
@@ -49,6 +49,7 @@ from apk_sources import search as apk_search, SourceError # noqa: E402
from apk_sources import fdroid as apk_fdroid # noqa: E402
import frame_apk_versions # noqa: E402
import frame_catalog # noqa: E402
import frame_clips # noqa: E402
import frame_devices # noqa: E402
import frame_steamgriddb
import frame_comfort # noqa: E402
@@ -746,6 +747,77 @@ def stream_command(query):
f"exec >&-; cat >/dev/null; kill $p 2>/dev/null; wait $p")
# Recordings of the headset view with its sound, and Steam's own clips, saved as .mp4
# in ~/Videos/SteamFrame (frame_clips.py has the commands and the Frame-side half).
def fetch_file(remote, local):
"""Copy one file from the Frame (a path relative to its home) to `local`."""
ensure_master()
try:
# Modern scp uses SFTP, so the remote path isn't parsed by a shell. -p keeps its time.
r = frame_host.run_ssh(["scp", "-p", *SSH[1:], f"{FRAME}:{remote}", str(local)], capture_output=True,
stdin=subprocess.DEVNULL, text=True, errors="replace", timeout=3600)
except subprocess.TimeoutExpired:
raise Failure(f"Copying {Path(remote).name} timed out")
if r.returncode != 0:
raise Failure(strip_ansi(r.stderr or r.stdout).strip() or f"scp exited {r.returncode}")
def _record_channel(command, stderr):
ensure_master()
proc = subprocess.Popen([*SSH, FRAME, command], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=stderr)
_live_tunnels.add(proc) # killed if the server stops: the Frame then finishes the file
return proc
recorder = frame_clips.Recorder(_record_channel, lambda cmd, timeout: ssh(cmd, timeout=timeout), fetch_file,
lambda fn: busy_thread(fn).start(), release=lambda proc: _live_tunnels.discard(proc))
def _clips_here():
if LOCAL:
raise Failure("Recordings are saved to a computer: use Frame Control on your computer", 400)
def record(body):
"""{"action": "start"|"stop"}: record what the headset shows, with its sound."""
_clips_here()
action = body.get("action")
if action not in ("start", "stop"):
raise Failure("action must be start or stop", 400)
try:
return recorder.start() if action == "start" else recorder.stop()
except frame_clips.ClipError as e:
raise Failure(str(e), e.status)
def _clips_frame(args, timeout):
return steam_frame(*args, timeout=timeout, script="frame_clips.py")
def list_clips():
_clips_here()
try:
return frame_clips.list_clips(_clips_frame)
except frame_clips.ClipError as e:
raise Failure(str(e), e.status)
def export_clip(body):
"""{"id"}: stitch one of Steam's recordings on the Frame and copy the .mp4 here (a job)."""
_clips_here()
clip_id = body.get("id")
name = frame_clips.export_name(clip_id) # ValueError, so 400, for a bad id
def work():
try:
return frame_clips.export_clip(clip_id, _clips_frame, lambda cmd, timeout: ssh(cmd, timeout=timeout),
fetch_file)
except frame_clips.ClipError as e:
raise Failure(str(e), e.status)
return start_job(f"Export {name}", work)
def launch(body):
appid = str(body.get("appid", ""))
if not APPID.match(appid):
@@ -856,22 +928,20 @@ def set_volume(body):
return {"message": "Volume updated"}
# The headset desktop's session bus. Verified 2026-09-25 (SteamOS 0.3.0 vr, build
# 20260922): the headset desktop is a nested Plasma Wayland session inside
# gamescope with its own D-Bus bus, and wl-copy/xclip are not installed. Klipper
# (org.kde.klipper, served by plasmashell) is reachable with qdbus6, so borrow
# plasmashell's bus address. Same as scripts/paste-to-frame.sh.
DESKTOP_BUS = r"""pid=$(pgrep -u "$(id -u)" -x plasmashell | head -n 1)
# Runs on the Frame, clipboard text on stdin. Verified 2026-09-25 (SteamOS 0.3.0
# vr, build 20260922): the headset desktop is a nested Plasma Wayland session
# inside gamescope with its own D-Bus bus, and wl-copy/xclip are not installed.
# Klipper (org.kde.klipper, served by plasmashell) is reachable with qdbus6, so
# borrow plasmashell's bus address. Same as scripts/paste-to-frame.sh.
PASTE = r"""set -u
text=$(cat; printf x); text=${text%x}
pid=$(pgrep -u "$(id -u)" -x plasmashell | head -n 1)
if [ -z "$pid" ]; then
echo "The headset's desktop isn't open (plasmashell is not running). Open the desktop in the headset, then try again." >&2
exit 2
fi
bus=$(tr '\0' '\n' < "/proc/$pid/environ" | sed -n 's/^DBUS_SESSION_BUS_ADDRESS=//p')
"""
# Runs on the Frame, clipboard text on stdin.
PASTE = r"""set -u
text=$(cat; printf x); text=${text%x}
""" + DESKTOP_BUS + r"""if DBUS_SESSION_BUS_ADDRESS=$bus qdbus6 org.kde.klipper /klipper \
if DBUS_SESSION_BUS_ADDRESS=$bus qdbus6 org.kde.klipper /klipper \
org.kde.klipper.klipper.setClipboardContents "$text" >/dev/null; then
echo "copied via Klipper (${#text} chars)"
else
@@ -879,77 +949,12 @@ else
exit 2
fi
"""
DESKTOP_CLOSED = "plasmashell is not running" # what PASTE and COPY say when the headset's desktop isn't open
DESKTOP_CLOSED = "plasmashell is not running" # what PASTE says when the headset's desktop isn't open
# base64 keeps the script intact through every local shell's quoting rules.
PASTE_CMD = 'bash -c "$(echo %s | base64 -d)"' % base64.b64encode(PASTE.encode()).decode()
# The other way: Klipper's getClipboardContents, printed by qdbus6 with a newline
# after it, sent back as base64 so no shell or console encoding touches it.
# Not tested on a headset: the method is Klipper's, the same interface PASTE uses.
CLIPBOARD_LIMIT = 1_000_000 # bytes of UTF-8, about a 500-page book
CLIPBOARD_TOO_BIG = "too much to copy"
COPY = r"""set -u
""" + DESKTOP_BUS + r"""out=$(mktemp) || exit 2
trap 'rm -f "$out"' EXIT
if ! DBUS_SESSION_BUS_ADDRESS=$bus qdbus6 org.kde.klipper /klipper \
org.kde.klipper.klipper.getClipboardContents >"$out"; then
echo "Klipper call failed (bus: ${bus:-none})" >&2
exit 2
fi
if [ "$(wc -c <"$out")" -gt %d ]; then
echo "The headset's clipboard holds %s (over 1 MB). Copy less, or send it as a file." >&2
exit 3
fi
base64 -w 0 <"$out"
""" % (CLIPBOARD_LIMIT + 1, CLIPBOARD_TOO_BIG)
COPY_CMD = 'bash -c "$(echo %s | base64 -d)"' % base64.b64encode(COPY.encode()).decode()
# Klipper's text for an image it holds, e.g. "▨ 640x480 32bpp" (HistoryImageItem::text in
# Klipper's source; not seen on a headset).
KLIPPER_IMAGE = re.compile(r"▨ \d+x\d+ \d+bpp")
CLIPBOARD_EMPTY = "The headset's clipboard is empty"
def desktop_failure(e):
"""The headset's desktop being closed is the person's to change, not a fault: a 409 isn't reported as one."""
if DESKTOP_CLOSED in str(e):
return Failure(str(e), 409)
if CLIPBOARD_TOO_BIG in str(e):
return Failure(str(e), 413)
return None
def headset_clipboard():
"""The text on the headset desktop's clipboard. Never logged: it can hold anything."""
try:
out = ssh(COPY_CMD, timeout=30)
except Failure as e:
raise desktop_failure(e) or e from None
try:
text = base64.b64decode(out.strip(), validate=True).decode("utf-8")
except (ValueError, UnicodeDecodeError):
raise Failure("The headset's clipboard couldn't be read as text", 502) from None
text = text[:-1] if text.endswith("\n") else text # qdbus6's newline, not the clipboard's
if KLIPPER_IMAGE.fullmatch(text):
raise Failure("The headset's clipboard holds an image. Only text can be copied.", 400)
if not text:
raise Failure(CLIPBOARD_EMPTY + " (or holds something other than text)", 400)
return text
def clipboard(body):
if body.get("read") is True:
# The page puts it on the clipboard where it runs (the app, or the iPhone).
text = headset_clipboard()
return {"text": text, "message": f"Copied {len(text):,} characters from the headset's clipboard"}
if body.get("toComputer") is True:
if LOCAL:
raise Failure("This server runs on the headset: copy from the Frame Control app instead", 400)
text = headset_clipboard()
try:
frame_host.set_clipboard_text(text)
except frame_host.HostError as e:
raise Failure(str(e), 500)
return {"message": f"Copied {len(text):,} characters from the headset to this computer's clipboard"}
if body.get("fromMac") or body.get("fromComputer"):
try:
text = frame_host.clipboard_text()
@@ -964,7 +969,10 @@ def clipboard(body):
try:
return {"message": ssh(PASTE_CMD, stdin=text, timeout=30).strip()}
except Failure as e:
raise desktop_failure(e) or e from None
if DESKTOP_CLOSED in str(e):
# The headset's state, for the person to change, not a fault: a 409 isn't reported as one.
raise Failure(str(e), 409) from None
raise
# ---- keyboard and pointer (KDE Connect on the Frame, see frame_input_agent.py) ----
@@ -1532,6 +1540,10 @@ def open_thing(body):
SHOTS_DIR.mkdir(parents=True, exist_ok=True)
frame_host.open_path(SHOTS_DIR)
return {"message": f"Opened {SHOTS_DIR} in {frame_host.FILE_MANAGER}"}
if what == "videos":
frame_clips.VIDEOS_DIR.mkdir(parents=True, exist_ok=True)
frame_host.open_path(frame_clips.VIDEOS_DIR)
return {"message": f"Opened {frame_clips.VIDEOS_DIR} in {frame_host.FILE_MANAGER}"}
if what == "shot":
saved = SHOTS_DIR / shot_path(body.get("id")).rsplit("/", 1)[-1]
if not saved.exists():
@@ -2773,6 +2785,7 @@ POST = {
"/api/settings/artwork": frame_steamgriddb.save_settings,
"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/quest": quest, "/api/meta": meta, "/api/queue": queue_action, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/mods": mods, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
"/api/record": record, "/api/clips/export": export_clip,
"/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
"/api/webinstall/cancel": webinstall_cancel,
"/api/telemetry": frame_telemetry.update_settings, "/api/telemetry/event": frame_telemetry.page_event,
@@ -3065,6 +3078,11 @@ class Handler(BaseHTTPRequestHandler):
self.send_json(list_shots())
elif path == "/api/shots/image":
self.send_bytes(*shot_image(url.query))
elif path == "/api/record":
_clips_here()
self.send_json(recorder.status())
elif path == "/api/clips":
self.send_json(list_clips())
elif path == "/api/stream":
self.stream_video(url.query)
elif path == "/api/screenshot" and parse_qs(url.query).get("view") == ["panel"]: