Compare commits

..
Author SHA1 Message Date
SAPHID 7dfe7777ac Group clip export changelog with screenshot copying 2026-10-25 06:21:36 +00:00
SAPHID a87b746dc1 Merge remote-tracking branch 'origin/main' into fix/steam-clip-export-collisions
# Conflicts:
#	CHANGELOG.md
2026-10-25 06:21:03 +00:00
Alex SouthwellandSAPHID da34d94009 Skip backup paths with symlinked ancestors (#215)
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-25 06:19:38 +00:00
Alex SouthwellandSAPHID 234b0a1814 Add managed Decky store and Frame plugin compatibility audit (#208)
* Add managed Decky store plugins and Frame compatibility audit

* Reject malformed flags and linked plugin folders

---------

Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-25 06:15:42 +00:00
Alex SouthwellandSAPHID 7a8e919546 Validate extension bridge API versions before loading panels (#209)
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-25 06:13:55 +00:00
SAPHID c49afdc625 Fix Steam clip export collisions across headset and session identities 2026-10-25 06:10:44 +00:00
15 changed files with 644 additions and 44 deletions

No files matched your search

+11
View File
@@ -2,6 +2,12 @@
## Unreleased
- Skip backup saves and settings reached through a symbolic link anywhere below
the home folder, and recheck paths before packing each file. Local synthetic
tests cover redirected Proton prefixes and links changed after discovery;
concurrent filesystem changes can still race the checks.
[Backup layout and limits](docs/backup-and-recovery.md).
- Import a base APK with supported native-library or asset splits from a local
Android backup or XAPK/APKS/APKM bundle. Check package and version before
install; refuse resource, code and feature splits and device-variant sets.
@@ -12,6 +18,11 @@
existing files in the old flat folder are left in place.
[Copying guide](docs/recording.md).
- Keep Steam clip exports separate for each headset, Steam account and nested
recording session. Manual export and automatic copying use the same folders;
existing flat exports stay untouched and do not mark unrelated clips saved.
[Recording guide](docs/recording.md#steam-clips).
- Compare two explicit Windows Wireless Adapter doctor reads in local memory.
Highlight exact PnP instances and known state changes after a dropout;
incomplete inventories stay unknown. A Forget button clears the comparison.
+23 -2
View File
@@ -100,7 +100,21 @@ copy one by hand if you need it.
- Steam Cloud's own folders (`userdata/<account>/<appid>`), shader caches and
screenshots (**Tools → Capture & Live** saves those to Pictures).
- Android apps themselves, their OBB files and shared storage (`/sdcard`).
- Symbolic links (never followed), and single files over 2 GiB.
- Symbolic links, including any ancestor below the home folder of a selected
save or settings path, and single files over 2 GiB. Discovery checks each
component, and packing checks it again before opening each file.
Steam backups use the normal `~/.local/share/Steam` directory tree. A separate
`~/.steam/steam` alias does not affect this route. A library, Proton prefix or
Wine save folder redirected by a symlink below the home folder is skipped,
even if its target is inside home; it is not resolved and backed up elsewhere.
The home folder itself is trusted and may use the host's normal path aliases.
These are path checks, not a filesystem snapshot: a concurrent rename or link
replacement between a check and a read can still race it. Close games and
avoid moving or relinking these folders during backup. This fix rejects links
already present when checked, including ones introduced after discovery; it
does not provide atomic protection against concurrent filesystem changes.
Backups aren't encrypted. Game folders can still hold a launcher's login, and
Android app data can hold an app's, so keep backups private.
@@ -192,11 +206,18 @@ If the button can't help, or the headset isn't reachable:
`tests/test_backup.py` runs the headset-side script (`ui/frame_backup_remote.py`)
for real against a folder laid out like the Frame's home, with only SSH faked:
which paths are picked, that sign-in files and keys never are (also inside game
folders), links never followed, the manifest and its checksums, restore checks
folders), links in save/settings roots and their ancestors refused, the manifest
and its checksums, restore checks
after a reset (game not installed, no Proton folders yet, not signed in), that a
restore replaces only its own files and keeps the old ones, unsafe archives
(`..`, absolute paths, links, other games' folders) writing nothing, backup
names that try to leave the backups folder, and the exact Game Mode command.
Synthetic symlink fixtures cover every component of a Proton save root below
home, including `compatdata/<appid>` and `steamuser/AppData`, plus directory
and file links introduced between discovery and packing. Ordinary Proton and
non-Steam saves, Linux saves, controller layouts, reference settings and home
aliases still pack with their original bytes. These are local fixture checks;
this path fix has not been tested on a headset.
`tests/e2e/test_recovery_kit.py` backs up a Proton save from the fake Frame through
the server, deletes it and restores it. Neither involves a real headset.
@@ -0,0 +1,70 @@
# Steam clip export collision fixtures, 25 October 2026
Base: `d79efb4` (Frame Control v0.4.30). Issue:
[#212](https://github.com/saphid/frame-control/issues/212).
Host: Linux 6.8.0-139-generic, Python 3.12.3, ffmpeg 6.1.1.
## Baseline reproduction
Loaded the unmodified `ui/frame_clips.py` from `d79efb4` into a temporary
Python module and exported three fake sources into a temporary directory:
```
1/clips/clip_620/video/bg
2/clips/clip_620/video/bg
1/clips/clip_620/video/other
```
Each source's payload was its own ID. All three exports returned the same
destination, `clip_620.mp4`; listing marked all three saved. The final file
contained only the last source ID. No SSH or headset was used.
## Regression coverage
`tests/test_clips.py` exports five IDs on each of two fake headsets, including
equal clip names across accounts, nested sessions with equal leaf names,
different leaf names under one clip and the background/clip distinction.
All ten payloads survive at distinct destinations. Listing marks only the
identities already exported. A pre-existing flat MP4 retains its original
bytes and does not mark any identity saved. Normal export still returns
`message` and `file`, removes its remote temporary export and clears local
staging; a failed re-export preserves the completed file.
`tests/test_clip_copy.py` exercises the server's list and manual export
functions and its actual auto-sync callbacks with fake transport. A manual
copy is recognised by auto-sync, all other account/session identities copy,
and switching to a second headset copies its own sources without overwriting
the first. A delayed manual job refuses a changed headset with status 409.
Both local phone mode and local headset runtime reject Steam clip list,
export and auto-sync requests with status 400. The separate `Recorder` path
is unchanged.
## Validation
- `PYTHONPATH=tests python3 -m unittest test_clips test_clip_copy test_autosync test_shot_copy test_headset_actions -v`
— 66 tests passed in 17.288 seconds, no skips. Includes real ffmpeg/ffprobe
export of a generated 65-second DASH fixture and recording stop/cleanup.
- `PYTHONPATH=tests python3 -m unittest test_server -v`
— 44 tests passed in 12.315 seconds, no skips.
- `node --test site/test/*.test.mjs` — 2 tests passed.
- `python3 -m unittest discover -s tests -v` — 1,518 tests run in
181.025 seconds; OK, 16 skipped for unavailable platform/tool prerequisites.
- `python3 -m py_compile ui/frame_clips.py ui/server.py tests/test_clips.py tests/test_clip_copy.py`
and `git diff --check` — passed.
The server and full Python suites require localhost sockets and were run
outside the network sandbox. Tests import `tests/sandbox.py` to isolate app
data, SSH settings, home directories and service URLs.
## Limits and upgrade behaviour
No real headset was accessed: the shared device lock is held by other work.
This is Linux fixture and server coverage, not a fresh packaged-app or
Windows/macOS verification. Existing real-headset export evidence in the
recording guide predates this destination change.
Old flat exports cannot prove headset/account/session identity and are left
untouched. Previously remembered auto-sync IDs retain their existing
watermark/history behaviour; **Copy everything now** or manual **Export**
creates an identified copy. No release, version bump, merge or social post
is included. PR #113 is outside this task.
+8
View File
@@ -34,11 +34,19 @@ The manifest ID must match the folder name:
"id": "your-id",
"name": "Your feature",
"version": "1.0.0",
"apiVersion": 1,
"description": "What the panel does.",
"entry": "index.html"
}
```
`apiVersion` selects the Frame Control bridge contract, independently of your
extension's own `version`. This app supports integer `1`; a missing field means
`1` for existing extensions. Unsupported versions appear as errors on Refresh,
and their assets are not served. Declare it in new extensions so a future API
change can fail clearly instead of running incompatible code. The runtime
`capabilities` reply below reports the bridge version too.
Put JavaScript in a separate `.js` file. Supported assets are HTML, JS, CSS,
JSON, PNG, JPEG, GIF, WebP and SVG, all in the extension's top-level folder.
Nested paths and symlinks are not served. No file may exceed 5 MiB. The panel
+28 -5
View File
@@ -69,11 +69,31 @@ yes-or-no answer about passthrough pixels. See the
**Steam clips** lists what Steam's game recording has kept on the Frame: saved
clips and background recordings, newest first, with the game, length and size.
**Export** joins one into a single `.mp4` in `~/Videos/SteamFrame`, named after
its folder (for example `clip_620_20261020_202134.mp4`), however long it is.
**Export** joins one into a single `.mp4` under `~/Videos/SteamFrame`, named after
its clip folder (for example `clip_620_20261020_202134.mp4`), however long it is.
Steam's own export is reported to stop at about 60 seconds. Exports run in the
background; ones already on your computer are marked.
Each export keeps its headset, Steam account and complete recording session ID:
```
~/Videos/SteamFrame/headset-<id>/<account>/clips/<clip>/video/<session>/<clip>.mp4
~/Videos/SteamFrame/headset-<id>/<account>/video/<session>/<session>.mp4
```
The headset folder uses the same short device-identity hash as screenshots.
All folders in the Steam clip ID are kept, including nested sessions, so equal
clip names across accounts, headsets or sessions keep separate files. Manual
**Export** and automatic copying use the same path and saved marker. Exporting
the same identified session again replaces only that session's export.
Existing exports in the old flat `~/Videos/SteamFrame/<clip>.mp4` layout stay
untouched. Their source identity cannot be proved, so they do not mark a clip
as saved. Export again or use **Copy everything now** to make an identified
copy. Automatic copying still remembers IDs copied before the upgrade; use
**Copy everything now** to copy those again. This destination change has local
fixture coverage; it has not been tested on a real headset.
How it works: Steam keeps each recording as MPEG-DASH segments, one folder per
recording session.
@@ -147,8 +167,10 @@ and the last problem it had. The screenshot and clip lists say when it's on.
`ui/frame_clips.py` holds the logic, so other features can reuse it (automatic
copying, in `ui/frame_autosync.py`, does):
- `list_clips(run_frame)` and `export_clip(id, run_frame, run, fetch)` are the
computer's side; the server passes how to reach the Frame.
- `list_clips(run_frame, device_id=...)` and
`export_clip(id, run_frame, run, fetch, device_id=...)` are the computer's
side; the server passes how to reach the Frame and its pinned device identity.
Both use `clip_local_path(id, device_id, folder=None)` for the destination.
- `list_recordings(userdata)` and `export(userdata, id)` are the Frame's side,
run as `python3 - list` / `python3 - export ID` with the file on stdin.
- `Recorder` runs one recording from Start to the copy.
@@ -171,7 +193,8 @@ The API, with the same guards as every other request:
They aren't available when the server runs on the Frame itself (the iPhone
app), since they save to a computer.
Tests: `tests/test_clips.py`, and `tests/test_autosync.py` for automatic
Tests: `tests/test_clips.py`, `tests/test_clip_copy.py` for server destinations
and collision coverage, and `tests/test_autosync.py` for automatic
copying (with a fake headset). They run the recording's shell under bash with a
stand-in `ffmpeg`, so Stop, the time limit and the clean-up are tested as they
run on the Frame. Where `ffmpeg` is installed, they also make a 65-second DASH
@@ -2,6 +2,7 @@
"id": "headset-glance",
"name": "Headset Glance",
"version": "1.0.0",
"apiVersion": 1,
"description": "Shows a small read-only summary of the connected Steam Frame.",
"entry": "index.html"
}
+102
View File
@@ -113,6 +113,108 @@ class Remote(unittest.TestCase):
os.symlink(str(outside), str(base / 'linked-file'))
self.assertFalse([f for f in self.files('saves:1234') if 'linked' in f])
def packed(self, part):
out = io.BytesIO()
remote.pack(part, out)
out.seek(0)
with tarfile.open(fileobj=out, mode='r:gz') as archive:
return {m.name: archive.extractfile(m).read() for m in archive}
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_backup_rejects_every_ancestor_of_a_save_root(self):
save = USER % 1234 + '/AppData/Roaming/Game/fixture.dat'
# Each component below HOME, including the selected root itself.
components = (USER % 1234 + '/AppData/Roaming').split('/')
for end in range(1, len(components) + 1):
ancestor = '/'.join(components[:end])
with self.subTest(ancestor=ancestor), tempfile.TemporaryDirectory() as tmp:
home = Path(tmp) / 'home'
manifest(home, 1234, 'Synthetic game')
path = write(home, save, b'synthetic fixture')
linked = home / ancestor
outside = Path(tmp) / 'outside/.ssh'
outside.parent.mkdir()
linked.rename(outside)
linked.symlink_to(outside, target_is_directory=True)
# Only synthetic bytes: the lexical archive path hides .ssh.
self.assertEqual(path.read_bytes(), b'synthetic fixture')
with patch.object(remote, 'HOME', home):
self.assertEqual(self.files('saves:1234'), [])
self.assertNotIn('saves:1234', [p['id'] for p in remote.plan()['parts']])
self.assertEqual(self.packed('saves:1234'), {})
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_pack_rechecks_links_changed_after_discovery(self):
save = USER % 1234 + '/Documents/Game/fixture.dat'
for ancestor in (remote.COMPAT + '/1234', USER % 1234 + '/Documents', save):
with self.subTest(ancestor=ancestor), tempfile.TemporaryDirectory() as tmp:
home = Path(tmp) / 'home'
write(home, save, b'original save')
linked = home / ancestor
outside = Path(tmp) / 'outside/.ssh'
outside.parent.mkdir()
part_files = remote.part_files
def discover_then_link(part):
result = part_files(part)
self.assertEqual(result[0], [(save, len(b'original save'))])
linked.rename(outside)
linked.symlink_to(outside, target_is_directory=outside.is_dir())
(home / save).write_bytes(b'synthetic redirected bytes')
return result
with patch.object(remote, 'HOME', home), \
patch.object(remote, 'part_files', side_effect=discover_then_link):
self.assertEqual(self.packed('saves:1234'), {})
def test_legitimate_steam_layout_and_reference_files_still_pack(self):
frame_home(self.home)
manifest(self.home, 2800000002, 'Synthetic non-Steam game')
save = USER % 2800000002 + '/Saved Games/fixture.dat'
write(self.home, save, b'non-Steam save')
controller = STEAM + '/steamapps/common/Steam Controller Configs/555/config/layout.vdf'
write(self.home, controller, b'controller layout')
for part in ('saves:1234', 'saves:2800000002', 'saves:linux',
'settings:steam', 'settings:system'):
with self.subTest(part=part):
expected = {rel: (self.home / rel).read_bytes() for rel in self.files(part)}
self.assertTrue(expected)
self.assertEqual(self.packed(part), expected)
self.assertEqual(self.packed('saves:2800000002'), {save: b'non-Steam save'})
self.assertIn(controller, self.packed('settings:steam'))
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_settings_discovery_and_manifest_reads_reject_links(self):
frame_home(self.home)
for ancestor, part in (('.config', 'settings:system'),
(STEAM + '/userdata', 'settings:steam')):
with self.subTest(ancestor=ancestor):
linked = self.home / ancestor
moved = linked.with_name(linked.name + '-original')
linked.rename(moved)
linked.symlink_to(moved, target_is_directory=True)
self.assertEqual(self.files(part), [])
self.assertEqual(self.packed(part), {})
if part == 'settings:steam':
self.assertEqual(remote.accounts(), [])
appmanifest = self.home / STEAM / 'steamapps/appmanifest_1234.acf'
moved = appmanifest.with_suffix('.original')
appmanifest.rename(moved)
appmanifest.symlink_to(moved)
with patch.object(Path, 'read_text', side_effect=AssertionError('must not read a link')):
self.assertIsNone(remote.game_name('1234'))
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_home_alias_and_unused_steam_alias_preserve_normal_backups(self):
frame_home(self.home)
alias = self.home / 'home-alias'
alias.symlink_to(self.home, target_is_directory=True)
(self.home / '.steam/steam').symlink_to(self.home / STEAM, target_is_directory=True)
expected = self.packed('saves:1234')
with patch.object(remote, 'HOME', alias):
self.assertEqual(self.packed('saves:1234'), expected)
self.assertEqual(remote.game_name('1234'), 'Test Game')
def test_big_files_are_listed_as_skipped(self):
frame_home(self.home)
with patch.object(remote, 'MAX_FILE', 3):
+115
View File
@@ -0,0 +1,115 @@
"""Clip destinations through the server and automatic copying, without a headset."""
import sandbox # noqa: F401
import shutil
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "ui"))
import frame_autosync
import frame_clips
import server
IDS = ["1/clips/clip_620/video/bg", "2/clips/clip_620/video/bg",
"1/clips/clip_620/video/other", "1/clips/other/video/bg"]
NOW = 1800000000
class ClipCopyTest(unittest.TestCase):
def setUp(self):
self.folder = Path(tempfile.mkdtemp(prefix="frame-clip-copy-"))
self.addCleanup(shutil.rmtree, self.folder, ignore_errors=True)
self.device = "frame-a"
self.link = mock.Mock()
self.link.active_device.side_effect = lambda: {"id": self.device, "name": self.device}
self.link.snapshot.return_value = {"phase": "connected"}
self.exports = []
self.payload = None
for name, value in (("LINK", self.link), ("LOCAL", False)):
patch = mock.patch.object(server, name, value)
patch.start()
self.addCleanup(patch.stop)
patch = mock.patch.object(frame_clips, "VIDEOS_DIR", self.folder)
patch.start()
self.addCleanup(patch.stop)
for name, fn in (("_clips_frame", self.frame), ("fetch_file", self.fetch),
("ssh", lambda *a, **k: "")):
patch = mock.patch.object(server, name, side_effect=fn)
patch.start()
self.addCleanup(patch.stop)
(self.folder / "clip_620.mp4").write_bytes(b"legacy export")
def frame(self, args, timeout):
if args == ["list"]:
return {"clips": [{"id": cid, "kind": "clip", "time": NOW + 5, "idle": 60} for cid in IDS]}
self.exports.append((self.device, args[1]))
self.payload = f"{self.device}: {args[1]}".encode()
return {"path": f"{frame_clips.WORK}/export-0123456789abcdef/clip_620.mp4",
"size": len(self.payload)}
def fetch(self, remote, target):
target.write_bytes(self.payload)
def test_manual_export_lists_only_matching_identity_and_keeps_job_shape(self):
with mock.patch.object(server, "start_job", side_effect=lambda label, work: work()) as job:
for device in ("frame-a", "frame-b"):
self.device = device
self.assertFalse(any(c["saved"] for c in server.list_clips()["clips"]))
result = server.export_clip({"id": IDS[0]})
self.assertEqual(set(result), {"message", "file"})
self.assertEqual(Path(result["file"]).read_bytes(), f"{device}: {IDS[0]}".encode())
self.assertEqual([c["saved"] for c in server.list_clips()["clips"]], [True, False, False, False])
self.assertEqual(job.call_args.args[0], "Export clip_620.mp4")
first = frame_clips.clip_local_path(IDS[0], "frame-a", self.folder)
self.assertEqual(first.read_bytes(), f"frame-a: {IDS[0]}".encode())
self.assertEqual((self.folder / "clip_620.mp4").read_bytes(), b"legacy export")
def test_auto_sync_uses_same_identity_as_manual_exports(self):
with mock.patch.object(server, "start_job", side_effect=lambda label, work: work()):
server.export_clip({"id": IDS[0]})
sync = frame_autosync.AutoSync(
server._sync_list(server.list_shots), server._sync_shots,
server._sync_list(server.list_clips), server._sync_clip,
server._sync_device, lambda: False, shots_folder=self.folder / "shots",
clips_folder=self.folder, path=self.folder / "auto-sync.json", clock=lambda: NOW,
space=lambda folder: 100 * 1024**3)
sync.update({"clips": True})
sync.tick()
self.assertEqual(self.exports, [("frame-a", cid) for cid in IDS])
self.device = "frame-b"
sync.tick()
self.assertEqual(self.exports, [(device, cid) for device in ("frame-a", "frame-b") for cid in IDS])
sync.tick()
self.assertEqual(len(self.exports), 8)
for device in ("frame-a", "frame-b"):
for cid in IDS:
target = frame_clips.clip_local_path(cid, device, self.folder)
self.assertEqual(target.read_bytes(), f"{device}: {cid}".encode())
self.assertEqual((self.folder / "clip_620.mp4").read_bytes(), b"legacy export")
def test_delayed_job_refuses_a_different_headset(self):
with mock.patch.object(server, "start_job", return_value={"job": "queued"}) as job:
self.assertEqual(server.export_clip({"id": IDS[0]}), {"job": "queued"})
self.device = "frame-b"
with self.assertRaises(server.Failure) as error:
job.call_args.args[1]()
self.assertEqual(error.exception.status, 409)
self.assertEqual(self.exports, [])
def test_local_and_headset_modes_reject_steam_clip_exports(self):
for headset in (False, True):
with self.subTest(headset=headset), mock.patch.object(server, "LOCAL", True), \
mock.patch.object(server, "HEADSET_RUNTIME", headset):
for action in (server.list_clips, lambda: server.export_clip({"id": IDS[0]}),
lambda: server.auto_sync({"action": "now"})):
with self.assertRaises(server.Failure) as error:
action()
self.assertEqual(error.exception.status, 400)
self.assertEqual(self.exports, [])
if __name__ == "__main__":
unittest.main()
+65 -8
View File
@@ -71,6 +71,17 @@ class Ids(unittest.TestCase):
with self.subTest(bad=bad), self.assertRaises(ValueError):
clips.remove_command(bad)
def test_local_path_keeps_validated_identity_and_hashes_device_names(self):
folder = Path("exports")
target = clips.clip_local_path("123/clips/clip/video/session", "../../device:name", folder)
self.assertRegex(target.parts[1], r"^headset-[0-9a-f]{12}$")
self.assertEqual(target.parts[2:], ("123", "clips", "clip", "video", "session", "clip.mp4"))
for bad in (None, "", 42):
with self.subTest(device=bad), self.assertRaises(ValueError):
clips.clip_local_path("123/video/bg", bad, folder)
with self.assertRaises(ValueError):
clips.clip_local_path("../clips/clip", "frame-a", folder)
def test_recording_names_checked(self):
for fn in (clips.finish_command, clips.cleanup_command, lambda n: clips.record_command(n, True)):
for bad in ("rec-1", "rec-20261020-201950; rm -rf ~", "../rec-20261020-201950", "*"):
@@ -156,10 +167,10 @@ class Sessions(unittest.TestCase):
def test_listing_marks_what_is_already_here(self):
here = self.dir / "here"
write(here / "clip_620_20261020_202134.mp4")
write(clips.clip_local_path("1/clips/clip_620_20261020_202134/video/bg", "frame-a", here))
frame = lambda args, timeout: {"clips": [{"id": "1/clips/clip_620_20261020_202134/video/bg"},
{"id": "1/video/bg_620_20261020_201950"}]}
got = clips.list_clips(frame, folder=here)
got = clips.list_clips(frame, folder=here, device_id="frame-a")
self.assertEqual([c["saved"] for c in got["clips"]], [True, False])
self.assertEqual(got["folder"], str(here))
@@ -261,12 +272,16 @@ class ComputerSide(unittest.TestCase):
frame = lambda args, timeout: calls.append(("frame", args)) or {"path": path, "size": 3}
run = lambda cmd, timeout: calls.append(("run", cmd))
fetch = lambda remote, local: calls.append(("fetch", remote)) or Path(local).write_bytes(b"mp4")
got = clips.export_clip("1/clips/clip_620/video/bg", frame, run, fetch, folder=self.dir)
got = clips.export_clip("1/clips/clip_620/video/bg", frame, run, fetch, folder=self.dir, device_id="frame-a")
self.assertEqual(got["message"], "Saved clip_620.mp4 to ~/Videos/SteamFrame")
self.assertEqual((self.dir / "clip_620.mp4").read_bytes(), b"mp4")
target = clips.clip_local_path("1/clips/clip_620/video/bg", "frame-a", self.dir)
self.assertEqual(got["file"], str(target))
self.assertEqual(set(got), {"message", "file"})
self.assertEqual(target.read_bytes(), b"mp4")
self.assertEqual([c[0] for c in calls], ["frame", "fetch", "run"])
self.assertEqual(calls[2][1], clips.remove_command(path))
self.assertEqual([p.name for p in self.dir.iterdir()], ["clip_620.mp4"])
self.assertEqual(list(self.dir.rglob("*.mp4")), [target])
self.assertFalse(list(self.dir.glob(".incoming-*")))
def test_failed_copy_leaves_no_partial_file_and_still_cleans_up(self):
path = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
@@ -277,20 +292,62 @@ class ComputerSide(unittest.TestCase):
raise RuntimeError("connection lost")
with self.assertRaisesRegex(RuntimeError, "connection lost"):
clips.export_clip("1/clips/clip_620/video/bg", lambda a, t: {"path": path, "size": 4},
lambda cmd, timeout: ran.append(cmd), fetch, folder=self.dir)
lambda cmd, timeout: ran.append(cmd), fetch, folder=self.dir, device_id="frame-a")
self.assertEqual(list(self.dir.iterdir()), [])
self.assertEqual(ran, [clips.remove_command(path)])
def test_same_names_keep_each_account_headset_kind_and_nested_session(self):
first = "1/clips/clip_620/video/bg"
ids = [first, "2/clips/clip_620/video/bg", "1/clips/clip_620/video/other",
"1/clips/clip_620/other/bg", "1/video/clip_620"]
legacy = write(self.dir / "clip_620.mp4", b"legacy export")
remote = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
frame = lambda args, timeout: {"clips": [{"id": cid} for cid in ids]}
stored = {}
for device in ("frame-a", "frame-b"):
self.assertFalse(any(c["saved"] for c in clips.list_clips(
frame, folder=self.dir, device_id=device)["clips"]))
for index, cid in enumerate(ids):
payload = f"{device} session {index}".encode()
result = clips.export_clip(cid, lambda a, t: {"path": remote, "size": len(payload)},
lambda c, t: None, lambda r, p: p.write_bytes(payload),
folder=self.dir, device_id=device)
stored[Path(result["file"])] = payload
listing = clips.list_clips(frame, folder=self.dir, device_id=device)
self.assertEqual([c["saved"] for c in listing["clips"]],
[n <= index for n in range(len(ids))])
self.assertEqual(listing["folder"], str(self.dir))
self.assertEqual(len(stored), len(ids) * 2)
for path, payload in stored.items():
self.assertEqual(path.read_bytes(), payload)
self.assertEqual(legacy.read_bytes(), b"legacy export")
self.assertFalse(list(self.dir.glob(".incoming-*")))
def test_failed_reexport_preserves_completed_export(self):
cid = "1/clips/clip_620/video/bg"
target = write(clips.clip_local_path(cid, "frame-a", self.dir), b"completed")
remote = f"{clips.WORK}/export-0123456789abcdef/clip_620.mp4"
def fetch(remote, local):
local.write_bytes(b"partial")
raise RuntimeError("connection lost")
with self.assertRaisesRegex(RuntimeError, "connection lost"):
clips.export_clip(cid, lambda a, t: {"path": remote}, lambda c, t: None, fetch,
folder=self.dir, device_id="frame-a")
self.assertEqual(target.read_bytes(), b"completed")
self.assertFalse(list(self.dir.glob(".incoming-*")))
def test_bad_id_never_reaches_the_frame(self):
never = mock.Mock(side_effect=AssertionError("reached the Frame"))
with self.assertRaises(ValueError):
clips.export_clip("1/video/../../.ssh", never, never, never, folder=self.dir)
clips.export_clip("1/video/../../.ssh", never, never, never, folder=self.dir, device_id="frame-a")
def test_unexpected_path_from_the_frame_refused(self):
never = mock.Mock(side_effect=AssertionError("deleted or copied"))
with self.assertRaises(ValueError):
clips.export_clip("1/video/bg", lambda a, t: {"path": ".ssh/id_ed25519", "size": 1}, never, never,
folder=self.dir)
folder=self.dir, device_id="frame-a")
class Commands(unittest.TestCase):
+107
View File
@@ -0,0 +1,107 @@
"""Extension compatibility and filesystem boundaries, without running plugin code."""
import sandbox # noqa: F401
import json
import shutil
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
import frame_extensions
class ExtensionFiles(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name) / 'extensions'
self.root.mkdir()
self.patch = patch.object(frame_extensions, 'root', return_value=self.root)
self.patch.start()
self.addCleanup(self.patch.stop)
def extension(self, name='test-panel', **fields):
folder = self.root / name
folder.mkdir(exist_ok=True)
manifest = {'id': name, 'name': 'Test panel', 'version': '1.0.0',
'description': 'A read-only panel', 'entry': 'index.html', **fields}
(folder / 'manifest.json').write_text(json.dumps(manifest), encoding='utf-8')
(folder / 'index.html').write_text('<script src="main.js"></script>', encoding='utf-8')
(folder / 'main.js').write_text('/* extension code */', encoding='utf-8')
return folder
def test_existing_and_explicit_v1_manifests_are_compatible(self):
self.extension('legacy')
self.extension('explicit', apiVersion=1)
result = frame_extensions.list_extensions()
self.assertEqual(result['errors'], [])
self.assertEqual([(x['id'], x['apiVersion']) for x in result['extensions']],
[('explicit', 1), ('legacy', 1)])
self.assertEqual(frame_extensions.asset('explicit', 'main.js'),
(b'/* extension code */', 'text/javascript; charset=utf-8'))
def test_unsupported_or_malformed_api_cannot_be_loaded_directly(self):
for version in (2, 0, -1, '1', None, True, 1.0, [], {}):
with self.subTest(apiVersion=version):
self.extension(apiVersion=version)
result = frame_extensions.list_extensions()
self.assertEqual(result['extensions'], [])
self.assertIn('unsupported apiVersion', result['errors'][0]['error'])
with self.assertRaisesRegex(ValueError, 'unsupported apiVersion'):
frame_extensions.asset('test-panel', 'index.html')
def test_one_incompatible_extension_does_not_hide_compatible_panels(self):
self.extension('future', apiVersion=2)
self.extension('working', apiVersion=1)
result = frame_extensions.list_extensions()
self.assertEqual([x['id'] for x in result['extensions']], ['working'])
self.assertEqual(result['errors'][0]['folder'], 'future')
def test_sample_is_discoverable_and_its_assets_are_served(self):
source = Path(__file__).resolve().parents[1] / 'examples/extensions/headset-glance'
shutil.copytree(source, self.root / source.name)
result = frame_extensions.list_extensions()
self.assertEqual(result['errors'], [])
self.assertEqual(result['extensions'][0]['id'], source.name)
for asset in ('index.html', 'main.js', 'style.css'):
self.assertEqual(frame_extensions.asset(source.name, asset)[0],
(source / asset).read_bytes())
def test_private_nested_and_traversal_paths_are_rejected(self):
self.extension()
for name in ('manifest.json', '../outside.js', 'sub/main.js', '.hidden.js'):
with self.subTest(name=name), self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', name)
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('../test-panel', 'main.js')
def test_oversized_assets_are_not_read(self):
folder = self.extension()
with (folder / 'large.js').open('wb') as output:
output.truncate(5 * 1024 * 1024 + 1)
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', 'large.js')
def test_symlinks_cannot_expose_files_outside_a_panel(self):
folder = self.extension()
outside = Path(self.tmp.name) / 'outside.js'
outside.write_text('private file', encoding='utf-8')
try:
(folder / 'escape.js').symlink_to(outside)
(self.root / 'alias').symlink_to(folder, target_is_directory=True)
except OSError:
self.skipTest('symlinks unavailable for this user')
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', 'escape.js')
with self.assertRaises(ValueError):
frame_extensions.asset('alias', 'main.js')
(folder / 'manifest.json').unlink()
(folder / 'manifest.json').symlink_to(outside)
with self.assertRaises(ValueError):
frame_extensions.asset('test-panel', 'main.js')
if __name__ == '__main__':
unittest.main()
+18
View File
@@ -126,6 +126,24 @@ class ServerGuards(unittest.TestCase):
self.assertEqual(code, 400)
self.assertIn('experimental', body['error'])
def test_incompatible_extension_is_reported_and_assets_are_unavailable(self):
folder = Path(os.environ['FRAME_CONTROL_DATA_DIR']) / 'extensions' / 'future-panel'
folder.mkdir(parents=True, exist_ok=True)
(folder / 'manifest.json').write_text(json.dumps({'id': 'future-panel', 'name': 'Future',
'version': '1.0', 'description': 'Requires a newer API', 'apiVersion': 2}))
(folder / 'index.html').write_text('<p>Incompatible panel</p>')
try:
self.assertEqual(self.request('GET', '/api/extensions')[0], 403)
code, _, body = self.request('GET', '/api/extensions', headers={'X-Frame-UI': '1'})
self.assertEqual(code, 200)
data = json.loads(body)
self.assertNotIn('future-panel', [x['id'] for x in data['extensions']])
self.assertTrue(any(x['folder'] == 'future-panel' and 'apiVersion' in x['error']
for x in data['errors']))
self.assertEqual(self.request('GET', '/extension/future-panel/index.html')[0], 404)
finally:
shutil.rmtree(folder)
def test_headset_page_and_install_acknowledgement(self):
code, _, body = self.request('GET', '/headset')
self.assertEqual(code, 200)
+60 -17
View File
@@ -19,6 +19,7 @@ import os
from pathlib import Path, PurePosixPath
import re
import shutil
import stat
import subprocess
import sys
import tarfile
@@ -88,28 +89,57 @@ def _under(rel, root):
return rel == root or rel.startswith(root + '/')
def _backup_stat(rel):
"""lstat a backup path only if every component below trusted HOME is link-free.
Rechecked at discovery and before reads; not atomic against concurrent renames.
HOME itself may use the host's normal home/temp-directory aliases.
"""
if not safe_rel(rel):
return None
path = HOME
try:
for component in rel.split('/'):
path = path / component
st = path.lstat()
if stat.S_ISLNK(st.st_mode):
return None
return st
except OSError:
return None
def _backup_dir(rel):
st = _backup_stat(rel)
return st is not None and stat.S_ISDIR(st.st_mode)
def _walk(rel_root, skip=()):
"""Regular files under HOME/rel_root, never following links: [(rel, size)], [skipped]."""
files, skipped = [], []
base = HOME / rel_root
if base.is_symlink() or not base.is_dir():
if base.is_file() and not base.is_symlink():
return [(rel_root, base.stat().st_size)], skipped
st = _backup_stat(rel_root)
if st is None or is_secret(rel_root):
return files, skipped
if stat.S_ISREG(st.st_mode):
if st.st_size > MAX_FILE:
skipped.append({'path': rel_root, 'reason': 'over 2 GiB'})
return files, skipped
return [(rel_root, st.st_size)], skipped
if not stat.S_ISDIR(st.st_mode):
return files, skipped
for top, dirs, names in os.walk(str(base)):
rel_top = PurePosixPath(rel_root, *Path(top).relative_to(base).parts).as_posix()
dirs[:] = sorted(d for d in dirs if not os.path.islink(os.path.join(top, d)) and
if not _backup_dir(rel_top):
dirs[:] = []
continue
dirs[:] = sorted(d for d in dirs if _backup_dir(rel_top + '/' + d) and
not any(_under(rel_top + '/' + d, s) for s in skip) and
not is_secret(rel_top + '/' + d + '/x'))
for name in sorted(names):
path, rel = os.path.join(top, name), rel_top + '/' + name
if os.path.islink(path) or is_secret(rel):
continue
try:
st = os.lstat(path)
except OSError:
continue
if not os.path.isfile(path):
rel = rel_top + '/' + name
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode) or is_secret(rel):
continue
if st.st_size > MAX_FILE:
skipped.append({'path': rel, 'reason': 'over 2 GiB'})
@@ -124,26 +154,34 @@ def _manifest_field(text, key):
def game_name(appid):
rel = '%s/steamapps/appmanifest_%s.acf' % (STEAM, appid)
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode):
return None
try:
text = (HOME / STEAM / 'steamapps' / ('appmanifest_%s.acf' % appid)).read_text(errors='replace')
text = (HOME / rel).read_text(errors='replace')
except OSError:
return None
return _manifest_field(text, 'name')
def accounts():
root = HOME / STEAM / 'userdata'
rel = STEAM + '/userdata'
if not _backup_dir(rel):
return []
try:
return sorted(p.name for p in root.iterdir() if p.name.isdigit() and p.is_dir()
and not p.is_symlink() and p.name != '0')
return sorted(p.name for p in (HOME / rel).iterdir() if p.name.isdigit() and
p.name != '0' and _backup_dir(rel + '/' + p.name))
except OSError:
return []
def _prefixes():
if not _backup_dir(COMPAT):
return []
try:
return sorted((p.name for p in (HOME / COMPAT).iterdir()
if p.name.isdigit() and (p / PREFIX_USER).is_dir()), key=int)
if p.name.isdigit() and _backup_dir(COMPAT + '/' + p.name + '/' + PREFIX_USER)), key=int)
except OSError:
return []
@@ -244,9 +282,14 @@ def pack(part, output):
with tarfile.open(fileobj=output, mode='w|gz') as archive:
for rel, _ in files:
path = HOME / rel
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode) or is_secret(rel):
continue
try:
with open(str(path), 'rb') as src:
st = os.fstat(src.fileno())
if not stat.S_ISREG(st.st_mode) or st.st_size > MAX_FILE:
continue
member = tarfile.TarInfo(rel)
member.size, member.mtime, member.mode = st.st_size, int(st.st_mtime), st.st_mode & 0o755
archive.addfile(member, src)
+20 -7
View File
@@ -26,6 +26,7 @@ that stream. The folder names are as users and Steam's own files describe them;
listing doesn't depend on them: any folder under clips/ or video/ (up to four levels)
holding init-stream*.m4s counts as a session.
"""
import hashlib
import json
import os
import re
@@ -363,10 +364,20 @@ def parse_id(clip_id):
def export_name(clip_id):
"""The .mp4's name on the computer: the clip's (or background session's) own folder name."""
"""The .mp4's basename; it alone does not identify a saved export."""
return parse_id(clip_id)[2] + ".mp4"
def clip_local_path(clip_id, device_id, folder=None):
"""Keep the headset, account and full Steam session identity in the destination."""
parts = parse_id(clip_id)
if not isinstance(device_id, str) or not device_id:
raise ValueError("missing clip headset identity")
headset = hashlib.sha256(device_id.encode("utf-8")).hexdigest()[:12]
folder = Path(folder) if folder else VIDEOS_DIR
return folder / f"headset-{headset}" / Path(*parts) / export_name(clip_id)
def read_session(folder):
"""{stream number: [init, chunk, chunk...]} for one session folder, chunks in number
order. A stream without its init segment or without chunks can't be played, so it's left out."""
@@ -552,19 +563,20 @@ def remove_command(path):
# run_frame(args, timeout) runs this file on the Frame with `args` and returns its JSON;
# run(command, timeout) and fetch(remote, local) as for Recorder.
def list_clips(run_frame, folder=None):
def list_clips(run_frame, folder=None, *, device_id):
"""Steam's recordings on the Frame, each marked saved if its .mp4 is already here."""
folder = Path(folder) if folder else VIDEOS_DIR
clips = run_frame(["list"], 60)["clips"]
for c in clips:
c["saved"] = (folder / export_name(c["id"])).exists()
c["saved"] = clip_local_path(c["id"], device_id, folder).is_file()
return {"clips": clips, "folder": str(folder)}
def export_clip(clip_id, run_frame, run, fetch, folder=None):
def export_clip(clip_id, run_frame, run, fetch, folder=None, *, device_id):
"""Stitch a Steam recording on the Frame and copy the .mp4 here. -> {"message", "file"}."""
folder = Path(folder) if folder else VIDEOS_DIR
name = export_name(clip_id) # checks the id before anything reaches the Frame
target = clip_local_path(clip_id, device_id, folder) # validate before reaching the Frame
name = target.name
made = run_frame(["export", clip_id], 1900)
path = made.get("path")
remove = remove_command(path) # refuses a path that isn't one of our exports
@@ -573,7 +585,8 @@ def export_clip(clip_id, run_frame, run, fetch, folder=None):
incoming = Path(tempfile.mkdtemp(prefix=".incoming-", dir=folder))
try:
fetch(path, incoming / name)
os.replace(incoming / name, folder / name)
target.parent.mkdir(parents=True, exist_ok=True)
os.replace(incoming / name, target)
finally:
shutil.rmtree(incoming, ignore_errors=True)
finally:
@@ -581,7 +594,7 @@ def export_clip(clip_id, run_frame, run, fetch, folder=None):
run(remove, 30)
except Exception:
pass # the next export sweeps it
return {"message": f"Saved {name} to ~/Videos/SteamFrame", "file": str(folder / name)}
return {"message": f"Saved {name} to ~/Videos/SteamFrame", "file": str(target)}
# ---- on the Frame -----------------------------------------------------------------
+8 -1
View File
@@ -7,6 +7,7 @@ import frame_host
ID = re.compile(r"[a-z][a-z0-9-]{0,39}\Z")
API_VERSION = 1
ASSET = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]*\Z")
TYPES = {".html": "text/html; charset=utf-8", ".js": "text/javascript; charset=utf-8",
".css": "text/css; charset=utf-8", ".json": "application/json",
@@ -27,6 +28,11 @@ def _manifest(folder):
data = json.loads(manifest.read_text(encoding="utf-8"))
if not isinstance(data, dict) or data.get("id") != folder.name:
raise ValueError("manifest id must match its folder")
# Older manifests predate this field and use bridge v1. Check again when
# serving assets, so an incompatible panel cannot bypass discovery.
api_version = data.get("apiVersion", 1)
if type(api_version) is not int or api_version != API_VERSION:
raise ValueError(f"unsupported apiVersion; this app supports {API_VERSION}")
for key, limit in (("name", 80), ("version", 40), ("description", 240)):
value = data.get(key)
if not isinstance(value, str) or not value.strip() or len(value) > limit:
@@ -34,7 +40,8 @@ def _manifest(folder):
entry = data.get("entry", "index.html")
if entry != "index.html" or (folder / entry).is_symlink() or not (folder / entry).is_file():
raise ValueError("entry must be a regular index.html")
return {key: data[key] for key in ("id", "name", "version", "description")}
return {**{key: data[key] for key in ("id", "name", "version", "description")},
"apiVersion": api_version}
def list_extensions():
+8 -4
View File
@@ -928,7 +928,9 @@ def _clips_frame(args, timeout):
def list_clips():
_clips_here()
try:
return frame_clips.list_clips(_clips_frame)
device = agent_target()["id"]
with working(device):
return frame_clips.list_clips(_clips_frame, device_id=device)
except frame_clips.ClipError as e:
raise Failure(str(e), e.status)
@@ -938,11 +940,13 @@ def export_clip(body):
_clips_here()
clip_id = body.get("id")
name = frame_clips.export_name(clip_id) # ValueError, so 400, for a bad id
device = agent_target()["id"]
def work():
try:
return frame_clips.export_clip(clip_id, _clips_frame, lambda cmd, timeout: ssh(cmd, timeout=timeout),
fetch_file)
with working(device):
return frame_clips.export_clip(clip_id, _clips_frame, lambda cmd, timeout: ssh(cmd, timeout=timeout),
fetch_file, device_id=device)
except frame_clips.ClipError as e:
raise Failure(str(e), e.status)
return start_job(f"Export {name}", work)
@@ -965,7 +969,7 @@ def _sync_shots(device, ids):
def _sync_clip(device, clip_id):
with working(device):
return frame_clips.export_clip(clip_id, _clips_frame, lambda cmd, timeout: ssh(cmd, timeout=timeout),
fetch_file)
fetch_file, device_id=device)
def _sync_list(listing):