Compare commits

..
16 changed files with 38 additions and 728 deletions

No files matched your search

-11
View File
@@ -2,12 +2,6 @@
## Unreleased
- Skip backup saves and settings reached through a symbolic link anywhere below
the home folder, and recheck paths before packing each file. Local synthetic
tests cover redirected Proton prefixes and links changed after discovery;
concurrent filesystem changes can still race the checks.
[Backup layout and limits](docs/backup-and-recovery.md).
- Import a base APK with supported native-library or asset splits from a local
Android backup or XAPK/APKS/APKM bundle. Check package and version before
install; refuse resource, code and feature splits and device-variant sets.
@@ -219,8 +213,3 @@
shared Wine services are left alone. Verified with local Linux process
regressions and a 13/13 real Frame smoke run that stopped five test-owned
Proton process PIDs before removing the Windows test title.
- Keep failed automatic screenshot and clip copies visible after three attempts.
**Resume** and **Copy everything now** allow another try without restarting;
headset switches cannot use another headset's list or saved-file history.
[Copying guide](docs/recording.md).
+2 -23
View File
@@ -100,21 +100,7 @@ copy one by hand if you need it.
- Steam Cloud's own folders (`userdata/<account>/<appid>`), shader caches and
screenshots (**Tools → Capture & Live** saves those to Pictures).
- Android apps themselves, their OBB files and shared storage (`/sdcard`).
- Symbolic links, including any ancestor below the home folder of a selected
save or settings path, and single files over 2 GiB. Discovery checks each
component, and packing checks it again before opening each file.
Steam backups use the normal `~/.local/share/Steam` directory tree. A separate
`~/.steam/steam` alias does not affect this route. A library, Proton prefix or
Wine save folder redirected by a symlink below the home folder is skipped,
even if its target is inside home; it is not resolved and backed up elsewhere.
The home folder itself is trusted and may use the host's normal path aliases.
These are path checks, not a filesystem snapshot: a concurrent rename or link
replacement between a check and a read can still race it. Close games and
avoid moving or relinking these folders during backup. This fix rejects links
already present when checked, including ones introduced after discovery; it
does not provide atomic protection against concurrent filesystem changes.
- Symbolic links (never followed), and single files over 2 GiB.
Backups aren't encrypted. Game folders can still hold a launcher's login, and
Android app data can hold an app's, so keep backups private.
@@ -206,18 +192,11 @@ If the button can't help, or the headset isn't reachable:
`tests/test_backup.py` runs the headset-side script (`ui/frame_backup_remote.py`)
for real against a folder laid out like the Frame's home, with only SSH faked:
which paths are picked, that sign-in files and keys never are (also inside game
folders), links in save/settings roots and their ancestors refused, the manifest
and its checksums, restore checks
folders), links never followed, the manifest and its checksums, restore checks
after a reset (game not installed, no Proton folders yet, not signed in), that a
restore replaces only its own files and keeps the old ones, unsafe archives
(`..`, absolute paths, links, other games' folders) writing nothing, backup
names that try to leave the backups folder, and the exact Game Mode command.
Synthetic symlink fixtures cover every component of a Proton save root below
home, including `compatdata/<appid>` and `steamuser/AppData`, plus directory
and file links introduced between discovery and packing. Ordinary Proton and
non-Steam saves, Linux saves, controller layouts, reference settings and home
aliases still pack with their original bytes. These are local fixture checks;
this path fix has not been tested on a headset.
`tests/e2e/test_recovery_kit.py` backs up a Proton save from the fake Frame through
the server, deletes it and restores it. Neither involves a real headset.
@@ -1,79 +0,0 @@
# Auto-sync retry and headset-selection reliability
Date: 25 October 2026. Base: `d79efb4`, Frame Control v0.4.30.
Issue: [#214](https://github.com/saphid/frame-control/issues/214).
Environment: Linux x86_64, Python 3.12.3, Node 24.15.0.
## Reproduced problems
The existing 19 auto-sync tests passed on the base revision. Added regressions
failed before the implementation: an exhausted clip lost its error, explicit
retry did not recover failed screenshots or clips, and switches around listing
or copying continued the old pass. The original callback interface also had no
way to carry the tick's captured device into the server list reservation.
The final fake-source tests exercise these boundaries:
- Fail screenshots or clips three times, keep checking without further copy
attempts, and retain the source error plus an explicit retry instruction.
- Recover the source and use either Resume or Copy everything now without
restarting. Successful copies stay deduplicated; Resume retains the ledger
even after a local file is deleted. Copy everything now retains its documented
ability to restore deleted files and include older media.
- Reset an exhausted budget while the source still fails: another three
attempts are allowed, rather than unlimited background attempts.
- Save an exhausted item by hand and clear its stale failure on the next list.
- Switch after tick captures A, during a list, after screenshot bookkeeping,
between media kinds, before a transfer, or before transfer bookkeeping.
Interrupted passes do not mark the other headset's IDs or continue copying.
- Switch to B exactly at callback entry through the production `_sync_list`
adapter. `working(A)` refuses before calling the source list, leaving both
ledgers and copy callbacks untouched. The same production reservation blocks
a device switch during a list and rejects stale screenshot/clip transfers.
- Execute the production Settings renderer in Node: Resume is available for
copy errors and disk-full pauses, and hidden for a healthy state.
## Implementation boundary
Each list callback now receives the captured device, like the copy callbacks.
The existing server `working(meant)` mechanism checks that intended headset
under `_work_lock` while reserving work; `devices_post` uses the same lock to
refuse switches during that work. The worker also checks for interruption
around lists and transfers. No new device lock or clip destination logic is
introduced. The server diff is confined to `_sync_list`.
The failed-attempt and error maps are in-memory and reset on explicit retry or
restart. The successful-copy ledger and settings persistence retain their
existing format. Disk-full pauses, offline backoff, clip settling and the
three-attempt limit remain covered by the original tests.
## Checks
- `python3 -m unittest discover -s tests -p test_autosync.py -v`: 33 passed.
- `python3 -m unittest discover -s tests -p test_shot_copy.py -v`: 2 passed.
- `python3 -m unittest discover -s tests -p test_clips.py -v`: 34 passed,
including local ffmpeg coverage.
- `python3 -m unittest discover -s tests -p test_service_regressions.py -v`:
36 passed.
- `python3 -m unittest discover -s tests -p test_page_scripts.py -v`: 5 passed.
- `node --test site/test/*.test.mjs`: 2 test files passed.
- `python3 -m py_compile ui/frame_autosync.py ui/server.py tests/test_autosync.py`
and `git diff --check`: passed.
- `python3 -m unittest discover -s tests -v`: 1,525 tests run in 184.876
seconds, passed with 16 skips. Ran with sandbox escalation for loopback
sockets; test isolation still used throwaway app/SSH data and mocked headset
transport. Final output:
```text
Ran 1525 tests in 184.876s
OK (skipped=16)
```
## Limits and handoff
All new regressions use fake sources and mocked transport; they do not prove
real-headset transfer or UI rendering. No headset was accessed: its shared test
lock was held by another task. The Docker fake-Frame end-to-end suite and
Windows/macOS runs were not performed locally. The coordinator owns review and
merge; this task does not release, change PR #113, or publish announcements.
-8
View File
@@ -34,19 +34,11 @@ The manifest ID must match the folder name:
"id": "your-id",
"name": "Your feature",
"version": "1.0.0",
"apiVersion": 1,
"description": "What the panel does.",
"entry": "index.html"
}
```
`apiVersion` selects the Frame Control bridge contract, independently of your
extension's own `version`. This app supports integer `1`; a missing field means
`1` for existing extensions. Unsupported versions appear as errors on Refresh,
and their assets are not served. Declare it in new extensions so a future API
change can fail clearly instead of running incompatible code. The runtime
`capabilities` reply below reports the bridge version too.
Put JavaScript in a separate `.js` file. Supported assets are HTML, JS, CSS,
JSON, PNG, JPEG, GIF, WebP and SVG, all in the extension's top-level folder.
Nested paths and symlinks are not served. No file may exceed 5 MiB. The panel
+1 -10
View File
@@ -137,19 +137,10 @@ case was tested with separate fake sources, not two real same-name screenshots.
- **When this computer is full** (under 512 MB free, or a copy runs out of
space) it pauses and says so in Settings and above the lists. Free some space
and press **Resume**.
- **When one file keeps failing** it tries at most three times, then leaves
the problem visible in Settings. Fix the source problem and press **Resume**
to allow another three attempts without restarting Frame Control. **Copy
everything now** also resets failed attempts and includes older files.
Resume keeps the history of successful copies, so it does not recopy them.
- **With several headsets** each check and copy belongs to the headset selected
when the check began. Switching between lists or copies stops that pass;
another headset's saved files cannot become part of its copy history.
- **With several headsets** it copies from the one you're connected to.
Settings shows when it last checked, how many files it last copied and when,
and the last problem it had. The screenshot and clip lists say when it's on.
Retry and switch handling have offline regression coverage with fake sources;
see the [test record](evidence/autosync-reliability-2026-10-25.md).
## For developers
-15
View File
@@ -12,21 +12,6 @@ A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/ste
## Unit tests
Automatic screenshot and clip copying has offline coverage in
`tests/test_autosync.py`:
```sh
python3 -m unittest discover -s tests -p test_autosync.py -v
```
Fake sources exercise the three-attempt limit, visible exhausted errors,
Resume/Copy everything now recovery, successful-copy deduplication, disk-full
pauses and headset changes around lists and transfers. Server tests use the
production `working(meant)` reservation and listing/copy adapters; a Node test
executes the Settings renderer to check that Resume is available after an error.
These tests use no headset. See the
[auto-sync reliability test record](evidence/autosync-reliability-2026-10-25.md).
The gamescope keyboard/trackpad route has focused suites in `test_input.py`,
`test_touch.py` and `test_service_regressions.py`. They cover pad translation,
ASCII/modifier keys, Unicode queuing/retries, stale panel targets, release
@@ -2,7 +2,6 @@
"id": "headset-glance",
"name": "Headset Glance",
"version": "1.0.0",
"apiVersion": 1,
"description": "Shows a small read-only summary of the connected Steam Frame.",
"entry": "index.html"
}
+2 -227
View File
@@ -7,13 +7,11 @@ import sandbox # noqa: F401 (first: keeps tests off real data and services)
import errno
import json
import shutil
import subprocess
import sys
import tempfile
import threading
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "ui"))
import frame_autosync as fa # noqa: E402
@@ -37,15 +35,13 @@ class FakeFrame:
self.device, self.busy, self.down = "frame-a", False, False
self.save_error = self.export_error = None
def list_shots(self, device):
assert device == self.device
def list_shots(self):
self.calls.append("list_shots")
if self.down:
raise RuntimeError("ssh: connect to host frame port 22: Connection timed out")
return {"shots": list(self.shots)}
def list_clips(self, device):
assert device == self.device
def list_clips(self):
self.calls.append("list_clips")
if self.down:
raise RuntimeError("ssh: connect to host frame port 22: Connection timed out")
@@ -267,134 +263,6 @@ class AutoSyncTest(unittest.TestCase):
for _ in range(fa.TRIES + 2):
self.sync.tick()
self.assertEqual(self.frame.calls.count("export_clip"), fa.TRIES)
self.assertIn("ffmpeg couldn't join", self.sync.state()["error"] or "")
self.assertIn("Resume", self.sync.state()["error"] or "")
def test_explicit_retry_recovers_without_recopying_successful_items(self):
for kind in ("shots", "clips"):
for action in ("resume", "copy_all"):
with self.subTest(kind=kind, action=action):
self.frame = FakeFrame()
f = self.frame
sync = self.make()
sync.update({"screenshots": True, "clips": True})
f.shots, f.clips = [shot(1, NOW + 5)], [clip(1, NOW + 5)]
error_attr = "save_error" if kind == "shots" else "export_error"
copy_call = "save_shots" if kind == "shots" else "export_clip"
setattr(f, error_attr, RuntimeError("source not ready"))
for _ in range(fa.TRIES + 2):
sync.tick()
self.assertEqual(f.calls.count(copy_call), fa.TRIES)
self.assertIn("source not ready", sync.state()["error"] or "")
setattr(f, error_attr, None)
sync.tick() # recovery alone must not create unlimited automatic retries
self.assertEqual(f.calls.count(copy_call), fa.TRIES)
getattr(sync, action)()
sync.tick()
self.assertEqual(f.saved, [("frame-a", [shot(1, 0)["id"]])])
self.assertEqual(f.exported, [("frame-a", clip(1, 0)["id"])])
self.assertIsNone(sync.state()["error"])
# Resume preserves the per-headset ledger even if a local file was deleted.
for item in f.shots + f.clips:
item["saved"] = False
sync.resume()
sync.tick()
self.assertEqual((len(f.saved), len(f.exported)), (1, 1))
self.sync.path.unlink(missing_ok=True)
def test_explicit_retry_is_still_bounded(self):
self.sync.update({"clips": True})
self.frame.clips = [clip(1, NOW + 5)]
self.frame.export_error = RuntimeError("bad source")
for action in ("resume", "copy_all"):
for _ in range(fa.TRIES + 2):
self.sync.tick()
before = self.frame.calls.count("export_clip")
getattr(self.sync, action)()
for _ in range(fa.TRIES + 2):
self.sync.tick()
self.assertEqual(self.frame.calls.count("export_clip") - before, fa.TRIES)
self.assertIn("bad source", self.sync.state()["error"] or "")
def test_failed_item_saved_by_hand_clears_error(self):
self.sync.update({"clips": True})
self.frame.clips = [clip(1, NOW + 5)]
self.frame.export_error = RuntimeError("bad source")
for _ in range(fa.TRIES + 1):
self.sync.tick()
self.frame.clips[0]["saved"] = True
self.sync.tick()
self.assertIsNone(self.sync.state()["error"])
self.assertIn(clip(1, 0)["id"], self.sync._synced("frame-a", "clips"))
def test_switch_after_capture_does_not_list_the_new_headset(self):
self.sync.update({"screenshots": True, "clips": True})
def switched():
self.frame.device = "frame-b"
return False
self.sync.busy = switched # tick has already captured frame-a
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual(self.frame.calls, [])
def test_list_callbacks_receive_the_captured_device(self):
self.sync.update({"screenshots": True, "clips": True})
self.sync.list_shots = mock.Mock(return_value={"shots": []})
self.sync.list_clips = mock.Mock(return_value={"clips": []})
self.sync.tick()
self.sync.list_shots.assert_called_once_with("frame-a")
self.sync.list_clips.assert_called_once_with("frame-a")
def test_switch_during_listing_does_not_mark_or_copy_its_results(self):
for kind, item in (("shots", shot(1, NOW + 5)), ("clips", clip(1, NOW + 5))):
with self.subTest(kind=kind):
self.frame.device = "frame-a"
self.sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
def listing(*_args):
self.frame.device = "frame-b"
return {kind: [dict(item, saved=True), dict(item, id="uncopied", saved=False)]}
setattr(self.sync, "list_" + kind, listing)
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual(self.sync._synced("frame-a", kind), set())
self.assertEqual(self.sync._synced("frame-b", kind), set())
self.assertEqual((self.frame.saved, self.frame.exported), ([], []))
def test_switch_between_kinds_does_not_read_clips_or_mark_transfer(self):
self.sync.update({"screenshots": True, "clips": True})
self.frame.shots = [shot(1, NOW + 5)]
self.frame.clips = [clip(1, NOW + 5, saved=True)]
save = self.sync.save_shots
def then_switch(device, ids):
save(device, ids)
self.frame.device = "frame-b"
self.sync.save_shots = then_switch
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertNotIn("list_clips", self.frame.calls)
self.assertEqual(self.sync._synced("frame-a", "clips"), set())
self.assertEqual(self.sync._synced("frame-a", "shots"), set())
def test_switch_after_bookkeeping_stops_before_next_kind(self):
self.sync.update({"screenshots": True, "clips": True})
self.frame.shots = [shot(1, NOW + 5, saved=True)]
mark = self.sync._mark
def then_switch(device, kind, ids):
mark(device, kind, ids)
self.frame.device = "frame-b"
self.sync._mark = then_switch
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertNotIn("list_clips", self.frame.calls)
def test_switch_before_transfer_does_not_copy(self):
for kind, item in (("shots", shot(1, NOW + 5)), ("clips", clip(1, NOW + 5))):
with self.subTest(kind=kind):
self.frame.device = "frame-a"
self.sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
setattr(self.frame, kind, [item])
def switched_space(_folder):
self.frame.device = "frame-b"
return 100 * GB
self.sync.space = switched_space
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual((self.frame.saved, self.frame.exported), ([], []))
def test_worker_stops(self):
stop = threading.Event()
@@ -407,75 +275,6 @@ class AutoSyncTest(unittest.TestCase):
class Server(unittest.TestCase):
def test_switch_at_list_callback_entry_does_not_read_or_mark_the_new_headset(self):
import server
with tempfile.TemporaryDirectory() as folder:
for kind in ("shots", "clips"):
with self.subTest(kind=kind):
active = {"id": "frame-a"}
link = mock.Mock()
link.active_device.side_effect = lambda: dict(active)
listing = mock.Mock(return_value={kind: [
shot(1, NOW + 5, saved=True) if kind == "shots" else clip(1, NOW + 5, saved=True)]})
def switch_then_list(device):
active["id"] = "frame-b"
return server._sync_list(listing)(device)
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]):
sync = fa.AutoSync(switch_then_list, mock.Mock(), switch_then_list, mock.Mock(),
lambda: active["id"], lambda: False,
path=Path(folder) / (kind + ".json"), clock=lambda: NOW)
sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
sync.tick()
listing.assert_not_called()
sync.save_shots.assert_not_called()
sync.export_clip.assert_not_called()
self.assertEqual(sync._synced("frame-a", kind), set())
self.assertEqual(sync._synced("frame-b", kind), set())
self.assertIn("switched headsets", sync.state()["error"])
self.assertEqual(server._work[0], 0)
def test_listing_rejects_switch_before_reserving_work(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-b"}
listing = mock.Mock(return_value={"shots": [shot(1, NOW, saved=True)]})
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]):
with self.assertRaisesRegex(server.Failure, "switched headsets"):
server._sync_list(listing)("frame-a")
listing.assert_not_called()
self.assertEqual(server._work[0], 0)
def test_listing_reserves_work_on_the_captured_device(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-a", "name": "Frame A"}
def listing():
self.assertGreater(server._work[0], 0)
with self.assertRaises(server.Failure):
server.devices_post({"action": "use", "id": "frame-b"})
return {"shots": []}
with mock.patch.object(server, "LINK", link), \
mock.patch.object(server, "PRIVATE", False), mock.patch.object(server, "_work", [0]):
self.assertEqual(server._sync_list(listing)("frame-a"), {"shots": []})
self.assertEqual(server._work[0], 0)
def test_copy_rejects_switch_after_listing(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-a"}
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]), \
mock.patch.object(server, "save_shots") as save, \
mock.patch.object(server.frame_clips, "export_clip") as export:
server._sync_list(lambda: {"shots": []})("frame-a")
link.active_device.return_value = {"id": "frame-b"}
for copy in (lambda: server._sync_shots("frame-a", [shot(1, NOW)["id"]]),
lambda: server._sync_clip("frame-a", clip(1, NOW)["id"])):
with self.assertRaisesRegex(server.Failure, "switched headsets"):
copy()
save.assert_not_called()
export.assert_not_called()
self.assertEqual(server._work[0], 0)
def test_route_and_settings(self):
import server
self.assertIn("/api/autosync", server.POST)
@@ -488,29 +287,5 @@ class Server(unittest.TestCase):
server.auto_sync({"action": "now"}) # nothing connected
@unittest.skipUnless(shutil.which("node"), "Node exercises the Settings renderer")
class Settings(unittest.TestCase):
def test_resume_is_available_after_a_copy_error(self):
page = (Path(__file__).resolve().parents[1] / "ui/index.html").read_text(encoding="utf-8")
script = page.split("const autoSync =", 1)[1].split("async function loadAutoSync", 1)[0]
harness = r'''
const assert = require('node:assert/strict');
const nodes = {};
const $ = id => nodes[id] ||= {};
const HOST = {mobile: false};
const esc = value => value;
'''
checks = r'''
for (const state of [{error: 'Could not copy clip'}, {paused: 'Disk full'}, {}]) {
renderAutoSync(state);
assert.equal($('syncResume').hidden, !(state.error || state.paused));
if (state.error) assert.match($('syncStatus').textContent, /Could not copy clip/);
}
'''
result = subprocess.run(["node", "-e", harness + "const autoSync =" + script + checks],
capture_output=True, text=True, timeout=10)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
if __name__ == "__main__":
unittest.main()
-102
View File
@@ -113,108 +113,6 @@ class Remote(unittest.TestCase):
os.symlink(str(outside), str(base / 'linked-file'))
self.assertFalse([f for f in self.files('saves:1234') if 'linked' in f])
def packed(self, part):
out = io.BytesIO()
remote.pack(part, out)
out.seek(0)
with tarfile.open(fileobj=out, mode='r:gz') as archive:
return {m.name: archive.extractfile(m).read() for m in archive}
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_backup_rejects_every_ancestor_of_a_save_root(self):
save = USER % 1234 + '/AppData/Roaming/Game/fixture.dat'
# Each component below HOME, including the selected root itself.
components = (USER % 1234 + '/AppData/Roaming').split('/')
for end in range(1, len(components) + 1):
ancestor = '/'.join(components[:end])
with self.subTest(ancestor=ancestor), tempfile.TemporaryDirectory() as tmp:
home = Path(tmp) / 'home'
manifest(home, 1234, 'Synthetic game')
path = write(home, save, b'synthetic fixture')
linked = home / ancestor
outside = Path(tmp) / 'outside/.ssh'
outside.parent.mkdir()
linked.rename(outside)
linked.symlink_to(outside, target_is_directory=True)
# Only synthetic bytes: the lexical archive path hides .ssh.
self.assertEqual(path.read_bytes(), b'synthetic fixture')
with patch.object(remote, 'HOME', home):
self.assertEqual(self.files('saves:1234'), [])
self.assertNotIn('saves:1234', [p['id'] for p in remote.plan()['parts']])
self.assertEqual(self.packed('saves:1234'), {})
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_pack_rechecks_links_changed_after_discovery(self):
save = USER % 1234 + '/Documents/Game/fixture.dat'
for ancestor in (remote.COMPAT + '/1234', USER % 1234 + '/Documents', save):
with self.subTest(ancestor=ancestor), tempfile.TemporaryDirectory() as tmp:
home = Path(tmp) / 'home'
write(home, save, b'original save')
linked = home / ancestor
outside = Path(tmp) / 'outside/.ssh'
outside.parent.mkdir()
part_files = remote.part_files
def discover_then_link(part):
result = part_files(part)
self.assertEqual(result[0], [(save, len(b'original save'))])
linked.rename(outside)
linked.symlink_to(outside, target_is_directory=outside.is_dir())
(home / save).write_bytes(b'synthetic redirected bytes')
return result
with patch.object(remote, 'HOME', home), \
patch.object(remote, 'part_files', side_effect=discover_then_link):
self.assertEqual(self.packed('saves:1234'), {})
def test_legitimate_steam_layout_and_reference_files_still_pack(self):
frame_home(self.home)
manifest(self.home, 2800000002, 'Synthetic non-Steam game')
save = USER % 2800000002 + '/Saved Games/fixture.dat'
write(self.home, save, b'non-Steam save')
controller = STEAM + '/steamapps/common/Steam Controller Configs/555/config/layout.vdf'
write(self.home, controller, b'controller layout')
for part in ('saves:1234', 'saves:2800000002', 'saves:linux',
'settings:steam', 'settings:system'):
with self.subTest(part=part):
expected = {rel: (self.home / rel).read_bytes() for rel in self.files(part)}
self.assertTrue(expected)
self.assertEqual(self.packed(part), expected)
self.assertEqual(self.packed('saves:2800000002'), {save: b'non-Steam save'})
self.assertIn(controller, self.packed('settings:steam'))
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_settings_discovery_and_manifest_reads_reject_links(self):
frame_home(self.home)
for ancestor, part in (('.config', 'settings:system'),
(STEAM + '/userdata', 'settings:steam')):
with self.subTest(ancestor=ancestor):
linked = self.home / ancestor
moved = linked.with_name(linked.name + '-original')
linked.rename(moved)
linked.symlink_to(moved, target_is_directory=True)
self.assertEqual(self.files(part), [])
self.assertEqual(self.packed(part), {})
if part == 'settings:steam':
self.assertEqual(remote.accounts(), [])
appmanifest = self.home / STEAM / 'steamapps/appmanifest_1234.acf'
moved = appmanifest.with_suffix('.original')
appmanifest.rename(moved)
appmanifest.symlink_to(moved)
with patch.object(Path, 'read_text', side_effect=AssertionError('must not read a link')):
self.assertIsNone(remote.game_name('1234'))
@unittest.skipIf(os.name == 'nt', 'symlinks need privileges on Windows')
def test_home_alias_and_unused_steam_alias_preserve_normal_backups(self):
frame_home(self.home)
alias = self.home / 'home-alias'
alias.symlink_to(self.home, target_is_directory=True)
(self.home / '.steam/steam').symlink_to(self.home / STEAM, target_is_directory=True)
expected = self.packed('saves:1234')
with patch.object(remote, 'HOME', alias):
self.assertEqual(self.packed('saves:1234'), expected)
self.assertEqual(remote.game_name('1234'), 'Test Game')
def test_big_files_are_listed_as_skipped(self):
frame_home(self.home)
with patch.object(remote, 'MAX_FILE', 3):
-107
View File
@@ -1,107 +0,0 @@
"""Extension compatibility and filesystem boundaries, without running plugin code."""
import sandbox # noqa: F401
import json
import shutil
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
import frame_extensions
class ExtensionFiles(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name) / 'extensions'
self.root.mkdir()
self.patch = patch.object(frame_extensions, 'root', return_value=self.root)
self.patch.start()
self.addCleanup(self.patch.stop)
def extension(self, name='test-panel', **fields):
folder = self.root / name
folder.mkdir(exist_ok=True)
manifest = {'id': name, 'name': 'Test panel', 'version': '1.0.0',
'description': 'A read-only panel', 'entry': 'index.html', **fields}
(folder / 'manifest.json').write_text(json.dumps(manifest), encoding='utf-8')
(folder / 'index.html').write_text('<script src="main.js"></script>', encoding='utf-8')
(folder / 'main.js').write_text('/* extension code */', encoding='utf-8')
return folder
def test_existing_and_explicit_v1_manifests_are_compatible(self):
self.extension('legacy')
self.extension('explicit', apiVersion=1)
result = frame_extensions.list_extensions()
self.assertEqual(result['errors'], [])
self.assertEqual([(x['id'], x['apiVersion']) for x in result['extensions']],
[('explicit', 1), ('legacy', 1)])
self.assertEqual(frame_extensions.asset('explicit', 'main.js'),
(b'/* extension code */', 'text/javascript; charset=utf-8'))
def test_unsupported_or_malformed_api_cannot_be_loaded_directly(self):
for version in (2, 0, -1, '1', None, True, 1.0, [], {}):
with self.subTest(apiVersion=version):
self.extension(apiVersion=version)
result = frame_extensions.list_extensions()
self.assertEqual(result['extensions'], [])
self.assertIn('unsupported apiVersion', result['errors'][0]['error'])
with self.assertRaisesRegex(ValueError, 'unsupported apiVersion'):
frame_extensions.asset('test-panel', 'index.html')
def test_one_incompatible_extension_does_not_hide_compatible_panels(self):
self.extension('future', apiVersion=2)
self.extension('working', apiVersion=1)
result = frame_extensions.list_extensions()
self.assertEqual([x['id'] for x in result['extensions']], ['working'])
self.assertEqual(result['errors'][0]['folder'], 'future')
def test_sample_is_discoverable_and_its_assets_are_served(self):
source = Path(__file__).resolve().parents[1] / 'examples/extensions/headset-glance'
shutil.copytree(source, self.root / source.name)
result = frame_extensions.list_extensions()
self.assertEqual(result['errors'], [])
self.assertEqual(result['extensions'][0]['id'], source.name)
for asset in ('index.html', 'main.js', 'style.css'):
self.assertEqual(frame_extensions.asset(source.name, asset)[0],
(source / asset).read_bytes())
def test_private_nested_and_traversal_paths_are_rejected(self):
self.extension()
for name in ('manifest.json', '../outside.js', 'sub/main.js', '.hidden.js'):
with self.subTest(name=name), self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', name)
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('../test-panel', 'main.js')
def test_oversized_assets_are_not_read(self):
folder = self.extension()
with (folder / 'large.js').open('wb') as output:
output.truncate(5 * 1024 * 1024 + 1)
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', 'large.js')
def test_symlinks_cannot_expose_files_outside_a_panel(self):
folder = self.extension()
outside = Path(self.tmp.name) / 'outside.js'
outside.write_text('private file', encoding='utf-8')
try:
(folder / 'escape.js').symlink_to(outside)
(self.root / 'alias').symlink_to(folder, target_is_directory=True)
except OSError:
self.skipTest('symlinks unavailable for this user')
with self.assertRaises(FileNotFoundError):
frame_extensions.asset('test-panel', 'escape.js')
with self.assertRaises(ValueError):
frame_extensions.asset('alias', 'main.js')
(folder / 'manifest.json').unlink()
(folder / 'manifest.json').symlink_to(outside)
with self.assertRaises(ValueError):
frame_extensions.asset('test-panel', 'main.js')
if __name__ == '__main__':
unittest.main()
-18
View File
@@ -126,24 +126,6 @@ class ServerGuards(unittest.TestCase):
self.assertEqual(code, 400)
self.assertIn('experimental', body['error'])
def test_incompatible_extension_is_reported_and_assets_are_unavailable(self):
folder = Path(os.environ['FRAME_CONTROL_DATA_DIR']) / 'extensions' / 'future-panel'
folder.mkdir(parents=True, exist_ok=True)
(folder / 'manifest.json').write_text(json.dumps({'id': 'future-panel', 'name': 'Future',
'version': '1.0', 'description': 'Requires a newer API', 'apiVersion': 2}))
(folder / 'index.html').write_text('<p>Incompatible panel</p>')
try:
self.assertEqual(self.request('GET', '/api/extensions')[0], 403)
code, _, body = self.request('GET', '/api/extensions', headers={'X-Frame-UI': '1'})
self.assertEqual(code, 200)
data = json.loads(body)
self.assertNotIn('future-panel', [x['id'] for x in data['extensions']])
self.assertTrue(any(x['folder'] == 'future-panel' and 'apiVersion' in x['error']
for x in data['errors']))
self.assertEqual(self.request('GET', '/extension/future-panel/index.html')[0], 404)
finally:
shutil.rmtree(folder)
def test_headset_page_and_install_acknowledgement(self):
code, _, body = self.request('GET', '/headset')
self.assertEqual(code, 200)
+12 -56
View File
@@ -16,9 +16,9 @@ saved Steam clips to ~/Videos/SteamFrame.
- When the headset can't be reached it waits longer each time, up to MAX_BACKOFF.
The server supplies how to reach the headset, so this can be tested without one:
list_shots(device) -> {"shots": [{"id", "time", "saved"}...]}
list_shots() -> {"shots": [{"id", "time", "saved"}...]}
save_shots(device, ids) copies those screenshots here
list_clips(device) -> {"clips": [{"id", "kind", "time", "idle", "saved"}...]}
list_clips() -> {"clips": [{"id", "kind", "time", "idle", "saved"}...]}
export_clip(device, id) exports one clip here
device() -> the connected headset's id, or None
busy() -> True while other work runs on the headset (installs, jobs, a recording)
@@ -41,7 +41,7 @@ SETTLE = 30 # a clip written to within this many seconds is still being recorde
MIN_FREE = 512 * 1024**2 # pause below this much free space on this computer
KEEP = 5000 # copied ids remembered per headset and kind
BATCH = 100 # screenshots per copy
TRIES = 3 # failed attempts per item until Resume, Copy everything now, or restart
TRIES = 3 # an item that fails this often is left until the app restarts
KINDS = ("shots", "clips")
@@ -86,7 +86,6 @@ class AutoSync:
self._running = False
self._backoff = 0
self._tries = {} # (device, id) -> failed attempts this run
self._failures = {} # last error for each failed item, including exhausted retries
self._errors = [] # this pass's, for the page
self._s = self._load()
@@ -144,19 +143,13 @@ class AutoSync:
"""Copy everything on the headset that isn't on this computer, once (also resumes)."""
with self._lock:
self._all, self._s["paused"] = True, None
self._tries.clear()
self._failures.clear()
self._s["error"] = None
self._save()
self._backoff = 0
self._wake.set()
return self.state()
def resume(self):
with self._lock:
self._s["paused"], self._s["error"] = None, None
self._tries.clear()
self._failures.clear()
self._save()
self._backoff = 0
self._wake.set()
@@ -222,9 +215,6 @@ class AutoSync:
def _mark(self, device, kind, ids):
with self._lock:
for item in ids:
self._tries.pop((device, item), None)
self._failures.pop((device, item), None)
per = self._s["synced"].setdefault(device, {})
have = per.get(kind, [])
new = [i for i in ids if i not in set(have)]
@@ -243,46 +233,24 @@ class AutoSync:
"then press Resume.")
if self.offline(error):
raise error
with self._lock:
key = (device, item)
self._tries[key] = self._tries.get(key, 0) + 1
message = f"Couldn't copy {item.rsplit('/', 1)[-1]}: {error}"
if self._tries[key] >= TRIES:
message += ". Automatic retries stopped. Press Resume to try again."
self._failures[key] = message
self._errors.append(message)
def _retry_allowed(self, device, item):
with self._lock:
key = (device, item)
if self._tries.get(key, 0) < TRIES:
return True
self._errors.append(self._failures[key])
return False
def _interrupted(self, device):
return self.busy() or self.device() != device
key = (device, item)
self._tries[key] = self._tries.get(key, 0) + 1
self._errors.append(f"Couldn't copy {item.rsplit('/', 1)[-1]}: {error}")
def _pass(self, device, shots, clips, since):
"""-> (copied, clips still being written, finished without stopping for other work)."""
copied = waiting = 0
if shots:
if self._interrupted(device):
return copied, waiting, False
listing = self.list_shots(device)["shots"]
if self._interrupted(device):
return copied, waiting, False
listing = self.list_shots()["shots"]
done = self._synced(device, "shots")
self._mark(device, "shots", [s["id"] for s in listing if s.get("saved")])
todo = [s["id"] for s in sorted(listing, key=lambda s: s["time"])
if not s.get("saved") and (since is None or (s["time"] >= since and s["id"] not in done))
and self._retry_allowed(device, s["id"])]
and self._tries.get((device, s["id"]), 0) < TRIES]
for i in range(0, len(todo), BATCH):
if self._interrupted(device):
if self.busy():
return copied, waiting, False
self._room(self.shots_folder)
if self._interrupted(device):
return copied, waiting, False
batch = todo[i:i + BATCH]
try:
self.save_shots(device, batch)
@@ -290,40 +258,28 @@ class AutoSync:
for item in batch:
self._failed(device, item, e)
continue
if self.device() != device:
return copied, waiting, False
self._mark(device, "shots", batch)
copied += len(batch)
if clips:
if self._interrupted(device):
return copied, waiting, False
listing = [c for c in self.list_clips(device)["clips"] if c.get("kind") == "clip"]
if self._interrupted(device):
return copied, waiting, False
listing = [c for c in self.list_clips()["clips"] if c.get("kind") == "clip"]
done = self._synced(device, "clips")
self._mark(device, "clips", [c["id"] for c in listing if c.get("saved")])
for c in sorted(listing, key=lambda c: c["time"]):
if c.get("saved"):
if c.get("saved") or self._tries.get((device, c["id"]), 0) >= TRIES:
continue
if since is not None and (c["time"] < since or c["id"] in done):
continue
if not self._retry_allowed(device, c["id"]):
continue
if c.get("idle") is None or c["idle"] < SETTLE:
waiting += 1 # still being written: copied on a later look
continue
if self._interrupted(device):
if self.busy():
return copied, waiting, False
self._room(self.clips_folder)
if self._interrupted(device):
return copied, waiting, False
try:
self.export_clip(device, c["id"])
except Exception as e:
self._failed(device, c["id"], e)
continue
if self.device() != device:
return copied, waiting, False
self._mark(device, "clips", [c["id"]])
copied += 1
return copied, waiting, True
+17 -60
View File
@@ -19,7 +19,6 @@ import os
from pathlib import Path, PurePosixPath
import re
import shutil
import stat
import subprocess
import sys
import tarfile
@@ -89,57 +88,28 @@ def _under(rel, root):
return rel == root or rel.startswith(root + '/')
def _backup_stat(rel):
"""lstat a backup path only if every component below trusted HOME is link-free.
Rechecked at discovery and before reads; not atomic against concurrent renames.
HOME itself may use the host's normal home/temp-directory aliases.
"""
if not safe_rel(rel):
return None
path = HOME
try:
for component in rel.split('/'):
path = path / component
st = path.lstat()
if stat.S_ISLNK(st.st_mode):
return None
return st
except OSError:
return None
def _backup_dir(rel):
st = _backup_stat(rel)
return st is not None and stat.S_ISDIR(st.st_mode)
def _walk(rel_root, skip=()):
"""Regular files under HOME/rel_root, never following links: [(rel, size)], [skipped]."""
files, skipped = [], []
base = HOME / rel_root
st = _backup_stat(rel_root)
if st is None or is_secret(rel_root):
return files, skipped
if stat.S_ISREG(st.st_mode):
if st.st_size > MAX_FILE:
skipped.append({'path': rel_root, 'reason': 'over 2 GiB'})
return files, skipped
return [(rel_root, st.st_size)], skipped
if not stat.S_ISDIR(st.st_mode):
if base.is_symlink() or not base.is_dir():
if base.is_file() and not base.is_symlink():
return [(rel_root, base.stat().st_size)], skipped
return files, skipped
for top, dirs, names in os.walk(str(base)):
rel_top = PurePosixPath(rel_root, *Path(top).relative_to(base).parts).as_posix()
if not _backup_dir(rel_top):
dirs[:] = []
continue
dirs[:] = sorted(d for d in dirs if _backup_dir(rel_top + '/' + d) and
dirs[:] = sorted(d for d in dirs if not os.path.islink(os.path.join(top, d)) and
not any(_under(rel_top + '/' + d, s) for s in skip) and
not is_secret(rel_top + '/' + d + '/x'))
for name in sorted(names):
rel = rel_top + '/' + name
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode) or is_secret(rel):
path, rel = os.path.join(top, name), rel_top + '/' + name
if os.path.islink(path) or is_secret(rel):
continue
try:
st = os.lstat(path)
except OSError:
continue
if not os.path.isfile(path):
continue
if st.st_size > MAX_FILE:
skipped.append({'path': rel, 'reason': 'over 2 GiB'})
@@ -154,34 +124,26 @@ def _manifest_field(text, key):
def game_name(appid):
rel = '%s/steamapps/appmanifest_%s.acf' % (STEAM, appid)
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode):
return None
try:
text = (HOME / rel).read_text(errors='replace')
text = (HOME / STEAM / 'steamapps' / ('appmanifest_%s.acf' % appid)).read_text(errors='replace')
except OSError:
return None
return _manifest_field(text, 'name')
def accounts():
rel = STEAM + '/userdata'
if not _backup_dir(rel):
return []
root = HOME / STEAM / 'userdata'
try:
return sorted(p.name for p in (HOME / rel).iterdir() if p.name.isdigit() and
p.name != '0' and _backup_dir(rel + '/' + p.name))
return sorted(p.name for p in root.iterdir() if p.name.isdigit() and p.is_dir()
and not p.is_symlink() and p.name != '0')
except OSError:
return []
def _prefixes():
if not _backup_dir(COMPAT):
return []
try:
return sorted((p.name for p in (HOME / COMPAT).iterdir()
if p.name.isdigit() and _backup_dir(COMPAT + '/' + p.name + '/' + PREFIX_USER)), key=int)
if p.name.isdigit() and (p / PREFIX_USER).is_dir()), key=int)
except OSError:
return []
@@ -282,14 +244,9 @@ def pack(part, output):
with tarfile.open(fileobj=output, mode='w|gz') as archive:
for rel, _ in files:
path = HOME / rel
st = _backup_stat(rel)
if st is None or not stat.S_ISREG(st.st_mode) or is_secret(rel):
continue
try:
with open(str(path), 'rb') as src:
st = os.fstat(src.fileno())
if not stat.S_ISREG(st.st_mode) or st.st_size > MAX_FILE:
continue
member = tarfile.TarInfo(rel)
member.size, member.mtime, member.mode = st.st_size, int(st.st_mtime), st.st_mode & 0o755
archive.addfile(member, src)
+1 -8
View File
@@ -7,7 +7,6 @@ import frame_host
ID = re.compile(r"[a-z][a-z0-9-]{0,39}\Z")
API_VERSION = 1
ASSET = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]*\Z")
TYPES = {".html": "text/html; charset=utf-8", ".js": "text/javascript; charset=utf-8",
".css": "text/css; charset=utf-8", ".json": "application/json",
@@ -28,11 +27,6 @@ def _manifest(folder):
data = json.loads(manifest.read_text(encoding="utf-8"))
if not isinstance(data, dict) or data.get("id") != folder.name:
raise ValueError("manifest id must match its folder")
# Older manifests predate this field and use bridge v1. Check again when
# serving assets, so an incompatible panel cannot bypass discovery.
api_version = data.get("apiVersion", 1)
if type(api_version) is not int or api_version != API_VERSION:
raise ValueError(f"unsupported apiVersion; this app supports {API_VERSION}")
for key, limit in (("name", 80), ("version", 40), ("description", 240)):
value = data.get(key)
if not isinstance(value, str) or not value.strip() or len(value) > limit:
@@ -40,8 +34,7 @@ def _manifest(folder):
entry = data.get("entry", "index.html")
if entry != "index.html" or (folder / entry).is_symlink() or not (folder / entry).is_file():
raise ValueError("entry must be a regular index.html")
return {**{key: data[key] for key in ("id", "name", "version", "description")},
"apiVersion": api_version}
return {key: data[key] for key in ("id", "name", "version", "description")}
def list_extensions():
+1 -1
View File
@@ -5610,7 +5610,7 @@ const syncWhen = t => new Date(t * 1000).toLocaleString([], { dateStyle: "medium
function renderAutoSync(s) {
autoSync.s = s;
$("syncShots").checked = s.screenshots; $("syncClips").checked = s.clips;
$("syncResume").hidden = !(s.paused || s.error);
$("syncResume").hidden = !s.paused;
const on = s.screenshots || s.clips, lines = [];
if (s.paused) lines.push(s.paused);
else if (s.error) lines.push(`Last problem: ${s.error}`);
+2 -2
View File
@@ -969,8 +969,8 @@ def _sync_clip(device, clip_id):
def _sync_list(listing):
def run(device):
with working(device):
def run():
with working(_sync_device()):
return listing()
return run