Compare commits

...
8 changed files with 395 additions and 18 deletions

No files matched your search

+5
View File
@@ -219,3 +219,8 @@
shared Wine services are left alone. Verified with local Linux process
regressions and a 13/13 real Frame smoke run that stopped five test-owned
Proton process PIDs before removing the Windows test title.
- Keep failed automatic screenshot and clip copies visible after three attempts.
**Resume** and **Copy everything now** allow another try without restarting;
headset switches cannot use another headset's list or saved-file history.
[Copying guide](docs/recording.md).
@@ -0,0 +1,79 @@
# Auto-sync retry and headset-selection reliability
Date: 25 October 2026. Base: `d79efb4`, Frame Control v0.4.30.
Issue: [#214](https://github.com/saphid/frame-control/issues/214).
Environment: Linux x86_64, Python 3.12.3, Node 24.15.0.
## Reproduced problems
The existing 19 auto-sync tests passed on the base revision. Added regressions
failed before the implementation: an exhausted clip lost its error, explicit
retry did not recover failed screenshots or clips, and switches around listing
or copying continued the old pass. The original callback interface also had no
way to carry the tick's captured device into the server list reservation.
The final fake-source tests exercise these boundaries:
- Fail screenshots or clips three times, keep checking without further copy
attempts, and retain the source error plus an explicit retry instruction.
- Recover the source and use either Resume or Copy everything now without
restarting. Successful copies stay deduplicated; Resume retains the ledger
even after a local file is deleted. Copy everything now retains its documented
ability to restore deleted files and include older media.
- Reset an exhausted budget while the source still fails: another three
attempts are allowed, rather than unlimited background attempts.
- Save an exhausted item by hand and clear its stale failure on the next list.
- Switch after tick captures A, during a list, after screenshot bookkeeping,
between media kinds, before a transfer, or before transfer bookkeeping.
Interrupted passes do not mark the other headset's IDs or continue copying.
- Switch to B exactly at callback entry through the production `_sync_list`
adapter. `working(A)` refuses before calling the source list, leaving both
ledgers and copy callbacks untouched. The same production reservation blocks
a device switch during a list and rejects stale screenshot/clip transfers.
- Execute the production Settings renderer in Node: Resume is available for
copy errors and disk-full pauses, and hidden for a healthy state.
## Implementation boundary
Each list callback now receives the captured device, like the copy callbacks.
The existing server `working(meant)` mechanism checks that intended headset
under `_work_lock` while reserving work; `devices_post` uses the same lock to
refuse switches during that work. The worker also checks for interruption
around lists and transfers. No new device lock or clip destination logic is
introduced. The server diff is confined to `_sync_list`.
The failed-attempt and error maps are in-memory and reset on explicit retry or
restart. The successful-copy ledger and settings persistence retain their
existing format. Disk-full pauses, offline backoff, clip settling and the
three-attempt limit remain covered by the original tests.
## Checks
- `python3 -m unittest discover -s tests -p test_autosync.py -v`: 33 passed.
- `python3 -m unittest discover -s tests -p test_shot_copy.py -v`: 2 passed.
- `python3 -m unittest discover -s tests -p test_clips.py -v`: 34 passed,
including local ffmpeg coverage.
- `python3 -m unittest discover -s tests -p test_service_regressions.py -v`:
36 passed.
- `python3 -m unittest discover -s tests -p test_page_scripts.py -v`: 5 passed.
- `node --test site/test/*.test.mjs`: 2 test files passed.
- `python3 -m py_compile ui/frame_autosync.py ui/server.py tests/test_autosync.py`
and `git diff --check`: passed.
- `python3 -m unittest discover -s tests -v`: 1,525 tests run in 184.876
seconds, passed with 16 skips. Ran with sandbox escalation for loopback
sockets; test isolation still used throwaway app/SSH data and mocked headset
transport. Final output:
```text
Ran 1525 tests in 184.876s
OK (skipped=16)
```
## Limits and handoff
All new regressions use fake sources and mocked transport; they do not prove
real-headset transfer or UI rendering. No headset was accessed: its shared test
lock was held by another task. The Docker fake-Frame end-to-end suite and
Windows/macOS runs were not performed locally. The coordinator owns review and
merge; this task does not release, change PR #113, or publish announcements.
+10 -1
View File
@@ -137,10 +137,19 @@ case was tested with separate fake sources, not two real same-name screenshots.
- **When this computer is full** (under 512 MB free, or a copy runs out of
space) it pauses and says so in Settings and above the lists. Free some space
and press **Resume**.
- **With several headsets** it copies from the one you're connected to.
- **When one file keeps failing** it tries at most three times, then leaves
the problem visible in Settings. Fix the source problem and press **Resume**
to allow another three attempts without restarting Frame Control. **Copy
everything now** also resets failed attempts and includes older files.
Resume keeps the history of successful copies, so it does not recopy them.
- **With several headsets** each check and copy belongs to the headset selected
when the check began. Switching between lists or copies stops that pass;
another headset's saved files cannot become part of its copy history.
Settings shows when it last checked, how many files it last copied and when,
and the last problem it had. The screenshot and clip lists say when it's on.
Retry and switch handling have offline regression coverage with fake sources;
see the [test record](evidence/autosync-reliability-2026-10-25.md).
## For developers
+15
View File
@@ -12,6 +12,21 @@ A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/ste
## Unit tests
Automatic screenshot and clip copying has offline coverage in
`tests/test_autosync.py`:
```sh
python3 -m unittest discover -s tests -p test_autosync.py -v
```
Fake sources exercise the three-attempt limit, visible exhausted errors,
Resume/Copy everything now recovery, successful-copy deduplication, disk-full
pauses and headset changes around lists and transfers. Server tests use the
production `working(meant)` reservation and listing/copy adapters; a Node test
executes the Settings renderer to check that Resume is available after an error.
These tests use no headset. See the
[auto-sync reliability test record](evidence/autosync-reliability-2026-10-25.md).
The gamescope keyboard/trackpad route has focused suites in `test_input.py`,
`test_touch.py` and `test_service_regressions.py`. They cover pad translation,
ASCII/modifier keys, Unicode queuing/retries, stale panel targets, release
+227 -2
View File
@@ -7,11 +7,13 @@ import sandbox # noqa: F401 (first: keeps tests off real data and services)
import errno
import json
import shutil
import subprocess
import sys
import tempfile
import threading
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "ui"))
import frame_autosync as fa # noqa: E402
@@ -35,13 +37,15 @@ class FakeFrame:
self.device, self.busy, self.down = "frame-a", False, False
self.save_error = self.export_error = None
def list_shots(self):
def list_shots(self, device):
assert device == self.device
self.calls.append("list_shots")
if self.down:
raise RuntimeError("ssh: connect to host frame port 22: Connection timed out")
return {"shots": list(self.shots)}
def list_clips(self):
def list_clips(self, device):
assert device == self.device
self.calls.append("list_clips")
if self.down:
raise RuntimeError("ssh: connect to host frame port 22: Connection timed out")
@@ -263,6 +267,134 @@ class AutoSyncTest(unittest.TestCase):
for _ in range(fa.TRIES + 2):
self.sync.tick()
self.assertEqual(self.frame.calls.count("export_clip"), fa.TRIES)
self.assertIn("ffmpeg couldn't join", self.sync.state()["error"] or "")
self.assertIn("Resume", self.sync.state()["error"] or "")
def test_explicit_retry_recovers_without_recopying_successful_items(self):
for kind in ("shots", "clips"):
for action in ("resume", "copy_all"):
with self.subTest(kind=kind, action=action):
self.frame = FakeFrame()
f = self.frame
sync = self.make()
sync.update({"screenshots": True, "clips": True})
f.shots, f.clips = [shot(1, NOW + 5)], [clip(1, NOW + 5)]
error_attr = "save_error" if kind == "shots" else "export_error"
copy_call = "save_shots" if kind == "shots" else "export_clip"
setattr(f, error_attr, RuntimeError("source not ready"))
for _ in range(fa.TRIES + 2):
sync.tick()
self.assertEqual(f.calls.count(copy_call), fa.TRIES)
self.assertIn("source not ready", sync.state()["error"] or "")
setattr(f, error_attr, None)
sync.tick() # recovery alone must not create unlimited automatic retries
self.assertEqual(f.calls.count(copy_call), fa.TRIES)
getattr(sync, action)()
sync.tick()
self.assertEqual(f.saved, [("frame-a", [shot(1, 0)["id"]])])
self.assertEqual(f.exported, [("frame-a", clip(1, 0)["id"])])
self.assertIsNone(sync.state()["error"])
# Resume preserves the per-headset ledger even if a local file was deleted.
for item in f.shots + f.clips:
item["saved"] = False
sync.resume()
sync.tick()
self.assertEqual((len(f.saved), len(f.exported)), (1, 1))
self.sync.path.unlink(missing_ok=True)
def test_explicit_retry_is_still_bounded(self):
self.sync.update({"clips": True})
self.frame.clips = [clip(1, NOW + 5)]
self.frame.export_error = RuntimeError("bad source")
for action in ("resume", "copy_all"):
for _ in range(fa.TRIES + 2):
self.sync.tick()
before = self.frame.calls.count("export_clip")
getattr(self.sync, action)()
for _ in range(fa.TRIES + 2):
self.sync.tick()
self.assertEqual(self.frame.calls.count("export_clip") - before, fa.TRIES)
self.assertIn("bad source", self.sync.state()["error"] or "")
def test_failed_item_saved_by_hand_clears_error(self):
self.sync.update({"clips": True})
self.frame.clips = [clip(1, NOW + 5)]
self.frame.export_error = RuntimeError("bad source")
for _ in range(fa.TRIES + 1):
self.sync.tick()
self.frame.clips[0]["saved"] = True
self.sync.tick()
self.assertIsNone(self.sync.state()["error"])
self.assertIn(clip(1, 0)["id"], self.sync._synced("frame-a", "clips"))
def test_switch_after_capture_does_not_list_the_new_headset(self):
self.sync.update({"screenshots": True, "clips": True})
def switched():
self.frame.device = "frame-b"
return False
self.sync.busy = switched # tick has already captured frame-a
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual(self.frame.calls, [])
def test_list_callbacks_receive_the_captured_device(self):
self.sync.update({"screenshots": True, "clips": True})
self.sync.list_shots = mock.Mock(return_value={"shots": []})
self.sync.list_clips = mock.Mock(return_value={"clips": []})
self.sync.tick()
self.sync.list_shots.assert_called_once_with("frame-a")
self.sync.list_clips.assert_called_once_with("frame-a")
def test_switch_during_listing_does_not_mark_or_copy_its_results(self):
for kind, item in (("shots", shot(1, NOW + 5)), ("clips", clip(1, NOW + 5))):
with self.subTest(kind=kind):
self.frame.device = "frame-a"
self.sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
def listing(*_args):
self.frame.device = "frame-b"
return {kind: [dict(item, saved=True), dict(item, id="uncopied", saved=False)]}
setattr(self.sync, "list_" + kind, listing)
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual(self.sync._synced("frame-a", kind), set())
self.assertEqual(self.sync._synced("frame-b", kind), set())
self.assertEqual((self.frame.saved, self.frame.exported), ([], []))
def test_switch_between_kinds_does_not_read_clips_or_mark_transfer(self):
self.sync.update({"screenshots": True, "clips": True})
self.frame.shots = [shot(1, NOW + 5)]
self.frame.clips = [clip(1, NOW + 5, saved=True)]
save = self.sync.save_shots
def then_switch(device, ids):
save(device, ids)
self.frame.device = "frame-b"
self.sync.save_shots = then_switch
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertNotIn("list_clips", self.frame.calls)
self.assertEqual(self.sync._synced("frame-a", "clips"), set())
self.assertEqual(self.sync._synced("frame-a", "shots"), set())
def test_switch_after_bookkeeping_stops_before_next_kind(self):
self.sync.update({"screenshots": True, "clips": True})
self.frame.shots = [shot(1, NOW + 5, saved=True)]
mark = self.sync._mark
def then_switch(device, kind, ids):
mark(device, kind, ids)
self.frame.device = "frame-b"
self.sync._mark = then_switch
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertNotIn("list_clips", self.frame.calls)
def test_switch_before_transfer_does_not_copy(self):
for kind, item in (("shots", shot(1, NOW + 5)), ("clips", clip(1, NOW + 5))):
with self.subTest(kind=kind):
self.frame.device = "frame-a"
self.sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
setattr(self.frame, kind, [item])
def switched_space(_folder):
self.frame.device = "frame-b"
return 100 * GB
self.sync.space = switched_space
self.assertEqual(self.sync.tick(), fa.BUSY_WAIT)
self.assertEqual((self.frame.saved, self.frame.exported), ([], []))
def test_worker_stops(self):
stop = threading.Event()
@@ -275,6 +407,75 @@ class AutoSyncTest(unittest.TestCase):
class Server(unittest.TestCase):
def test_switch_at_list_callback_entry_does_not_read_or_mark_the_new_headset(self):
import server
with tempfile.TemporaryDirectory() as folder:
for kind in ("shots", "clips"):
with self.subTest(kind=kind):
active = {"id": "frame-a"}
link = mock.Mock()
link.active_device.side_effect = lambda: dict(active)
listing = mock.Mock(return_value={kind: [
shot(1, NOW + 5, saved=True) if kind == "shots" else clip(1, NOW + 5, saved=True)]})
def switch_then_list(device):
active["id"] = "frame-b"
return server._sync_list(listing)(device)
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]):
sync = fa.AutoSync(switch_then_list, mock.Mock(), switch_then_list, mock.Mock(),
lambda: active["id"], lambda: False,
path=Path(folder) / (kind + ".json"), clock=lambda: NOW)
sync.update({"screenshots": kind == "shots", "clips": kind == "clips"})
sync.tick()
listing.assert_not_called()
sync.save_shots.assert_not_called()
sync.export_clip.assert_not_called()
self.assertEqual(sync._synced("frame-a", kind), set())
self.assertEqual(sync._synced("frame-b", kind), set())
self.assertIn("switched headsets", sync.state()["error"])
self.assertEqual(server._work[0], 0)
def test_listing_rejects_switch_before_reserving_work(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-b"}
listing = mock.Mock(return_value={"shots": [shot(1, NOW, saved=True)]})
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]):
with self.assertRaisesRegex(server.Failure, "switched headsets"):
server._sync_list(listing)("frame-a")
listing.assert_not_called()
self.assertEqual(server._work[0], 0)
def test_listing_reserves_work_on_the_captured_device(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-a", "name": "Frame A"}
def listing():
self.assertGreater(server._work[0], 0)
with self.assertRaises(server.Failure):
server.devices_post({"action": "use", "id": "frame-b"})
return {"shots": []}
with mock.patch.object(server, "LINK", link), \
mock.patch.object(server, "PRIVATE", False), mock.patch.object(server, "_work", [0]):
self.assertEqual(server._sync_list(listing)("frame-a"), {"shots": []})
self.assertEqual(server._work[0], 0)
def test_copy_rejects_switch_after_listing(self):
import server
link = mock.Mock()
link.active_device.return_value = {"id": "frame-a"}
with mock.patch.object(server, "LINK", link), mock.patch.object(server, "_work", [0]), \
mock.patch.object(server, "save_shots") as save, \
mock.patch.object(server.frame_clips, "export_clip") as export:
server._sync_list(lambda: {"shots": []})("frame-a")
link.active_device.return_value = {"id": "frame-b"}
for copy in (lambda: server._sync_shots("frame-a", [shot(1, NOW)["id"]]),
lambda: server._sync_clip("frame-a", clip(1, NOW)["id"])):
with self.assertRaisesRegex(server.Failure, "switched headsets"):
copy()
save.assert_not_called()
export.assert_not_called()
self.assertEqual(server._work[0], 0)
def test_route_and_settings(self):
import server
self.assertIn("/api/autosync", server.POST)
@@ -287,5 +488,29 @@ class Server(unittest.TestCase):
server.auto_sync({"action": "now"}) # nothing connected
@unittest.skipUnless(shutil.which("node"), "Node exercises the Settings renderer")
class Settings(unittest.TestCase):
def test_resume_is_available_after_a_copy_error(self):
page = (Path(__file__).resolve().parents[1] / "ui/index.html").read_text(encoding="utf-8")
script = page.split("const autoSync =", 1)[1].split("async function loadAutoSync", 1)[0]
harness = r'''
const assert = require('node:assert/strict');
const nodes = {};
const $ = id => nodes[id] ||= {};
const HOST = {mobile: false};
const esc = value => value;
'''
checks = r'''
for (const state of [{error: 'Could not copy clip'}, {paused: 'Disk full'}, {}]) {
renderAutoSync(state);
assert.equal($('syncResume').hidden, !(state.error || state.paused));
if (state.error) assert.match($('syncStatus').textContent, /Could not copy clip/);
}
'''
result = subprocess.run(["node", "-e", harness + "const autoSync =" + script + checks],
capture_output=True, text=True, timeout=10)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
if __name__ == "__main__":
unittest.main()
+56 -12
View File
@@ -16,9 +16,9 @@ saved Steam clips to ~/Videos/SteamFrame.
- When the headset can't be reached it waits longer each time, up to MAX_BACKOFF.
The server supplies how to reach the headset, so this can be tested without one:
list_shots() -> {"shots": [{"id", "time", "saved"}...]}
list_shots(device) -> {"shots": [{"id", "time", "saved"}...]}
save_shots(device, ids) copies those screenshots here
list_clips() -> {"clips": [{"id", "kind", "time", "idle", "saved"}...]}
list_clips(device) -> {"clips": [{"id", "kind", "time", "idle", "saved"}...]}
export_clip(device, id) exports one clip here
device() -> the connected headset's id, or None
busy() -> True while other work runs on the headset (installs, jobs, a recording)
@@ -41,7 +41,7 @@ SETTLE = 30 # a clip written to within this many seconds is still being recorde
MIN_FREE = 512 * 1024**2 # pause below this much free space on this computer
KEEP = 5000 # copied ids remembered per headset and kind
BATCH = 100 # screenshots per copy
TRIES = 3 # an item that fails this often is left until the app restarts
TRIES = 3 # failed attempts per item until Resume, Copy everything now, or restart
KINDS = ("shots", "clips")
@@ -86,6 +86,7 @@ class AutoSync:
self._running = False
self._backoff = 0
self._tries = {} # (device, id) -> failed attempts this run
self._failures = {} # last error for each failed item, including exhausted retries
self._errors = [] # this pass's, for the page
self._s = self._load()
@@ -143,13 +144,19 @@ class AutoSync:
"""Copy everything on the headset that isn't on this computer, once (also resumes)."""
with self._lock:
self._all, self._s["paused"] = True, None
self._tries.clear()
self._failures.clear()
self._s["error"] = None
self._save()
self._backoff = 0
self._wake.set()
return self.state()
def resume(self):
with self._lock:
self._s["paused"], self._s["error"] = None, None
self._tries.clear()
self._failures.clear()
self._save()
self._backoff = 0
self._wake.set()
@@ -215,6 +222,9 @@ class AutoSync:
def _mark(self, device, kind, ids):
with self._lock:
for item in ids:
self._tries.pop((device, item), None)
self._failures.pop((device, item), None)
per = self._s["synced"].setdefault(device, {})
have = per.get(kind, [])
new = [i for i in ids if i not in set(have)]
@@ -233,24 +243,46 @@ class AutoSync:
"then press Resume.")
if self.offline(error):
raise error
key = (device, item)
self._tries[key] = self._tries.get(key, 0) + 1
self._errors.append(f"Couldn't copy {item.rsplit('/', 1)[-1]}: {error}")
with self._lock:
key = (device, item)
self._tries[key] = self._tries.get(key, 0) + 1
message = f"Couldn't copy {item.rsplit('/', 1)[-1]}: {error}"
if self._tries[key] >= TRIES:
message += ". Automatic retries stopped. Press Resume to try again."
self._failures[key] = message
self._errors.append(message)
def _retry_allowed(self, device, item):
with self._lock:
key = (device, item)
if self._tries.get(key, 0) < TRIES:
return True
self._errors.append(self._failures[key])
return False
def _interrupted(self, device):
return self.busy() or self.device() != device
def _pass(self, device, shots, clips, since):
"""-> (copied, clips still being written, finished without stopping for other work)."""
copied = waiting = 0
if shots:
listing = self.list_shots()["shots"]
if self._interrupted(device):
return copied, waiting, False
listing = self.list_shots(device)["shots"]
if self._interrupted(device):
return copied, waiting, False
done = self._synced(device, "shots")
self._mark(device, "shots", [s["id"] for s in listing if s.get("saved")])
todo = [s["id"] for s in sorted(listing, key=lambda s: s["time"])
if not s.get("saved") and (since is None or (s["time"] >= since and s["id"] not in done))
and self._tries.get((device, s["id"]), 0) < TRIES]
and self._retry_allowed(device, s["id"])]
for i in range(0, len(todo), BATCH):
if self.busy():
if self._interrupted(device):
return copied, waiting, False
self._room(self.shots_folder)
if self._interrupted(device):
return copied, waiting, False
batch = todo[i:i + BATCH]
try:
self.save_shots(device, batch)
@@ -258,28 +290,40 @@ class AutoSync:
for item in batch:
self._failed(device, item, e)
continue
if self.device() != device:
return copied, waiting, False
self._mark(device, "shots", batch)
copied += len(batch)
if clips:
listing = [c for c in self.list_clips()["clips"] if c.get("kind") == "clip"]
if self._interrupted(device):
return copied, waiting, False
listing = [c for c in self.list_clips(device)["clips"] if c.get("kind") == "clip"]
if self._interrupted(device):
return copied, waiting, False
done = self._synced(device, "clips")
self._mark(device, "clips", [c["id"] for c in listing if c.get("saved")])
for c in sorted(listing, key=lambda c: c["time"]):
if c.get("saved") or self._tries.get((device, c["id"]), 0) >= TRIES:
if c.get("saved"):
continue
if since is not None and (c["time"] < since or c["id"] in done):
continue
if not self._retry_allowed(device, c["id"]):
continue
if c.get("idle") is None or c["idle"] < SETTLE:
waiting += 1 # still being written: copied on a later look
continue
if self.busy():
if self._interrupted(device):
return copied, waiting, False
self._room(self.clips_folder)
if self._interrupted(device):
return copied, waiting, False
try:
self.export_clip(device, c["id"])
except Exception as e:
self._failed(device, c["id"], e)
continue
if self.device() != device:
return copied, waiting, False
self._mark(device, "clips", [c["id"]])
copied += 1
return copied, waiting, True
+1 -1
View File
@@ -5610,7 +5610,7 @@ const syncWhen = t => new Date(t * 1000).toLocaleString([], { dateStyle: "medium
function renderAutoSync(s) {
autoSync.s = s;
$("syncShots").checked = s.screenshots; $("syncClips").checked = s.clips;
$("syncResume").hidden = !s.paused;
$("syncResume").hidden = !(s.paused || s.error);
const on = s.screenshots || s.clips, lines = [];
if (s.paused) lines.push(s.paused);
else if (s.error) lines.push(`Last problem: ${s.error}`);
+2 -2
View File
@@ -969,8 +969,8 @@ def _sync_clip(device, clip_id):
def _sync_list(listing):
def run():
with working(_sync_device()):
def run(device):
with working(device):
return listing()
return run