Compare commits

..
Author SHA1 Message Date
SAPHID 08c99c2d7d docs: scope Game Sync ARM64 limit to cloud sync 2026-10-29 01:00:28 +00:00
SAPHID b3e3e21f46 test: remove unverified rclone path assertion 2026-10-29 00:58:58 +00:00
SAPHID 84bb7fa0b1 test: verify Game Sync configuration in native Decky 2026-10-29 00:55:54 +00:00
SAPHID 130d82128b test: include RadiYo native probe in runtime inventory 2026-10-29 00:38:22 +00:00
SAPHID 65e629f8c3 Verify RadiYo backend through isolated native Decky 2026-10-29 00:32:26 +00:00
Alex Southwell e33ed00945 Merge pull request #397 from saphid/fix/release-all-tag-runs
Reject earlier failed builds for a release tag
2026-10-29 00:09:11 +00:00
SAPHID 419d69ef8a Reject prior failed tagged release runs before publishing 2026-10-28 23:58:30 +00:00
Alex Southwell 8cf2fb9da3 Merge pull request #395 from saphid/ci/frame-smoke-source-commit
Bind Frame smoke evidence to release commit and lock cleanup
2026-10-28 23:39:18 +00:00
SAPHID ff2eb87ac1 Merge remote-tracking branch 'origin/main' into ci/frame-smoke-source-commit 2026-10-28 23:30:51 +00:00
Alex SouthwellandSAPHID c611f4aeb7 Run live Windows wireless inventory only in dedicated CI job (#396)
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-28 23:30:18 +00:00
SAPHID f981ba9315 Require exact locked Frame smoke before publication 2026-10-28 23:19:35 +00:00
SAPHID 014e6b3531 Merge remote-tracking branch 'origin/main' into ci/frame-smoke-source-commit 2026-10-28 23:15:37 +00:00
SAPHID 8b6c32ba37 Bind Frame smoke evidence to release commit and lock cleanup 2026-10-28 23:15:01 +00:00
22 changed files with 474 additions and 37 deletions

No files matched your search

@@ -0,0 +1,24 @@
# Game Sync 1.0.4 isolated backend check, 29 October 2026
The official Decky archive for Game Sync (ID 104) is 2,228,806 bytes with
SHA-256 `65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd`.
Its unmodified ZIP is pinned as `tests/fixtures/framey-official/104.zip`.
`python3 scripts/framey-test-decky-runtime.py --case 104 --output /tmp/framey-gamesync-native-104.json`
passed with the published Decky 3.3.0 x86_64 loader, SHA-256
`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`.
The disposable container had no external network or host devices, dropped
capabilities and used a private home. The real loader discovered the exact
plugin. Its `get_config` returned defaults with automatic game sync and
capture upload disabled. A private `advanced_mode` setting changed from false
to true and remained true after the loader restarted. The runner removed its
container and image; a separate `docker ps` and image-list audit found no
owned resources.
The original opaque-iframe browser audit blocked `get_config` before panel
initialization completed, so it did not prove a usable frontend. This native
test confirms only a configuration backend path. No cloud authentication,
rclone download or sync, screenshot deletion, process signal, game event,
real Frame backend, VR visibility or physical controller was exercised.
The published `rclone_manager.py` names an amd64 rclone download, so this
result does not establish ARM64 sync support. Game Sync remains **partial**.
@@ -0,0 +1,24 @@
# RadiYo! 1.3 isolated backend check, 29 October 2026
The official Decky store archive for RadiYo! (ID 24) is 33,686 bytes with
SHA-256 `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43`.
Its unmodified ZIP is pinned as `tests/fixtures/framey-official/24.zip` and
passes Frame Control's archive validation and fake Frame panel packaging.
`python3 scripts/framey-test-decky-runtime.py --case 24 --output /tmp/framey-radiyo-decky-runtime-20261029.json`
ran the published Decky 3.3.0 x86_64 loader, digest
`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`,
in a disposable container with `--network none`, no host devices, dropped
capabilities and a private home. The real loader discovered the exact plugin
and dispatched its legacy `fetch` and `getRedirectUrl` RPCs. A server bound to
container loopback returned a fixed station marker: regular fetch returned it,
custom-station fetch omitted the body as upstream intends, and a redirect
resolved to the same fixture. The runner removed its container and image.
The previously recorded opaque-iframe browser audit showed frontend
initialization, but denied backend and network calls. This native probe adds
backend transport evidence only. No external radio station, HTTPS endpoint,
audio playback, real Frame service, VR visibility or controller was tested.
The published backend sets `check_hostname=False` and `CERT_NONE` for its
outbound HTTPS requests; the loopback HTTP test did not exercise that path.
RadiYo! remains **partial**. The compatibility count does not change.
+2 -2
View File
@@ -56,7 +56,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 21: PowerTools 2.0.3 | legacy / legacy | N/A | Not supported on Frame: Targets AMD Ryzen power controls; Frame uses an ARM64 SoC. | static archive, local browser, fake-Frame |
| 22: Memory Deck 0.1.8 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: memory-deck/bin/libscanmem.so.1.0.0. | static archive, local browser, fake-Frame |
| 23: Animation Changer 1.3.2 | legacy / legacy | partial | Initializer succeeds; forced backend failure prevents loading animations. Boot-video changes untested. | static archive, local browser, fake-Frame |
| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds; radio discovery and playback remain unverified. | static archive, local browser, fake-Frame |
| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made. | static archive, local browser, fake-Frame, isolated Decky 3.3.0 x86_64 backend |
| 26: HLTB for Deck 2.0.12 | 1 / 2 | partial | Settings render without the prior IndexedDB SecurityError; Steam library badges have no patch target, and live lookup is unverified. | static archive, local browser, fake-Frame |
| 27: AutoSuspend 2.2.0 | legacy / legacy | broken | Frontend audit failed: Cannot read properties of null (reading '6'). The exact frontend also contains SteamOS suspend and shutdown paths; threshold behavior and safe Frame operation are unverified. | static archive, local browser, fake-Frame |
| 28: Emuchievements 2.0.5 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: Emuchievements/bin/hash. | static archive, local browser, fake-Frame |
@@ -118,7 +118,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 101: XIVOmega 1.0.2 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
| 102: DeckSP 1.1.3 | 1 / 2 | broken | Optional UI-audio lookup succeeds, but the published frontend raises "Cannot access PluginManager before initialization" when Steam MobX reaction lookup is unavailable. JamesDSP installation/audio routing and ARM64 remain unverified. | static archive, local browser, fake-Frame |
| 103: Decky Ludusavi 1.2.2 | 1 / 2 | partial | Initializer succeeds; Flatpak Ludusavi discovery and backup/restore unverified. | static archive, local browser, fake-Frame |
| 104: Game Sync 1.0.4 | 1 / 2 | partial | Backend get_config is intentionally rejected by the audit before initialization completes; actual backend compatibility is unknown. | static archive, local browser, fake-Frame |
| 104: Game Sync 1.0.4 | 1 / 2 | partial | Exact official backend discovers under Decky 3.3.0 amd64; default get_config and a private setting read/write survive a loader restart. The isolated browser baseline still stops at its denied get_config call. Cloud login, rclone download/sync (including ARM64), process signals, real Frame and VR are unverified. The published updater names an amd64 rclone download, so no ARM64 sync claim follows. | static archive, local browser, fake-Frame, native Decky amd64 configuration |
| 105: WiFi Locker 1.0.2 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
| 106: LetMeReShade 1.8 | 1 / 2 | partial | Initializer succeeds; Deck-model assumptions, shader downloads and game modifications unverified. | static archive, local browser, fake-Frame |
| 107: Decky Sunshine 2025.10.27-dddf365 | legacy / legacy | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
+13 -3
View File
@@ -57,6 +57,16 @@ count. So a build reaches people only when you publish it, after testing it.
publish unless every required job succeeded; it cannot prove physical
headset or installer behavior on its own.
Supply the saved smoke result to the publisher. Both its dry run and publish
command reject a result from another or dirty commit, incomplete title or
cleanup steps, a skip or failure, or an unconfirmed headset lock release:
```sh
export FRAME_SMOKE_RESULT=/path/to/smoke-result.json
python3 scripts/verify-frame-smoke-result.py "$FRAME_SMOKE_RESULT" \
"$(git rev-parse 'v0.4.0^{commit}')"
```
```sh
scripts/publish-release.sh --dry-run v0.4.0 # checks and shows update.json, changes nothing
scripts/publish-release.sh v0.4.0
@@ -64,9 +74,9 @@ count. So a build reaches people only when you publish it, after testing it.
The script checks that the tag is on GitHub, that exact-commit main CI
(including e2e), native amd64/ARM64 Decky runtime checks, and the tagged
release workflow are green with no failed,
cancelled, skipped or running jobs in any run attempt, and that all ten installers
are attached, each with the SHA-256 digest GitHub records. It also checks
release workflow are green with no failed, cancelled, skipped or running
jobs in any exact-tag push run or attempt, and that all ten installers are
attached, each with the SHA-256 digest GitHub records. It also checks
that the draft does not begin with a `WITHDRAWN` notice, so the preserved
v0.4.48 draft cannot accidentally be offered again. It checks
that the uploaded `SHA256SUMS` digest matches a fresh list of those ten
+4 -1
View File
@@ -215,7 +215,10 @@ acquires the shared `/tmp/frame-test.lock` with a unique owner token. It reads
SteamVR activity, proximity and battery while locked. An awake headset, unknown
activity, battery below 20%, or a busy lock stops the run without installing
anything. It releases only its own lock after cleanup. The JSON result includes
the read-only preflight for completed smoke runs.
the read-only preflight and the source checkout's commit and clean/dirty state.
For a release, run it from a clean checkout of the exact release commit;
`scripts/verify-frame-smoke-result.py` rejects a different or dirty checkout,
missing steps, skipped or failed launches, and an unconfirmed lock release.
It checks `properties.json` and the status, then installs, launches and
removes three tiny titles built from bytes by `tests/smoke/tiny_programs.py`
+9 -2
View File
@@ -70,10 +70,15 @@ def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--loader', type=Path, help='Optional cached, digest-verified native Decky 3.3.0 binary')
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--case', type=int, help='Run one official plugin case locally; CI runs all cases')
args = parser.parse_args()
if sys.platform != 'linux' or not shutil.which('docker'):
parser.error('Linux with Docker is required')
cases = check_coverage()
if args.case is not None:
cases = [case for case in cases if case['id'] == args.case]
if not cases:
parser.error('Unknown official plugin case')
data, digest = loader_bytes(args.loader)
tag = 'framey-decky-runtime:'+uuid.uuid4().hex[:12]
report = {'decky': RELEASE, 'architecture': platform.machine(), 'loaderSha256': digest, 'cases': []}
@@ -85,8 +90,10 @@ def main():
loader.write_bytes(data)
loader.chmod(0o644) # Private container user copies/chmods its own executable.
probes = [(case['id'], None) for case in cases] + [
(111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
(60, '--reproduce-fileserver-empty-flag-regression')]
(plugin_id, negative) for plugin_id, negative in [
(111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
(60, '--reproduce-fileserver-empty-flag-regression')]
if args.case is None or args.case == plugin_id]
for plugin_id, negative in probes:
name = 'framey-decky-'+uuid.uuid4().hex[:12]
command = ['docker', 'run', '--rm', '--name', name, '--network', 'none', '--read-only',
+3 -3
View File
@@ -3,7 +3,7 @@
# (docs/releasing.md). Checks every installer is attached with a SHA-256
# digest first, since the app's updater refuses assets without one, then
# attaches update.json, the manifest the updater reads.
# Usage: scripts/publish-release.sh [--dry-run] v0.4.0
# Usage: FRAME_SMOKE_RESULT=/path/to/result.json scripts/publish-release.sh [--dry-run] v0.4.0
#
# Everything goes through the REST API (gh api), not `gh release view/edit`:
# those look a draft up by its tag, and a draft can show tag_name
@@ -33,8 +33,8 @@ gh api "repos/$repo/git/ref/tags/$tag" >/dev/null 2>&1 \
# A complete asset list is insufficient: a cancelled installer build can leave
# old uploaded bytes behind. Check the exact tagged commit's main/e2e jobs and
# the latest exact-tag release attempt before any release mutation.
python3 "$(dirname "$0")/verify-release-gates.py" "$tag"
# every exact-tag push run and attempt before any release mutation.
python3 "$(dirname "$0")/verify-release-gates.py" "$tag" "${FRAME_SMOKE_RESULT:-}"
# Every release, drafts included, one JSON object per line.
gh api --paginate "repos/$repo/releases?per_page=100" --jq '.[]' > "$tmp/releases"
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Check a locked Frame smoke result against the exact release commit."""
import json
import re
import sys
from pathlib import Path
REQUIRED_STEPS = {
'cleanup: leftovers from earlier runs',
'devkit service: properties.json',
'status',
'cleanup: everything this run installed',
*(f'{kind}: {action}' for kind in ('arm64', 'x86_64', 'exe')
for action in ('install', 'launch', 'remove')),
}
def verify(result, commit):
if not isinstance(result, dict):
raise ValueError('smoke result is not an object')
if not re.fullmatch(r'[0-9a-f]{40}', commit):
raise ValueError('expected a full lowercase commit SHA-1')
if result.get('source') != {'commit': commit, 'clean': True}:
raise ValueError('smoke source is not the exact clean release commit')
if result.get('lock_released') is not True:
raise ValueError('smoke lock release was not recorded')
steps = result.get('steps')
if not isinstance(steps, list) or not steps:
raise ValueError('smoke steps are missing')
names = [step.get('step') for step in steps if isinstance(step, dict)]
if (len(names) != len(steps) or any(not isinstance(name, str) for name in names) or
len(names) != len(set(names)) or not REQUIRED_STEPS.issubset(names)):
raise ValueError('smoke steps are missing or duplicated')
passed = sum(step.get('ok') is True and not step.get('skipped') for step in steps)
if (passed != len(steps) or result.get('passed') != passed or
result.get('skipped') != 0 or result.get('failed') != 0):
raise ValueError('smoke contains a failed, skipped, or inconsistent step')
state = result.get('preflight') or {}
if (not isinstance(state, dict) or state.get('activity') != 3 or
not isinstance(state.get('proximity'), list) or not state['proximity'] or
any(type(value) not in (int, float) or value < 0 or value > 10
for value in state['proximity']) or
not isinstance(state.get('battery'), list) or not state['battery'] or
any(type(value) is not int or value < 20 or value > 100
for value in state['battery'])):
raise ValueError('locked standby preflight is missing')
def main(argv):
if len(argv) != 3:
raise ValueError('usage: verify-frame-smoke-result.py RESULT.json RELEASE_COMMIT_SHA')
result = json.loads(Path(argv[1]).read_text(encoding='utf-8'))
verify(result, argv[2])
print(f'Frame smoke passed for clean release commit {argv[2]}')
if __name__ == '__main__':
try:
main(sys.argv)
except (OSError, ValueError) as error:
sys.exit(f'Frame smoke gate failed: {error}')
+37 -8
View File
@@ -1,13 +1,14 @@
#!/usr/bin/env python3
"""Refuse publication unless the exact tag commit and tagged installers passed CI.
"""Refuse publication unless CI, tagged installers, and exact Frame smoke passed.
This checks GitHub's current run and job records, not a prior PR's check marks.
The real Frame smoke and independent package checks remain release-procedure
Independent package checks and release verification remain release-procedure
gates documented in docs/releasing.md.
"""
import json
import subprocess
import sys
from pathlib import Path
from urllib.parse import quote
@@ -60,14 +61,29 @@ def tag_commit(tag):
raise GateError(f"{tag} has too many nested tag objects")
def exact_run(workflow, branch, sha, event):
def exact_runs(workflow, branch, sha, event):
query = f"actions/workflows/{workflow}/runs?branch={quote(branch, safe='')}&head_sha={sha}&per_page=100"
runs = api(query).get("workflow_runs") or []
runs = []
for page in range(1, 21):
data = api(f"{query}&page={page}")
batch = data.get("workflow_runs")
total = data.get("total_count")
if not isinstance(batch, list) or type(total) is not int or total < 0:
raise GateError(f"invalid {workflow} run list for {branch} at {sha}")
runs.extend(batch)
if len(runs) >= total or len(batch) < 100:
break
if len(runs) != total or len({run.get('id') for run in runs}) != len(runs):
raise GateError(f"incomplete or duplicate {workflow} run list for {branch} at {sha}")
matching = [run for run in runs if run.get("head_sha") == sha
and run.get("head_branch") == branch and run.get("event") == event]
if not matching:
raise GateError(f"no exact {workflow} {event} run for {branch} at {sha}")
run = max(matching, key=lambda item: item["id"])
return sorted(matching, key=lambda item: item["id"])
def exact_run(workflow, branch, sha, event):
run = exact_runs(workflow, branch, sha, event)[-1]
if run.get("status") != "completed" or run.get("conclusion") != "success":
raise GateError(f"{workflow} run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
return run
@@ -98,18 +114,31 @@ def successful_jobs(run, required):
print(f"gate passed: {run.get('html_url')} ({len(latest_jobs)} jobs, {attempt} attempts)")
def main(tag):
def main(tag, smoke_result):
if not tag.startswith("v") or not tag[1:]:
raise GateError("expected a version tag beginning with v")
if not smoke_result:
raise GateError("FRAME_SMOKE_RESULT is required for the locked real Frame smoke gate")
sha = tag_commit(tag)
successful_jobs(exact_run("checks.yml", "main", sha, "push"), MAIN_JOBS)
successful_jobs(exact_run("decky-runtime.yml", "main", sha, "push"), DECKY_JOBS)
successful_jobs(exact_run("release.yml", tag, sha, "push"), RELEASE_JOBS)
# A fresh run must not hide a failed, cancelled, or skipped installer in an
# earlier separate push run for the same tag and commit.
for run in exact_runs("release.yml", tag, sha, "push"):
if run.get("status") != "completed" or run.get("conclusion") != "success":
raise GateError(f"release.yml run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
successful_jobs(run, RELEASE_JOBS)
command = [sys.executable, str(Path(__file__).with_name("verify-frame-smoke-result.py")),
smoke_result, sha]
checked = subprocess.run(command, text=True, capture_output=True, check=False)
if checked.returncode:
raise GateError(checked.stderr.strip() or "Frame smoke result verification failed")
print(checked.stdout.strip())
print(f"exact release commit: {sha}")
if __name__ == "__main__":
try:
main(sys.argv[1])
main(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else None)
except (IndexError, GateError) as error:
sys.exit(f"release gate failed: {error}")
+10
View File
@@ -14,6 +14,16 @@
"shim": null,
"probe": "toggle-settings"
},
{
"id": 24,
"shim": null,
"probe": "radio-local"
},
{
"id": 104,
"shim": null,
"probe": "game-sync-config"
},
{
"id": 63,
"shim": null,
+66
View File
@@ -2,6 +2,7 @@
import argparse
import asyncio
import hashlib
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
@@ -11,6 +12,7 @@ import socket
import ssl
import subprocess
import sys
import threading
import time
import urllib.request
import zipfile
@@ -265,6 +267,70 @@ def main():
assert current('read_deals') == {}
report['result'] = ('discovery, default settings, empty local deals, '
'private setting write/read and restart persistence passed; feed, notifications and VR unverified')
elif case['probe'] == 'game-sync-config':
# Exercise only private settings. This plugin also exposes
# rclone cloud sync and arbitrary process signals; neither
# belongs in a passive compatibility probe.
settings = current('get_config')
assert settings['sync_on_game_start'] is False, settings
assert settings['sync_on_game_stop'] is False, settings
assert settings['capture_upload'] is False, settings
assert settings['advanced_mode'] is False, settings
assert current('set_config', 'advanced_mode', True) is None
assert current('get_config')['advanced_mode'] is True
os.killpg(backend.pid, signal.SIGTERM)
backend.wait(timeout=8)
backend = subprocess.Popen([str(loader)], env=env, stdout=log, stderr=log,
start_new_session=True)
processes.append(backend)
until_ready(lambda: rpc('loader/get_plugins'), 'restarted Decky discovery')
assert until_ready(lambda: current('get_config')['advanced_mode'],
'private Game Sync setting after restart') is True
report['result'] = ('discovery, default config and private setting restart persistence passed; '
'rclone, cloud authentication, sync, process signals, real Frame and VR unverified')
elif case['probe'] == 'radio-local':
# The published backend accepts arbitrary URLs and disables
# HTTPS verification. Exercise only a loopback fixture inside
# this network-isolated container; no station or audio opens.
marker = 'Framey private radio station\n'
class RadioHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/redirect':
self.send_response(302)
self.send_header('Location', '/station')
self.end_headers()
elif self.path == '/station':
data = marker.encode()
self.send_response(200)
self.send_header('Content-Type', 'text/plain')
self.send_header('Content-Length', str(len(data)))
self.end_headers()
self.wfile.write(data)
else:
self.send_error(404)
def log_message(self, *_):
pass
server = ThreadingHTTPServer(('127.0.0.1', 0), RadioHandler)
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()
try:
base_url = f'http://127.0.0.1:{server.server_port}'
station = legacy('fetch', url=base_url+'/station', customStation=False)
assert station['status'] == 200 and station['body'] == marker, station
assert station['headers']['Content-Type'] == 'text/plain', station
custom = legacy('fetch', url=base_url+'/station', customStation=True)
assert custom['status'] == 200 and custom['body'] == '', custom
redirect = legacy('getRedirectUrl', url=base_url+'/redirect')
assert redirect == {'response': base_url+'/station'}, redirect
finally:
server.shutdown()
server.server_close()
worker.join(timeout=3)
report['result'] = ('actual Decky discovery and local station fetch/redirect passed; '
'external radio, HTTPS, audio playback, Frame and VR unverified')
elif case['probe'] == 'toggle-settings':
def call(method, *values):
result = rpc('loader/call_plugin_method', record['name'], method, *values)
+14 -3
View File
@@ -96,15 +96,26 @@ elif method == "GET" and endpoint.startswith(repo + "actions/workflows/"):
"status": gate.get("decky_status" if run_id == 503 else ("main_status" if is_main else "release_status"), "completed"),
"conclusion": gate.get("decky_conclusion" if run_id == 503 else ("main_conclusion" if is_main else "release_conclusion"), "success"),
"html_url": "https://github.com/saphid/frame-control/actions/runs/%s" % run_id}
print(json.dumps({"workflow_runs": [run]}))
runs = [run]
if run_id == 502 and gate.get("prior_release_run_bad_job"):
runs.append(dict(run, id=500, run_attempt=1, status="completed",
conclusion=gate.get("prior_release_run_conclusion", "failure"),
html_url="https://github.com/saphid/frame-control/actions/runs/500"))
if run_id == 502 and gate.get("release_run_list_incomplete"):
print(json.dumps({"total_count": len(runs) + 1, "workflow_runs": runs}))
else:
print(json.dumps({"total_count": len(runs), "workflow_runs": runs}))
elif method == "GET" and endpoint.startswith(repo + "actions/runs/"):
parts = endpoint[len(repo + "actions/runs/"):].split("/")
run_id = parts[0]
attempt = int(parts[2])
names = {"501": main_jobs, "502": release_jobs, "503": decky_jobs}[run_id]
names = {"500": release_jobs, "501": main_jobs, "502": release_jobs,
"503": decky_jobs}[run_id]
names = [name for name in names if name != gate.get("omit_job")]
jobs = [{"name": name, "status": "completed",
"conclusion": (gate.get("prior_bad_job_conclusion", "failure")
"conclusion": (gate.get("prior_release_run_bad_conclusion", "cancelled")
if run_id == "500" and name == gate.get("prior_release_run_bad_job")
else gate.get("prior_bad_job_conclusion", "failure")
if run_id == "502" and attempt == 1 and name == gate.get("prior_bad_job")
else gate.get("bad_job_conclusion", "success") if name == gate.get("bad_job")
else "success")}
Binary file not shown.
Binary file not shown.
+10 -1
View File
@@ -3,7 +3,7 @@
These are unmodified ZIPs downloaded from the official Decky store CDN on
2026-10-25, except `96.zip` and `111.zip` downloaded on 2026-10-26 and `60.zip` on
2026-10-27; `13.zip` and `125.zip` were added from the digest-verified audit cache on 2026-10-27,
and `67.zip` and `114.zip` on 2026-10-28.
and `67.zip` and `114.zip` on 2026-10-28. `24.zip` and `104.zip` were added on 2026-10-29.
Each ZIP includes its upstream `LICENSE`. They are test data, not plugins shipped or installed by
Frame Control. The fake Frame tests serve them
from a local fixture so CI does not depend on the store being online.
@@ -11,6 +11,8 @@ from a local fixture so CI does not depend on the store being online.
| File | Official plugin | Version | Store SHA-256 |
| --- | --- | --- | --- |
| `13.zip` | Pause Games | 1.0.3 | `4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62` |
| `24.zip` | RadiYo! | 1.3 | `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43` |
| `104.zip` | Game Sync | 1.0.4 | `65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd` |
| `60.zip` | DeckyFileServer | 1.2.0 | `7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0` |
| `125.zip` | Decky Localsend | 0.4.5 | `e271ccb55eb23f918b99a69f79aec905df4c9b2ef7b23bbf4ec1c2d1a6f0c6b7` |
| `114.zip` | Network Sentinel | 1.0.4 | `d3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5` |
@@ -60,6 +62,13 @@ Pause Games runs through actual published Decky 3.3.0 discovery and instance RPC
for private settings and a synthetic exclusion list. No process-control method
is called. The official frontend's standalone Toggle component separately
checks saved true/false state and matching settings callbacks; effects are inert.
RadiYo! runs through actual published Decky 3.3.0 legacy RPC in a network-isolated
container. Only loopback station fetch, custom-station fetch and redirect are
tested. External radio, HTTPS and audio playback remain unverified.
Game Sync runs through actual published Decky 3.3.0 instance RPC in the same
isolation. Only default configuration and a private setting surviving restart
are checked. No rclone, cloud sync, process signal, Frame backend or VR action
is tested.
Network Sentinel runs through the same isolated native Decky gate. Its exact
archive gets plugin-local psutil and a user-service manifest; the probe checks
discovery, private settings and one monitoring-loop sample without external
+4
View File
@@ -10,6 +10,10 @@ PLUGINS = (
'folder': 'network-sentinel', 'sha256': 'd3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5'},
{'id': 13, 'name': 'Pause Games', 'version': '1.0.3', 'api_version': 1,
'folder': 'SDH-PauseGames', 'sha256': '4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62'},
{'id': 24, 'name': 'RadiYo!', 'version': '1.3', 'api_version': 'legacy',
'folder': 'radiyo-steam-deck', 'sha256': '647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43'},
{'id': 104, 'name': 'Game Sync', 'version': '1.0.4', 'api_version': 1,
'folder': 'SDH-GameSync', 'sha256': '65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd'},
{'id': 60, 'name': 'DeckyFileServer', 'version': '1.2.0', 'api_version': 1,
'folder': 'DeckyFileServer', 'sha256': '7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0'},
{'id': 125, 'name': 'Decky Localsend', 'version': '0.4.5', 'api_version': 1,
+29 -4
View File
@@ -35,6 +35,7 @@ import argparse
import json
import os
import platform
import re
import secrets
import shlex
import subprocess
@@ -61,6 +62,20 @@ PREFIX = 'fc_smoke_'
LOCK = '/tmp/frame-test.lock'
def source_revision():
"""Record which checkout produced the smoke result for release review."""
try:
head = subprocess.run(['git', '-C', str(ROOT), 'rev-parse', 'HEAD'],
capture_output=True, text=True, timeout=10, check=True).stdout.strip()
status = subprocess.run(['git', '-C', str(ROOT), 'status', '--porcelain'],
capture_output=True, text=True, timeout=10, check=True).stdout
except (OSError, subprocess.CalledProcessError, subprocess.TimeoutExpired):
return {'commit': None, 'clean': False}
if not isinstance(head, str) or not re.fullmatch(r'[0-9a-f]{40}', head):
return {'commit': None, 'clean': False}
return {'commit': head, 'clean': not bool(status.strip())}
def remote(command, *, input=None):
"""Use a bounded SSH call for shared-lock operations."""
return subprocess.run(['ssh', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=8', ALIAS, command],
@@ -420,12 +435,14 @@ def run_smoke(args, before):
failed = len(smoke.steps) - passed - skipped
RESULTS.mkdir(exist_ok=True)
out = RESULTS / f"{started.replace(':', '')}-{build}.json"
out.write_text(json.dumps({'started': started, 'alias': ALIAS, 'preflight': before,
out.write_text(json.dumps({'started': started, 'alias': ALIAS, 'source': source_revision(),
'preflight': before,
'os_release': release, 'paired': args.pair,
'passed': passed, 'skipped': skipped, 'failed': failed,
'lock_released': False,
'devkit_utils_there_before': had_utils, 'steps': smoke.steps}, indent=1))
print(f'==> {passed} passed, {skipped} skipped, {failed} failed on build {build}; results in {out}')
return 0 if failed == 0 and skipped == 0 else 1
return (0 if failed == 0 and skipped == 0 else 1), out
def main():
@@ -440,12 +457,12 @@ def main():
if not locked:
print('Shared Frame lock busy or unreachable; no headset status read or smoke run.', file=sys.stderr)
return 2
result = 1
result, output = 1, None
try:
try:
before = preflight()
print('Locked read-only preflight: ' + json.dumps(before), flush=True)
result = run_smoke(args, before)
result, output = run_smoke(args, before)
except (OSError, ValueError, RuntimeError, subprocess.TimeoutExpired) as failure:
print(str(failure), file=sys.stderr)
result = 2
@@ -457,6 +474,14 @@ def main():
if not released:
print('ERROR: could not release owned Frame lock.', file=sys.stderr)
result = 1
elif output is not None:
try:
saved = json.loads(output.read_text())
saved['lock_released'] = True
output.write_text(json.dumps(saved, indent=1))
except (OSError, ValueError) as failure:
print(f'ERROR: could not record Frame lock release: {failure}', file=sys.stderr)
result = 1
return result
+1 -1
View File
@@ -21,7 +21,7 @@ class RuntimeCoverage(unittest.TestCase):
self.assertIn(case['id'], official)
self.assertIn(case['probe'], ('mpris', 'screenshots', 'system-monitor', 'timer-lifecycle',
'toggle-settings', 'fileserver', 'localsend', 'network-sentinel',
'battery-tracker', 'free-loader'))
'battery-tracker', 'free-loader', 'radio-local', 'game-sync-config'))
def test_new_shim_without_runtime_case_fails_coverage(self):
import frame_decky_store
+57 -1
View File
@@ -38,15 +38,35 @@ def draft(tag_name="untagged-c6ddfed7f75d67db2e99", name="Frame Control 9.8.7",
class PublishRelease(unittest.TestCase):
def run_script(self, releases, *args, tags="v9.8.7", gates=None,
corrupt_manifest_readback=False):
corrupt_manifest_readback=False, smoke=None):
d = Path(tempfile.mkdtemp())
(d / "releases.json").write_text(json.dumps(releases))
(d / "gates.json").write_text(json.dumps(gates or {}))
steps = ["cleanup: leftovers from earlier runs", "devkit service: properties.json",
"status", "cleanup: everything this run installed"] + [
f"{kind}: {action}" for kind in ("arm64", "x86_64", "exe")
for action in ("install", "launch", "remove")]
clean_smoke = {"source": {"commit": "a" * 40, "clean": True},
"lock_released": True,
"steps": [{"step": name, "ok": True} for name in steps],
"passed": len(steps), "skipped": 0, "failed": 0,
"preflight": {"activity": 3, "proximity": [2.3125], "battery": [100]}}
if smoke == "skipped":
clean_smoke["steps"][0].update(ok=False, skipped=True)
clean_smoke.update(passed=len(steps) - 1, skipped=1)
elif smoke == "unreleased":
clean_smoke["lock_released"] = False
if smoke is not False:
(d / "smoke.json").write_text(json.dumps(
clean_smoke if smoke is None or isinstance(smoke, str) else smoke))
env = dict(os.environ, PATH="%s:%s" % (FAKEGH, os.environ["PATH"]),
FAKEGH_RELEASES=str(d / "releases.json"), FAKEGH_TAGS=tags,
FAKEGH_LOG=str(d / "log"), FAKEGH_UPLOAD=str(d / "upload.json"),
FAKEGH_GATES=str(d / "gates.json"),
FAKEGH_CORRUPT_MANIFEST_READBACK="1" if corrupt_manifest_readback else "0")
env.pop("FRAME_SMOKE_RESULT", None)
if smoke is not False:
env["FRAME_SMOKE_RESULT"] = str(d / "smoke.json")
p = subprocess.run(["sh", str(SCRIPT), *args], env=env, capture_output=True, text=True, timeout=30)
log = [json.loads(line) for line in (d / "log").read_text().splitlines()] if (d / "log").exists() else []
upload = json.loads((d / "upload.json").read_text()) if (d / "upload.json").exists() else None
@@ -156,6 +176,18 @@ class PublishRelease(unittest.TestCase):
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_missing_or_skipped_frame_smoke_before_mutation(self):
for smoke in (False,
{"source": {"commit": "b" * 40, "clean": True}},
{"source": {"commit": "a" * 40, "clean": False}},
"skipped", "unreleased"):
with self.subTest(smoke=smoke):
p, log, upload = self.run_script([draft()], "v9.8.7", smoke=smoke)
self.assertNotEqual(p.returncode, 0)
self.assertIn("release gate failed", p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_a_prior_cancelled_installer_after_successful_rerun(self):
gates = {"release_attempts": 2,
"prior_bad_job": "build (macos-latest, dist, app/dist/*.dmg app/dist/*.zip)",
@@ -167,6 +199,30 @@ class PublishRelease(unittest.TestCase):
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_bad_installer_in_separate_prior_tag_run(self):
for run_conclusion, job_conclusion, expected in (
("failure", "cancelled", "failure"),
("success", "skipped", "skipped")):
with self.subTest(run_conclusion=run_conclusion, job_conclusion=job_conclusion):
gates = {"prior_release_run_bad_job":
"build (macos-latest, dist, app/dist/*.dmg app/dist/*.zip)",
"prior_release_run_conclusion": run_conclusion,
"prior_release_run_bad_conclusion": job_conclusion}
p, log, upload = self.run_script([draft()], "v9.8.7", gates=gates)
self.assertNotEqual(p.returncode, 0)
self.assertIn("run 500", p.stderr)
self.assertIn(expected, p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_incomplete_tag_run_list(self):
p, log, upload = self.run_script([draft()], "v9.8.7",
gates={"release_run_list_incomplete": True})
self.assertNotEqual(p.returncode, 0)
self.assertIn("incomplete", p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_missing_or_nonexact_gate_results(self):
for gates in ({"omit_job": "e2e"}, {"main_sha": "c" * 40},
{"decky_conclusion": "failure"}, {"release_conclusion": "failure"},
+36 -4
View File
@@ -95,6 +95,18 @@ class RealProcesses(unittest.TestCase):
class Safety(unittest.TestCase):
def test_source_revision_records_clean_checkout_and_rejects_invalid_head(self):
head = 'a' * 40
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout=head + '\n'), mock.Mock(stdout='')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': head, 'clean': True})
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout=head + '\n'), mock.Mock(stdout=' M tests/smoke/frame_smoke.py\n')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': head, 'clean': False})
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout='not-a-commit\n'), mock.Mock(stdout='')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': None, 'clean': False})
def test_locked_preflight_rejects_active_or_unknown_proximity_before_titles(self):
for proximity in ([], [88.0625], [4.6875, 88.0625], [None]):
with self.subTest(proximity=proximity), mock.patch.object(
@@ -147,6 +159,8 @@ class Safety(unittest.TestCase):
else:
smoke.step(f'{kind}: launch', lambda: {'outcome': 'running'})
with mock.patch.object(frame_smoke, 'RESULTS', output), \
mock.patch.object(frame_smoke, 'source_revision', return_value={
'commit': 'a' * 40, 'clean': True}), \
mock.patch.object(frame_smoke, 'os_release', return_value={
'BUILD_ID': 'fake', 'VERSION_ID': 'fake', 'VARIANT_ID': 'fake'}), \
mock.patch.object(frame_smoke, 'ssh', return_value='yes'), \
@@ -155,10 +169,14 @@ class Safety(unittest.TestCase):
mock.patch.object(frame_smoke, 'status', return_value={}), \
mock.patch.object(frame_smoke, 'title_cycle', side_effect=cycle), \
mock.patch.object(frame_smoke.tempfile, 'mkdtemp', return_value=str(folder)):
self.assertEqual(frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity': 3, 'proximity': [], 'battery': [100]}), 1)
code, saved_path = frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity': 3, 'proximity': [], 'battery': [100]})
self.assertEqual(code, 1)
result = json.loads(next(output.glob('*-fake.json')).read_text())
self.assertEqual(saved_path.parent, output)
self.assertEqual((result['passed'], result['skipped'], result['failed']), (6, 1, 0))
self.assertEqual(result['source'], {'commit': 'a' * 40, 'clean': True})
self.assertFalse(result['lock_released'])
self.assertEqual(next(s for s in result['steps'] if s['step'] == 'x86_64: launch')
['detail']['outcome'], 'runtime missing')
@@ -207,13 +225,27 @@ class Safety(unittest.TestCase):
mock.patch.object(frame_smoke, 'cleanup', side_effect=RuntimeError('cannot stop')), \
mock.patch.object(frame_smoke, 'title_cycle') as cycle, \
mock.patch('builtins.print'):
self.assertEqual(frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity':3,'proximity':[],'battery':[100]}), 1)
code, _ = frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity':3,'proximity':[],'battery':[100]})
self.assertEqual(code, 1)
cycle.assert_not_called()
self.assertEqual(len(list(Path(folder).glob('*-fake.json'))), 1)
class SharedSmokeLock(unittest.TestCase):
def test_result_marks_release_only_after_owned_lock_is_removed(self):
with tempfile.TemporaryDirectory() as folder:
result_path = Path(folder) / 'smoke.json'
for released in (False, True):
result_path.write_text('{"lock_released": false}')
with mock.patch.object(frame_smoke.sys, 'argv', ['frame_smoke.py']), \
mock.patch.object(frame_smoke, 'acquire_lock', return_value=True), \
mock.patch.object(frame_smoke, 'preflight', return_value={}), \
mock.patch.object(frame_smoke, 'run_smoke', return_value=(0, result_path)), \
mock.patch.object(frame_smoke, 'release_lock', return_value=released):
self.assertEqual(frame_smoke.main(), 0 if released else 1)
self.assertIs(json.loads(result_path.read_text())['lock_released'], released)
def test_busy_lock_prevents_any_status_read_or_title_change(self):
with mock.patch.object(frame_smoke.sys, 'argv', ['frame_smoke.py']), \
mock.patch.object(frame_smoke, 'acquire_lock', return_value=False), \
+62
View File
@@ -0,0 +1,62 @@
"""The release smoke gate requires a complete result from its exact checkout."""
import importlib.util
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
SPEC = importlib.util.spec_from_file_location(
'verify_frame_smoke_result', ROOT / 'scripts' / 'verify-frame-smoke-result.py')
gate = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(gate)
SHA = 'a' * 40
def result():
steps = [{'step': name, 'ok': True} for name in sorted(gate.REQUIRED_STEPS)]
return {'source': {'commit': SHA, 'clean': True}, 'lock_released': True, 'steps': steps,
'passed': len(steps), 'skipped': 0, 'failed': 0,
'preflight': {'activity': 3, 'proximity': [2.3125], 'battery': [100]}}
class VerifyFrameSmokeResult(unittest.TestCase):
def test_accepts_complete_clean_result_from_exact_commit(self):
gate.verify(result(), SHA)
def test_rejects_wrong_commit_dirty_checkout_or_missing_step(self):
for changed in ('commit', 'clean', 'step'):
with self.subTest(changed=changed):
data = result()
if changed == 'commit':
data['source']['commit'] = 'b' * 40
elif changed == 'clean':
data['source']['clean'] = False
else:
data['steps'].pop()
data['passed'] -= 1
with self.assertRaises(ValueError):
gate.verify(data, SHA)
def test_rejects_skipped_launch_even_when_counts_claim_success(self):
data = result()
launch = next(step for step in data['steps'] if step['step'] == 'x86_64: launch')
launch.update(ok=False, skipped=True)
data.update(passed=len(data['steps']) - 1, skipped=1)
with self.assertRaisesRegex(ValueError, 'failed, skipped'):
gate.verify(data, SHA)
def test_rejects_missing_locked_preflight(self):
data = result()
data['preflight']['activity'] = 1
with self.assertRaisesRegex(ValueError, 'preflight'):
gate.verify(data, SHA)
def test_rejects_missing_lock_release_record(self):
data = result()
data['lock_released'] = False
with self.assertRaisesRegex(ValueError, 'lock release'):
gate.verify(data, SHA)
if __name__ == '__main__':
unittest.main()
+6 -4
View File
@@ -1479,12 +1479,13 @@
"https://github.com/isiah-lloyd/radiyo-steam-deck"
],
"status": "partial",
"reason": "Initializer succeeds; radio discovery and playback remain unverified.",
"reason": "Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame"
"fake-Frame",
"isolated Decky 3.3.0 x86_64 backend"
],
"browser": {
"id": 24,
@@ -8879,12 +8880,13 @@
"https://github.com/GedasFX/decky-cloud-save"
],
"status": "partial",
"reason": "Backend get_config is intentionally rejected by the audit before initialization completes; actual backend compatibility is unknown.",
"reason": "Exact official backend discovers under Decky 3.3.0 amd64; default get_config and a private setting read/write survive a loader restart. The isolated browser baseline still stops at its denied get_config call. Cloud login, rclone download/sync (including ARM64), process signals, real Frame and VR are unverified. The published updater names an amd64 rclone download, so no ARM64 sync claim follows.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame"
"fake-Frame",
"native Decky amd64 configuration"
],
"browser": {
"id": 104,