test: include RadiYo native probe in runtime inventory

Reject earlier failed builds for a release tag

docs/evidence/framey-radiyo-native-2026-10-29.md

@@ -0,0 +1,24 @@
+# RadiYo! 1.3 isolated backend check, 29 October 2026
+
+The official Decky store archive for RadiYo! (ID 24) is 33,686 bytes with
+SHA-256 `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43`.
+Its unmodified ZIP is pinned as `tests/fixtures/framey-official/24.zip` and
+passes Frame Control's archive validation and fake Frame panel packaging.
+
+`python3 scripts/framey-test-decky-runtime.py --case 24 --output /tmp/framey-radiyo-decky-runtime-20261029.json`
+ran the published Decky 3.3.0 x86_64 loader, digest
+`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`,
+in a disposable container with `--network none`, no host devices, dropped
+capabilities and a private home. The real loader discovered the exact plugin
+and dispatched its legacy `fetch` and `getRedirectUrl` RPCs. A server bound to
+container loopback returned a fixed station marker: regular fetch returned it,
+custom-station fetch omitted the body as upstream intends, and a redirect
+resolved to the same fixture. The runner removed its container and image.
+
+The previously recorded opaque-iframe browser audit showed frontend
+initialization, but denied backend and network calls. This native probe adds
+backend transport evidence only. No external radio station, HTTPS endpoint,
+audio playback, real Frame service, VR visibility or controller was tested.
+The published backend sets `check_hostname=False` and `CERT_NONE` for its
+outbound HTTPS requests; the loopback HTTP test did not exercise that path.
+RadiYo! remains **partial**. The compatibility count does not change.

docs/framey-compatibility.md

@@ -56,7 +56,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
 | 21: PowerTools 2.0.3 | legacy / legacy | N/A | Not supported on Frame: Targets AMD Ryzen power controls; Frame uses an ARM64 SoC. | static archive, local browser, fake-Frame |
 | 22: Memory Deck 0.1.8 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: memory-deck/bin/libscanmem.so.1.0.0. | static archive, local browser, fake-Frame |
 | 23: Animation Changer 1.3.2 | legacy / legacy | partial | Initializer succeeds; forced backend failure prevents loading animations. Boot-video changes untested. | static archive, local browser, fake-Frame |
-| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds; radio discovery and playback remain unverified. | static archive, local browser, fake-Frame |
+| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made. | static archive, local browser, fake-Frame, isolated Decky 3.3.0 x86_64 backend |
 | 26: HLTB for Deck 2.0.12 | 1 / 2 | partial | Settings render without the prior IndexedDB SecurityError; Steam library badges have no patch target, and live lookup is unverified. | static archive, local browser, fake-Frame |
 | 27: AutoSuspend 2.2.0 | legacy / legacy | broken | Frontend audit failed: Cannot read properties of null (reading '6'). The exact frontend also contains SteamOS suspend and shutdown paths; threshold behavior and safe Frame operation are unverified. | static archive, local browser, fake-Frame |
 | 28: Emuchievements 2.0.5 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: Emuchievements/bin/hash. | static archive, local browser, fake-Frame |

docs/releasing.md

@@ -74,9 +74,9 @@ count. So a build reaches people only when you publish it, after testing it.
 
    The script checks that the tag is on GitHub, that exact-commit main CI
    (including e2e), native amd64/ARM64 Decky runtime checks, and the tagged
-   release workflow are green with no failed,
-   cancelled, skipped or running jobs in any run attempt, and that all ten installers
-   are attached, each with the SHA-256 digest GitHub records. It also checks
+   release workflow are green with no failed, cancelled, skipped or running
+   jobs in any exact-tag push run or attempt, and that all ten installers are
+   attached, each with the SHA-256 digest GitHub records. It also checks
    that the draft does not begin with a `WITHDRAWN` notice, so the preserved
    v0.4.48 draft cannot accidentally be offered again. It checks
    that the uploaded `SHA256SUMS` digest matches a fresh list of those ten

scripts/framey-test-decky-runtime.py

@@ -70,10 +70,15 @@ def main():
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument('--loader', type=Path, help='Optional cached, digest-verified native Decky 3.3.0 binary')
     parser.add_argument('--output', type=Path, required=True)
+    parser.add_argument('--case', type=int, help='Run one official plugin case locally; CI runs all cases')
     args = parser.parse_args()
     if sys.platform != 'linux' or not shutil.which('docker'):
         parser.error('Linux with Docker is required')
     cases = check_coverage()
+    if args.case is not None:
+        cases = [case for case in cases if case['id'] == args.case]
+        if not cases:
+            parser.error('Unknown official plugin case')
     data, digest = loader_bytes(args.loader)
     tag = 'framey-decky-runtime:'+uuid.uuid4().hex[:12]
     report = {'decky': RELEASE, 'architecture': platform.machine(), 'loaderSha256': digest, 'cases': []}
@@ -85,8 +90,10 @@ def main():
             loader.write_bytes(data)
             loader.chmod(0o644)  # Private container user copies/chmods its own executable.
             probes = [(case['id'], None) for case in cases] + [
-                (111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
-                (60, '--reproduce-fileserver-empty-flag-regression')]
+                (plugin_id, negative) for plugin_id, negative in [
+                    (111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
+                    (60, '--reproduce-fileserver-empty-flag-regression')]
+                if args.case is None or args.case == plugin_id]
             for plugin_id, negative in probes:
                 name = 'framey-decky-'+uuid.uuid4().hex[:12]
                 command = ['docker', 'run', '--rm', '--name', name, '--network', 'none', '--read-only',

scripts/publish-release.sh

@@ -33,7 +33,7 @@ gh api "repos/$repo/git/ref/tags/$tag" >/dev/null 2>&1 \
 
 # A complete asset list is insufficient: a cancelled installer build can leave
 # old uploaded bytes behind. Check the exact tagged commit's main/e2e jobs and
-# the latest exact-tag release attempt before any release mutation.
+# every exact-tag push run and attempt before any release mutation.
 python3 "$(dirname "$0")/verify-release-gates.py" "$tag" "${FRAME_SMOKE_RESULT:-}"
 
 # Every release, drafts included, one JSON object per line.

scripts/verify-release-gates.py

@@ -61,14 +61,29 @@ def tag_commit(tag):
     raise GateError(f"{tag} has too many nested tag objects")
 
 
-def exact_run(workflow, branch, sha, event):
+def exact_runs(workflow, branch, sha, event):
     query = f"actions/workflows/{workflow}/runs?branch={quote(branch, safe='')}&head_sha={sha}&per_page=100"
-    runs = api(query).get("workflow_runs") or []
+    runs = []
+    for page in range(1, 21):
+        data = api(f"{query}&page={page}")
+        batch = data.get("workflow_runs")
+        total = data.get("total_count")
+        if not isinstance(batch, list) or type(total) is not int or total < 0:
+            raise GateError(f"invalid {workflow} run list for {branch} at {sha}")
+        runs.extend(batch)
+        if len(runs) >= total or len(batch) < 100:
+            break
+    if len(runs) != total or len({run.get('id') for run in runs}) != len(runs):
+        raise GateError(f"incomplete or duplicate {workflow} run list for {branch} at {sha}")
     matching = [run for run in runs if run.get("head_sha") == sha
                 and run.get("head_branch") == branch and run.get("event") == event]
     if not matching:
         raise GateError(f"no exact {workflow} {event} run for {branch} at {sha}")
-    run = max(matching, key=lambda item: item["id"])
+    return sorted(matching, key=lambda item: item["id"])
+
+
+def exact_run(workflow, branch, sha, event):
+    run = exact_runs(workflow, branch, sha, event)[-1]
     if run.get("status") != "completed" or run.get("conclusion") != "success":
         raise GateError(f"{workflow} run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
     return run
@@ -107,7 +122,12 @@ def main(tag, smoke_result):
     sha = tag_commit(tag)
     successful_jobs(exact_run("checks.yml", "main", sha, "push"), MAIN_JOBS)
     successful_jobs(exact_run("decky-runtime.yml", "main", sha, "push"), DECKY_JOBS)
-    successful_jobs(exact_run("release.yml", tag, sha, "push"), RELEASE_JOBS)
+    # A fresh run must not hide a failed, cancelled, or skipped installer in an
+    # earlier separate push run for the same tag and commit.
+    for run in exact_runs("release.yml", tag, sha, "push"):
+        if run.get("status") != "completed" or run.get("conclusion") != "success":
+            raise GateError(f"release.yml run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
+        successful_jobs(run, RELEASE_JOBS)
     command = [sys.executable, str(Path(__file__).with_name("verify-frame-smoke-result.py")),
                smoke_result, sha]
     checked = subprocess.run(command, text=True, capture_output=True, check=False)

tests/decky-runtime/cases.json

@@ -14,6 +14,11 @@
     "shim": null,
     "probe": "toggle-settings"
   },
+  {
+    "id": 24,
+    "shim": null,
+    "probe": "radio-local"
+  },
   {
     "id": 63,
     "shim": null,

tests/decky-runtime/check.py

@@ -2,6 +2,7 @@
 import argparse
 import asyncio
 import hashlib
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 import json
 import os
 from pathlib import Path
@@ -11,6 +12,7 @@ import socket
 import ssl
 import subprocess
 import sys
+import threading
 import time
 import urllib.request
 import zipfile
@@ -265,6 +267,49 @@ def main():
                 assert current('read_deals') == {}
                 report['result'] = ('discovery, default settings, empty local deals, '
                     'private setting write/read and restart persistence passed; feed, notifications and VR unverified')
+            elif case['probe'] == 'radio-local':
+                # The published backend accepts arbitrary URLs and disables
+                # HTTPS verification. Exercise only a loopback fixture inside
+                # this network-isolated container; no station or audio opens.
+                marker = 'Framey private radio station\n'
+
+                class RadioHandler(BaseHTTPRequestHandler):
+                    def do_GET(self):
+                        if self.path == '/redirect':
+                            self.send_response(302)
+                            self.send_header('Location', '/station')
+                            self.end_headers()
+                        elif self.path == '/station':
+                            data = marker.encode()
+                            self.send_response(200)
+                            self.send_header('Content-Type', 'text/plain')
+                            self.send_header('Content-Length', str(len(data)))
+                            self.end_headers()
+                            self.wfile.write(data)
+                        else:
+                            self.send_error(404)
+
+                    def log_message(self, *_):
+                        pass
+
+                server = ThreadingHTTPServer(('127.0.0.1', 0), RadioHandler)
+                worker = threading.Thread(target=server.serve_forever, daemon=True)
+                worker.start()
+                try:
+                    base_url = f'http://127.0.0.1:{server.server_port}'
+                    station = legacy('fetch', url=base_url+'/station', customStation=False)
+                    assert station['status'] == 200 and station['body'] == marker, station
+                    assert station['headers']['Content-Type'] == 'text/plain', station
+                    custom = legacy('fetch', url=base_url+'/station', customStation=True)
+                    assert custom['status'] == 200 and custom['body'] == '', custom
+                    redirect = legacy('getRedirectUrl', url=base_url+'/redirect')
+                    assert redirect == {'response': base_url+'/station'}, redirect
+                finally:
+                    server.shutdown()
+                    server.server_close()
+                    worker.join(timeout=3)
+                report['result'] = ('actual Decky discovery and local station fetch/redirect passed; '
+                    'external radio, HTTPS, audio playback, Frame and VR unverified')
             elif case['probe'] == 'toggle-settings':
                 def call(method, *values):
                     result = rpc('loader/call_plugin_method', record['name'], method, *values)

tests/fakegh/gh

@@ -96,15 +96,26 @@ elif method == "GET" and endpoint.startswith(repo + "actions/workflows/"):
            "status": gate.get("decky_status" if run_id == 503 else ("main_status" if is_main else "release_status"), "completed"),
            "conclusion": gate.get("decky_conclusion" if run_id == 503 else ("main_conclusion" if is_main else "release_conclusion"), "success"),
            "html_url": "https://github.com/saphid/frame-control/actions/runs/%s" % run_id}
-    print(json.dumps({"workflow_runs": [run]}))
+    runs = [run]
+    if run_id == 502 and gate.get("prior_release_run_bad_job"):
+        runs.append(dict(run, id=500, run_attempt=1, status="completed",
+                         conclusion=gate.get("prior_release_run_conclusion", "failure"),
+                         html_url="https://github.com/saphid/frame-control/actions/runs/500"))
+    if run_id == 502 and gate.get("release_run_list_incomplete"):
+        print(json.dumps({"total_count": len(runs) + 1, "workflow_runs": runs}))
+    else:
+        print(json.dumps({"total_count": len(runs), "workflow_runs": runs}))
 elif method == "GET" and endpoint.startswith(repo + "actions/runs/"):
     parts = endpoint[len(repo + "actions/runs/"):].split("/")
     run_id = parts[0]
     attempt = int(parts[2])
-    names = {"501": main_jobs, "502": release_jobs, "503": decky_jobs}[run_id]
+    names = {"500": release_jobs, "501": main_jobs, "502": release_jobs,
+             "503": decky_jobs}[run_id]
     names = [name for name in names if name != gate.get("omit_job")]
     jobs = [{"name": name, "status": "completed",
-             "conclusion": (gate.get("prior_bad_job_conclusion", "failure")
+             "conclusion": (gate.get("prior_release_run_bad_conclusion", "cancelled")
+                            if run_id == "500" and name == gate.get("prior_release_run_bad_job")
+                            else gate.get("prior_bad_job_conclusion", "failure")
                             if run_id == "502" and attempt == 1 and name == gate.get("prior_bad_job")
                             else gate.get("bad_job_conclusion", "success") if name == gate.get("bad_job")
                             else "success")}

tests/fixtures/framey-official/README.md

@@ -3,7 +3,7 @@
 These are unmodified ZIPs downloaded from the official Decky store CDN on
 2026-10-25, except `96.zip` and `111.zip` downloaded on 2026-10-26 and `60.zip` on
 2026-10-27; `13.zip` and `125.zip` were added from the digest-verified audit cache on 2026-10-27,
-and `67.zip` and `114.zip` on 2026-10-28.
+and `67.zip` and `114.zip` on 2026-10-28. `24.zip` was added on 2026-10-29.
 Each ZIP includes its upstream `LICENSE`. They are test data, not plugins shipped or installed by
 Frame Control. The fake Frame tests serve them
 from a local fixture so CI does not depend on the store being online.
@@ -11,6 +11,7 @@ from a local fixture so CI does not depend on the store being online.
 | File | Official plugin | Version | Store SHA-256 |
 | --- | --- | --- | --- |
 | `13.zip` | Pause Games | 1.0.3 | `4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62` |
+| `24.zip` | RadiYo! | 1.3 | `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43` |
 | `60.zip` | DeckyFileServer | 1.2.0 | `7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0` |
 | `125.zip` | Decky Localsend | 0.4.5 | `e271ccb55eb23f918b99a69f79aec905df4c9b2ef7b23bbf4ec1c2d1a6f0c6b7` |
 | `114.zip` | Network Sentinel | 1.0.4 | `d3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5` |
@@ -60,6 +61,9 @@ Pause Games runs through actual published Decky 3.3.0 discovery and instance RPC
 for private settings and a synthetic exclusion list. No process-control method
 is called. The official frontend's standalone Toggle component separately
 checks saved true/false state and matching settings callbacks; effects are inert.
+RadiYo! runs through actual published Decky 3.3.0 legacy RPC in a network-isolated
+container. Only loopback station fetch, custom-station fetch and redirect are
+tested. External radio, HTTPS and audio playback remain unverified.
 Network Sentinel runs through the same isolated native Decky gate. Its exact
 archive gets plugin-local psutil and a user-service manifest; the probe checks
 discovery, private settings and one monitoring-loop sample without external

tests/framey_official.py

@@ -10,6 +10,8 @@ PLUGINS = (
      'folder': 'network-sentinel', 'sha256': 'd3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5'},
     {'id': 13, 'name': 'Pause Games', 'version': '1.0.3', 'api_version': 1,
      'folder': 'SDH-PauseGames', 'sha256': '4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62'},
+    {'id': 24, 'name': 'RadiYo!', 'version': '1.3', 'api_version': 'legacy',
+     'folder': 'radiyo-steam-deck', 'sha256': '647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43'},
     {'id': 60, 'name': 'DeckyFileServer', 'version': '1.2.0', 'api_version': 1,
      'folder': 'DeckyFileServer', 'sha256': '7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0'},
     {'id': 125, 'name': 'Decky Localsend', 'version': '0.4.5', 'api_version': 1,

tests/test_framey_decky_runtime.py

@@ -21,7 +21,7 @@ class RuntimeCoverage(unittest.TestCase):
             self.assertIn(case['id'], official)
             self.assertIn(case['probe'], ('mpris', 'screenshots', 'system-monitor', 'timer-lifecycle',
                                          'toggle-settings', 'fileserver', 'localsend', 'network-sentinel',
-                                         'battery-tracker', 'free-loader'))
+                                         'battery-tracker', 'free-loader', 'radio-local'))
 
     def test_new_shim_without_runtime_case_fails_coverage(self):
         import frame_decky_store

tests/test_publish_release.py

@@ -199,6 +199,30 @@ class PublishRelease(unittest.TestCase):
         self.assertIsNone(upload)
         self.assertFalse(any("-X" in call for call in log))
 
+    def test_refuses_bad_installer_in_separate_prior_tag_run(self):
+        for run_conclusion, job_conclusion, expected in (
+                ("failure", "cancelled", "failure"),
+                ("success", "skipped", "skipped")):
+            with self.subTest(run_conclusion=run_conclusion, job_conclusion=job_conclusion):
+                gates = {"prior_release_run_bad_job":
+                         "build (macos-latest, dist, app/dist/*.dmg app/dist/*.zip)",
+                         "prior_release_run_conclusion": run_conclusion,
+                         "prior_release_run_bad_conclusion": job_conclusion}
+                p, log, upload = self.run_script([draft()], "v9.8.7", gates=gates)
+                self.assertNotEqual(p.returncode, 0)
+                self.assertIn("run 500", p.stderr)
+                self.assertIn(expected, p.stderr)
+                self.assertIsNone(upload)
+                self.assertFalse(any("-X" in call for call in log))
+
+    def test_refuses_incomplete_tag_run_list(self):
+        p, log, upload = self.run_script([draft()], "v9.8.7",
+                                         gates={"release_run_list_incomplete": True})
+        self.assertNotEqual(p.returncode, 0)
+        self.assertIn("incomplete", p.stderr)
+        self.assertIsNone(upload)
+        self.assertFalse(any("-X" in call for call in log))
+
     def test_refuses_missing_or_nonexact_gate_results(self):
         for gates in ({"omit_job": "e2e"}, {"main_sha": "c" * 40},
                       {"decky_conclusion": "failure"}, {"release_conclusion": "failure"},

ui/framey-compatibility.json

@@ -1479,12 +1479,13 @@
         "https://github.com/isiah-lloyd/radiyo-steam-deck"
       ],
       "status": "partial",
-      "reason": "Initializer succeeds; radio discovery and playback remain unverified.",
+      "reason": "Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made.",
       "blockInstall": false,
       "verification": [
         "static archive",
         "local browser",
-        "fake-Frame"
+        "fake-Frame",
+        "isolated Decky 3.3.0 x86_64 backend"
       ],
       "browser": {
         "id": 24,