Compare commits

...
Author SHA1 Message Date
SAPHID 35b59e62da Document Frame smoke sensor gate evidence 2026-10-29 01:29:48 +00:00
SAPHID be07dbeeeb Reject unreadable Frame smoke sensor values 2026-10-29 01:25:26 +00:00
Alex Southwell b123f5ff34 Merge pull request #399 from saphid/test/framey-gamesync-native
Verify Game Sync configuration in native Decky
2026-10-29 01:07:59 +00:00
SAPHID 08c99c2d7d docs: scope Game Sync ARM64 limit to cloud sync 2026-10-29 01:00:28 +00:00
SAPHID b3e3e21f46 test: remove unverified rclone path assertion 2026-10-29 00:58:58 +00:00
SAPHID 84bb7fa0b1 test: verify Game Sync configuration in native Decky 2026-10-29 00:55:54 +00:00
Alex Southwell 01e5d8b2db Merge pull request #398 from saphid/test/framey-radiyo-native
Verify RadiYo backend through isolated Decky
2026-10-29 00:49:26 +00:00
SAPHID 130d82128b test: include RadiYo native probe in runtime inventory 2026-10-29 00:38:22 +00:00
SAPHID 65e629f8c3 Verify RadiYo backend through isolated native Decky 2026-10-29 00:32:26 +00:00
Alex Southwell e33ed00945 Merge pull request #397 from saphid/fix/release-all-tag-runs
Reject earlier failed builds for a release tag
2026-10-29 00:09:11 +00:00
SAPHID 419d69ef8a Reject prior failed tagged release runs before publishing 2026-10-28 23:58:30 +00:00
Alex Southwell 8cf2fb9da3 Merge pull request #395 from saphid/ci/frame-smoke-source-commit
Bind Frame smoke evidence to release commit and lock cleanup
2026-10-28 23:39:18 +00:00
SAPHID ff2eb87ac1 Merge remote-tracking branch 'origin/main' into ci/frame-smoke-source-commit 2026-10-28 23:30:51 +00:00
Alex SouthwellandSAPHID c611f4aeb7 Run live Windows wireless inventory only in dedicated CI job (#396)
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-28 23:30:18 +00:00
SAPHID f981ba9315 Require exact locked Frame smoke before publication 2026-10-28 23:19:35 +00:00
SAPHID 014e6b3531 Merge remote-tracking branch 'origin/main' into ci/frame-smoke-source-commit 2026-10-28 23:15:37 +00:00
SAPHID 8b6c32ba37 Bind Frame smoke evidence to release commit and lock cleanup 2026-10-28 23:15:01 +00:00
Alex SouthwellandSAPHID 96294f5137 Clarify two official Framey compatibility limits (#394)
Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-28 23:13:01 +00:00
Alex SouthwellandSAPHID 6f544474ce Document v0.4.54 x86 runtime blocker (#393)
* Document v0.4.54 x86 runtime blocker

* Clarify FEX wrapper runtime dependency

---------

Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-28 22:54:41 +00:00
Alex SouthwellandSAPHID b40bf89565 Test Shotty screenshot copies through native Decky RPC (#392)
* Test Shotty screenshot copies through native Decky RPC

* Clarify local Shotty RPC evidence limit

---------

Co-authored-by: SAPHID <saphid@users.noreply.github.com>
2026-10-28 22:35:12 +00:00
Alex Southwell 760c89fec8 Merge pull request #391 from saphid/docs/framey-shotty-native-evidence
Align native Decky evidence for four official plugins
2026-10-28 22:23:51 +00:00
SAPHID 584970b82f Align native Decky evidence for managed plugins 2026-10-28 22:11:14 +00:00
SAPHID d319c5ef4d Correct Shotty native Decky compatibility evidence 2026-10-28 22:04:02 +00:00
Alex Southwell dad5f78a02 Merge pull request #390 from saphid/test/framey-e2e-isolation
Clear managed Decky state between fake Frame tests
2026-10-28 21:58:29 +00:00
33 changed files with 697 additions and 67 deletions

No files matched your search

+5
View File
@@ -77,6 +77,11 @@ jobs:
- name: OBS lifecycle tests
run: node app/test/obs-source.test.js
- name: Server tests
# The dedicated wireless-windows job runs the live inventory on a fresh
# Windows runner; do not repeat it after the full server suite has
# consumed this runner's probe budget.
env:
FRAME_SKIP_WINDOWS_WIRELESS_LIVE: "1"
run: python -m unittest discover -s tests -v
# Fast, PC-only Windows query validation, without capture libraries or a headset.
@@ -0,0 +1,20 @@
# Frame smoke sensor validation, 29 October 2026
The locked real Frame smoke reads SteamVR activity, IIO proximity and battery
before installing a test title. The proximity reader converts sensor text to a
Python float. Python accepts `NaN` and infinity as floats, while range
comparisons with `NaN` do not reject it. Before this change, a proximity array
containing `NaN` could pass the standby preflight and the saved-result release
verifier.
Focused local tests supplied `NaN`, positive and negative infinity to both
the live preflight parser and the release-result verifier. They also supplied
boolean, nonfinite, below-threshold and above-100 battery values. Every invalid
sample was rejected. Valid standby, finite proximity and 99–100% integer
battery samples continued to pass. The 25 smoke/cleanup tests, six result-gate
tests and 16 publisher tests passed locally.
This evidence uses simulated sensor responses. It does not record a real Frame
smoke, a worn-headset check, or controller input. Release publication still
requires a complete locked smoke from the exact clean release commit, including
the x86 launch that was skipped in the held v0.4.54 candidate.
@@ -9,3 +9,13 @@ Under a unique owner token on `/tmp/frame-test.lock`, a **read-only** real Frame
In network/device-isolated, unprivileged **actual Decky 3.3.0 amd64**, the exact adapted archive was discovered. A private Frame-shaped supply fixture produced history and per-app power through public legacy RPC. The complete native regression runner, including existing negative controls, passed. Local managed install/remove tests and the exact-source/manifest guard passed. CI's native ARM64 and fake Frame e2e jobs remain required before merge.
This supports **partial** compatibility. The official frontend initialized in the earlier opaque Chromium audit with backend calls denied; it has not been tested with these live samples. Actual managed recording on Frame ARM64, long-term history, foreground-app attribution, panel controls, and worn VR remain unverified. Existing copies need update or reinstall.
## Hosted ARM64 follow-up
The [main native Decky run](https://github.com/saphid/frame-control/actions/runs/40776292569)
on `dad5f78a02dd477a689689865cd4ef4205b87e22` passed the exact Battery
Tracker case on native amd64 **and ARM64** runners. Both logs report `PASS 67`
for discovery, Frame-shaped private battery samples, history and per-app power
through actual Decky 3.3.0 RPC. These were disposable fixture supplies inside
isolated containers. The managed plugin still has not recorded a real Frame
battery sample; the remaining limits above still apply.
@@ -23,3 +23,13 @@ notification delivery, real Frame ARM64 service, rendered deal cards, worn VR
or controller interaction. The earlier opaque Chromium check had denied backend
and network calls, so it could not establish that deal cards render with live
data. No headset was touched for this check.
## Hosted ARM64 follow-up
The [main native Decky run](https://github.com/saphid/frame-control/actions/runs/40776292569)
on `dad5f78a02dd477a689689865cd4ef4205b87e22` passed the exact Free Loader
case on native amd64 **and ARM64** runners. Both logs report `PASS 63` for
discovery, default settings, empty local deals, private setting write/read and
restart persistence. The runners used separate network and device isolated
containers; no feed was fetched and no real Frame service was started. The
remaining partial-compatibility limits above still apply.
@@ -0,0 +1,24 @@
# Game Sync 1.0.4 isolated backend check, 29 October 2026
The official Decky archive for Game Sync (ID 104) is 2,228,806 bytes with
SHA-256 `65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd`.
Its unmodified ZIP is pinned as `tests/fixtures/framey-official/104.zip`.
`python3 scripts/framey-test-decky-runtime.py --case 104 --output /tmp/framey-gamesync-native-104.json`
passed with the published Decky 3.3.0 x86_64 loader, SHA-256
`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`.
The disposable container had no external network or host devices, dropped
capabilities and used a private home. The real loader discovered the exact
plugin. Its `get_config` returned defaults with automatic game sync and
capture upload disabled. A private `advanced_mode` setting changed from false
to true and remained true after the loader restarted. The runner removed its
container and image; a separate `docker ps` and image-list audit found no
owned resources.
The original opaque-iframe browser audit blocked `get_config` before panel
initialization completed, so it did not prove a usable frontend. This native
test confirms only a configuration backend path. No cloud authentication,
rclone download or sync, screenshot deletion, process signal, game event,
real Frame backend, VR visibility or physical controller was exercised.
The published `rclone_manager.py` names an amd64 rclone download, so this
result does not establish ARM64 sync support. Game Sync remains **partial**.
@@ -9,3 +9,13 @@ The managed install checks the complete archive digest, manifest and Python sour
Local transaction tests installed and removed the exact archive, checked the native wheel and unmodified license, and rejected an altered archive. In network/device-isolated, unprivileged **actual Decky 3.3.0 amd64**, the managed plugin was discovered, imported native psutil, returned default and updated settings, reported empty history and bandwidth, started its monitoring loop, produced a history sample with quality and bandwidth fields, then stopped. The full native regression runner also passed its other cases, including FileServer and negative controls. The first run's existing large FileServer install timed out at 15 seconds under host load; its bounded timeout is now 30 seconds and the complete rerun passed. No external network call succeeded in the isolated container. Native ARM64 and fake Frame install/remove are CI gates for this change; neither is claimed by this local evidence.
The preexisting local Chromium audit initialized the exact frontend with backend calls denied. It did not prove a functioning panel. External latency/DNS, Wi-Fi scans, actual Frame ARM64 runtime, frontend controls, and worn VR remain unverified. This is a **partial** compatibility result, not a full monitoring or headset-usability claim. No real Frame, normal service, power value, or physical network setting was touched in this check.
## Hosted ARM64 follow-up
The [main native Decky run](https://github.com/saphid/frame-control/actions/runs/40776292569)
on `dad5f78a02dd477a689689865cd4ef4205b87e22` passed the exact Network
Sentinel case on native amd64 **and ARM64** runners. Both logs report `PASS 114`
for discovery, native psutil import, private settings and one isolated
monitoring-loop sample through actual Decky 3.3.0 RPC. The containers had no
external network or Frame device access. Real Frame monitoring, external
latency/DNS, panel controls and worn VR remain unverified.
@@ -0,0 +1,24 @@
# RadiYo! 1.3 isolated backend check, 29 October 2026
The official Decky store archive for RadiYo! (ID 24) is 33,686 bytes with
SHA-256 `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43`.
Its unmodified ZIP is pinned as `tests/fixtures/framey-official/24.zip` and
passes Frame Control's archive validation and fake Frame panel packaging.
`python3 scripts/framey-test-decky-runtime.py --case 24 --output /tmp/framey-radiyo-decky-runtime-20261029.json`
ran the published Decky 3.3.0 x86_64 loader, digest
`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`,
in a disposable container with `--network none`, no host devices, dropped
capabilities and a private home. The real loader discovered the exact plugin
and dispatched its legacy `fetch` and `getRedirectUrl` RPCs. A server bound to
container loopback returned a fixed station marker: regular fetch returned it,
custom-station fetch omitted the body as upstream intends, and a redirect
resolved to the same fixture. The runner removed its container and image.
The previously recorded opaque-iframe browser audit showed frontend
initialization, but denied backend and network calls. This native probe adds
backend transport evidence only. No external radio station, HTTPS endpoint,
audio playback, real Frame service, VR visibility or controller was tested.
The published backend sets `check_hostname=False` and `CERT_NONE` for its
outbound HTTPS requests; the loopback HTTP test did not exercise that path.
RadiYo! remains **partial**. The compatibility count does not change.
@@ -0,0 +1,28 @@
# Shotty native Decky backend check, 28 October 2026
The exact official Shotty 0.1.0 archive (store ID 53, SHA-256
`ab7f6618b264a0cb766525e788cef6c3e1478c0fef98a701225e5e8af441e2fe`)
receives Frame Control's digest-scoped `shotty-click-app-dir` module during a
managed install. The [earlier isolated Python check](framey-shotty-python-2026-10-26.md)
copied one synthetic screenshot into a temporary `Pictures` folder.
The [main native Decky run](https://github.com/saphid/frame-control/actions/runs/40776292569)
on commit `dad5f78a02dd477a689689865cd4ef4205b87e22` passed Shotty's
`tests/decky-runtime/cases.json` case on both native amd64 and ARM64 runners.
The test used the digest-verified published Decky 3.3.0 loader and exact archive
in separate disposable, network and device isolated containers. It installed
the managed archive, checked the shim receipt and plugin discovery, then called
the legacy backend methods through Decky's real websocket RPC:
| Runner | Observed result |
| --- | --- |
| Ubuntu amd64 | `was_rescued` returned `false`; `aggregate_all(allapps=[])` returned `0` |
| Ubuntu ARM64 | `was_rescued` returned `false`; `aggregate_all(allapps=[])` returned `0` |
Both job logs report `PASS 53: discovery, backend import and empty screenshot
aggregation passed`. The containers were removed by the test runner. This
corrects the prior catalog statement that actual Decky ARM64 backend loading
was unverified. The empty RPCs do not exercise a real screenshot or the
continuous rescuer. The managed backend on a real Frame, live screenshot
notifications and copies, wearer-visible panel and physical controllers remain
unverified, so Shotty stays **partial**.
@@ -0,0 +1,27 @@
# Shotty screenshot copy through native Decky RPC, 28 October 2026
The exact official Shotty 0.1.0 archive (store ID 53, SHA-256
`ab7f6618b264a0cb766525e788cef6c3e1478c0fef98a701225e5e8af441e2fe`)
was installed with Frame Control's digest-scoped `shotty-click-app-dir` shim.
The local `scripts/framey-test-decky-runtime.py` run used the published,
digest-verified Decky 3.3.0 amd64 loader in a disposable unprivileged container
with no network, host devices or writable host mount. Its full 14-case native
suite, including three negative controls, exited successfully.
The Shotty case placed two small marker files under a private synthetic Steam
screenshots directory in the container's temporary home. Calls went through
Decky's real legacy websocket RPC. `set_id_map_fronend` supplied a private game
name; `copy_screenshot` returned `true` for the selected file. Independent
reads of `Pictures/Screenshots/Private Game/selected.jpg` and
`Pictures/Screenshots/most_recent.jpg` matched the marker bytes. A later
`aggregate_all` returned `1`, and the historical file in the named folder
also matched its source bytes. Empty aggregation and `was_rescued=false` still
passed. The runner removed its owned containers and image; no headset or user
screenshot was accessed.
This local run checks the real Decky RPC and same-filesystem copy path with
synthetic files on amd64. The PR's hosted native ARM64 gate must pass before
merge; this local result alone does not establish ARM64 behavior. Steam's live
screenshot notification, cross-filesystem copies, the continuous rescuer, real
Frame installation, wearer-visible VR and physical controller use remain
unverified. Shotty remains **partial**.
@@ -0,0 +1,27 @@
# Static limits for two official Decky archives, 28 October 2026
The published archives below were downloaded from Decky's version CDN and
checked against the catalog's exact SHA-256 before inspection. No plugin code,
backend, plugin-initiated network request or headset operation ran in this
audit. Their status labels remain unchanged.
## IsThereAnyDeal for Deck 1.1.1
Store ID 81, SHA-256
`18e133b2e6203f16fd1ee2ed83b5a90bad34491515bef7ec1c82d525f0bfadef`.
The backend exposes settings load/save. The published frontend uses
`fetchNoCors` for deal lookups with an embedded upstream API key and attempts
to find a Steam store debugger tab through `localhost:8080/json`. The key was
not decoded or used. The existing real Frame country read and local settings
render remain the verified scope; live deals, settings writes and Steam page
integration remain unverified. Status stays **partial**.
## Decky Clipboard 1.0.1
Store ID 119, SHA-256
`d283bd9a5c35b3fac3788391736a6763f3e8c4615831a7c560c9fa6e6c0950ac`.
The manifest requests root and the ZIP bundles an x86-64 `bin/xclip`. In the
published `main.py`, `auto_start` defaults to true; `WebServer.start` binds
`0.0.0.0:8765`, and the clipboard GET/POST routes have no authentication
check. A managed adaptation would need to address that behavior before any
Frame install. The archive remains **broken** and was not installed or run.
@@ -22,6 +22,17 @@ smoke title directories or owned test processes. It did not wake the Frame or
send input. Nobody confirmed wearing the Frame, so no wearer-visible or
physical-controller result follows.
Follow-up probes at about 22:40–22:45 UTC each acquired a new unique owner in
the shared headset lock before reading status. SteamVR stayed at activity 3
(standby), proximity was 2.3125–2.375 and battery was 100%. A bounded,
read-only `FEXBash --help` probe reported that `SteamLinuxRuntime_4` did not
exist in Steam's common directory. A separate read of `/usr/bin/FEXBash` found
that its wrapper exits when that directory is absent. The probes did not
install a runtime, launch a title, wake the headset or send input. Each owner
released its lock. The available FEX wrapper therefore did not provide an
immediate x86 Linux launch path; this does not establish that every possible
runtime source is absent.
**Release decision: hold v0.4.54.** The smoke script at the candidate commit
returned exit 0 even though it recorded a skipped launch. Under the current
release rule, a skip cannot satisfy the smoke gate. The accompanying change
+11 -11
View File
@@ -25,13 +25,13 @@ python3 scripts/framey-audit-browser.py --cache /tmp/framey-archives
Open the browser audit URL in local Chromium, run `runAudit()` and inspect/download `results`. All backend/network actions remain denied.
A [Free Loader duplicate-entry follow-up](evidence/framey-free-loader-duplicates-2026-10-26.md) accepts only byte-identical duplicate ZIP members whose modes differ at most in execute bits. The exact archive passed private local and focused fake-Frame install/remove; its frontend rendered in opaque Chromium. The earlier full 116-plugin fake-Frame audit blocked this archive before the fix. A later [native Decky 3.3.0 check](evidence/framey-free-loader-native-2026-10-28.md) verified plugin discovery, default settings, empty local deals and private setting persistence after restart in a network-isolated amd64 container. Live feed updates, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified.
A [Free Loader duplicate-entry follow-up](evidence/framey-free-loader-duplicates-2026-10-26.md) accepts only byte-identical duplicate ZIP members whose modes differ at most in execute bits. The exact archive passed private local and focused fake-Frame install/remove; its frontend rendered in opaque Chromium. The earlier full 116-plugin fake-Frame audit blocked this archive before the fix. A later [native Decky 3.3.0 check](evidence/framey-free-loader-native-2026-10-28.md) verified plugin discovery, default settings, empty local deals and private setting persistence after restart in network-isolated amd64 and ARM64 containers. Live feed updates, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified.
A [2026-10-28 static action-limit review](evidence/framey-static-action-limits-2026-10-28.md) hash-checked the exact AutoSuspend and CheatDeck archives. AutoSuspend contains Steam suspend/shutdown paths, and CheatDeck's advertised trainer flow needs a Steam game context menu and launch-option writes. Neither plugin was executed; both retain their broken labels.
A [Network Sentinel user-service check](evidence/framey-network-sentinel-2026-10-28.md) adapts only the exact 1.0.4 digest. In isolated actual Decky 3.3.0 amd64, the pinned plugin-local psutil wheel, private settings RPC and one monitoring-loop sample pass. External ping/DNS, Frame ARM64, panel controls and worn VR remain unverified; this is partial compatibility.
A [Network Sentinel user-service check](evidence/framey-network-sentinel-2026-10-28.md) adapts only the exact 1.0.4 digest. In isolated actual Decky 3.3.0 amd64 and ARM64, the pinned plugin-local psutil wheel, private settings RPC and one monitoring-loop sample pass. External ping/DNS, a real Frame backend, panel controls and worn VR remain unverified; this is partial compatibility.
A [Battery Tracker user-service check](evidence/framey-battery-tracker-2026-10-28.md) adapts only the exact 0.2.0 digest. A locked read-only Frame probe confirmed a Battery-type supply with the required fields; actual Decky 3.3.0 amd64 recorded private Frame-shaped samples and returned history/per-app power. The managed plugin has not run on the real Frame, and panel controls and worn VR remain unverified; this is partial compatibility.
A [Battery Tracker user-service check](evidence/framey-battery-tracker-2026-10-28.md) adapts only the exact 0.2.0 digest. A locked read-only Frame probe confirmed a Battery-type supply with the required fields; actual Decky 3.3.0 amd64 and ARM64 recorded private Frame-shaped samples and returned history/per-app power. The managed plugin has not run on the real Frame, and panel controls and worn VR remain unverified; this is partial compatibility.
The current published digests are pinned in the JSON; if the store changed, retain/download those exact archives for reproduction. Audit downloads permit larger ZIPs for inspection; production installer limits remain 50 MiB ZIP / 150 MiB unpacked / 30 MiB per member.
@@ -56,7 +56,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 21: PowerTools 2.0.3 | legacy / legacy | N/A | Not supported on Frame: Targets AMD Ryzen power controls; Frame uses an ARM64 SoC. | static archive, local browser, fake-Frame |
| 22: Memory Deck 0.1.8 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: memory-deck/bin/libscanmem.so.1.0.0. | static archive, local browser, fake-Frame |
| 23: Animation Changer 1.3.2 | legacy / legacy | partial | Initializer succeeds; forced backend failure prevents loading animations. Boot-video changes untested. | static archive, local browser, fake-Frame |
| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds; radio discovery and playback remain unverified. | static archive, local browser, fake-Frame |
| 24: RadiYo! 1.3 | legacy / legacy | partial | Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made. | static archive, local browser, fake-Frame, isolated Decky 3.3.0 x86_64 backend |
| 26: HLTB for Deck 2.0.12 | 1 / 2 | partial | Settings render without the prior IndexedDB SecurityError; Steam library badges have no patch target, and live lookup is unverified. | static archive, local browser, fake-Frame |
| 27: AutoSuspend 2.2.0 | legacy / legacy | broken | Frontend audit failed: Cannot read properties of null (reading '6'). The exact frontend also contains SteamOS suspend and shutdown paths; threshold behavior and safe Frame operation are unverified. | static archive, local browser, fake-Frame |
| 28: Emuchievements 2.0.5 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: Emuchievements/bin/hash. | static archive, local browser, fake-Frame |
@@ -74,18 +74,18 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 49: Decky Cloud Save 1.4.2 | legacy / legacy | broken | The exact official ZIP contains a 60,776,600-byte x86_64 rclone member, above the 30 MiB member limit and unusable on ARM64. Installation remains blocked. | static archive, local browser, fake-Frame |
| 50: Storage Cleaner 1.4.0 | 1 / 2 | partial | Empty-data panel rendered; cleanup was deliberately not run because it can remove local data. | static archive, local browser, fake-Frame |
| 52: Game Theme Music 1.7.1-1 | 1 / 2 | partial | An inert local component lets the exact frontend initialize and render its Settings view in opaque Chromium. Backend get_setting and plugin network were denied in the audit; Steam game-page theme injection has no verified target in Framey. Music search, playback, downloads, settings persistence and worn VR remain unverified. | static archive, local browser, fake-Frame |
| 53: Shotty 0.1.0 | legacy / legacy | partial | The frontend starts with the read-only Frame library snapshot. Frame Control adds a digest-scoped click.get_app_dir compatibility module to the managed install; the exact backend then imports and copies a synthetic Steam screenshot into a temporary Pictures folder in isolated Python. Actual Decky 3.3.0 ARM64 loading, live screenshots/rescue, in-VR panel and physical controller use remain unverified. | static archive, local browser, fake-Frame, real Frame SSH, isolated Python backend |
| 53: Shotty 0.1.0 | legacy / legacy | partial | The frontend starts with the read-only Frame library snapshot. The exact-digest click.get_app_dir shim lets actual Decky 3.3.0 in isolated amd64 and ARM64 containers discover Shotty and complete empty screenshot RPCs; isolated Python also copied a synthetic screenshot into a temporary Pictures folder. The managed backend on the real Frame, live screenshots/rescue, in-VR panel and physical controller use remain unverified. | static archive, local browser, fake-Frame, real Frame SSH, isolated Python backend, isolated Decky 3.3.0 amd64/ARM64 empty backend RPCs |
| 56: MangoPeel 0.0.7 | 1 / 2 | partial | Initializer succeeds; gamescope/MangoHud actions and backend are unverified. | static archive, local browser, fake-Frame |
| 58: Volume Boost 0.0.7 | legacy / legacy | works | Default-sink controls pass in local Chromium through real Decky 3.3.0 ARM64: linked slider/keyboard and numeric writes, separate channels, re-link and reopen, with independent Pulse readback including 125%/150% while muted. Exact original volume/mute restored. Audible output, Bluetooth sinks and worn VR unverified. | static archive, local browser, fake-Frame, isolated Linux backend, real Frame SSH |
| 59: Web Browser 1.4.3 | legacy / legacy | broken | The exact frontend now initializes with a null Steam React-root lookup. The browser itself still requires Steam GamepadUI CDP injection and a backend WebSocket unavailable in Framey; no working browser tab was verified. | static archive, local browser, fake-Frame, focused local browser |
| 60: DeckyFileServer 1.2.0 | 1 / 2 | partial | Managed install supplies a native ARM64 helper, removes the exact archive's unnecessary root request, and displays the backend-returned IP with HTTPS. Real Decky 3.3.0 on Frame passes private read-only HTTPS listing/download and upload denial. The Steam folder picker remains unavailable; use a preconfigured directory. Uploads, generated thumbnails and worn VR are unverified. Existing managed copies need update or reinstall for the address fix. | static archive, local browser, fake-Frame, real Decky 3.3.0 amd64, real Frame SSH |
| 61: TabMaster 2.15.1 | 1 / 2 | partial | Local Chromium rendered the QAM and navigable Overview/Filters documentation with an actual Decky 3.3.0 amd64 backend. Library-tab editing remains unavailable: Steam login/library patches and MobX reactions have no target. ARM64 and worn VR unverified. | static archive, local browser, fake-Frame |
| 62: TS3 QuickAccess 1.2.0 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: ts3-qs4sd/bin/ts3-qs4sd.so. | static archive, local browser, fake-Frame |
| 63: Free Loader 1.6.0 | 1 / 2 | partial | The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in a private network-isolated container discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made. | static archive, local browser, fake-Frame pinned install/remove, local exact-archive transaction, isolated Python backend, isolated Linux backend |
| 63: Free Loader 1.6.0 | 1 / 2 | partial | The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in private network-isolated amd64 and ARM64 containers discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made. | static archive, local browser, fake-Frame pinned install/remove, local exact-archive transaction, isolated Python backend, isolated Decky 3.3.0 amd64/ARM64 backend |
| 64: Syncthing 0.2.7 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: decky-syncthing/bin/decky-syncthing-watchdog. Shared Toggle value rendering is repaired; other listed blockers remain. | static archive, local browser, fake-Frame |
| 65: Decky Terminal 0.5.3 | 1 / 2 | partial | Local routes initialize after parameter-route support; no shell commands were run. | static archive, local browser, fake-Frame |
| 66: PlayTime 3.2.5 | 1 / 2 | partial | Statistics and routed settings render with a detached real Frame library/user snapshot and actual Decky 3.3.0 amd64 backend in isolation. User selection and empty statistics reads pass. Running snapshots are now exposed, but its private MobX observation is unsupported; automatic recording and ARM64 backend remain unverified. | static archive, local browser, fake-Frame, real Frame SSH |
| 67: Battery Tracker 0.2.0 | legacy / legacy | partial | Exact managed install removes the unnecessary root flag, selects a readable Battery-type power supply, and repairs empty history and per-app sample grouping. Actual Decky 3.3.0 amd64 recorded Frame-shaped private battery samples and returned history/power data. A locked real Frame read verified the required sysfs fields, not the managed plugin. Live ARM64 recording, long-term battery history, foreground-app labeling, panel controls and worn VR remain unverified; existing copies need update or reinstall. | static archive, local browser, fake-Frame, isolated Decky amd64 backend, real Frame read-only sysfs |
| 67: Battery Tracker 0.2.0 | legacy / legacy | partial | Exact managed install removes the unnecessary root flag, selects a readable Battery-type power supply, and repairs empty history and per-app sample grouping. Actual Decky 3.3.0 in isolated amd64 and ARM64 containers recorded Frame-shaped private battery samples and returned history/per-app power. A locked real Frame read verified the required sysfs fields, not the managed plugin. Real Frame ARM64 recording, long-term history, foreground-app labeling, panel controls and worn VR remain unverified; existing copies need update or reinstall. | static archive, local browser, fake-Frame, isolated Decky 3.3.0 amd64/ARM64 backend, real Frame read-only sysfs |
| 68: Wine Cellar 0.2.1 | 1 / 2 | broken | Plugin contains a non-ARM64 Linux binary: decky-wine-cellar/bin/backend. Frontend audit failed: Audit blocks WebSocket. | static archive, local browser, fake-Frame |
| 70: MicroSDeck 0.11.2 | legacy / 2 | broken | Plugin contains a non-ARM64 Linux binary: MicroSDeck/bin/backend. | static archive, local browser, fake-Frame |
| 72: ScreenshotUploader 0.0.3 | legacy / legacy | partial | Initializer succeeds; screenshot upload requires external credentials/network and was not run. | static archive, local browser, fake-Frame |
@@ -97,7 +97,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 78: Brightness Bar 1.0.4 | legacy / legacy | N/A | Not supported on Frame: Targets Steam/QAM brightness shortcuts and a global Steam overlay; neither is attached to the Framey panel. | static archive, local browser, fake-Frame |
| 79: Decky Notifications 1.0.1 | legacy / legacy | partial | Initializer succeeds; device/account setup and notification delivery unverified. | static archive, local browser, fake-Frame |
| 80: MagicBlack 2.0.2 | 1 / 2 | N/A | Not supported on Frame: Targets a global Steam Deck screen-black overlay; the isolated panel cannot black out the Frame compositor. | static archive, local browser, fake-Frame |
| 81: IsThereAnyDeal for Deck 1.1.1 | legacy / legacy | partial | Country settings render and select the real Frame Steam country through the read-only bridge. Settings writes, deal lookups and Steam page integration remain unverified; plugin backend calls were denied. | static archive, local browser, fake-Frame, real Frame SSH |
| 81: IsThereAnyDeal for Deck 1.1.1 | legacy / legacy | partial | Country settings render and select the real Frame Steam country through the read-only bridge. The published frontend attempts deal lookups with an embedded upstream API key and a Steam store CDP tab; neither was exercised in Framey. Settings writes and Steam page integration remain unverified. | static archive, local browser, fake-Frame, real Frame SSH |
| 82: MagicPods 2.0.21 | 1 / 2 | broken | Plugin contains a non-ARM64 Linux binary: MagicPodsDecky/bin/magicpodscore. Frontend audit failed: Audit blocks WebSocket. | static archive, local browser, fake-Frame |
| 84: Junk-Store 1.1.13 | legacy / legacy | partial | Official Decky 3.3.0 ARM64 static menu/backend calls pass; local Chromium Games/Epic navigation, About/Dependencies tabs, Back and reopen render with real Frame library metadata. Legendary is absent; dependency install, login, game operations, RSS, optional websocket and worn VR remain unverified. Existing installs need update/reinstall for script permissions. | static archive, local browser, fake-Frame, isolated Linux backend, real Frame SSH |
| 85: Decky-Undervolt 1.0.8 | 1 / 2 | N/A | Not supported on Frame: RyzenAdj Curve Optimizer requires an AMD CPU; Frame is ARM64. | static archive, local browser, fake-Frame |
@@ -118,7 +118,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 101: XIVOmega 1.0.2 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
| 102: DeckSP 1.1.3 | 1 / 2 | broken | Optional UI-audio lookup succeeds, but the published frontend raises "Cannot access PluginManager before initialization" when Steam MobX reaction lookup is unavailable. JamesDSP installation/audio routing and ARM64 remain unverified. | static archive, local browser, fake-Frame |
| 103: Decky Ludusavi 1.2.2 | 1 / 2 | partial | Initializer succeeds; Flatpak Ludusavi discovery and backup/restore unverified. | static archive, local browser, fake-Frame |
| 104: Game Sync 1.0.4 | 1 / 2 | partial | Backend get_config is intentionally rejected by the audit before initialization completes; actual backend compatibility is unknown. | static archive, local browser, fake-Frame |
| 104: Game Sync 1.0.4 | 1 / 2 | partial | Exact official backend discovers under Decky 3.3.0 amd64; default get_config and a private setting read/write survive a loader restart. The isolated browser baseline still stops at its denied get_config call. Cloud login, rclone download/sync (including ARM64), process signals, real Frame and VR are unverified. The published updater names an amd64 rclone download, so no ARM64 sync claim follows. | static archive, local browser, fake-Frame, native Decky amd64 configuration |
| 105: WiFi Locker 1.0.2 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
| 106: LetMeReShade 1.8 | 1 / 2 | partial | Initializer succeeds; Deck-model assumptions, shader downloads and game modifications unverified. | static archive, local browser, fake-Frame |
| 107: Decky Sunshine 2025.10.27-dddf365 | legacy / legacy | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
@@ -128,12 +128,12 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 111: Music Control 1.1.7 | legacy / legacy | partial | Managed reinstall preserves Decky’s legacy calling convention and repairs the Frame OS shell library-path failure. Chromium transport/seek/volume against a silent MPRIS fixture and real ARM64 empty-player discovery pass; actual Frame media playback, album art and worn VR remain unverified. Reinstall existing copies to apply. | static archive, local browser, fake-Frame, isolated Python backend, isolated Decky 3.3.0 + MPRIS fixture, real Frame SSH |
| 112: Better Keyboard 0.3.4 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. Frontend audit failed: window.SteamClient.UI.GetUIMode is not a function. | static archive, local browser, fake-Frame |
| 113: Decky LSFG-VK 0.12.2 | 1 / 2 | partial | Initializer succeeds; runtime LSFG/Windows DLL downloads and Vulkan integration are not ARM64-validated. Shared Toggle value rendering is repaired; other listed blockers remain. | static archive, local browser, fake-Frame |
| 114: Network Sentinel 1.0.4 | 1 / 2 | partial | Exact managed install removes the unnecessary root flag and adds pinned plugin-local psutil wheels. Actual Decky 3.3.0 amd64 in network isolation loads the backend and records a network sample; private settings and read-only RPCs pass. External ping/DNS, real Frame ARM64, Wi-Fi scan behavior, panel controls and worn VR remain unverified. Existing copies need update or reinstall. | static archive, local browser, fake-Frame, isolated Decky amd64 backend |
| 114: Network Sentinel 1.0.4 | 1 / 2 | partial | Exact managed install removes the unnecessary root flag and adds pinned plugin-local psutil wheels. Actual Decky 3.3.0 in isolated amd64 and ARM64 containers loads the backend and records a private network sample; settings and read-only RPCs pass. External ping/DNS, real Frame ARM64 backend, Wi-Fi scan behavior, panel controls and worn VR remain unverified. Existing copies need update or reinstall. | static archive, local browser, fake-Frame, isolated Decky 3.3.0 amd64/ARM64 backend |
| 115: Decky Clash 1.3.0 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. | static archive, local browser, fake-Frame |
| 116: Muon 0.2.1 | 1 / 2 | broken | This plugin needs root and is not supported by the Frame user service. Shared Toggle value rendering is repaired; other listed blockers remain. | static archive, local browser, fake-Frame |
| 117: DeckyParty File Manager 1.0.0 | legacy / legacy | partial | Initializer succeeds; privileged file operations and backend are unverified. No file changes were run. | static archive, local browser, fake-Frame |
| 118: Speed Test 1.0.2 | 1 / 2 | partial | Initializer succeeds; automatic bandwidth tests were blocked by the audit network policy. | static archive, local browser, fake-Frame |
| 119: Decky Clipboard 1.0.1 | 1 / 2 | broken | Plugin contains a non-ARM64 Linux binary: decky-clipboard/bin/xclip. | static archive, local browser, fake-Frame |
| 119: Decky Clipboard 1.0.1 | 1 / 2 | broken | The exact archive is blocked by its x86-64 xclip binary and root request. Its published backend defaults to auto-starting an unauthenticated clipboard GET/POST API on 0.0.0.0:8765; this was inspected statically and never run on Frame. | static archive, local browser, fake-Frame |
| 120: Reroll 0.2.1 | 1 / 2 | broken | Frontend now initializes with inert game-action subscriptions, but the archive contains x86_64 memscan.so and requests root. No process-memory access was run. | static archive, local browser, fake-Frame |
| 121: DeckPass 1.1.1 | 1 / 2 | partial | Initializer succeeds; KeePass Flatpak discovery and secrets integration were not exercised. | static archive, local browser, fake-Frame |
| 123: Steam Status 1.2.3 | 2 / 2 | partial | Settings/public status rendered locally; cache errors fixed. Read-only get_settings RPC returned actual Decky 3.3.0 defaults on ARM64 Frame. Persistence, live backend diagnostics and VR controls unverified. | static archive, local browser, fake-Frame, real Frame SSH |
+13 -3
View File
@@ -57,6 +57,16 @@ count. So a build reaches people only when you publish it, after testing it.
publish unless every required job succeeded; it cannot prove physical
headset or installer behavior on its own.
Supply the saved smoke result to the publisher. Both its dry run and publish
command reject a result from another or dirty commit, incomplete title or
cleanup steps, a skip or failure, or an unconfirmed headset lock release:
```sh
export FRAME_SMOKE_RESULT=/path/to/smoke-result.json
python3 scripts/verify-frame-smoke-result.py "$FRAME_SMOKE_RESULT" \
"$(git rev-parse 'v0.4.0^{commit}')"
```
```sh
scripts/publish-release.sh --dry-run v0.4.0 # checks and shows update.json, changes nothing
scripts/publish-release.sh v0.4.0
@@ -64,9 +74,9 @@ count. So a build reaches people only when you publish it, after testing it.
The script checks that the tag is on GitHub, that exact-commit main CI
(including e2e), native amd64/ARM64 Decky runtime checks, and the tagged
release workflow are green with no failed,
cancelled, skipped or running jobs in any run attempt, and that all ten installers
are attached, each with the SHA-256 digest GitHub records. It also checks
release workflow are green with no failed, cancelled, skipped or running
jobs in any exact-tag push run or attempt, and that all ten installers are
attached, each with the SHA-256 digest GitHub records. It also checks
that the draft does not begin with a `WITHDRAWN` notice, so the preserved
v0.4.48 draft cannot accidentally be offered again. It checks
that the uploaded `SHA256SUMS` digest matches a fresh list of those ten
+4 -1
View File
@@ -215,7 +215,10 @@ acquires the shared `/tmp/frame-test.lock` with a unique owner token. It reads
SteamVR activity, proximity and battery while locked. An awake headset, unknown
activity, battery below 20%, or a busy lock stops the run without installing
anything. It releases only its own lock after cleanup. The JSON result includes
the read-only preflight for completed smoke runs.
the read-only preflight and the source checkout's commit and clean/dirty state.
For a release, run it from a clean checkout of the exact release commit;
`scripts/verify-frame-smoke-result.py` rejects a different or dirty checkout,
missing steps, skipped or failed launches, and an unconfirmed lock release.
It checks `properties.json` and the status, then installs, launches and
removes three tiny titles built from bytes by `tests/smoke/tiny_programs.py`
+3 -3
View File
@@ -40,10 +40,10 @@ def render(data):
'python3 scripts/framey-audit-browser.py --cache /tmp/framey-archives',
'```', '',
'Open the browser audit URL in local Chromium, run `runAudit()` and inspect/download `results`. All backend/network actions remain denied.', '',
'A [Free Loader duplicate-entry follow-up](evidence/framey-free-loader-duplicates-2026-10-26.md) accepts only byte-identical duplicate ZIP members whose modes differ at most in execute bits. The exact archive passed private local and focused fake-Frame install/remove; its frontend rendered in opaque Chromium. The earlier full 116-plugin fake-Frame audit blocked this archive before the fix. A later [native Decky 3.3.0 check](evidence/framey-free-loader-native-2026-10-28.md) verified plugin discovery, default settings, empty local deals and private setting persistence after restart in a network-isolated amd64 container. Live feed updates, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified.', '',
'A [Free Loader duplicate-entry follow-up](evidence/framey-free-loader-duplicates-2026-10-26.md) accepts only byte-identical duplicate ZIP members whose modes differ at most in execute bits. The exact archive passed private local and focused fake-Frame install/remove; its frontend rendered in opaque Chromium. The earlier full 116-plugin fake-Frame audit blocked this archive before the fix. A later [native Decky 3.3.0 check](evidence/framey-free-loader-native-2026-10-28.md) verified plugin discovery, default settings, empty local deals and private setting persistence after restart in network-isolated amd64 and ARM64 containers. Live feed updates, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified.', '',
'A [2026-10-28 static action-limit review](evidence/framey-static-action-limits-2026-10-28.md) hash-checked the exact AutoSuspend and CheatDeck archives. AutoSuspend contains Steam suspend/shutdown paths, and CheatDeck\'s advertised trainer flow needs a Steam game context menu and launch-option writes. Neither plugin was executed; both retain their broken labels.', '',
'A [Network Sentinel user-service check](evidence/framey-network-sentinel-2026-10-28.md) adapts only the exact 1.0.4 digest. In isolated actual Decky 3.3.0 amd64, the pinned plugin-local psutil wheel, private settings RPC and one monitoring-loop sample pass. External ping/DNS, Frame ARM64, panel controls and worn VR remain unverified; this is partial compatibility.', '',
'A [Battery Tracker user-service check](evidence/framey-battery-tracker-2026-10-28.md) adapts only the exact 0.2.0 digest. A locked read-only Frame probe confirmed a Battery-type supply with the required fields; actual Decky 3.3.0 amd64 recorded private Frame-shaped samples and returned history/per-app power. The managed plugin has not run on the real Frame, and panel controls and worn VR remain unverified; this is partial compatibility.', '',
'A [Network Sentinel user-service check](evidence/framey-network-sentinel-2026-10-28.md) adapts only the exact 1.0.4 digest. In isolated actual Decky 3.3.0 amd64 and ARM64, the pinned plugin-local psutil wheel, private settings RPC and one monitoring-loop sample pass. External ping/DNS, a real Frame backend, panel controls and worn VR remain unverified; this is partial compatibility.', '',
'A [Battery Tracker user-service check](evidence/framey-battery-tracker-2026-10-28.md) adapts only the exact 0.2.0 digest. A locked read-only Frame probe confirmed a Battery-type supply with the required fields; actual Decky 3.3.0 amd64 and ARM64 recorded private Frame-shaped samples and returned history/per-app power. The managed plugin has not run on the real Frame, and panel controls and worn VR remain unverified; this is partial compatibility.', '',
'The current published digests are pinned in the JSON; if the store changed, retain/download those exact archives for reproduction. '
'Audit downloads permit larger ZIPs for inspection; production installer limits remain 50 MiB ZIP / 150 MiB unpacked / 30 MiB per member.', '',
'After the VR job released the shared lock, this audit acquired it and used a private temporary Decky 3.3.0 ARM64 user service. '
+9 -2
View File
@@ -70,10 +70,15 @@ def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--loader', type=Path, help='Optional cached, digest-verified native Decky 3.3.0 binary')
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--case', type=int, help='Run one official plugin case locally; CI runs all cases')
args = parser.parse_args()
if sys.platform != 'linux' or not shutil.which('docker'):
parser.error('Linux with Docker is required')
cases = check_coverage()
if args.case is not None:
cases = [case for case in cases if case['id'] == args.case]
if not cases:
parser.error('Unknown official plugin case')
data, digest = loader_bytes(args.loader)
tag = 'framey-decky-runtime:'+uuid.uuid4().hex[:12]
report = {'decky': RELEASE, 'architecture': platform.machine(), 'loaderSha256': digest, 'cases': []}
@@ -85,8 +90,10 @@ def main():
loader.write_bytes(data)
loader.chmod(0o644) # Private container user copies/chmods its own executable.
probes = [(case['id'], None) for case in cases] + [
(111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
(60, '--reproduce-fileserver-empty-flag-regression')]
(plugin_id, negative) for plugin_id, negative in [
(111, '--reproduce-bound-method-regression'), (96, '--reproduce-timer-regression'),
(60, '--reproduce-fileserver-empty-flag-regression')]
if args.case is None or args.case == plugin_id]
for plugin_id, negative in probes:
name = 'framey-decky-'+uuid.uuid4().hex[:12]
command = ['docker', 'run', '--rm', '--name', name, '--network', 'none', '--read-only',
+3 -3
View File
@@ -3,7 +3,7 @@
# (docs/releasing.md). Checks every installer is attached with a SHA-256
# digest first, since the app's updater refuses assets without one, then
# attaches update.json, the manifest the updater reads.
# Usage: scripts/publish-release.sh [--dry-run] v0.4.0
# Usage: FRAME_SMOKE_RESULT=/path/to/result.json scripts/publish-release.sh [--dry-run] v0.4.0
#
# Everything goes through the REST API (gh api), not `gh release view/edit`:
# those look a draft up by its tag, and a draft can show tag_name
@@ -33,8 +33,8 @@ gh api "repos/$repo/git/ref/tags/$tag" >/dev/null 2>&1 \
# A complete asset list is insufficient: a cancelled installer build can leave
# old uploaded bytes behind. Check the exact tagged commit's main/e2e jobs and
# the latest exact-tag release attempt before any release mutation.
python3 "$(dirname "$0")/verify-release-gates.py" "$tag"
# every exact-tag push run and attempt before any release mutation.
python3 "$(dirname "$0")/verify-release-gates.py" "$tag" "${FRAME_SMOKE_RESULT:-}"
# Every release, drafts included, one JSON object per line.
gh api --paginate "repos/$repo/releases?per_page=100" --jq '.[]' > "$tmp/releases"
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""Check a locked Frame smoke result against the exact release commit."""
import json
import math
import re
import sys
from pathlib import Path
REQUIRED_STEPS = {
'cleanup: leftovers from earlier runs',
'devkit service: properties.json',
'status',
'cleanup: everything this run installed',
*(f'{kind}: {action}' for kind in ('arm64', 'x86_64', 'exe')
for action in ('install', 'launch', 'remove')),
}
def verify(result, commit):
if not isinstance(result, dict):
raise ValueError('smoke result is not an object')
if not re.fullmatch(r'[0-9a-f]{40}', commit):
raise ValueError('expected a full lowercase commit SHA-1')
if result.get('source') != {'commit': commit, 'clean': True}:
raise ValueError('smoke source is not the exact clean release commit')
if result.get('lock_released') is not True:
raise ValueError('smoke lock release was not recorded')
steps = result.get('steps')
if not isinstance(steps, list) or not steps:
raise ValueError('smoke steps are missing')
names = [step.get('step') for step in steps if isinstance(step, dict)]
if (len(names) != len(steps) or any(not isinstance(name, str) for name in names) or
len(names) != len(set(names)) or not REQUIRED_STEPS.issubset(names)):
raise ValueError('smoke steps are missing or duplicated')
passed = sum(step.get('ok') is True and not step.get('skipped') for step in steps)
if (passed != len(steps) or result.get('passed') != passed or
result.get('skipped') != 0 or result.get('failed') != 0):
raise ValueError('smoke contains a failed, skipped, or inconsistent step')
state = result.get('preflight') or {}
if (not isinstance(state, dict) or state.get('activity') != 3 or
not isinstance(state.get('proximity'), list) or not state['proximity'] or
any(type(value) not in (int, float) or not math.isfinite(value) or
value < 0 or value > 10
for value in state['proximity']) or
not isinstance(state.get('battery'), list) or not state['battery'] or
any(type(value) is not int or value < 20 or value > 100
for value in state['battery'])):
raise ValueError('locked standby preflight is missing')
def main(argv):
if len(argv) != 3:
raise ValueError('usage: verify-frame-smoke-result.py RESULT.json RELEASE_COMMIT_SHA')
result = json.loads(Path(argv[1]).read_text(encoding='utf-8'))
verify(result, argv[2])
print(f'Frame smoke passed for clean release commit {argv[2]}')
if __name__ == '__main__':
try:
main(sys.argv)
except (OSError, ValueError) as error:
sys.exit(f'Frame smoke gate failed: {error}')
+37 -8
View File
@@ -1,13 +1,14 @@
#!/usr/bin/env python3
"""Refuse publication unless the exact tag commit and tagged installers passed CI.
"""Refuse publication unless CI, tagged installers, and exact Frame smoke passed.
This checks GitHub's current run and job records, not a prior PR's check marks.
The real Frame smoke and independent package checks remain release-procedure
Independent package checks and release verification remain release-procedure
gates documented in docs/releasing.md.
"""
import json
import subprocess
import sys
from pathlib import Path
from urllib.parse import quote
@@ -60,14 +61,29 @@ def tag_commit(tag):
raise GateError(f"{tag} has too many nested tag objects")
def exact_run(workflow, branch, sha, event):
def exact_runs(workflow, branch, sha, event):
query = f"actions/workflows/{workflow}/runs?branch={quote(branch, safe='')}&head_sha={sha}&per_page=100"
runs = api(query).get("workflow_runs") or []
runs = []
for page in range(1, 21):
data = api(f"{query}&page={page}")
batch = data.get("workflow_runs")
total = data.get("total_count")
if not isinstance(batch, list) or type(total) is not int or total < 0:
raise GateError(f"invalid {workflow} run list for {branch} at {sha}")
runs.extend(batch)
if len(runs) >= total or len(batch) < 100:
break
if len(runs) != total or len({run.get('id') for run in runs}) != len(runs):
raise GateError(f"incomplete or duplicate {workflow} run list for {branch} at {sha}")
matching = [run for run in runs if run.get("head_sha") == sha
and run.get("head_branch") == branch and run.get("event") == event]
if not matching:
raise GateError(f"no exact {workflow} {event} run for {branch} at {sha}")
run = max(matching, key=lambda item: item["id"])
return sorted(matching, key=lambda item: item["id"])
def exact_run(workflow, branch, sha, event):
run = exact_runs(workflow, branch, sha, event)[-1]
if run.get("status") != "completed" or run.get("conclusion") != "success":
raise GateError(f"{workflow} run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
return run
@@ -98,18 +114,31 @@ def successful_jobs(run, required):
print(f"gate passed: {run.get('html_url')} ({len(latest_jobs)} jobs, {attempt} attempts)")
def main(tag):
def main(tag, smoke_result):
if not tag.startswith("v") or not tag[1:]:
raise GateError("expected a version tag beginning with v")
if not smoke_result:
raise GateError("FRAME_SMOKE_RESULT is required for the locked real Frame smoke gate")
sha = tag_commit(tag)
successful_jobs(exact_run("checks.yml", "main", sha, "push"), MAIN_JOBS)
successful_jobs(exact_run("decky-runtime.yml", "main", sha, "push"), DECKY_JOBS)
successful_jobs(exact_run("release.yml", tag, sha, "push"), RELEASE_JOBS)
# A fresh run must not hide a failed, cancelled, or skipped installer in an
# earlier separate push run for the same tag and commit.
for run in exact_runs("release.yml", tag, sha, "push"):
if run.get("status") != "completed" or run.get("conclusion") != "success":
raise GateError(f"release.yml run {run['id']} is {run.get('status')}/{run.get('conclusion')}")
successful_jobs(run, RELEASE_JOBS)
command = [sys.executable, str(Path(__file__).with_name("verify-frame-smoke-result.py")),
smoke_result, sha]
checked = subprocess.run(command, text=True, capture_output=True, check=False)
if checked.returncode:
raise GateError(checked.stderr.strip() or "Frame smoke result verification failed")
print(checked.stdout.strip())
print(f"exact release commit: {sha}")
if __name__ == "__main__":
try:
main(sys.argv[1])
main(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else None)
except (IndexError, GateError) as error:
sys.exit(f"release gate failed: {error}")
+10
View File
@@ -14,6 +14,16 @@
"shim": null,
"probe": "toggle-settings"
},
{
"id": 24,
"shim": null,
"probe": "radio-local"
},
{
"id": 104,
"shim": null,
"probe": "game-sync-config"
},
{
"id": 63,
"shim": null,
+82 -1
View File
@@ -2,6 +2,7 @@
import argparse
import asyncio
import hashlib
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
@@ -11,6 +12,7 @@ import socket
import ssl
import subprocess
import sys
import threading
import time
import urllib.request
import zipfile
@@ -265,6 +267,70 @@ def main():
assert current('read_deals') == {}
report['result'] = ('discovery, default settings, empty local deals, '
'private setting write/read and restart persistence passed; feed, notifications and VR unverified')
elif case['probe'] == 'game-sync-config':
# Exercise only private settings. This plugin also exposes
# rclone cloud sync and arbitrary process signals; neither
# belongs in a passive compatibility probe.
settings = current('get_config')
assert settings['sync_on_game_start'] is False, settings
assert settings['sync_on_game_stop'] is False, settings
assert settings['capture_upload'] is False, settings
assert settings['advanced_mode'] is False, settings
assert current('set_config', 'advanced_mode', True) is None
assert current('get_config')['advanced_mode'] is True
os.killpg(backend.pid, signal.SIGTERM)
backend.wait(timeout=8)
backend = subprocess.Popen([str(loader)], env=env, stdout=log, stderr=log,
start_new_session=True)
processes.append(backend)
until_ready(lambda: rpc('loader/get_plugins'), 'restarted Decky discovery')
assert until_ready(lambda: current('get_config')['advanced_mode'],
'private Game Sync setting after restart') is True
report['result'] = ('discovery, default config and private setting restart persistence passed; '
'rclone, cloud authentication, sync, process signals, real Frame and VR unverified')
elif case['probe'] == 'radio-local':
# The published backend accepts arbitrary URLs and disables
# HTTPS verification. Exercise only a loopback fixture inside
# this network-isolated container; no station or audio opens.
marker = 'Framey private radio station\n'
class RadioHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/redirect':
self.send_response(302)
self.send_header('Location', '/station')
self.end_headers()
elif self.path == '/station':
data = marker.encode()
self.send_response(200)
self.send_header('Content-Type', 'text/plain')
self.send_header('Content-Length', str(len(data)))
self.end_headers()
self.wfile.write(data)
else:
self.send_error(404)
def log_message(self, *_):
pass
server = ThreadingHTTPServer(('127.0.0.1', 0), RadioHandler)
worker = threading.Thread(target=server.serve_forever, daemon=True)
worker.start()
try:
base_url = f'http://127.0.0.1:{server.server_port}'
station = legacy('fetch', url=base_url+'/station', customStation=False)
assert station['status'] == 200 and station['body'] == marker, station
assert station['headers']['Content-Type'] == 'text/plain', station
custom = legacy('fetch', url=base_url+'/station', customStation=True)
assert custom['status'] == 200 and custom['body'] == '', custom
redirect = legacy('getRedirectUrl', url=base_url+'/redirect')
assert redirect == {'response': base_url+'/station'}, redirect
finally:
server.shutdown()
server.server_close()
worker.join(timeout=3)
report['result'] = ('actual Decky discovery and local station fetch/redirect passed; '
'external radio, HTTPS, audio playback, Frame and VR unverified')
elif case['probe'] == 'toggle-settings':
def call(method, *values):
result = rpc('loader/call_plugin_method', record['name'], method, *values)
@@ -358,7 +424,22 @@ def main():
assert case['probe'] == 'screenshots'
assert legacy('was_rescued') is False
assert legacy('aggregate_all', allapps=[]) == 0
report['result'] = 'discovery, backend import and empty screenshot aggregation passed'
screenshots = home/'.local/share/Steam/userdata/9001/760/remote/4242/screenshots'
screenshots.mkdir(parents=True)
marker = b'Framey private screenshot fixture\n'
(screenshots/'selected.jpg').write_bytes(marker)
legacy('set_id_map_fronend', allapps=[[4242, 'Private Game']])
assert legacy('copy_screenshot', app_id=4242,
url='https://localhost/selected.jpg') is True
copied = home/'Pictures/Screenshots/Private Game/selected.jpg'
assert copied.read_bytes() == marker
assert (home/'Pictures/Screenshots/most_recent.jpg').read_bytes() == marker
(screenshots/'historical.jpg').write_bytes(b'Framey historical fixture\n')
assert legacy('aggregate_all', allapps=[[4242, 'Private Game']]) == 1
assert (copied.parent/'historical.jpg').read_bytes() == b'Framey historical fixture\n'
assert legacy('was_rescued') is False
report['result'] = ('discovery, backend import, empty aggregation, private screenshot '
'copy and historical aggregation passed')
log.flush()
log.seek(0)
assert 'Starting Decky version v3.3.0' in log.read()
+14 -3
View File
@@ -96,15 +96,26 @@ elif method == "GET" and endpoint.startswith(repo + "actions/workflows/"):
"status": gate.get("decky_status" if run_id == 503 else ("main_status" if is_main else "release_status"), "completed"),
"conclusion": gate.get("decky_conclusion" if run_id == 503 else ("main_conclusion" if is_main else "release_conclusion"), "success"),
"html_url": "https://github.com/saphid/frame-control/actions/runs/%s" % run_id}
print(json.dumps({"workflow_runs": [run]}))
runs = [run]
if run_id == 502 and gate.get("prior_release_run_bad_job"):
runs.append(dict(run, id=500, run_attempt=1, status="completed",
conclusion=gate.get("prior_release_run_conclusion", "failure"),
html_url="https://github.com/saphid/frame-control/actions/runs/500"))
if run_id == 502 and gate.get("release_run_list_incomplete"):
print(json.dumps({"total_count": len(runs) + 1, "workflow_runs": runs}))
else:
print(json.dumps({"total_count": len(runs), "workflow_runs": runs}))
elif method == "GET" and endpoint.startswith(repo + "actions/runs/"):
parts = endpoint[len(repo + "actions/runs/"):].split("/")
run_id = parts[0]
attempt = int(parts[2])
names = {"501": main_jobs, "502": release_jobs, "503": decky_jobs}[run_id]
names = {"500": release_jobs, "501": main_jobs, "502": release_jobs,
"503": decky_jobs}[run_id]
names = [name for name in names if name != gate.get("omit_job")]
jobs = [{"name": name, "status": "completed",
"conclusion": (gate.get("prior_bad_job_conclusion", "failure")
"conclusion": (gate.get("prior_release_run_bad_conclusion", "cancelled")
if run_id == "500" and name == gate.get("prior_release_run_bad_job")
else gate.get("prior_bad_job_conclusion", "failure")
if run_id == "502" and attempt == 1 and name == gate.get("prior_bad_job")
else gate.get("bad_job_conclusion", "success") if name == gate.get("bad_job")
else "success")}
Binary file not shown.
Binary file not shown.
+10 -1
View File
@@ -3,7 +3,7 @@
These are unmodified ZIPs downloaded from the official Decky store CDN on
2026-10-25, except `96.zip` and `111.zip` downloaded on 2026-10-26 and `60.zip` on
2026-10-27; `13.zip` and `125.zip` were added from the digest-verified audit cache on 2026-10-27,
and `67.zip` and `114.zip` on 2026-10-28.
and `67.zip` and `114.zip` on 2026-10-28. `24.zip` and `104.zip` were added on 2026-10-29.
Each ZIP includes its upstream `LICENSE`. They are test data, not plugins shipped or installed by
Frame Control. The fake Frame tests serve them
from a local fixture so CI does not depend on the store being online.
@@ -11,6 +11,8 @@ from a local fixture so CI does not depend on the store being online.
| File | Official plugin | Version | Store SHA-256 |
| --- | --- | --- | --- |
| `13.zip` | Pause Games | 1.0.3 | `4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62` |
| `24.zip` | RadiYo! | 1.3 | `647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43` |
| `104.zip` | Game Sync | 1.0.4 | `65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd` |
| `60.zip` | DeckyFileServer | 1.2.0 | `7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0` |
| `125.zip` | Decky Localsend | 0.4.5 | `e271ccb55eb23f918b99a69f79aec905df4c9b2ef7b23bbf4ec1c2d1a6f0c6b7` |
| `114.zip` | Network Sentinel | 1.0.4 | `d3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5` |
@@ -60,6 +62,13 @@ Pause Games runs through actual published Decky 3.3.0 discovery and instance RPC
for private settings and a synthetic exclusion list. No process-control method
is called. The official frontend's standalone Toggle component separately
checks saved true/false state and matching settings callbacks; effects are inert.
RadiYo! runs through actual published Decky 3.3.0 legacy RPC in a network-isolated
container. Only loopback station fetch, custom-station fetch and redirect are
tested. External radio, HTTPS and audio playback remain unverified.
Game Sync runs through actual published Decky 3.3.0 instance RPC in the same
isolation. Only default configuration and a private setting surviving restart
are checked. No rclone, cloud sync, process signal, Frame backend or VR action
is tested.
Network Sentinel runs through the same isolated native Decky gate. Its exact
archive gets plugin-local psutil and a user-service manifest; the probe checks
discovery, private settings and one monitoring-loop sample without external
+4
View File
@@ -10,6 +10,10 @@ PLUGINS = (
'folder': 'network-sentinel', 'sha256': 'd3e07f40a0c800e72d15594e03c8c61e9bf3092524a578a0260520e735b971a5'},
{'id': 13, 'name': 'Pause Games', 'version': '1.0.3', 'api_version': 1,
'folder': 'SDH-PauseGames', 'sha256': '4aefdfb0ef25dea9c6d9fd037731c8b99ee8a1be966c2f7d5b0602112ef29f62'},
{'id': 24, 'name': 'RadiYo!', 'version': '1.3', 'api_version': 'legacy',
'folder': 'radiyo-steam-deck', 'sha256': '647111578aa7a5ecd16af8d7fe6c068a6d7ea8737c6dce5ac2d52e84a549ec43'},
{'id': 104, 'name': 'Game Sync', 'version': '1.0.4', 'api_version': 1,
'folder': 'SDH-GameSync', 'sha256': '65e021ebb37d2ebfbfbe3dbc8af95aef9556cf484730362c6b26c4ecc3e62ccd'},
{'id': 60, 'name': 'DeckyFileServer', 'version': '1.2.0', 'api_version': 1,
'folder': 'DeckyFileServer', 'sha256': '7dc2ecdcd05aa415d1ac90afaff28ecb17f77809a2bc7a77aee8e92505b976f0'},
{'id': 125, 'name': 'Decky Localsend', 'version': '0.4.5', 'api_version': 1,
+35 -6
View File
@@ -33,8 +33,10 @@ headset isn't reachable. A skipped launch is not a release smoke pass.
"""
import argparse
import json
import math
import os
import platform
import re
import secrets
import shlex
import subprocess
@@ -61,6 +63,20 @@ PREFIX = 'fc_smoke_'
LOCK = '/tmp/frame-test.lock'
def source_revision():
"""Record which checkout produced the smoke result for release review."""
try:
head = subprocess.run(['git', '-C', str(ROOT), 'rev-parse', 'HEAD'],
capture_output=True, text=True, timeout=10, check=True).stdout.strip()
status = subprocess.run(['git', '-C', str(ROOT), 'status', '--porcelain'],
capture_output=True, text=True, timeout=10, check=True).stdout
except (OSError, subprocess.CalledProcessError, subprocess.TimeoutExpired):
return {'commit': None, 'clean': False}
if not isinstance(head, str) or not re.fullmatch(r'[0-9a-f]{40}', head):
return {'commit': None, 'clean': False}
return {'commit': head, 'clean': not bool(status.strip())}
def remote(command, *, input=None):
"""Use a bounded SSH call for shared-lock operations."""
return subprocess.run(['ssh', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=8', ALIAS, command],
@@ -111,10 +127,13 @@ def preflight():
proximity = state.get('proximity')
if (not isinstance(proximity, list) or not proximity or
any(not isinstance(value, (int, float)) or isinstance(value, bool) or
value < 0 or value > 10 for value in proximity)):
not math.isfinite(value) or value < 0 or value > 10
for value in proximity)):
raise RuntimeError('Frame proximity is active or unknown; active session untouched')
battery = state.get('battery')
if not battery or min(battery) < 20:
if (not isinstance(battery, list) or not battery or
any(type(value) is not int or value < 20 or value > 100
for value in battery)):
raise RuntimeError('Frame battery is unavailable or below the smoke threshold')
return state
@@ -420,12 +439,14 @@ def run_smoke(args, before):
failed = len(smoke.steps) - passed - skipped
RESULTS.mkdir(exist_ok=True)
out = RESULTS / f"{started.replace(':', '')}-{build}.json"
out.write_text(json.dumps({'started': started, 'alias': ALIAS, 'preflight': before,
out.write_text(json.dumps({'started': started, 'alias': ALIAS, 'source': source_revision(),
'preflight': before,
'os_release': release, 'paired': args.pair,
'passed': passed, 'skipped': skipped, 'failed': failed,
'lock_released': False,
'devkit_utils_there_before': had_utils, 'steps': smoke.steps}, indent=1))
print(f'==> {passed} passed, {skipped} skipped, {failed} failed on build {build}; results in {out}')
return 0 if failed == 0 and skipped == 0 else 1
return (0 if failed == 0 and skipped == 0 else 1), out
def main():
@@ -440,12 +461,12 @@ def main():
if not locked:
print('Shared Frame lock busy or unreachable; no headset status read or smoke run.', file=sys.stderr)
return 2
result = 1
result, output = 1, None
try:
try:
before = preflight()
print('Locked read-only preflight: ' + json.dumps(before), flush=True)
result = run_smoke(args, before)
result, output = run_smoke(args, before)
except (OSError, ValueError, RuntimeError, subprocess.TimeoutExpired) as failure:
print(str(failure), file=sys.stderr)
result = 2
@@ -457,6 +478,14 @@ def main():
if not released:
print('ERROR: could not release owned Frame lock.', file=sys.stderr)
result = 1
elif output is not None:
try:
saved = json.loads(output.read_text())
saved['lock_released'] = True
output.write_text(json.dumps(saved, indent=1))
except (OSError, ValueError) as failure:
print(f'ERROR: could not record Frame lock release: {failure}', file=sys.stderr)
result = 1
return result
+1 -1
View File
@@ -21,7 +21,7 @@ class RuntimeCoverage(unittest.TestCase):
self.assertIn(case['id'], official)
self.assertIn(case['probe'], ('mpris', 'screenshots', 'system-monitor', 'timer-lifecycle',
'toggle-settings', 'fileserver', 'localsend', 'network-sentinel',
'battery-tracker', 'free-loader'))
'battery-tracker', 'free-loader', 'radio-local', 'game-sync-config'))
def test_new_shim_without_runtime_case_fails_coverage(self):
import frame_decky_store
+57 -1
View File
@@ -38,15 +38,35 @@ def draft(tag_name="untagged-c6ddfed7f75d67db2e99", name="Frame Control 9.8.7",
class PublishRelease(unittest.TestCase):
def run_script(self, releases, *args, tags="v9.8.7", gates=None,
corrupt_manifest_readback=False):
corrupt_manifest_readback=False, smoke=None):
d = Path(tempfile.mkdtemp())
(d / "releases.json").write_text(json.dumps(releases))
(d / "gates.json").write_text(json.dumps(gates or {}))
steps = ["cleanup: leftovers from earlier runs", "devkit service: properties.json",
"status", "cleanup: everything this run installed"] + [
f"{kind}: {action}" for kind in ("arm64", "x86_64", "exe")
for action in ("install", "launch", "remove")]
clean_smoke = {"source": {"commit": "a" * 40, "clean": True},
"lock_released": True,
"steps": [{"step": name, "ok": True} for name in steps],
"passed": len(steps), "skipped": 0, "failed": 0,
"preflight": {"activity": 3, "proximity": [2.3125], "battery": [100]}}
if smoke == "skipped":
clean_smoke["steps"][0].update(ok=False, skipped=True)
clean_smoke.update(passed=len(steps) - 1, skipped=1)
elif smoke == "unreleased":
clean_smoke["lock_released"] = False
if smoke is not False:
(d / "smoke.json").write_text(json.dumps(
clean_smoke if smoke is None or isinstance(smoke, str) else smoke))
env = dict(os.environ, PATH="%s:%s" % (FAKEGH, os.environ["PATH"]),
FAKEGH_RELEASES=str(d / "releases.json"), FAKEGH_TAGS=tags,
FAKEGH_LOG=str(d / "log"), FAKEGH_UPLOAD=str(d / "upload.json"),
FAKEGH_GATES=str(d / "gates.json"),
FAKEGH_CORRUPT_MANIFEST_READBACK="1" if corrupt_manifest_readback else "0")
env.pop("FRAME_SMOKE_RESULT", None)
if smoke is not False:
env["FRAME_SMOKE_RESULT"] = str(d / "smoke.json")
p = subprocess.run(["sh", str(SCRIPT), *args], env=env, capture_output=True, text=True, timeout=30)
log = [json.loads(line) for line in (d / "log").read_text().splitlines()] if (d / "log").exists() else []
upload = json.loads((d / "upload.json").read_text()) if (d / "upload.json").exists() else None
@@ -156,6 +176,18 @@ class PublishRelease(unittest.TestCase):
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_missing_or_skipped_frame_smoke_before_mutation(self):
for smoke in (False,
{"source": {"commit": "b" * 40, "clean": True}},
{"source": {"commit": "a" * 40, "clean": False}},
"skipped", "unreleased"):
with self.subTest(smoke=smoke):
p, log, upload = self.run_script([draft()], "v9.8.7", smoke=smoke)
self.assertNotEqual(p.returncode, 0)
self.assertIn("release gate failed", p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_a_prior_cancelled_installer_after_successful_rerun(self):
gates = {"release_attempts": 2,
"prior_bad_job": "build (macos-latest, dist, app/dist/*.dmg app/dist/*.zip)",
@@ -167,6 +199,30 @@ class PublishRelease(unittest.TestCase):
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_bad_installer_in_separate_prior_tag_run(self):
for run_conclusion, job_conclusion, expected in (
("failure", "cancelled", "failure"),
("success", "skipped", "skipped")):
with self.subTest(run_conclusion=run_conclusion, job_conclusion=job_conclusion):
gates = {"prior_release_run_bad_job":
"build (macos-latest, dist, app/dist/*.dmg app/dist/*.zip)",
"prior_release_run_conclusion": run_conclusion,
"prior_release_run_bad_conclusion": job_conclusion}
p, log, upload = self.run_script([draft()], "v9.8.7", gates=gates)
self.assertNotEqual(p.returncode, 0)
self.assertIn("run 500", p.stderr)
self.assertIn(expected, p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_incomplete_tag_run_list(self):
p, log, upload = self.run_script([draft()], "v9.8.7",
gates={"release_run_list_incomplete": True})
self.assertNotEqual(p.returncode, 0)
self.assertIn("incomplete", p.stderr)
self.assertIsNone(upload)
self.assertFalse(any("-X" in call for call in log))
def test_refuses_missing_or_nonexact_gate_results(self):
for gates in ({"omit_job": "e2e"}, {"main_sha": "c" * 40},
{"decky_conclusion": "failure"}, {"release_conclusion": "failure"},
+42 -6
View File
@@ -95,8 +95,21 @@ class RealProcesses(unittest.TestCase):
class Safety(unittest.TestCase):
def test_source_revision_records_clean_checkout_and_rejects_invalid_head(self):
head = 'a' * 40
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout=head + '\n'), mock.Mock(stdout='')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': head, 'clean': True})
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout=head + '\n'), mock.Mock(stdout=' M tests/smoke/frame_smoke.py\n')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': head, 'clean': False})
with mock.patch.object(frame_smoke.subprocess, 'run', side_effect=[
mock.Mock(stdout='not-a-commit\n'), mock.Mock(stdout='')]):
self.assertEqual(frame_smoke.source_revision(), {'commit': None, 'clean': False})
def test_locked_preflight_rejects_active_or_unknown_proximity_before_titles(self):
for proximity in ([], [88.0625], [4.6875, 88.0625], [None]):
for proximity in ([], [88.0625], [4.6875, 88.0625], [None],
[float('nan')], [float('inf')], [float('-inf')]):
with self.subTest(proximity=proximity), mock.patch.object(
frame_smoke, 'remote', return_value=mock.Mock(
returncode=0, stdout=json.dumps({
@@ -147,6 +160,8 @@ class Safety(unittest.TestCase):
else:
smoke.step(f'{kind}: launch', lambda: {'outcome': 'running'})
with mock.patch.object(frame_smoke, 'RESULTS', output), \
mock.patch.object(frame_smoke, 'source_revision', return_value={
'commit': 'a' * 40, 'clean': True}), \
mock.patch.object(frame_smoke, 'os_release', return_value={
'BUILD_ID': 'fake', 'VERSION_ID': 'fake', 'VARIANT_ID': 'fake'}), \
mock.patch.object(frame_smoke, 'ssh', return_value='yes'), \
@@ -155,10 +170,14 @@ class Safety(unittest.TestCase):
mock.patch.object(frame_smoke, 'status', return_value={}), \
mock.patch.object(frame_smoke, 'title_cycle', side_effect=cycle), \
mock.patch.object(frame_smoke.tempfile, 'mkdtemp', return_value=str(folder)):
self.assertEqual(frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity': 3, 'proximity': [], 'battery': [100]}), 1)
code, saved_path = frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity': 3, 'proximity': [], 'battery': [100]})
self.assertEqual(code, 1)
result = json.loads(next(output.glob('*-fake.json')).read_text())
self.assertEqual(saved_path.parent, output)
self.assertEqual((result['passed'], result['skipped'], result['failed']), (6, 1, 0))
self.assertEqual(result['source'], {'commit': 'a' * 40, 'clean': True})
self.assertFalse(result['lock_released'])
self.assertEqual(next(s for s in result['steps'] if s['step'] == 'x86_64: launch')
['detail']['outcome'], 'runtime missing')
@@ -207,13 +226,27 @@ class Safety(unittest.TestCase):
mock.patch.object(frame_smoke, 'cleanup', side_effect=RuntimeError('cannot stop')), \
mock.patch.object(frame_smoke, 'title_cycle') as cycle, \
mock.patch('builtins.print'):
self.assertEqual(frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity':3,'proximity':[],'battery':[100]}), 1)
code, _ = frame_smoke.run_smoke(mock.Mock(pair=False),
{'activity':3,'proximity':[],'battery':[100]})
self.assertEqual(code, 1)
cycle.assert_not_called()
self.assertEqual(len(list(Path(folder).glob('*-fake.json'))), 1)
class SharedSmokeLock(unittest.TestCase):
def test_result_marks_release_only_after_owned_lock_is_removed(self):
with tempfile.TemporaryDirectory() as folder:
result_path = Path(folder) / 'smoke.json'
for released in (False, True):
result_path.write_text('{"lock_released": false}')
with mock.patch.object(frame_smoke.sys, 'argv', ['frame_smoke.py']), \
mock.patch.object(frame_smoke, 'acquire_lock', return_value=True), \
mock.patch.object(frame_smoke, 'preflight', return_value={}), \
mock.patch.object(frame_smoke, 'run_smoke', return_value=(0, result_path)), \
mock.patch.object(frame_smoke, 'release_lock', return_value=released):
self.assertEqual(frame_smoke.main(), 0 if released else 1)
self.assertIs(json.loads(result_path.read_text())['lock_released'], released)
def test_busy_lock_prevents_any_status_read_or_title_change(self):
with mock.patch.object(frame_smoke.sys, 'argv', ['frame_smoke.py']), \
mock.patch.object(frame_smoke, 'acquire_lock', return_value=False), \
@@ -229,7 +262,10 @@ class SharedSmokeLock(unittest.TestCase):
for state in ({'activity':1,'proximity':[1],'battery':[100]},
{'activity':None,'proximity':[],'battery':[100]},
{'activity':3,'proximity':[],'battery':[19]},
{'activity':3,'proximity':[],'battery':[]}):
{'activity':3,'proximity':[],'battery':[]},
{'activity':3,'proximity':[0],'battery':[True]},
{'activity':3,'proximity':[0],'battery':[float('nan')]},
{'activity':3,'proximity':[0],'battery':[101]}):
with self.subTest(state=state), mock.patch.object(frame_smoke, 'remote',
return_value=mock.Mock(returncode=0, stdout=json.dumps(state))):
with self.assertRaises(RuntimeError):
+77
View File
@@ -0,0 +1,77 @@
"""The release smoke gate requires a complete result from its exact checkout."""
import importlib.util
import math
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
SPEC = importlib.util.spec_from_file_location(
'verify_frame_smoke_result', ROOT / 'scripts' / 'verify-frame-smoke-result.py')
gate = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(gate)
SHA = 'a' * 40
def result():
steps = [{'step': name, 'ok': True} for name in sorted(gate.REQUIRED_STEPS)]
return {'source': {'commit': SHA, 'clean': True}, 'lock_released': True, 'steps': steps,
'passed': len(steps), 'skipped': 0, 'failed': 0,
'preflight': {'activity': 3, 'proximity': [2.3125], 'battery': [100]}}
class VerifyFrameSmokeResult(unittest.TestCase):
def test_accepts_complete_clean_result_from_exact_commit(self):
gate.verify(result(), SHA)
def test_rejects_wrong_commit_dirty_checkout_or_missing_step(self):
for changed in ('commit', 'clean', 'step'):
with self.subTest(changed=changed):
data = result()
if changed == 'commit':
data['source']['commit'] = 'b' * 40
elif changed == 'clean':
data['source']['clean'] = False
else:
data['steps'].pop()
data['passed'] -= 1
with self.assertRaises(ValueError):
gate.verify(data, SHA)
def test_rejects_skipped_launch_even_when_counts_claim_success(self):
data = result()
launch = next(step for step in data['steps'] if step['step'] == 'x86_64: launch')
launch.update(ok=False, skipped=True)
data.update(passed=len(data['steps']) - 1, skipped=1)
with self.assertRaisesRegex(ValueError, 'failed, skipped'):
gate.verify(data, SHA)
def test_rejects_missing_locked_preflight(self):
data = result()
data['preflight']['activity'] = 1
with self.assertRaisesRegex(ValueError, 'preflight'):
gate.verify(data, SHA)
def test_rejects_nonfinite_proximity_and_invalid_battery(self):
for proximity in (math.nan, math.inf, -math.inf):
with self.subTest(proximity=proximity):
data = result()
data['preflight']['proximity'] = [proximity]
with self.assertRaisesRegex(ValueError, 'preflight'):
gate.verify(data, SHA)
for battery in ([True], [math.nan], [101], [19], ['99']):
with self.subTest(battery=battery):
data = result()
data['preflight']['battery'] = battery
with self.assertRaisesRegex(ValueError, 'preflight'):
gate.verify(data, SHA)
def test_rejects_missing_lock_release_record(self):
data = result()
data['lock_released'] = False
with self.assertRaisesRegex(ValueError, 'lock release'):
gate.verify(data, SHA)
if __name__ == '__main__':
unittest.main()
+3
View File
@@ -7,6 +7,7 @@ import base64
import copy
import http.client
import json
import os
import shutil
import subprocess
import sys
@@ -189,6 +190,8 @@ class Probe(unittest.TestCase):
finally:
wireless._lock.release()
@unittest.skipIf(os.environ.get('FRAME_SKIP_WINDOWS_WIRELESS_LIVE') == '1',
'dedicated wireless-windows CI job owns the live inventory')
@unittest.skipUnless(wireless.frame_host.WINDOWS, 'Live read-only Windows inventory runs on Windows CI')
def test_live_windows_inventory(self):
r = wireless.diagnose()
+21 -17
View File
@@ -1479,12 +1479,13 @@
"https://github.com/isiah-lloyd/radiyo-steam-deck"
],
"status": "partial",
"reason": "Initializer succeeds; radio discovery and playback remain unverified.",
"reason": "Initializer succeeds. Exact official legacy backend passed published Decky 3.3.0 x86_64 discovery and loopback-only station fetch, custom-station fetch and redirect in a network-isolated container. Real radio services, HTTPS behavior, audio playback, real Frame backend and worn VR remain unverified. The published backend disables HTTPS certificate verification; no external request was made.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame"
"fake-Frame",
"isolated Decky 3.3.0 x86_64 backend"
],
"browser": {
"id": 24,
@@ -3903,14 +3904,15 @@
"https://github.com/safijari/Shotty"
],
"status": "partial",
"reason": "The frontend starts with the read-only Frame library snapshot. Frame Control adds a digest-scoped click.get_app_dir compatibility module to the managed install; the exact backend then imports and copies a synthetic Steam screenshot into a temporary Pictures folder in isolated Python. Actual Decky 3.3.0 ARM64 loading, live screenshots/rescue, in-VR panel and physical controller use remain unverified.",
"reason": "The frontend starts with the read-only Frame library snapshot. The exact-digest click.get_app_dir shim lets actual Decky 3.3.0 in isolated amd64 and ARM64 containers discover Shotty and complete empty screenshot RPCs; isolated Python also copied a synthetic screenshot into a temporary Pictures folder. The managed backend on the real Frame, live screenshots/rescue, in-VR panel and physical controller use remain unverified.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame",
"real Frame SSH",
"isolated Python backend"
"isolated Python backend",
"isolated Decky 3.3.0 amd64/ARM64 empty backend RPCs"
],
"browser": {
"id": 53,
@@ -3943,11 +3945,12 @@
"frontendReady": true
},
"backendFollowUp": {
"evidence": "docs/evidence/framey-shotty-python-2026-10-26.md",
"mode": "exact archive managed transaction and isolated Python import/copy with temporary HOME",
"evidence": "docs/evidence/framey-shotty-native-2026-10-28.md",
"mode": "exact archive managed transaction, isolated Python copy, and actual Decky 3.3.0 amd64/ARM64 discovery with empty backend RPCs",
"compatShim": "shotty-click-app-dir",
"syntheticScreenshotCopied": true,
"actualDeckyArm64": false
"actualDeckyArm64": true,
"actualFrameBackend": false
}
},
{
@@ -4704,7 +4707,7 @@
"https://github.com/jwhitlow45/free-loader.git"
],
"status": "partial",
"reason": "The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in a private network-isolated container discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made.",
"reason": "The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in private network-isolated amd64 and ARM64 containers discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made.",
"blockInstall": false,
"verification": [
"static archive",
@@ -4712,7 +4715,7 @@
"fake-Frame pinned install/remove",
"local exact-archive transaction",
"isolated Python backend",
"isolated Linux backend"
"isolated Decky 3.3.0 amd64/ARM64 backend"
],
"browser": {
"id": 63,
@@ -5179,7 +5182,7 @@
"https://github.com/safijari/steam-deck-battery-tracker.git"
],
"status": "partial",
"reason": "Exact managed install removes the unnecessary root flag, selects a readable Battery-type power supply, and repairs empty history and per-app sample grouping. Actual Decky 3.3.0 amd64 recorded Frame-shaped private battery samples and returned history/power data. A locked real Frame read verified the required sysfs fields, not the managed plugin. Live ARM64 recording, long-term battery history, foreground-app labeling, panel controls and worn VR remain unverified; existing copies need update or reinstall.",
"reason": "Exact managed install removes the unnecessary root flag, selects a readable Battery-type power supply, and repairs empty history and per-app sample grouping. Actual Decky 3.3.0 in isolated amd64 and ARM64 containers recorded Frame-shaped private battery samples and returned history/per-app power. A locked real Frame read verified the required sysfs fields, not the managed plugin. Real Frame ARM64 recording, long-term history, foreground-app labeling, panel controls and worn VR remain unverified; existing copies need update or reinstall.",
"blockInstall": false,
"requiredShims": [
"battery-tracker-user-supply"
@@ -5188,7 +5191,7 @@
"static archive",
"local browser",
"fake-Frame",
"isolated Decky amd64 backend",
"isolated Decky 3.3.0 amd64/ARM64 backend",
"real Frame read-only sysfs"
],
"browser": {
@@ -6320,7 +6323,7 @@
"https://github.com/JtdeGraaf/IsThereAnyDeal-DeckyPlugin.git"
],
"status": "partial",
"reason": "Country settings render and select the real Frame Steam country through the read-only bridge. Settings writes, deal lookups and Steam page integration remain unverified; plugin backend calls were denied.",
"reason": "Country settings render and select the real Frame Steam country through the read-only bridge. The published frontend attempts deal lookups with an embedded upstream API key and a Steam store CDP tab; neither was exercised in Framey. Settings writes and Steam page integration remain unverified.",
"blockInstall": false,
"verification": [
"static archive",
@@ -8877,12 +8880,13 @@
"https://github.com/GedasFX/decky-cloud-save"
],
"status": "partial",
"reason": "Backend get_config is intentionally rejected by the audit before initialization completes; actual backend compatibility is unknown.",
"reason": "Exact official backend discovers under Decky 3.3.0 amd64; default get_config and a private setting read/write survive a loader restart. The isolated browser baseline still stops at its denied get_config call. Cloud login, rclone download/sync (including ARM64), process signals, real Frame and VR are unverified. The published updater names an amd64 rclone download, so no ARM64 sync claim follows.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame"
"fake-Frame",
"native Decky amd64 configuration"
],
"browser": {
"id": 104,
@@ -9976,7 +9980,7 @@
"https://github.com/SteamDeckHomebrew/decky-loader"
],
"status": "partial",
"reason": "Exact managed install removes the unnecessary root flag and adds pinned plugin-local psutil wheels. Actual Decky 3.3.0 amd64 in network isolation loads the backend and records a network sample; private settings and read-only RPCs pass. External ping/DNS, real Frame ARM64, Wi-Fi scan behavior, panel controls and worn VR remain unverified. Existing copies need update or reinstall.",
"reason": "Exact managed install removes the unnecessary root flag and adds pinned plugin-local psutil wheels. Actual Decky 3.3.0 in isolated amd64 and ARM64 containers loads the backend and records a private network sample; settings and read-only RPCs pass. External ping/DNS, real Frame ARM64 backend, Wi-Fi scan behavior, panel controls and worn VR remain unverified. Existing copies need update or reinstall.",
"blockInstall": false,
"requiredShims": [
"network-sentinel-user-psutil"
@@ -9985,7 +9989,7 @@
"static archive",
"local browser",
"fake-Frame",
"isolated Decky amd64 backend"
"isolated Decky 3.3.0 amd64/ARM64 backend"
],
"browser": {
"id": 114,
@@ -10544,7 +10548,7 @@
"https://github.com/astrand/xclip"
],
"status": "broken",
"reason": "Plugin contains a non-ARM64 Linux binary: decky-clipboard/bin/xclip.",
"reason": "The exact archive is blocked by its x86-64 xclip binary and root request. Its published backend defaults to auto-starting an unauthenticated clipboard GET/POST API on 0.0.0.0:8765; this was inspected statically and never run on Frame.",
"blockInstall": true,
"verification": [
"static archive",