Compare commits

..
Author SHA1 Message Date
SAPHID 89e5ee610d Merge remote-tracking branch 'origin/main' into test/framey-freeloader-native 2026-10-28 19:58:54 +00:00
SAPHID 2b8836929e Test official Free Loader in native Decky runtime 2026-10-28 19:58:54 +00:00
Alex Southwell 248e5b3382 Merge pull request #384 from saphid/fix/mac-updater-preserve-quarantine
Preserve macOS quarantine metadata during auto-update
2026-10-28 19:58:16 +00:00
SAPHID 3dcba52e91 Preserve macOS quarantine metadata during auto-update 2026-10-28 19:41:56 +00:00
Alex Southwell efda14776d Merge pull request #383 from saphid/ci/audit-published-update-manifest
Verify the published update manifest in release audits
2026-10-28 19:25:21 +00:00
SAPHID 6bcce9cbf7 Gate publication on uploaded manifest readback 2026-10-28 19:14:10 +00:00
SAPHID 8f1a811e1d Verify published update manifest in release asset audit 2026-10-28 19:08:59 +00:00
Alex Southwell 9658f38382 Merge pull request #382 from saphid/ci/reusable-release-assets-audit
Make release installer checks repeatable
2026-10-28 18:55:14 +00:00
13 changed files with 199 additions and 13 deletions

No files matched your search

-1
View File
@@ -181,7 +181,6 @@ old="$stage/old.app"
if mv "$app" "$old"; then
if mv "$new" "$app"; then rm -rf "$old"; else mv "$old" "$app"; fi
fi
xattr -dr com.apple.quarantine "$app" 2>/dev/null
rm -rf "$stage"
open "$app"
`;
@@ -0,0 +1,25 @@
# Free Loader 1.6.0: private Decky backend check
The official store archive for ID 63 was downloaded and matched SHA-256
`05208aee6f6ba824391d8ddc81b1168b2599a9711e7528cdaa789f6cc710b42a`.
The published Decky 3.3.0 amd64 loader matched SHA-256
`e80d03e641d1766a33a817cb27ab8d2e0a4633685e5d27e4de948a6687526181`.
An unprivileged, disposable Debian container ran that loader and the exact
official plugin with no network, host devices, host session or writable host
mount. Decky discovered Free Loader as enabled. Its API 1 methods returned 14
default settings and an empty local deals map. A private `larger_text` setting
changed to `true` and read back; startup created only the container's private
settings file, and `read_deals` created an empty private deals cache. No feed,
notification, Steam store or external account method was called. The local
probe exited successfully; its container was removed. The committed native
Decky runtime case repeats these calls and also checks persistence after a
loader restart on the CI runner's native architecture. The local x86_64 run
passed all 14 runtime cases, including Free Loader and the existing negative
regressions. The runner removed its temporary containers and image.
This strengthens the **partial** label. It does not verify a live feed refresh,
notification delivery, real Frame ARM64 service, rendered deal cards, worn VR
or controller interaction. The earlier opaque Chromium check had denied backend
and network calls, so it could not establish that deal cards render with live
data. No headset was touched for this check.
+1 -1
View File
@@ -81,7 +81,7 @@ The API column shows manifest / actual bundled frontend API. Many manifests say
| 60: DeckyFileServer 1.2.0 | 1 / 2 | partial | Managed install supplies a native ARM64 helper and removes the exact archive's unnecessary root request. Real Decky 3.3.0 on Frame passes private read-only HTTPS listing/download and upload denial. Frontend address/picker limitations remain; use HTTPS and a preconfigured directory. Uploads, generated thumbnails and worn VR are unverified. Shared Toggle value rendering is repaired; other listed blockers remain. | static archive, local browser, fake-Frame, real Decky 3.3.0 amd64, real Frame SSH |
| 61: TabMaster 2.15.1 | 1 / 2 | partial | Local Chromium rendered the QAM and navigable Overview/Filters documentation with an actual Decky 3.3.0 amd64 backend. Library-tab editing remains unavailable: Steam login/library patches and MobX reactions have no target. ARM64 and worn VR unverified. | static archive, local browser, fake-Frame |
| 62: TS3 QuickAccess 1.2.0 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: ts3-qs4sd/bin/ts3-qs4sd.so. | static archive, local browser, fake-Frame |
| 63: Free Loader 1.6.0 | 1 / 2 | partial | The exact archive's duplicate names contain identical bytes and differ only in execute bits; the guarded installer now accepts them, and a private local install/remove transaction passed. The current opaque Chromium host renders Free Games but cannot load deals with backend/network denied. Exact official Python settings and empty local-deals methods passed in a no-network container with a Decky stub. Real Decky, feed refresh, notifications and worn VR remain unverified. | static archive, local browser, fake-Frame pinned install/remove, local exact-archive transaction, isolated Python backend |
| 63: Free Loader 1.6.0 | 1 / 2 | partial | The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in a private network-isolated container discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made. | static archive, local browser, fake-Frame pinned install/remove, local exact-archive transaction, isolated Python backend, isolated Linux backend |
| 64: Syncthing 0.2.7 | legacy / legacy | broken | Plugin contains a non-ARM64 Linux binary: decky-syncthing/bin/decky-syncthing-watchdog. Shared Toggle value rendering is repaired; other listed blockers remain. | static archive, local browser, fake-Frame |
| 65: Decky Terminal 0.5.3 | 1 / 2 | partial | Local routes initialize after parameter-route support; no shell commands were run. | static archive, local browser, fake-Frame |
| 66: PlayTime 3.2.5 | 1 / 2 | partial | Statistics and routed settings render with a detached real Frame library/user snapshot and actual Decky 3.3.0 amd64 backend in isolation. User selection and empty statistics reads pass. Running snapshots are now exposed, but its private MobX observation is unsupported; automatic recording and ARM64 backend remain unverified. | static archive, local browser, fake-Frame, real Frame SSH |
+9 -1
View File
@@ -72,7 +72,9 @@ count. So a build reaches people only when you publish it, after testing it.
that the uploaded `SHA256SUMS` digest matches a fresh list of those ten
installer digests. It attaches
`update.json` (the version, the notes, the release page and each
installer's digest), then publishes the release and marks it latest. It
installer's digest), verifies the uploaded asset's SHA-256 and downloads
its bytes for an exact readback before publishing the release and marking
it latest. It
refuses an already published release before replacing its live update
manifest; use a new tested release for changes after publication. It
uses only the REST API: `gh release view` can't find a draft whose
@@ -84,6 +86,12 @@ count. So a build reaches people only when you publish it, after testing it.
seconds after starting, then every 6 hours, and anyone can use **Check for
Updates…** (the app menu on macOS, the Help menu elsewhere).
After publication, rerun `audit-release-assets.py` with a new report path.
For a published release it also downloads `update.json`, verifies its
GitHub SHA-256, and checks that the offered version, release page, notes,
installer names, sizes, and digests match that release. This checks the
updater channel's published bytes, which the draft audit cannot inspect.
To pull a bad release, mark the previous one as latest
(`gh release edit v0.3.9 --latest`) or turn the bad one back into a draft.
Copies that already updated stay on it. Nothing downgrades them.
+40 -1
View File
@@ -111,6 +111,38 @@ def download_asset(asset, path):
return digest
def validate_update_manifest(path, release):
"""Check the bytes running copies read against the published release."""
try:
manifest = json.loads(path.read_text(encoding="utf-8"))
except (UnicodeError, json.JSONDecodeError) as error:
raise AuditError(f"update.json is not valid UTF-8 JSON: {error}") from error
if not isinstance(manifest, dict):
raise AuditError("update.json is not an object")
tag = release.get("tag_name")
if not isinstance(tag, str) or not re.fullmatch(r"v\d+\.\d+\.\d+", tag):
raise AuditError("published release has an invalid tag")
if manifest.get("version") != tag[1:]:
raise AuditError("update.json version differs from the release tag")
page = f"https://github.com/{REPO}/releases/tag/{tag}"
if manifest.get("page") != page:
raise AuditError("update.json release page differs from the tag page")
if manifest.get("notes") != (release.get("body") or "")[:4000]:
raise AuditError("update.json notes differ from the release notes")
expected = {asset["name"]: {key: asset[key] for key in ("name", "size", "digest")}
for asset in release["assets"] if asset["name"] in INSTALLERS}
assets = manifest.get("assets")
if not isinstance(assets, list) or len(assets) != len(expected):
raise AuditError("update.json has an incomplete installer list")
names = [asset.get("name") for asset in assets if isinstance(asset, dict)]
if (len(names) != len(assets) or not all(isinstance(name, str) for name in names)
or len(names) != len(set(names))):
raise AuditError("update.json has invalid or duplicate installer entries")
if {asset["name"]: asset for asset in assets} != expected:
raise AuditError("update.json installers differ from the release assets")
return {"version": manifest["version"], "installers": len(assets)}
def inspect_package(path, name, version):
if name.endswith(".zip"):
with zipfile.ZipFile(path) as archive:
@@ -190,6 +222,12 @@ def audit(tag, report_path, launch_x86_appimage=False):
sums_hash = download_asset(by_name["SHA256SUMS"], sums_path)
if sums_path.read_bytes() != expected_sums:
raise AuditError("SHA256SUMS bytes differ from the ten GitHub asset digests")
manifest_detail = None
if not release.get("draft"):
manifest_path = scratch / "update.json"
manifest_digest = download_asset(by_name["update.json"], manifest_path)
manifest_detail = validate_update_manifest(manifest_path, release)
manifest_detail["sha256"] = manifest_digest
rows = []
for name in INSTALLERS:
path = scratch / name
@@ -210,7 +248,8 @@ def audit(tag, report_path, launch_x86_appimage=False):
print(f"verified {name}", flush=True)
report = {"tag": tag, "releaseId": release["id"],
"tagNameAtAudit": release.get("tag_name"), "draftAtAudit": release.get("draft"),
"sha256sumsSha256": sums_hash, "assets": rows}
"sha256sumsSha256": sums_hash, "updateManifest": manifest_detail,
"assets": rows}
with report_path.open("x", encoding="utf-8") as output:
json.dump(report, output, indent=2)
output.write("\n")
+17 -1
View File
@@ -139,7 +139,23 @@ for asset in $old; do
gh api -X DELETE "repos/$repo/releases/assets/$asset" >/dev/null
done
gh api -X POST "https://uploads.github.com/repos/$repo/releases/$id/assets?name=update.json" \
-H "Content-Type: application/json" --input "$tmp/update.json" >/dev/null
-H "Content-Type: application/json" --input "$tmp/update.json" > "$tmp/uploaded-manifest.json"
asset_id=$(python3 - "$tmp/uploaded-manifest.json" "$tmp/update.json" <<'EOF'
import hashlib, json, pathlib, sys
asset = json.load(open(sys.argv[1], encoding="utf-8"))
body = pathlib.Path(sys.argv[2]).read_bytes()
digest = "sha256:" + hashlib.sha256(body).hexdigest()
if (asset.get("name") != "update.json" or not isinstance(asset.get("id"), int)
or asset["id"] <= 0 or asset.get("size") != len(body)
or asset.get("digest") != digest):
sys.exit("not publishing: uploaded update.json metadata differs from its source bytes")
print(asset["id"])
EOF
)
gh api -H "Accept: application/octet-stream" "repos/$repo/releases/assets/$asset_id" > "$tmp/update-readback.json"
cmp -s "$tmp/update.json" "$tmp/update-readback.json" || {
echo "not publishing: downloaded update.json differs from its source bytes" >&2; exit 1;
}
echo "ok update.json"
gh api -X PATCH "repos/$repo/releases/$id" -f tag_name="$tag" -f body="$(cat "$tmp/release-body.txt")" -F draft=false -F prerelease=false \
+5
View File
@@ -14,6 +14,11 @@
"shim": null,
"probe": "toggle-settings"
},
{
"id": 63,
"shim": null,
"probe": "free-loader"
},
{
"id": 12,
"shim": "music-control-system-dbus-env",
+24
View File
@@ -240,6 +240,30 @@ def main():
report['result'] = ('original empty argument bug reproduced: thumbnail policy ignored'
if args.reproduce_fileserver_empty_flag_regression else
'discovery, API1 settings, native HTTPS listing/download, thumbnail policy and stop passed; uploads disabled')
elif case['probe'] == 'free-loader':
settings = current('settings_read')
assert settings['notify_on_free_games'] is True, settings
assert settings['larger_text'] is False, settings
assert current('read_deals') == {}
# This plugin commits defaults on startup and writes an empty
# deals cache on first read. Both paths are private to this
# disposable container; no feed, notification or store call.
settings_dir = base/'settings'/record['folder']
assert (settings_dir/'settings.json').is_file()
assert (settings_dir/'deal_db.json').is_file()
current('settings_setSetting', 'larger_text', True)
assert current('settings_read')['larger_text'] is True
os.killpg(backend.pid, signal.SIGTERM)
backend.wait(timeout=8)
backend = subprocess.Popen([str(loader)], env=env, stdout=log, stderr=log,
start_new_session=True)
processes.append(backend)
until_ready(lambda: rpc('loader/get_plugins'), 'restarted Decky discovery')
assert until_ready(lambda: current('settings_read')['larger_text'],
'private Free Loader setting after restart') is True
assert current('read_deals') == {}
report['result'] = ('discovery, default settings, empty local deals, '
'private setting write/read and restart persistence passed; feed, notifications and VR unverified')
elif case['probe'] == 'toggle-settings':
def call(method, *values):
result = rpc('loader/call_plugin_method', record['name'], method, *values)
+11 -3
View File
@@ -9,6 +9,7 @@ $FAKEGH_NEW_ID (default 9001) without adding it to the listing, like GitHub's li
lagging a fresh draft. With jq installed, --jq filters a response the way gh does
(raw strings, compact JSON)."""
import json
import hashlib
import os
import shutil
import subprocess
@@ -120,10 +121,17 @@ elif method == "POST" and endpoint == repo + "releases":
answer(new, json.dumps(new))
elif method == "DELETE" and endpoint.startswith(repo + "releases/assets/"):
pass
elif method == "GET" and endpoint == repo + "releases/assets/901":
data = open(os.environ["FAKEGH_UPLOAD"], "rb").read()
if os.environ.get("FAKEGH_CORRUPT_MANIFEST_READBACK") == "1":
data += b" "
sys.stdout.buffer.write(data)
elif method == "POST" and endpoint.startswith("https://uploads.github.com/" + repo + "releases/"):
with open(inp) as src, open(os.environ["FAKEGH_UPLOAD"], "w") as dst:
dst.write(src.read())
print("{}")
data = open(inp, "rb").read()
with open(os.environ["FAKEGH_UPLOAD"], "wb") as dst:
dst.write(data)
print(json.dumps({"id": 901, "name": "update.json", "size": len(data),
"digest": "sha256:" + hashlib.sha256(data).hexdigest()}))
elif method == "PATCH" and endpoint.startswith(repo + "releases/"):
rid = endpoint[len(repo + "releases/"):]
if not rid.isdigit(): # what GitHub said to v0.4.2's PATCH of releases/ (an empty id)
+49
View File
@@ -2,6 +2,7 @@
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import hashlib
import importlib.util
import json
import plistlib
import struct
import tempfile
@@ -33,6 +34,34 @@ def release_assets():
class AssetAudit(unittest.TestCase):
def test_published_manifest_matches_release_bytes(self):
assets, _ = release_assets()
release = {"tag_name": "v9.8.7", "body": "Release notes", "assets": assets}
manifest = {
"version": "9.8.7",
"page": "https://github.com/saphid/frame-control/releases/tag/v9.8.7",
"notes": "Release notes",
"assets": [{key: asset[key] for key in ("name", "size", "digest")}
for asset in assets if asset["name"] in audit.INSTALLERS],
}
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "update.json"
path.write_text(json.dumps(manifest), encoding="utf-8")
self.assertEqual(audit.validate_update_manifest(path, release),
{"version": "9.8.7", "installers": 10})
for change in (
{"version": "9.8.6"},
{"page": "https://github.com/elsewhere/releases/tag/v9.8.7"},
{"notes": "stale notes"},
{"assets": manifest["assets"][:-1]},
{"assets": [dict(manifest["assets"][0], digest="sha256:" + "0" * 64),
*manifest["assets"][1:]]},
):
with self.subTest(change=change):
path.write_text(json.dumps({**manifest, **change}), encoding="utf-8")
with self.assertRaises(audit.AuditError):
audit.validate_update_manifest(path, release)
def test_complete_metadata_uses_release_checksum_order(self):
assets, sums = release_assets()
by_name, expected = audit.validate_assets({"assets": assets})
@@ -118,6 +147,26 @@ class AssetAudit(unittest.TestCase):
audit.audit("v9.8.7", report)
self.assertFalse(report.exists())
def test_bad_published_manifest_creates_no_report(self):
assets, sums = release_assets()
assets.append({"id": 100, "name": "update.json", "size": 2,
"digest": "sha256:" + hashlib.sha256(b"{}").hexdigest()})
release = {"id": 42, "tag_name": "v9.8.7", "draft": False,
"body": "Release notes", "assets": assets}
def download(asset, path):
path.write_bytes(sums if asset["name"] == "SHA256SUMS" else b"{}")
return asset["digest"][7:]
with tempfile.TemporaryDirectory() as directory:
report = Path(directory) / "report.json"
with mock.patch.object(audit, "find_release", return_value=release), \
mock.patch.object(audit, "download_asset", side_effect=download), \
mock.patch.object(audit.shutil, "which", return_value="/fake/tool"):
with self.assertRaisesRegex(audit.AuditError, "version differs"):
audit.audit("v9.8.7", report)
self.assertFalse(report.exists())
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -21,7 +21,7 @@ class RuntimeCoverage(unittest.TestCase):
self.assertIn(case['id'], official)
self.assertIn(case['probe'], ('mpris', 'screenshots', 'system-monitor', 'timer-lifecycle',
'toggle-settings', 'fileserver', 'localsend', 'network-sentinel',
'battery-tracker'))
'battery-tracker', 'free-loader'))
def test_new_shim_without_runtime_case_fails_coverage(self):
import frame_decky_store
+14 -2
View File
@@ -37,14 +37,16 @@ def draft(tag_name="untagged-c6ddfed7f75d67db2e99", name="Frame Control 9.8.7",
class PublishRelease(unittest.TestCase):
def run_script(self, releases, *args, tags="v9.8.7", gates=None):
def run_script(self, releases, *args, tags="v9.8.7", gates=None,
corrupt_manifest_readback=False):
d = Path(tempfile.mkdtemp())
(d / "releases.json").write_text(json.dumps(releases))
(d / "gates.json").write_text(json.dumps(gates or {}))
env = dict(os.environ, PATH="%s:%s" % (FAKEGH, os.environ["PATH"]),
FAKEGH_RELEASES=str(d / "releases.json"), FAKEGH_TAGS=tags,
FAKEGH_LOG=str(d / "log"), FAKEGH_UPLOAD=str(d / "upload.json"),
FAKEGH_GATES=str(d / "gates.json"))
FAKEGH_GATES=str(d / "gates.json"),
FAKEGH_CORRUPT_MANIFEST_READBACK="1" if corrupt_manifest_readback else "0")
p = subprocess.run(["sh", str(SCRIPT), *args], env=env, capture_output=True, text=True, timeout=30)
log = [json.loads(line) for line in (d / "log").read_text().splitlines()] if (d / "log").exists() else []
upload = json.loads((d / "upload.json").read_text()) if (d / "upload.json").exists() else None
@@ -60,11 +62,21 @@ class PublishRelease(unittest.TestCase):
self.assertEqual(sorted(a["name"] for a in upload["assets"]), sorted(INSTALLERS))
self.assertIn(["api", "-X", "DELETE", "repos/saphid/frame-control/releases/assets/7"], log)
patch = next(c for c in log if "PATCH" in c)
readback = next(c for c in log if any("releases/assets/901" in arg for arg in c))
self.assertLess(log.index(readback), log.index(patch))
self.assertEqual(patch[3], "repos/saphid/frame-control/releases/42")
for f in ("tag_name=v9.8.7", "draft=false", "prerelease=false", "make_latest=true"):
self.assertIn(f, patch)
self.assertTrue(all(c[0] == "api" for c in log)) # never `gh release ...` (GraphQL)
def test_refuses_mismatched_manifest_readback_before_publication(self):
p, log, upload = self.run_script([draft()], "v9.8.7",
corrupt_manifest_readback=True)
self.assertNotEqual(p.returncode, 0)
self.assertIn("downloaded update.json differs", p.stderr)
self.assertIsNotNone(upload)
self.assertFalse(any("PATCH" in call for call in log))
def test_an_untagged_draft_may_carry_a_subtitle(self):
p, log, upload = self.run_script([draft(name="Frame Control 9.8.7: faster")], "v9.8.7")
self.assertEqual(p.returncode, 0, p.stderr + p.stdout)
+3 -2
View File
@@ -4703,14 +4703,15 @@
"https://github.com/jwhitlow45/free-loader.git"
],
"status": "partial",
"reason": "The exact archive's duplicate names contain identical bytes and differ only in execute bits; the guarded installer now accepts them, and a private local install/remove transaction passed. The current opaque Chromium host renders Free Games but cannot load deals with backend/network denied. Exact official Python settings and empty local-deals methods passed in a no-network container with a Decky stub. Real Decky, feed refresh, notifications and worn VR remain unverified.",
"reason": "The exact archive's byte-identical duplicate ZIP members pass guarded install/remove. Actual Decky 3.3.0 in a private network-isolated container discovers the plugin, reads default settings and empty local deals, and retains a settings change after restart. Live deal-feed refresh, notifications, Steam store integration, real Frame ARM64 service and worn VR remain unverified; no feed request was made.",
"blockInstall": false,
"verification": [
"static archive",
"local browser",
"fake-Frame pinned install/remove",
"local exact-archive transaction",
"isolated Python backend"
"isolated Python backend",
"isolated Linux backend"
],
"browser": {
"id": 63,