Source logins: refuse addresses with a backslash and a login; decode escapes in one pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -213,7 +213,6 @@ Frame's software fits together, all checked against a real headset and labelled
 | [Scripts and headset setup](docs/scripts.md) | The command-line helpers, minimum typing, streaming options, floating panels |
 | [How the Frame works](docs/how-the-frame-works.md) | SteamVR → gamescope → Plasma, verified facts, debugging |
 | [Android apps (Lepton)](docs/apks.md) | Sideloading, the rated F-Droid catalogue, per-app instances |
-| [Import from my Quest](docs/quest-import.md) | Copy games you own off your Quest by USB; what runs, and why licence-checked games don't |
 | [Sideloading Linux and Windows games](docs/sideloading.md) | A .zip, folder or .exe as a Steam Devkit Game, runtime detection |
 | [Install links for websites](docs/web-install.md) | `frame-control://install` links and manifests, the rules, a button to paste |
 | [VR comfort and HUD](docs/vr-utilities.md) · [Steam games](docs/steam-games.md) · [VR video](docs/vr-video.md) · [WebXR in Chromium](docs/webxr-chromium.md) | Installing and buying, watching VR180/360, the Chromium build |

app/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "frame-control",
-  "version": "0.4.5",
+  "version": "0.4.4",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "frame-control",
-      "version": "0.4.5",
+      "version": "0.4.4",
       "license": "MIT",
       "devDependencies": {
         "electron": "^44.4.5",

app/package.json

@@ -1,7 +1,7 @@
 {
   "name": "frame-control",
   "productName": "Frame Control",
-  "version": "0.4.5",
+  "version": "0.4.4",
   "description": "Desktop app for managing a Valve Steam Frame over SSH",
   "private": true,
   "main": "main.js",

docs/apk-repos.md

@@ -58,8 +58,61 @@ form shows the pinned one ("Trusted on first use: …") so you can compare it.
 Re-adding an existing URL preserves its pin; changing it requires deliberately
 removing and re-adding it.
 
+### Repositories behind a login
+
+Some self-hosted repositories sit behind HTTP basic auth. As in F-Droid's own
+client, the login can be part of the address
+(`https://user:pass@example.org/fdroid/repo`), or given separately: in the
+store's **Add a source** form under "This repository needs a login", in the
+server API (`username` and `password` beside `url`), or on the command line,
+where the password is read from standard input so it stays out of shell history
+and `ps`:
+
+```sh
+printf '%s\n' "$PASSWORD" | python3 ui/apk_sources/fdroid.py add 'https://example.org/fdroid/repo' --username me --password-stdin
+```
+
+How the login is kept and used:
+
+- **Kept out of the address.** The `user:pass@` part is removed before anything
+  is saved, so the saved repository URL, its id, the source list, job results
+  and the page never contain it. The store only says "Uses a login".
+- **Stored separately.** Logins live in
+  `frame_host.data_dir('source-credentials.json')`
+  (`~/Library/Application Support/Frame Control/source-credentials.json` on
+  macOS), written with mode 0600 and keyed by source id. They are stored in
+  plain text there, protected by file permissions only (on Windows,
+  the folder's normal ACLs). Removing the repository deletes its login, and
+  the file goes when the last login is removed. Each change holds a lock
+  file (`source-credentials.lock`), so the app and the command line can't undo
+  each other's saves or removals.
+- **HTTPS only.** An `http://` address with a login is refused before any request.
+- **Inside the repository only.** The `Authorization` header is decided
+  separately for every request, redirects included: index, `entry.jar`,
+  APKs and the store's artwork proxy. It is sent only when the request's
+  scheme, host and port match the repository's and its path is inside the
+  repository's directory by whole path segments. A redirect to another host
+  or port, or to another path on the same server, gets no login, and that
+  request's own `Authorization` header is removed. Paths with `.` or `..`
+  segments (plain or percent-encoded), encoded `/` or `\`, double encoding
+  or `;` get no login, since a server could normalise them to a path
+  outside the repository. Index file names like that are refused outright.
+- **Never echoed.** Logins are not written to logs, error messages, API
+  responses or the cache. The CLI prints only the saved source.
+- **Checked before saving.** As with any repository, the whole index must
+  download and verify before anything is saved. HTTP 401 (or 403 when a login
+  was sent) stops the add with "check the username and password" and saves
+  nothing: no repository, no login, no rollback state. A 403 with a login is
+  not treated as rate limiting.
+- **Same verification.** The login changes only who may download. Signature,
+  fingerprint pin or TOFU, index hash and rollback checks apply exactly as for
+  any other repository.
+
+Re-adding the same address with a new login replaces the saved one. Re-adding
+it without a login keeps the saved one.
+
 The API for the search/server integration is in `ui/apk_sources/fdroid.py`:
-`add_repo(url, fingerprint=None, name=None)`, `remove_repo(source_id)`,
+`add_repo(url, fingerprint=None, name=None, username=None, password=None)`, `remove_repo(source_id)`,
 `set_enabled(source_id, enabled)`, and `user_repos()`. The module also exposes
 `sources`, `search`, `details`, and `download` from the shared source contract.
 This change supplies the CLI and API; the integrated source-management UI is

docs/apk-sources.md

@@ -45,10 +45,23 @@ search. Results outside the curated list stay page-only, even if a repository
 claims an open-source license. This prevents an arbitrary tagged mirror from
 becoming a trusted downloader. Extend the curated JSON after provenance review.
 
-Set optional `FRAME_GITHUB_TOKEN` in the process environment for a higher API
-quota. Tokens are sent only to `api.github.com`, never written to the cache,
-never sent to asset hosts, and removed on redirects. The adapter does not
-read `gh` credentials automatically. Metadata is cached for one hour under
+For a higher API quota, add a GitHub access token in the store's source
+settings (the GitHub row's token field) or through the server API
+(`POST /api/sources` with `action: "set-token"` or `"clear-token"` and
+`source: "github"`). `FRAME_GITHUB_TOKEN` in the process environment still
+works and wins over a saved token. A saved token is kept with repository
+logins in `source-credentials.json` (mode 0600, see
+[APK repositories](apk-repos.md#repositories-behind-a-login)). The source list
+reports only whether a token is set, where it came from and whether GitHub
+rejected it, never its value. Tokens are sent as `Authorization: Bearer` only to
+`api.github.com`, never written to the cache, never sent to asset hosts (public
+release downloads don't need them), and removed on redirects. If GitHub answers
+401 to the token, the request is retried once without it, as Obtainium does.
+The rejection is logged without the token and shown in source settings. The
+token is not tried again until it is saved again or the app restarts. Rate-limit
+backoff is kept per login (anonymous, or each token), so saving a token after
+anonymous requests were limited lets the next request go out with it. The
+adapter does not read `gh` credentials automatically. Metadata is cached for one hour under
 `frame_host.cache_dir('apk-sources', 'publisher')`. A cold details request
 fetches at most ten releases. Rate-limit errors are surfaced without retry
 loops. Asset IDs and release tags are not Android version codes: metadata

docs/apks.md

@@ -39,12 +39,6 @@ section below), instead of going into Lepton Development:
    run at once alongside Lepton Development, each in its own container
    (`lepton-steamlaunch-<id>`, ADB on 5556, 5557, …).
 
-Split APKs (a base plus `split_*.apk` files) install as one app:
-`python3 ui/frame_android.py install base.apk split1.apk …`. Lepton runs one
-APK, so splits that add native libraries or assets are merged into the base
-and re-signed; density and language splits are refused. See
-[quest-import.md](quest-import.md#split-apks). Not yet tried on a real Frame.
-
 Verified with AntennaPod and Tabletop Tools: installed in about 7 s, launched
 from the shortcut, stopped, and relaunched with their data intact. ADB and
 Lepton Development aren't involved.

docs/privacy.md

@@ -33,12 +33,6 @@ computer, exactly as they were sent.
   yours.
 - Every event includes the app version, OS name (macOS, Windows or Linux),
   CPU architecture and Python version.
-- `frame_connected` carries `headset_id`: the first 16 hex digits of a
-  SHA-256 hash of the headset's `/etc/machine-id`, worked out on the headset,
-  so the machine-id itself never leaves it. It is the same for one headset on
-  every computer, which lets us count headsets rather than installs; it can't
-  be turned back into the machine-id or a serial number. `headsets_saved` is how
-  many headsets this copy of Frame Control has set up.
 
 ## Usage events
 
@@ -47,7 +41,7 @@ computer, exactly as they were sent.
 | `app_installed` | First run | |
 | `app_updated` | First run of a new version | `from_version` |
 | `app_opened` | At most once a day | |
-| `frame_connected` | The first time a SteamOS build is seen on each headset | `steamos_build`, `steamos_version`, `headset_id`, `headsets_saved` |
+| `frame_connected` | The first time a SteamOS build is seen | `steamos_build`, `steamos_version` |
 | `tab_viewed` | The first click on each tab in a session | `tab` |
 | `install_finished` | Any install finishes, working or not | `kind` (apk, flatpak, steam, title, web), `ok`, `seconds`, `error_category` (only when `ok` is false), `installer_code`, `xr_layer_missing`, and see below |
 | `update_offered`, `update_started`, `update_failed` | The update banner | `to_version`, `error_category` |
@@ -57,15 +51,12 @@ error becomes one category from this fixed list, plus Android's own
 `INSTALL_FAILED_…` code (`installer_code`) when there is one:
 `android_installer`, `apk_needs_newer_android`, `apk_wrong_abi`,
 `layer_missing`, `apk_repack_failed`, `tool_missing`, `apk_unreadable`,
-`cant_run_on_frame`, `steam_shortcut`, `steam_not_running`,
-`steam_not_responding`, `steam_js_error`, `artwork_failed`, `copy_timeout`,
-`frame_not_set_up`, `frame_auth`, `frame_unreachable`, `frame_disk_full`,
-`copy_failed`, `download_failed`, `flatpak`, `cancelled`, `lepton`,
-`internal_error` (a mistake in Frame Control's own code) or `other`. A VR
-APK that installed without Frame Control's OpenXR compatibility layer,
-because this copy of the app is missing it, carries `xr_layer_missing: true`;
-otherwise that field is left out. It names what was installed only when
-that's already public:
+`cant_run_on_frame`, `steam_shortcut`, `frame_not_set_up`, `frame_auth`,
+`frame_unreachable`, `frame_disk_full`, `download_failed`, `flatpak`,
+`cancelled`, `lepton` or `other`. A VR APK that installed without Frame
+Control's OpenXR compatibility layer, because this copy of the app is missing
+it, carries `xr_layer_missing: true`; otherwise that field is left out. It
+names what was installed only when that's already public:
 
 - F-Droid catalogue apps: `package`. Never the version, since a local build can reuse a
   catalogue app's package name

docs/quest-import.md

@@ -1,195 +0,0 @@
-# Import from my Quest
-
-Move games you own from your own Meta Quest onto the Steam Frame. Plug the
-Quest into your computer by USB; Frame Control copies the game off it with
-adb, checks it can run in Lepton, and installs it on the Frame like any other
-APK, with its own Lepton instance and Steam library entry
-([apks.md](apks.md), [vr-apks.md](vr-apks.md)).
-
-There is no licence-check bypass, and there won't be. A game that checks its
-Meta store licence is refused, with the reason. See
-[What won't run](#what-wont-run-and-why) below.
-
-## Before you start: developer mode on the Quest
-
-adb only reaches a Quest with developer mode on:
-
-1. Make a free Meta developer organisation at
-   [developers.meta.com](https://developers.meta.com/horizon/) on a **verified**
-   Meta account (Meta asks for a phone number or payment card to verify).
-2. In the Meta Horizon phone app, open the headset's settings and switch
-   **Developer Mode** on. Restart the Quest.
-3. Plug the Quest into the computer that runs Frame Control with a USB cable
-   that carries data (some charging cables don't).
-4. Put the headset on. Accept **Allow USB debugging**, and tick **Always allow
-   from this computer**.
-
-Frame Control uses the adb it already uses for the Frame (an adb you
-installed, else the one it bundles). On Linux, adb also needs permission to
-open the Quest: add a udev rule for Meta's USB vendor id `2833`, for example
-`SUBSYSTEM=="usb", ATTR{idVendor}=="2833", MODE="0660", GROUP="plugdev"`, and
-be in that group. Then unplug and replug.
-
-The Frame's own adb connections (Frame Control's SSH tunnels to Lepton,
-`127.0.0.1:<port>`) are never touched. Only a device that reports itself as
-Oculus or Meta, or a Quest model, counts as a Quest.
-
-## Using it
-
-In the app: **Android → On your Frame → Import from my Quest**. Choose **Look
-for my Quest**, then **Import** next to a game. Tick **Also copy each game's
-files in Android/data** to bring the game's external files too (some games
-keep saves there). Progress shows in the panel; the result and any reason it
-stopped appear in the message log.
-
-A game with game data (an OBB folder, or Android/data when ticked) takes two
-imports, because Android can only take that data while the game's Lepton
-instance is running, and there is no way to start the instance without the
-game. Frame Control never starts the game itself:
-
-1. **Import** installs the game and says it has data to add.
-2. Start the game from the Steam library and leave it on its first screen.
-3. **Import** again. Frame Control sees the game open with the same version,
-   copies the data in, and closes the game. Start it again to play.
-
-A game that quits straight away without its data can't take it this way.
-
-From a terminal:
-
-```sh
-python3 ui/frame_quest.py devices                      # what adb sees, and which is the Quest
-python3 ui/frame_quest.py list                         # apps on the Quest: split or not, OBB and data sizes
-python3 ui/frame_quest.py import com.example.game      # copy one to the Frame
-python3 ui/frame_quest.py import com.example.game --data --serial 1WMHH000XXXXXX
-```
-
-The API is `GET /api/quest` (the Quest, or guidance when there isn't one, and
-its apps) and `POST /api/quest` with
-`{"action": "import", "package": "...", "serial": "...", "data": false}`,
-which returns a background job (`/api/job`) with a stage and percentage.
-
-## What an import does
-
-1. Finds the Quest (`adb devices -l`, then `ro.product.manufacturer` and
-   `ro.product.model`). If none is ready it says why: nothing connected, not
-   authorised yet (the **Allow USB debugging** prompt), no USB permission
-   (Linux), offline, another Android device, or several Quests.
-2. Lists third-party apps (`pm list packages -3`), each with `pm path` (one
-   line is a single APK, several a split APK) and the size of
-   `/sdcard/Android/obb/<package>` and `/sdcard/Android/data/<package>`.
-   Labels aren't shown before import: reading them means copying the APK.
-3. Checks there is room on the computer (the APKs, a merged and a patched copy
-   of them counted uncompressed, the OBB folder and the Android/data folder
-   once, since that is streamed to the Frame), then copies the APK and its splits
-   into a per-user cache folder (`Frame Control/quest-import` under the
-   system's cache folder).
-4. **Checks the APK before anything else is copied**, refusing:
-   - Meta's Platform SDK licence check (`libovrplatformloader.so`, in the base
-     or any split);
-   - 32-bit-only builds (native code but no `arm64-v8a`);
-   - apps that need an Android newer than Lepton's (API 30);
-   - split APKs Frame Control can't merge (below).
-   VrApi-only builds (`libvrapi.so`, no OpenXR loader) are flagged, as for any
-   APK: they install, but won't run.
-5. If the game isn't open on the Frame: installs it through the normal APK
-   install (`frame_android.install`: VR detection, the launcher fix, the
-   OpenXR compatibility layer, artwork), and says whether it has game data
-   to add. An older Quest copy never replaces a newer version on the Frame
-   (compared by version code, which installs now record; for installs made
-   before that, it is read from the installed APK on the Frame) unless you
-   tick **Allow replacing a newer version** (`--allow-downgrade`,
-   `"allowDowngrade": true`). If the installed version can't be read at all,
-   that permission is needed too.
-6. If the game is open on the Frame (the second import): refuses unless
-   Frame Control's install record is for this package, in this package's own
-   instance, at the Quest's exact version code (an open game is never
-   replaced), otherwise copies the OBB folder (and `Android/data` if ticked)
-   from the Quest, waits for Android to have finished booting, copies standard
-   `main|patch.<version>.<package>.obb` files with the existing OBB helper
-   (checked by SHA-256 inside Android, [sidequest.md](sidequest.md#obb-files))
-   once Android's own `dumpsys package` in that instance shows the same
-   package at the same version code,
-   streams the `Android/data` files in as a tar written straight into the SSH
-   transfer, then closes the game so its next start reads them, and checks
-   its instance has stopped; if it hasn't, or the check itself fails (a
-   failed `podman ps` never reads as "stopped"), the result says so. The game
-   is closed whatever happens during the copy, including a local read error
-   or Ctrl-C, and a failed close is reported without hiding the original
-   error. Everything
-   from the second check of the install record (after the pulls, which can
-   be long) to closing the game holds the same lock as installs and removals.
-   Other files in the OBB folder are listed as not copied.
-7. Deletes the cache folder, whether the import worked or not. A folder left
-   by a crash is removed by a later import after six hours.
-
-Only one import runs at a time. If copying game data fails, the game is
-closed and the message says so; start it and import again.
-
-### Split APKs
-
-Lepton runs one APK per instance, so the shared install path now merges a
-split APK into one (`frame_android.install(base, splits=[...])`, or
-`python3 ui/frame_android.py install base.apk split1.apk ...`). Merging copies
-each split's native libraries and assets into the base, drops the base
-manifest's `isSplitRequired` flag, and re-signs with Frame Control's key, as
-the launcher fix already does. Only splits that hold nothing but
-`lib/<abi>/*.so`, `assets/` and their own signature, manifest and an empty
-resource table are merged. Refused, with the reason, before anything reaches
-the Frame: splits with resources (screen density or language), code
-(`classes*.dex`), any other file, a manifest that declares components or
-libraries (`<service>`, `<uses-library>`...), feature splits, and splits of
-another package. The merged files are held in memory while the APK is
-repacked and signed (as for every patched APK), so a merge adds at most
-512 MB per file and 1 GB in all. Each file is read from its raw bytes and
-checked against both of its ZIP headers: the local header must name the same
-file with the same method, sizes and checksum as the central directory (or a
-data descriptor must carry them; ZIP64 sizes are understood), the data must
-end before the next entry, a stored file must be exactly its declared size, a
-compressed one must end exactly at the end of its data with nothing after it,
-and its checksum must match. So a forged header can't truncate a file or fill
-memory. A resource table counts as empty only
-if it defines nothing at all (styles and other complex entries included). A merged, re-signed APK can't update a
-copy signed by the publisher.
-
-## What won't run, and why
-
-- **Games that check their Meta store licence.** Store games call Meta's
-  Platform SDK (`libovrplatformloader.so`) to confirm you bought them. That
-  needs Meta's services, which only exist on Meta headsets, so on the Frame
-  the game quits. Frame Control refuses these and **won't remove or work
-  around the check**, and doesn't bundle, call or link to any tool that does.
-  Ask the developer for a build without it, or get the game from Steam or
-  itch.io. Most Meta Store games have it; most SideQuest, itch.io and
-  open-source games don't.
-- **32-bit games** (Quest 1 era): the Frame runs 64-bit ARM only.
-- **VrApi-only games**: the Frame has OpenXR, not Meta's older VrApi.
-- **Anything else that needs Meta**: avatars, Quest-only device checks,
-  passthrough and other `XR_FB_*` features SteamVR lacks
-  ([vr-apks.md](vr-apks.md#what-steamvrs-android-runtime-supports-verified)).
-- **Private app data** (`/data/data/<package>`) can't be read over adb without
-  root, so only saves in `Android/data` come across.
-
-What usually works: OpenXR games without Meta's licence check, for example
-open-source and itch.io builds you sideloaded onto the Quest.
-
-Only import games you own, for your own use.
-
-## Evidence and limits
-
-**Verified locally** with a fake adb (`tests/fakeadb.py`, `tests/test_quest.py`):
-device detection (Quest, the Frame's tunnel, the Frame over USB, other
-Android devices, unauthorised, no permissions, offline, none, several),
-package listing and split detection, the refusals, the order of copies
-(APK checked before the OBB is copied), the two-step install then data copy
-(the game is never started by Frame Control, never replaced while open, and
-closed after its data is copied), split merging and signing, the split
-refusals and size limits (including a forged zip bomb), the space estimate,
-and cache clean-up after success and failure.
-
-**Not verified:** a real Quest (none was available), Quest system apps in
-`pm list packages -3`, real split APKs from the Meta Store, OBB and
-`Android/data` writes into a running Lepton instance on a real Frame
-(the OBB helper itself is also unverified there), whether games stay open
-without their OBB long enough for the second import, and the Linux udev rule.
-Large games are copied into memory while being merged or patched, as every
-patched APK already is.

docs/vr-apks.md

@@ -214,8 +214,7 @@ source-search endpoint remain unverified (the public install seam is tested).
 
 - **Meta entitlement.** Apps that call the Oculus Platform SDK
   (`libovrplatformloader.so`) to check the Quest store licence need Meta's
-  services. Frame Control won't work around that. [Import from my
-  Quest](quest-import.md) refuses these before copying anything to the Frame.
+  services. Frame Control won't work around that.
 - **VrApi-era apps** (`libvrapi.so`, before OpenXR) need an API translator,
   not a patch.
 

tests/fakeadb.py

@@ -1,100 +0,0 @@
-#!/usr/bin/env python3
-"""A stand-in for Android's adb, for tests/test_quest.py: just enough of `devices -l`,
-`shell`, and `pull` for frame_quest.py, run as a real subprocess.
-
-$FAKEADB_STATE is a JSON file:
-  devices   [{"serial", "state", "extra" (the rest of the `devices -l` line), "props" {getprop name: value}}]
-  root      a folder standing in for each device's file system (remote /x/y is root/x/y)
-  packages  {package: [APK paths, as `pm path` prints them]}
-  fail_pull a remote path whose pull fails
-Every call is appended to $FAKEADB_LOG as a JSON line. Shell commands are the simple
-`; `-separated ones frame_quest sends, each answered here: no real shell runs.
-"""
-import json
-import os
-import shutil
-import sys
-
-args = sys.argv[1:]
-with open(os.environ['FAKEADB_LOG'], 'a') as f:
-    f.write(json.dumps(args) + '\n')
-with open(os.environ['FAKEADB_STATE']) as f:
-    state = json.load(f)
-root = state.get('root', '')
-serial = None
-if args[:1] == ['-s']:
-    serial, args = args[1], args[2:]
-
-
-def fail(message, code=1):
-    sys.stderr.write(message + '\n')
-    sys.exit(code)
-
-
-def local(remote):
-    return os.path.join(root, remote.lstrip('/'))
-
-
-def size_kb(path):
-    total = 0
-    for dirpath, _, files in os.walk(path):
-        total += sum(os.path.getsize(os.path.join(dirpath, n)) for n in files)
-    return max(4, total // 1024)
-
-
-if args == ['devices', '-l']:
-    print('List of devices attached')
-    for d in state.get('devices', []):
-        print(f"{d['serial']:<22} {d['state']} {d.get('extra', '')}".rstrip())
-    sys.exit(0)
-
-device = next((d for d in state.get('devices', []) if d['serial'] == serial), None)
-if serial is None or device is None:
-    fail(f"adb: device '{serial}' not found")
-if device['state'] == 'unauthorized':
-    fail('adb: device unauthorized.\nThis adb server\'s $ADB_VENDOR_KEYS is not set')
-if device['state'] != 'device':
-    fail(f"adb: device {device['state']}")
-
-if args[:1] == ['shell']:
-    out = []
-    for command in ' '.join(args[1:]).split('; '):
-        command = command.replace(' 2>/dev/null', '').strip()
-        words = command.split()
-        if not words or command == 'true':
-            continue
-        if words[0] == 'getprop':
-            out.append(device.get('props', {}).get(words[1], ''))
-        elif words[0] == 'echo':
-            out.append(' '.join(words[1:]))
-        elif command == 'pm list packages -3':
-            out += ['package:' + p for p in state.get('packages', {})]
-        elif words[:2] == ['pm', 'path']:
-            out += ['package:' + p for p in state.get('packages', {}).get(words[2], [])]
-        elif words[:2] == ['du', '-sk']:
-            if os.path.isdir(local(words[2])):
-                out.append(f'{size_kb(local(words[2]))}\t{words[2]}')
-        elif words[:3] == ['stat', '-c', '%s']:
-            out += [str(os.path.getsize(local(p.strip("'")))) for p in words[3:]]
-        else:
-            fail(f'/system/bin/sh: {words[0]}: inaccessible or not found', 127)
-    print('\n'.join(out))
-    sys.exit(0)
-
-if args[:1] == ['pull'] and len(args) == 3:
-    remote, dest = args[1], args[2]
-    if remote == state.get('fail_pull'):
-        fail(f"adb: error: failed to copy '{remote}' to '{dest}': remote Connection reset")
-    src = local(remote)
-    if not os.path.exists(src):
-        fail(f"adb: error: failed to stat remote object '{remote}': No such file or directory")
-    if os.path.isdir(dest):
-        dest = os.path.join(dest, os.path.basename(remote.rstrip('/')))
-    if os.path.isdir(src):
-        shutil.copytree(src, dest)
-    else:
-        shutil.copyfile(src, dest)
-    print(f'{remote}: 1 file pulled.')
-    sys.exit(0)
-
-fail('adb: unknown command ' + ' '.join(args))

tests/page/source_login.mjs

@@ -0,0 +1,79 @@
+// The store's "Add a source" login handling and GitHub token field, run in node against
+// the real functions from ui/index.html. A login typed into the address must leave the
+// address field (into the login fields) before anything is sent, and stay out of it.
+import { readFileSync } from "fs";
+const src = readFileSync(new URL("../../ui/index.html", import.meta.url), "utf8");
+const lines = src.split("\n");
+const fail = msg => { console.log("FAIL " + msg); process.exit(1); };
+
+function grab(name) {
+  const start = lines.findIndex(l => new RegExp(`^(?:(?:async )?function ${name}\\(|const ${name} = )`).test(l));
+  if (start < 0) fail(`no top-level ${name} in index.html`);
+  let depth = 0;
+  for (let i = start; i < lines.length; i++) {
+    for (const c of lines[i]) depth += c === "{" ? 1 : c === "}" ? -1 : 0;
+    if (depth <= 0) return lines.slice(start, i + 1).join("\n") + "\n";
+  }
+  fail(`unbalanced ${name}`);
+}
+const { moveLogin, sourceToken } = new Function(
+  "const esc = s => String(s).replace(/[&<>\"']/g, c => `&#${c.charCodeAt(0)};`); const storeName = s => s;\n" +
+  grab("decodeLenient") + grab("moveLogin") + grab("sourceToken") + "return { moveLogin, sourceToken };")();
+
+function form(url, username = "", password = "") {
+  const details = { open: false };
+  const field = value => ({ value, closest: () => details });
+  return { details, elements: { url: field(url), username: field(username), password: field(password) } };
+}
+
+let f = form("https://alice:s3cr%40t@repo.example/fdroid/repo");
+moveLogin(f);
+if (f.elements.url.value !== "https://repo.example/fdroid/repo") fail("address still holds the login: " + f.elements.url.value);
+if (f.elements.username.value !== "alice" || f.elements.password.value !== "s3cr@t") fail("login not moved into its fields");
+if (!f.details.open) fail("login fields not shown");
+
+f = form("https://alice:fromurl@repo.example/repo", "typed", "typedpw");
+moveLogin(f);
+if (f.elements.url.value.includes("fromurl") || f.elements.username.value !== "typed" || f.elements.password.value !== "typedpw")
+  fail("typed login replaced, or address kept the login");
+
+// Undecodable logins and unusual spellings the URL parser accepts still lose their login; undecodable bytes stay as typed.
+for (const [url, address, user, pass] of [
+  ["https://alice:100%done@repo.example/private/repo", "https://repo.example/private/repo", "alice", "100%done"],
+  ["https:\t//alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
+  ["ht\ntps://alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
+  ["https:alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
+  ["\u0000https://alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
+  ["https://alice:%FF@repo.example/private/repo", "https://repo.example/private/repo", "alice", "%FF"],
+  ["https://al%20ice:p%40ss:word@repo.example/r", "https://repo.example/r", "al ice", "p@ss:word"],
+  ["https://alice:p%FF\u00e4@repo.example/repo", "https://repo.example/repo", "alice", "p%FF\u00e4"],
+  ["fdroidrepos://alice:pw@repo.example/repo?fingerprint=AB", "fdroidrepos://repo.example/repo?fingerprint=AB", "alice", "pw"],
+  ["https://alice@repo.example/repo", "https://repo.example/repo", "alice", ""],
+]) {
+  f = form(url);
+  moveLogin(f);
+  if (f.elements.url.value !== address) fail(`address kept the login: ${url} -> ${f.elements.url.value}`);
+  if (f.elements.username.value !== user || f.elements.password.value !== pass)
+    fail(`login for ${url}: ${f.elements.username.value} / ${f.elements.password.value}`);
+}
+
+// An address the parser rejects that may hold a login is refused and left as typed, never sent.
+for (const url of ["https://alice\\name:pw@repo.example/repo", "https:\\\\alice:s3cret@repo.example/repo", "fdroidrepos://alice:p\\w@repo.example/repo", "https://alice:s3cret@repo example/repo", "https:alice:s3cret@repo example/repo", "https://repo.example:bad\\a@b"]) {
+  f = form(url);
+  if (moveLogin(f) !== false) fail("accepted an invalid address with a login: " + url);
+  if (f.elements.username.value || f.elements.password.value) fail("guessed a login from an invalid address: " + url);
+}
+
+for (const url of ["https://repo.example/repo", "not a url", "https://repo.example/a@b", "fdroidrepos:///a@b", "fdroidrepos:/a@b", "https://repo.example/r?x=a@b#c@d"]) {
+  f = form(url);
+  moveLogin(f);
+  if (f.elements.url.value !== url || f.elements.username.value) fail("changed an address without a login: " + url);
+}
+
+const html = sourceToken({ id: "github", name: "GitHub releases", token: { set: true, from: "saved", rejected: true } });
+if (!html.includes("rejected") || !html.includes('data-clear-token="github"')) fail("token row: " + html);
+if (sourceToken({ id: "fdroid", name: "F-Droid" }) !== "") fail("token field on a source without one");
+const slow = "https://a:" + "%FF".repeat(4000) + "@repo.example/r", t0 = Date.now();
+f = form(slow); moveLogin(f);
+if (Date.now() - t0 > 200 || f.elements.password.value !== "%FF".repeat(4000)) fail("decoding malformed escapes is slow or wrong");
+console.log("ok");

tests/test_apk_search.py

@@ -325,7 +325,7 @@ class EndpointTests(SettingsTest):
             mod.add_repo.side_effect = SourceError('repository fingerprint mismatch')
             job = self.wait(self.request('POST', '/api/sources', {'action': 'add', 'url': link})[1]['job'])
             self.assertEqual(job['error'], 'repository fingerprint mismatch')  # no "SourceError:" prefix
-            for url in ('http://repo.example/repo', 'fdroidrepo://repo.example/repo', 'https://u@repo.example/'):
+            for url in ('http://repo.example/repo', 'fdroidrepo://repo.example/repo', 'https://repo.example:x/'):
                 self.assertEqual(self.request('POST', '/api/sources', {'action': 'add', 'url': url})[0], 400)
             self.assertEqual(self.request('POST', '/api/sources', {'action': 'remove', 'source': 'fdroid'})[0], 200)
             mod.remove_repo.assert_called_once_with(source_id='fdroid')

tests/test_comfort.py

@@ -256,47 +256,3 @@ class SensorTests(unittest.TestCase):
                 self.assertIsNone(status.activity_level())
         with patch.object(status, 'run', return_value='[{"operation":"status","activity_level":3}]'):
             self.assertEqual(status.activity_level(), 3)
-
-
-class DesktopBundleTests(unittest.TestCase):
-    """server.comfort runs the copy cached on the headset; the sources go only when it's missing."""
-
-    @unittest.skipUnless(os.name == "posix", "checks POSIX permissions")
-    def test_sources_are_sent_once_per_version(self):
-        import server
-        with tempfile.TemporaryDirectory() as tmp:
-            here, home = Path(tmp) / 'ui', Path(tmp) / 'frame'
-            here.mkdir()
-            home.mkdir()
-            for name in server.COMFORT_FILES:
-                (here / name).write_text('# ' + name + '\n')
-            (here / 'frame_comfort.py').write_text('import json, sys\nprint(json.dumps({"active": False, "got": sys.argv[1]}))\n')
-            sent = []
-
-            def ssh(remote, *, stdin, timeout):  # the headset: the script run under its own home
-                sent.append(stdin)
-                return subprocess.run([sys.executable, '-'], input=stdin, capture_output=True, text=True, check=True,
-                                      env={**os.environ, 'HOME': str(home)}).stdout
-
-            with patch.object(server, 'HERE', here), patch.object(server, 'ssh', side_effect=ssh):
-                first = server.comfort({'action': 'status'})
-                self.assertEqual(json.loads(first['got']), {'action': 'status'})
-                self.assertEqual(len(sent), 2)  # the check, then the sources
-                self.assertNotIn('# frame_steam.py', sent[0])
-                self.assertIn('# frame_steam.py', sent[1])
-                self.assertEqual(server.comfort({'action': 'status'}), first)
-                self.assertEqual(len(sent), 3)
-                self.assertNotIn('# frame_steam.py', sent[2])
-                cached, = (home / '.cache/frame-control/comfort').iterdir()
-                self.assertEqual(sorted(p.name for p in cached.iterdir()), sorted(server.COMFORT_FILES))
-                self.assertEqual(cached.stat().st_mode & 0o777, 0o700)
-                self.assertTrue(all(p.stat().st_mode & 0o777 == 0o600 for p in cached.iterdir()))
-                # A copy cut short is completed; a changed helper is a new version.
-                (cached / 'frame_status.py').unlink()
-                server.comfort({'action': 'status'})
-                self.assertEqual(len(sent), 5)
-                self.assertTrue((cached / 'frame_status.py').exists())
-                (here / 'frame_steam.py').write_text('# frame_steam.py, changed\n')
-                server.comfort({'action': 'status'})
-                self.assertEqual(len(sent), 7)
-                self.assertEqual(len(list((home / '.cache/frame-control/comfort').iterdir())), 2)

tests/test_comfort_ui.py

@@ -59,32 +59,3 @@ const log = ()=>{}, toast = ()=>{};
 '''
         result = subprocess.run(['node', '-e', setup + code + checks], capture_output=True, text=True)
         self.assertEqual(result.returncode, 0, result.stderr)
-
-    def test_poll_waits_while_the_frame_is_unreachable(self):
-        page = (ROOT / 'ui/index.html').read_text(encoding='utf-8')
-        start = page.index('async function pollComfort()')
-        code = page[start:page.index('$("comfortForm").onsubmit', start)]
-        js = r'''
-const assert = require('node:assert/strict');
-const el = {textContent: 'Checking session…'};
-const $ = () => el;
-let online = false, comfortBusy = false, calls = 0, timers = 0;
-const api = async () => { calls++; return {}; };
-const renderComfort = () => {};
-const setTimeout = () => { timers++; };
-''' + code + r'''
-(async () => {
-  await pollComfort();
-  assert.equal(calls, 0);  // no ssh attempt while the status check can't reach the Frame
-  assert.match(el.textContent, /when the Frame is connected/);
-  assert.equal(timers, 1);  // still scheduled, to resume when it's back
-  online = true;
-  await pollComfort();
-  assert.equal(calls, 1);
-  online = null;  // a status check that failed for another reason: keep polling
-  await pollComfort();
-  assert.equal(calls, 2);
-})().catch(e => { console.error(e); process.exitCode = 1; });
-'''
-        result = subprocess.run(['node', '-e', js], capture_output=True, text=True)
-        self.assertEqual(result.returncode, 0, result.stderr)

tests/test_fdroid_sources.py

@@ -81,7 +81,7 @@ class Repositories(unittest.TestCase):
             state.clear()
             self.addCleanup(state.clear)
 
-    def fetch(self, url, path, maximum):
+    def fetch(self, url, path, maximum, auth=None):
         name = url.rsplit('/', 1)[-1]
         if self.v1 and name == 'entry.jar':
             raise urllib.error.HTTPError(url, 404, 'missing', None, None)
@@ -311,7 +311,7 @@ class Repositories(unittest.TestCase):
         source = self.add()
         other = dict(source, id='other-repo')
         started, release = threading.Event(), threading.Event()
-        def fetch(url, path, maximum):
+        def fetch(url, path, maximum, auth=None):
             if threading.current_thread().name == 'slow':
                 started.set()
                 release.wait(5)
@@ -355,7 +355,7 @@ class Repositories(unittest.TestCase):
         self.files['entry.jar'], _ = entry_jar(50)
         source = fdroid.add_repo(URL)
         jars = {'older': entry_jar(100)[0], 'newer': entry_jar(200)[0]}
-        def fetch(url, path, maximum):
+        def fetch(url, path, maximum, auth=None):
             name = threading.current_thread().name
             if name in jars and url.endswith('entry.jar'):
                 Path(path).write_bytes(jars[name])
@@ -401,7 +401,7 @@ class Repositories(unittest.TestCase):
         source = dict(id='overlap', name='Overlap', url=URL, fingerprint=None)
         self.v1 = True
         inner = []
-        def fetch(url, path, maximum):
+        def fetch(url, path, maximum, auth=None):
             if url.endswith('index-v1.jar') and not inner:
                 inner.append(1)  # the v1 load passed its fallback check; a v2 load finishes now
                 self.v1 = False
@@ -471,7 +471,7 @@ class Repositories(unittest.TestCase):
         source = self.add()
         self.expire(source)
         release = threading.Event()
-        def slow(url, path, maximum):
+        def slow(url, path, maximum, auth=None):
             release.wait(5)
             self.fetch(url, path, maximum)
         self.fetch_mock.side_effect = slow
@@ -528,7 +528,7 @@ class Repositories(unittest.TestCase):
         self.expire(source)
         before = self.fetch_mock.call_count
         release = __import__('threading').Event()
-        def slow(url, path, maximum):
+        def slow(url, path, maximum, auth=None):
             release.wait(5)
             self.fetch(url, path, maximum)
         self.fetch_mock.side_effect = slow

tests/test_frame_android_library.py

@@ -423,19 +423,6 @@ class InstallTests(unittest.TestCase):
         self.assertIn(('remove', '3346865537'), [c.args for c in api.call_args_list])
         self.assertTrue(any(c.args[0] == 'rm -rf Applications/Android/org.test.vr' for c in ssh.call_args_list))
 
-    def test_failed_update_removes_partial_apk_and_keeps_the_app(self):
-        with patch.object(android.frame_artwork, 'prepare', return_value=(self.images, [])), \
-                patch.object(android, 'read_meta', return_value=dict(self.existing)), \
-                patch.object(android, '_copy', side_effect=android.FrameError('copying x.apk to the Frame timed out')), \
-                patch.object(android, 'ssh', return_value='') as ssh, \
-                patch.object(android, 'shortcut_tool') as api:
-            with self.assertRaisesRegex(android.FrameError, 'timed out'):
-                android._install('x.apk', self.info, 'org.test.vr', False, None, None)
-        commands = [c.args[0] for c in ssh.call_args_list]
-        self.assertIn('rm -f Applications/Android/org.test.vr/app.apk.part', commands)
-        self.assertFalse(any(c.startswith('rm -rf') for c in commands))
-        api.assert_not_called()
-
     def test_remove_keeps_data_when_requested_and_survives_steam_failure(self):
         with patch.object(android, '_meta_or_fail', side_effect=lambda pkg: dict(self.existing)), \
                 patch.object(android, 'stop'), \
@@ -499,49 +486,6 @@ class InstallTests(unittest.TestCase):
         self.assertIn('podman stop -t 5 lepton-steamlaunch-2800000001', ssh.call_args.args[0])
 
 
-class CopyAndSshTests(unittest.TestCase):
-    """The commands behind an install fail as FrameErrors with readable words, and a large APK
-    gets the time it needs."""
-
-    def fake(self, stderr, code=1, stdout=b''):
-        """frame_host.run_ssh running a stand-in that writes these bytes, with the caller's options."""
-        script = (f'import sys; sys.stdout.buffer.write({stdout!r}); sys.stderr.buffer.write({stderr!r}); '
-                  f'sys.exit({code})')
-        return lambda argv, **kwargs: subprocess.run([sys.executable, '-c', script], **kwargs)
-
-    def test_large_apk_copy_timeout_grows_with_its_size(self):
-        seen = {}
-        with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
-                patch.object(android.frame_host, 'run_ssh', side_effect=lambda argv, **kw: seen.update(kw)), \
-                patch.object(android.os.path, 'getsize', return_value=2 * 1024 ** 3):
-            android._copy(f.name, 'Applications/Android/x/app.apk.part')
-        # 2 GB at the slowest rate allowed: 4096 s, not the old fixed 600 (300 MB at 4 Mbit/s).
-        self.assertGreaterEqual(seen['timeout'], 2 * 1024 ** 3 / (512 * 1024))
-        with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
-                patch.object(android.frame_host, 'run_ssh', side_effect=lambda argv, **kw: seen.update(kw)):
-            android._copy(f.name, 'x/launch.sh', timeout=120)
-        self.assertEqual(seen['timeout'], 120)  # a small file keeps the least time given
-
-    def test_copy_failure_in_another_encoding_is_a_frame_error(self):
-        # A message from a Windows code page, or a file name, isn't UTF-8.
-        with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
-                patch.object(android.shutil, 'which', return_value=None), \
-                patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'scp: Verbindung \x81ber\xff\n')):
-            with self.assertRaisesRegex(android.FrameError, r'\Acopying .* to the Frame failed: scp: Verbindung'):
-                android._copy(f.name, 'x/app.apk.part')
-
-    def test_ssh_failure_messages_are_text(self):
-        with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'mv: \xff cannot move\n')):
-            with self.assertRaisesRegex(android.FrameError, r'\Amv: .* cannot move\Z'):
-                android.ssh('mv a b')
-        with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'cat: write error: No space left on device\n')):
-            with self.assertRaises(android.FrameError) as caught:
-                android.ssh('cat > x', input=b'\x89PNG')  # bytes in, bytes out
-        self.assertEqual(str(caught.exception), 'cat: write error: No space left on device')
-        with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'', code=0, stdout=b'ok \xff\n')):
-            self.assertEqual(android.ssh('echo ok').strip(), 'ok �')
-
-
 class SteamAPITests(unittest.TestCase):
     def test_artwork_api_enums_and_safe_serialization(self):
         with patch.object(shortcuts, 'evaluate', return_value={'warnings': []}) as evaluate:

tests/test_frame_apk.py

@@ -93,20 +93,6 @@ class ApkInfo(unittest.TestCase):
             frame_apk.MAX_MANIFEST = limit
         self.assertLess(peak, 8 * 1024**2)
 
-    def test_damaged_download_is_an_apk_error(self):
-        # Corrupt deflate data used to escape as zlib.error, which the upload path didn't catch
-        # (a 500, and no install report) and frame_android.install reported as 'other'.
-        buf = io.BytesIO()
-        with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as z:
-            z.writestr('AndroidManifest.xml', manifest('com.example.bad', 0x7f010000, 0x7f010001, 21) * 20)
-        data = bytearray(buf.getvalue())
-        at = data.index(b'PK\x03\x04')
-        start = at + 30 + sum(struct.unpack_from('<HH', data, at + 26))  # past the name and extra field
-        for i in range(start, start + 40):
-            data[i] ^= 0x5a
-        with self.assertRaisesRegex(frame_apk.ApkError, 'could not read the APK manifest'):
-            self.read(bytes(data))
-
     def test_refuses_compression_android_cant_read(self):
         for method in (zipfile.ZIP_BZIP2, zipfile.ZIP_LZMA):
             buf = io.BytesIO()

tests/test_link.py

@@ -46,17 +46,6 @@ def explain(msg):
     return None
 
 
-class FailureDetail(unittest.TestCase):
-    def test_an_address_that_failed_says_why_in_a_few_words(self):
-        # #87: the banner put each address's detail after the headline, and with the whole
-        # message as the detail it said everything twice ("...may be answering here..").
-        self.assertEqual(fl.failure_detail({"message": "A long sentence.", "short": "SSH identity changed"}),
-                         "SSH identity changed")
-        self.assertEqual(fl.failure_detail({"message": "The connection to the Frame dropped. Try again."}),
-                         "The connection to the Frame dropped. Try again.")
-        self.assertEqual(fl.failure_detail(None), "SSH failed")
-
-
 class Probe(unittest.TestCase):
     def test_answers_refusals_and_unknown_names(self):
         with socket.socket() as srv:
@@ -211,8 +200,7 @@ class Connecting(unittest.TestCase):
         rows = {p["host"]: p for p in s["probes"]}
         self.assertEqual(rows["nothing.invalid"]["state"], "unresolved")
         self.assertEqual(rows["::1"]["state"], "sshfailed")
-        # Its row says why in a few words: the banner's headline already has the whole sentence (#87).
-        self.assertEqual(rows["::1"]["detail"], "SSH identity changed")
+        self.assertIn("different SSH identity", rows["::1"]["detail"])
         # Every ssh command was pointed at the winner, with the host key pinned per device.
         alias, opts = self.routes[-1]
         self.assertEqual(alias, "frame-t")
@@ -272,8 +260,6 @@ class Connecting(unittest.TestCase):
         self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "login"))
         self.assertIn("SSH key", s["error"]["message"])
         self.assertEqual(s["error"]["auth"], "denied")  # the page offers Set Up Connection
-        self.assertEqual(s["error"]["short"], "This computer's key refused")  # the header pill's words (#87)
-        self.assertEqual(s["probes"][0]["detail"], "This computer's key refused")
 
     def test_pinned_identity_is_checked_strictly(self):
         d = self.device("localhost")
@@ -363,7 +349,6 @@ class Connecting(unittest.TestCase):
                 self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "identity"))
                 self.assertEqual(s["error"]["hostkey"]["state"], "missing")
                 self.assertIn("missing or can't be read", s["error"]["message"])
-                self.assertEqual(s["error"]["short"], "Saved SSH identity missing")
                 view = next(x for x in fl.devices_view(self.link)["devices"] if x["id"] == d["id"])
                 self.assertTrue(view["identity_lost"])
                 scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
@@ -649,7 +634,6 @@ class Connecting(unittest.TestCase):
         self.assertEqual((shown["state"], shown["type"], shown["host"]), ("changed", "ssh-ed25519", "localhost"))
         self.assertTrue(shown["fingerprint"].startswith("SHA256:"))
         self.assertIn("Devices tab", s["error"]["message"])
-        self.assertEqual(s["error"]["short"], "SSH identity changed")
         self.assertEqual(fd.known_hosts(d["id"]).read_text(), saved)  # never accepted on its own
         r = self.command(d)
         self.assertEqual(r.returncode, 255)

tests/test_link_ui.py

@@ -1,136 +0,0 @@
-"""The connection pill, the offline banner and the Devices tab's selection: the page's own
-functions from ui/index.html, run in Node against a minimal DOM (#87)."""
-import sandbox  # noqa: F401  (first: keeps tests off real data and services)
-import pathlib
-import re
-import shutil
-import subprocess
-import unittest
-
-ROOT = pathlib.Path(__file__).resolve().parents[1]
-PAGE = (ROOT / "ui/index.html").read_text(encoding="utf-8")
-
-
-def between(start, end):
-    return PAGE[PAGE.index(start):PAGE.index(end)]
-
-
-SETUP = r'''
-const assert = require('node:assert/strict');
-const elements = new Map();
-const $ = id => {
-  if (!elements.has(id)) elements.set(id, {id, textContent: '', innerHTML: '', title: '', hidden: true, value: ''});
-  return elements.get(id);
-};
-const esc = s => String(s).replace(/[&<>"']/g, c => ({'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;'}[c]));
-const serverNow = () => 1000;
-const activeStage = s => (s.stages || []).find(x => x.state === 'active');
-const ago = () => 'a minute ago';
-const openTrust = () => {};
-const window = {};
-let online = false;
-const link = {live: true, s: null};
-const dv = {list: [], sel: null, data: null, found: 'x', editing: 'y'};
-let rendered = 0;
-const renderDevices = () => { rendered++; };
-let refuse = false;
-const act = async (label, fn) => refuse ? undefined : fn();
-const api = async () => ({ok: true});
-const STAGES = [{id: 'find', label: 'Finding the headset', state: 'done'},
-                {id: 'identity', label: "Checking the headset's identity", state: 'failed'}];
-const changed = {phase: 'failed', retry_at: 1012, device: {id: 'd1', name: 'Office Frame'}, network: {name: 'Home'},
-  stages: STAGES, probes: [{host: 'frame.local', detail: 'SSH identity changed'}],
-  error: {stage: 'identity', short: 'SSH identity changed', hostkey: {state: 'changed'},
-          message: 'This address answered with a different SSH identity from the one saved for Office Frame. '
-                 + 'If not, a different device may be answering here.'}};
-'''
-
-
-@unittest.skipUnless(shutil.which("node"), "Node runs the page's own functions")
-class LinkUI(unittest.TestCase):
-    def run_page(self, checks):
-        code = (between("function netLabel(s) {", "function renderConnDlg() {")
-                + between("async function useDevice(id) {", "async function devAction(")
-                + between("function deviceStatus(d) {", "function renderDevices() {"))
-        script = SETUP + code + "\n(async () => {\n" + checks + "\n})().catch(e => { console.error(e); process.exitCode = 1; });"
-        r = subprocess.run(["node", "-e", script], capture_output=True, text=True)
-        self.assertEqual(r.returncode, 0, r.stderr)
-
-    def test_the_pill_leads_with_the_reason(self):
-        # The header pill cut off "Offline: checking the headset's identity failed · retry in…":
-        # the reason comes first, in a few words, and only the countdown gets cut.
-        self.run_page(r'''
-  link.s = changed;
-  renderPill();
-  assert.match($('conn').innerHTML, /<b>Office Frame<\/b><span>SSH identity changed · retry in 12 s<\/span>/);
-  assert.equal($('conn').title, 'Office Frame is offline: SSH identity changed · retry in 12 s. Click for details.');
-  // Without a short reason: the stage that failed.
-  link.s = {...changed, retry_at: null, error: {stage: 'identity', message: 'Something.'}};
-  renderPill();
-  assert.match($('conn').innerHTML, /<span>Checking the headset&#39;s identity failed<\/span>/);
-''')
-
-    def test_the_banner_never_doubles_a_full_stop(self):
-        # "…may be answering here.." and "…(Developer Mode must be on).." (#87).
-        self.run_page(r'''
-  link.s = changed;
-  renderBanner();
-  assert.equal($('offDetail').textContent, 'Office Frame on Home. Tried frame.local: SSH identity changed. Trying again in 12 s.');
-  link.s = {...changed, probes: [
-    {host: 'frame.local', detail: "The Frame didn't accept this computer's SSH key. Set Up Connection pairs this computer with it again (Developer Mode must be on)."},
-    {host: '100.64.0.2', detail: 'No answer'}]};
-  renderBanner();
-  const text = $('offDetail').textContent;
-  assert.ok(!/\.\./.test(text), text);
-  assert.ok(text.includes("frame.local: the Frame didn't accept"), text);
-  assert.ok(text.includes('must be on); 100.64.0.2: no answer. Trying'), text);
-  assert.equal(asClause('SSH identity changed.'), 'SSH identity changed');  // abbreviations keep their case
-  assert.equal(asClause('Answered in 3 ms'), 'answered in 3 ms');
-''')
-
-    def test_the_devices_list_keeps_the_whole_message_within_reach(self):
-        self.run_page(r'''
-  link.s = changed;
-  const [dot, text, full] = deviceStatus({active: true, addresses: []});
-  assert.equal(dot, 'off');
-  assert.equal(text, 'SSH identity changed');
-  assert.equal(full, changed.error.message);  // the row's tooltip
-''')
-
-    def test_using_a_headset_selects_it(self):
-        # After "Use this headset" on Office Frame, the last headset's details stayed on show.
-        self.run_page(r'''
-  dv.list = [{id: 'd0', name: 'Living Room Frame', active: true}, {id: 'd1', name: 'Office Frame', active: false}];
-  dv.sel = 'd0';
-  await useDevice('d1');
-  assert.equal(dv.sel, 'd1');
-  assert.equal(dv.found, null);
-  assert.equal(dv.editing, null);
-  assert.equal(rendered, 1);
-  // Refused (an install is running): the selection stays, and the header shows the one in use.
-  dv.list = [{id: 'd0', name: 'Living Room Frame', active: true}, {id: 'd2', name: 'Spare', active: false}];
-  dv.sel = 'd0'; refuse = true;
-  await useDevice('d2');
-  assert.equal(dv.sel, 'd0');
-  assert.equal($('devSel').value, 'd0');
-''')
-
-
-class HeaderLayout(unittest.TestCase):
-    def test_the_devices_tab_explanation_wraps(self):
-        # It was cut to one line with no way to read the rest.
-        self.assertIn(".dev-problem .s { white-space: normal; }", PAGE)
-        self.assertEqual(len(re.findall(r'<div class="item dev-problem" role="alert">', PAGE)), 2)
-
-    def test_the_header_buttons_dont_give_way(self):
-        # Refresh overhung the right edge at 1280 px: the pill (and the headset picker) shrink, not the buttons.
-        header = between("<header>", "</header>")
-        end = header[header.index('<div class="hdr-end">'):]
-        for el in ('id="conn"', 'id="devSel"', 'id="battChip"', 'id="reportBtn"', 'id="refreshAll"'):
-            self.assertIn(el, end)
-        self.assertIn(".hdr-end > :not(.pill, .devsel) { flex: none; }", PAGE)
-        self.assertRegex(PAGE, r"\.hdr-end \{[^}]*min-width: 0;[^}]*margin-left: auto;")
-
-
-if __name__ == "__main__":
-    unittest.main()

tests/test_source_logins.py

@@ -0,0 +1,566 @@
+"""Repositories and sources behind a login: F-Droid basic auth and GitHub tokens.
+
+A fake HTTPS layer stands in for the network (urllib's real redirect, error and
+request processing still run), so these tests see every request and its headers.
+"""
+import sandbox  # noqa: F401  (first: keeps tests off real data and services)
+import base64
+import contextlib
+import http.client
+import io
+import json
+import os
+from pathlib import Path
+import shutil
+import stat
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+import unittest
+from unittest.mock import Mock, patch
+import urllib.request
+import urllib.response
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
+from apk_sources import SourceAuthError, SourceError, SourceLimited, _credentials, _images, _web, fdroid, github, search
+import server
+
+FIXTURES = Path(__file__).parent / 'fixtures' / 'fdroid'
+PIN = (FIXTURES / 'fingerprint.txt').read_text().strip()
+REPO = 'https://repo.example/fdroid/repo/'
+USER, PASSWORD = 'alice', 's3cr@t-pässword'
+TOKEN = 'ghp_frameControlTestToken0123456789'
+SECRETS = (PASSWORD, TOKEN, base64.b64encode((USER + ':' + PASSWORD).encode()).decode())
+REAL_BUILD_OPENER = urllib.request.build_opener
+
+
+def basic(user, password):
+    return 'Basic ' + base64.b64encode((user + ':' + password).encode()).decode()
+
+
+class FakeHTTPS(urllib.request.HTTPSHandler):
+    """Answers https requests from route(url, authorization) -> (status, headers, body)."""
+
+    def __init__(self, route):
+        super().__init__()
+        self.route, self.seen = route, []
+
+    def https_open(self, req):
+        auth = req.get_header('Authorization')
+        self.seen.append((req.full_url, auth))
+        status, headers, body = self.route(req.full_url, auth)
+        message = http.client.HTTPMessage()
+        for key, value in headers.items():
+            message[key] = value
+        response = urllib.response.addinfourl(io.BytesIO(body), message, req.full_url, status)
+        response.msg = 'fake'
+        return response
+
+
+class Base(unittest.TestCase):
+    def setUp(self):
+        tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(tmp.cleanup)
+        self.root = Path(tmp.name)
+        for name, value in [('data_dir', lambda *p: self.root.joinpath('data', *p)),
+                            ('cache_dir', lambda *p: self.root.joinpath('cache', *p))]:
+            mock = patch.object(fdroid.frame_host, name, value)
+            mock.start()
+            self.addCleanup(mock.stop)
+        self.web = FakeHTTPS(self.route)
+        mock = patch('urllib.request.build_opener', lambda *handlers: REAL_BUILD_OPENER(*handlers, self.web))
+        mock.start()
+        self.addCleanup(mock.stop)
+        mock = patch.dict(os.environ, {github.ENV: ''})
+        mock.start()
+        self.addCleanup(mock.stop)
+        for state in (_web._limited, fdroid._stale, fdroid._retry_at, fdroid._refreshing, github._rejected):
+            state.clear()
+            self.addCleanup(state.clear)
+        self.stderr = io.StringIO()
+        mock = patch('sys.stderr', self.stderr)
+        mock.start()
+        self.addCleanup(mock.stop)
+
+    def route(self, url, auth):
+        raise AssertionError('unexpected request ' + url)
+
+    def assertNoSecrets(self, *texts):
+        for text in texts:
+            for secret in SECRETS + (USER + ':',):
+                self.assertNotIn(secret, text)
+
+    def disk(self):
+        """Everything saved under the data and cache folders except the credentials file."""
+        out = []
+        for path in self.root.rglob('*'):
+            if path.is_file() and path.name != 'source-credentials.json':
+                out.append(path.read_bytes().decode('utf-8', 'replace'))
+        return '\n'.join(out)
+
+    def credentials_file(self):
+        return self.root / 'data' / 'source-credentials.json'
+
+
+class Repository(Base):
+    """A repository at REPO that wants USER and PASSWORD."""
+    def setUp(self):
+        super().setUp()
+        self.login = basic(USER, PASSWORD)
+        self.redirect = None  # file name -> absolute URL it redirects to
+        self.forbidden = 401
+
+    def serve(self, name):
+        return (FIXTURES / ('example.apk' if name.endswith('.apk') else name)).read_bytes()
+
+    def route(self, url, auth):
+        for mirror in ('https://mirror.example/fdroid/repo/', 'https://repo.example/elsewhere/'):
+            if url.startswith(mirror):  # serves anyone; a login must never get here unless same-origin
+                return 200, {}, self.serve(url[len(mirror):])
+        self.assertTrue(url.startswith(REPO), url)
+        if auth != self.login:
+            return self.forbidden, {'WWW-Authenticate': 'Basic realm="repo"'}, b'no'
+        name = url[len(REPO):]
+        if self.redirect and name in self.redirect:
+            return 302, {'Location': self.redirect[name]}, b''
+        return 200, {}, self.serve(name)
+
+    def add(self, **kw):
+        return fdroid.add_repo('https://alice:s3cr%40t-p%C3%A4ssword@repo.example/fdroid/repo', PIN, **kw)
+
+
+class RepositoryLogin(Repository):
+    def test_add_search_download_with_login_in_the_url(self):
+        source = self.add()
+        self.assertEqual(source['url'], REPO)
+        self.assertEqual(source['fingerprint'], PIN)
+        saved = fdroid.user_repos()[0]
+        self.assertEqual(saved['url'], REPO)
+        self.assertNotIn('login', saved)
+        self.assertEqual(len(fdroid.search(source, 'example offline')), 1)
+        downloaded = fdroid.download(source, 'org.example.app', 1)
+        self.assertTrue(downloaded['verified'])
+        self.assertTrue(self.web.seen)
+        self.assertTrue(all(auth == self.login for url, auth in self.web.seen))
+        self.assertTrue(all('@' not in url for url, auth in self.web.seen))  # never in the URL
+        self.assertTrue(next(s for s in fdroid.sources() if s['id'] == source['id'])['login'])
+        self.assertNoSecrets(json.dumps(source), json.dumps(fdroid.sources()), self.disk(), self.stderr.getvalue())
+        # The settings file holds no part of the login; the credentials file holds it, privately.
+        settings = (self.root / 'data' / 'apk-repos.json').read_text()
+        self.assertNotIn(USER, settings)
+        self.assertTrue(self.credentials_file().is_file())
+        if os.name != 'nt':
+            self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
+
+    def test_login_given_apart_and_saved_login_reused_on_refresh(self):
+        source = fdroid.add_repo(REPO, PIN, username=USER, password=PASSWORD)
+        self.assertEqual(source['url'], REPO)
+        self.web.seen.clear()
+        fdroid._load(fdroid.user_repos()[0], force=True)  # as a background refresh would
+        self.assertTrue(self.web.seen)
+        self.assertTrue(all(auth == self.login for url, auth in self.web.seen))
+        fdroid.add_repo(REPO)  # re-adding without a login keeps the saved one
+        self.assertTrue(_credentials.has(source['id']))
+
+    def test_wrong_or_missing_login_saves_nothing(self):
+        for kw, message in [({'username': USER, 'password': 'wrong'}, 'check the username and password'),
+                            ({}, 'needs a username and password')]:
+            with self.subTest(kw=kw):
+                with self.assertRaisesRegex(SourceAuthError, message) as caught:
+                    fdroid.add_repo(REPO, PIN, **kw)
+                self.assertNoSecrets(str(caught.exception), repr(caught.exception))
+                self.assertNotIn('wrong', str(caught.exception))
+                self.assertEqual(fdroid.user_repos(), [])
+                self.assertFalse(self.credentials_file().exists())
+                self.assertFalse((self.root / 'data' / 'apk-repo-state.json').exists())
+                self.assertEqual(_web._limited, {})  # an auth failure isn't a rate limit
+
+    def test_403_with_a_login_is_a_login_error_not_a_rate_limit(self):
+        self.forbidden = 403
+        with self.assertRaisesRegex(SourceAuthError, 'check the username and password'):
+            fdroid.add_repo(REPO, PIN, username=USER, password='wrong')
+        self.assertEqual(_web._limited, {})
+        self.assertEqual(fdroid.user_repos(), [])
+
+    def test_signature_checks_still_apply(self):
+        with self.assertRaisesRegex(SourceError, 'fingerprint mismatch'):
+            fdroid.add_repo(REPO, '0' * 64, username=USER, password=PASSWORD)
+        self.assertEqual(fdroid.user_repos(), [])
+        self.assertFalse(self.credentials_file().exists())
+
+    def test_cross_origin_redirect_never_gets_the_login(self):
+        self.redirect = {'entry.jar': 'https://mirror.example/fdroid/repo/entry.jar',
+                         'index-v2.json': 'https://repo.example/elsewhere/index-v2.json'}
+        source = self.add()
+        self.assertEqual(source['fingerprint'], PIN)  # the mirror's copy still had to verify
+        mirror = [auth for url, auth in self.web.seen if url.startswith('https://mirror.example/')]
+        same_origin = [auth for url, auth in self.web.seen if url.startswith('https://repo.example/elsewhere/')]
+        self.assertEqual(mirror, [None])
+        self.assertEqual(same_origin, [None])  # same origin but outside the repository: no login either
+
+    def test_traversal_names_are_refused_and_never_get_the_login(self):
+        for name in ('%252e%252e/%252e%252e/evil/collect.apk', '..;/..;/evil/collect.apk', 'a%2f..%2f..%2fevil.apk',
+                     'a%5c..%5cevil.apk', '%2e%2e/evil.apk', 'x;y.apk', '100%25.apk'):
+            with self.subTest(name=name), self.assertRaisesRegex(SourceError, 'unsafe|outside'):
+                fdroid._child(REPO, name)
+        self.assertEqual(fdroid._child(REPO, '/app/en-US/phoneScreenshots/#0 a.png'),
+                         REPO + 'app/en-US/phoneScreenshots/%230%20a.png')
+        # Even if such a URL reached the downloader directly, the login stays home.
+        login = _credentials.basic(REPO, USER, PASSWORD)
+        for url in (REPO + '%252e%252e/%252e%252e/evil/collect.apk', REPO + '..;/..;/evil/collect.apk'):
+            with self.subTest(url=url), tempfile.TemporaryDirectory() as tmp:
+                self.web.seen.clear()
+                with contextlib.suppress(SourceError):  # the repository answers 401 without the login
+                    fdroid._fetch(url, Path(tmp) / 'out', 1 << 20, auth=login)
+                self.assertEqual(self.web.seen, [(url, None)])
+
+    def test_same_origin_redirect_outside_the_repository_gets_no_login(self):
+        self.redirect = {'example1.apk': 'https://repo.example/elsewhere/example1.apk'}
+        source = self.add()
+        self.web.seen.clear()
+        self.assertTrue(fdroid.download(source, 'org.example.app', 1)['verified'])
+        self.assertEqual(self.web.seen, [(REPO + 'example1.apk', self.login),
+                                         ('https://repo.example/elsewhere/example1.apk', None)])
+
+    def test_other_ports_and_http_never_get_the_login(self):
+        login = _credentials.basic(REPO, USER, PASSWORD)
+        for url in ('https://repo.example:8443/fdroid/repo/x', 'http://repo.example/fdroid/repo/x',
+                    'https://other.example/fdroid/repo/x', 'https://repo.example.evil/x'):
+            self.assertFalse(login.matches(url), url)
+        self.assertTrue(login.matches('https://REPO.example:443/anything'))
+        self.assertNoSecrets(repr(login))
+        handler = _credentials.Handler(login)
+        req = urllib.request.Request('https://other.example/x', headers={'Authorization': 'Basic stale'})
+        self.assertIsNone(handler.https_request(req).get_header('Authorization'))
+
+    def test_http_and_bad_logins_are_refused_before_any_request(self):
+        for url, kw in [('http://alice:pw@repo.example/fdroid/repo', {}),
+                        ('http://repo.example/fdroid/repo', {'username': USER, 'password': PASSWORD}),
+                        (REPO, {'username': 'a:b', 'password': PASSWORD}),
+                        (REPO, {'username': USER, 'password': 'x\r\nX-Evil: 1'}),
+                        (REPO, {'password': PASSWORD}),
+                        ('https://alice:pw@repo.example/fdroid/repo', {'username': 'bob'})]:
+            with self.subTest(url=url, kw=kw), self.assertRaises(SourceError) as caught:
+                fdroid.add_repo(url, PIN, **kw)
+            self.assertNotIn('pw', str(caught.exception).replace('password', ''))
+            self.assertNoSecrets(str(caught.exception))
+        self.assertEqual(self.web.seen, [])
+
+    def test_remove_deletes_the_login(self):
+        source = self.add()
+        other = _credentials.bearer('https://api.github.com/', TOKEN)
+        _credentials.put('github', other)
+        fdroid.remove_repo(source['id'])
+        self.assertFalse(_credentials.has(source['id']))
+        self.assertNotIn(PASSWORD, self.credentials_file().read_text())
+        _credentials.remove('github')
+        self.assertFalse(self.credentials_file().exists())  # nothing left: the file goes
+
+    def test_cli_reads_the_password_from_stdin(self):
+        out = io.StringIO()
+        argv = ['fdroid.py', 'add', REPO, '--fingerprint', PIN, '--username', USER, '--password-stdin']
+        with patch.object(sys, 'argv', argv), patch.object(sys, 'stdin', io.StringIO(PASSWORD + '\n')), \
+                contextlib.redirect_stdout(out):
+            fdroid.main()
+        self.assertEqual(json.loads(out.getvalue())['url'], REPO)
+        self.assertNoSecrets(out.getvalue())
+        self.assertTrue(_credentials.has(fdroid.user_repos()[0]['id']))
+
+    def test_artwork_from_the_repository_gets_the_login_and_only_there(self):
+        source = self.add()
+        self.assertEqual(_credentials.for_url(REPO + 'icons/a.png').header(), self.login)
+        self.assertTrue(_credentials.has(source['id']))
+        self.assertIsNone(_credentials.for_url('https://repo.example/other/a.png'))  # outside the repository
+        self.assertIsNone(_credentials.for_url('https://cdn.example/fdroid/repo/a.png'))
+
+
+class GitHubToken(Base):
+    def setUp(self):
+        super().setUp()
+        self.accept = {TOKEN}
+        self.releases = json.dumps([]).encode()
+        self.anonymous_limited = False
+
+    def route(self, url, auth):
+        if url.startswith('https://api.github.com/'):
+            if auth is not None and auth not in {'Bearer ' + t for t in self.accept}:
+                return 401, {}, b'{"message": "Bad credentials"}'
+            if auth is None and self.anonymous_limited:
+                return 403, {'X-RateLimit-Remaining': '0', 'X-RateLimit-Reset': str(int(time.time()) + 1800)}, b'{}'
+            if url.endswith('/redirected'):
+                return 302, {'Location': 'https://github.com/landed'}, b''
+            return 200, {}, self.releases
+        if url == 'https://github.com/landed':
+            return 200, {}, b'{"ok": true}'
+        raise AssertionError('unexpected request ' + url)
+
+    def test_saved_token_goes_only_to_the_api(self):
+        github.set_token(TOKEN)
+        self.assertEqual(github._api('/repos/a/b/releases'), [])
+        self.assertEqual(self.web.seen, [('https://api.github.com/repos/a/b/releases', 'Bearer ' + TOKEN)])
+        auth = github._credential(github._token()[0])
+        self.assertEqual(_web.read('https://api.github.com/redirected', ('api.github.com', 'github.com'), auth=auth),
+                         b'{"ok": true}')
+        self.assertEqual(self.web.seen[-2:], [('https://api.github.com/redirected', 'Bearer ' + TOKEN),
+                                              ('https://github.com/landed', None)])  # redirect: no token
+        status = github.sources()[0]['token']
+        self.assertEqual(status, {'set': True, 'from': 'saved', 'rejected': False})
+        self.assertNoSecrets(json.dumps(github.sources()), self.disk(), self.stderr.getvalue())
+        if os.name != 'nt':
+            self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
+
+    def test_rejected_token_retries_once_without_it(self):
+        github.set_token(TOKEN)
+        self.accept = set()
+        self.assertEqual(github._api('/repos/a/b/releases'), [])
+        self.assertEqual([auth for _, auth in self.web.seen], ['Bearer ' + TOKEN, None])
+        self.assertTrue(github.token_status()['rejected'])
+        self.assertIn('rejected', self.stderr.getvalue())
+        self.assertNoSecrets(self.stderr.getvalue())
+        self.web.seen.clear()
+        github._api('/repos/a/b/other')
+        self.assertEqual([auth for _, auth in self.web.seen], [None])  # not tried again this run
+        github.set_token(TOKEN)  # saving it again gives it a fresh chance
+        self.assertFalse(github.token_status()['rejected'])
+
+    def test_environment_variable_wins(self):
+        github.set_token(TOKEN)
+        env = 'ghp_fromTheEnvironment'
+        self.accept = {env}
+        with patch.dict(os.environ, {github.ENV: env}):
+            github._api('/repos/a/b/releases')
+            self.assertEqual(self.web.seen[-1][1], 'Bearer ' + env)
+            self.assertEqual(github.token_status(), {'set': True, 'from': 'env', 'rejected': False})
+        self.assertEqual(github.token_status()['from'], 'saved')
+
+    def test_clear_removes_it_from_disk(self):
+        github.set_token(TOKEN)
+        github.clear_token()
+        self.assertFalse(self.credentials_file().exists())
+        self.assertEqual(github.token_status(), {'set': False, 'from': None, 'rejected': False})
+        github._api('/repos/a/b/releases')
+        self.assertEqual(self.web.seen[-1][1], None)
+
+    def test_saving_a_token_escapes_the_anonymous_rate_limit(self):
+        self.anonymous_limited = True
+        with self.assertRaisesRegex(SourceLimited, 'GitHub is limiting requests'):
+            github._api('/repos/a/b/releases')
+        self.assertEqual(self.web.seen[-1][1], None)
+        github.set_token(TOKEN)
+        self.assertEqual(github._api('/repos/a/b/other'), [])  # tried, with the token's own quota
+        self.assertEqual(self.web.seen[-1], ('https://api.github.com/repos/a/b/other', 'Bearer ' + TOKEN))
+        github.clear_token()
+        count = len(self.web.seen)
+        with self.assertRaisesRegex(SourceLimited, 'GitHub is limiting requests'):
+            github._api('/repos/a/b/third')
+        self.assertEqual(len(self.web.seen), count)  # anonymous is still backing off: no request
+
+    def test_bad_tokens_are_refused(self):
+        for token in ('', '   ', 'two words', 'x\nAuthorization: y'):
+            with self.subTest(token=token), self.assertRaises(SourceError):
+                github.set_token(token)
+        self.assertFalse(self.credentials_file().exists())
+
+
+PNG = b'\x89PNG\r\n\x1a\nfixture'
+PRIVATE = 'https://repo.example/private/repo/'
+TRAVERSALS = [PRIVATE + '%2e%2e/%2e%2e/evil/collect.png', PRIVATE + '%2E%2E/x.png', PRIVATE + '../../evil/x.png',
+              PRIVATE + './icons/a.png', PRIVATE + '..%2f..%2fevil/x.png', PRIVATE + 'a%5c..%5c..%5cevil.png',
+              PRIVATE + 'a;b/../../evil.png', PRIVATE + '%252e%252e/%252e%252e/evil.png',
+              'https://repo.example/private/repository/x.png', 'https://repo.example/private/repo%2fx.png']
+
+
+class ArtworkScope(Base):
+    """The artwork proxy: actual outgoing headers, through _images.get's own HTTP code."""
+
+    def setUp(self):
+        super().setUp()
+        self.sent, self.redirects = [], {}
+        _credentials.put('fdroid-user-private', _credentials.basic(PRIVATE, USER, PASSWORD))
+        test = self
+
+        class Response:
+            def __init__(self, status, location=''):
+                self.status, self.location, self.body = status, location, PNG if status == 200 else b''
+
+            def getheader(self, name, default=''):
+                return self.location if name == 'Location' else default
+
+            def read1(self, n):
+                chunk, self.body = self.body[:n], self.body[n:]
+                return chunk
+
+        class Connection:
+            def __init__(self, host, port, timeout=None):
+                self.host, self.port = host, port
+
+            def request(self, method, path, headers):
+                url = 'https://%s%s' % (self.host, path)
+                test.sent.append((url, headers.get('Authorization')))
+                self.url = url
+
+            def getresponse(self):
+                if self.url in test.redirects:
+                    return Response(302, test.redirects[self.url])
+                return Response(200)
+
+            def close(self):
+                pass
+
+        # A public address, no real socket or TLS, and Connection in place of http.client's.
+        for target, name, value in [(_images, '_resolve', lambda host, port, timeout: [(2, 1, 6, '', ('93.184.216.34', port))]),
+                                    (_images.socket, 'create_connection', lambda *a, **k: Mock()),
+                                    (_images.ssl, 'create_default_context', lambda: Mock()),
+                                    (_images.http.client, 'HTTPConnection', Connection)]:
+            mock = patch.object(target, name, value)
+            mock.start()
+            self.addCleanup(mock.stop)
+
+    def test_artwork_in_the_repository_gets_the_login(self):
+        self.assertEqual(_images.get(PRIVATE + 'icons/a%20b.png'), PNG)
+        self.assertEqual(self.sent, [(PRIVATE + 'icons/a%20b.png', basic(USER, PASSWORD))])
+
+    def test_paths_a_server_could_normalise_outside_the_repository_get_nothing(self):
+        for url in TRAVERSALS:
+            with self.subTest(url=url):
+                self.sent.clear()
+                self.assertEqual(_images.get(url), PNG)
+                self.assertEqual(self.sent, [(url, None)])
+                self.assertIsNone(_credentials.for_url(url))
+
+    def test_redirect_to_a_traversal_gets_nothing(self):
+        start = PRIVATE + 'icons/a.png'
+        self.redirects[start] = PRIVATE + '%2e%2e/%2e%2e/evil/collect.png'
+        self.assertEqual(_images.get(start), PNG)
+        self.assertEqual(self.sent, [(start, basic(USER, PASSWORD)), (self.redirects[start], None)])
+
+
+class ScopedHandler(Base):
+    """Handler(scoped=True), for callers (artwork-like) that must stay inside a scope, redirects included."""
+
+    def route(self, url, auth):
+        if url == PRIVATE + 'icons/a.png':
+            return 302, {'Location': PRIVATE + '%2e%2e/%2e%2e/evil/collect.png'}, b''
+        return 200, {}, PNG
+
+    def test_redirect_into_a_traversal_is_stripped(self):
+        login = _credentials.basic(PRIVATE, USER, PASSWORD)
+        opener = urllib.request.build_opener(_credentials.Handler(login, scoped=True))
+        with opener.open(PRIVATE + 'icons/a.png') as r:
+            self.assertEqual(r.read(), PNG)
+        self.assertEqual(self.web.seen, [(PRIVATE + 'icons/a.png', basic(USER, PASSWORD)),
+                                         (PRIVATE + '%2e%2e/%2e%2e/evil/collect.png', None)])
+        for url in TRAVERSALS:
+            self.assertFalse(_credentials.in_scope(login, url), url)
+        self.assertTrue(_credentials.in_scope(login, PRIVATE + 'a/b%20c.png?x=1'))
+        self.assertTrue(_credentials.in_scope(login, PRIVATE.rstrip('/')))
+
+
+CHURN = '''
+import sys
+sys.path.insert(0, sys.argv[1])
+from apk_sources import _credentials
+for i in range(int(sys.argv[3])):
+    key = sys.argv[2] + str(i)
+    _credentials.put(key, _credentials.basic('https://h.example/', 'u', 'p'))
+    if i % 2:
+        _credentials.remove(key)
+'''
+
+
+class TwoProcesses(Base):
+    def test_interleaved_saves_and_removals_are_not_lost(self):
+        ui = str(Path(__file__).resolve().parents[1] / 'ui')
+        env = dict(os.environ, FRAME_CONTROL_DATA_DIR=str(self.root / 'data'))
+        count = 60
+        procs = [subprocess.Popen([sys.executable, '-c', CHURN, ui, prefix, str(count)], env=env,
+                                  stdout=subprocess.PIPE, stderr=subprocess.PIPE) for prefix in ('a', 'b')]
+        for p in procs:
+            out, err = p.communicate(timeout=120)
+            self.assertEqual(p.returncode, 0, err.decode())
+        saved = set(json.loads(self.credentials_file().read_text()))
+        self.assertEqual(saved, {p + str(i) for p in 'ab' for i in range(0, count, 2)})
+        if os.name != 'nt':
+            self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
+
+
+@unittest.skipUnless(shutil.which('node'), 'needs node')
+class PageLogin(unittest.TestCase):
+    def test_login_leaves_the_address_field_before_sending(self):
+        """tests/page/source_login.mjs runs the real page functions from index.html."""
+        r = subprocess.run(['node', str(Path(__file__).parent / 'page' / 'source_login.mjs')],
+                           capture_output=True, text=True)
+        self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
+        self.assertIn('ok', r.stdout)
+
+
+class Endpoints(Repository):
+    """The server API: logins in, never out, in responses or the server log."""
+
+    def setUp(self):
+        super().setUp()
+        mock = patch.object(search, 'modules', return_value=([fdroid, github], []))
+        mock.start()
+        self.addCleanup(mock.stop)
+        self.httpd = server.ThreadingHTTPServer(('127.0.0.1', 0), server.Handler)
+        threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
+        self.addCleanup(self.httpd.server_close)
+        self.addCleanup(self.httpd.shutdown)
+
+    def request(self, method, path, body=None):
+        c = http.client.HTTPConnection('127.0.0.1', self.httpd.server_port)
+        c.request(method, path, json.dumps(body) if body is not None else None,
+                  {'X-Frame-UI': '1', 'Content-Type': 'application/json'})
+        r = c.getresponse()
+        raw = r.read().decode()
+        c.close()
+        self.replies.append(raw)
+        return r.status, json.loads(raw)
+
+    def job(self, reply):
+        for _ in range(500):
+            job = self.request('GET', '/api/job?id=' + reply['job'])[1]
+            if job['done']:
+                return job
+            time.sleep(.01)
+        self.fail('job did not finish')
+
+    def test_add_with_login_and_token_settings(self):
+        self.replies = []
+        bad = self.job(self.request('POST', '/api/sources', {'action': 'add', 'url': REPO, 'fingerprint': PIN,
+                                                             'username': USER, 'password': 'wrong'})[1])
+        self.assertIn('check the username and password', bad['error'])
+        self.assertEqual(fdroid.user_repos(), [])
+        code, reply = self.request('POST', '/api/sources', {'action': 'add', 'url': REPO, 'fingerprint': PIN,
+                                                            'username': USER, 'password': PASSWORD})
+        self.assertEqual(code, 200)
+        job = self.job(reply)
+        self.assertIsNone(job['error'])
+        listed = self.request('GET', '/api/sources')[1]['sources']
+        repo = next(s for s in listed if s['id'] == job['result']['source']['id'])
+        self.assertEqual((repo['url'], repo['login']), (REPO, True))
+        self.assertEqual(self.request('POST', '/api/sources', {'action': 'set-token', 'source': 'github',
+                                                               'token': TOKEN})[0], 200)
+        github_row = next(s for s in self.request('GET', '/api/sources')[1]['sources'] if s['id'] == 'github')
+        self.assertEqual(github_row['token'], {'set': True, 'from': 'saved', 'rejected': False})
+        self.assertEqual(self.request('POST', '/api/sources', {'action': 'set-token', 'source': 'fdroid',
+                                                               'token': TOKEN})[0], 400)
+        self.assertEqual(self.request('POST', '/api/sources', {'action': 'clear-token', 'source': 'github'})[0], 200)
+        self.assertFalse(_credentials.has('github'))
+        self.assertEqual(self.request('POST', '/api/sources', {'action': 'remove', 'source': repo['id']})[0], 200)
+        self.assertFalse(self.credentials_file().exists())
+        self.assertNoSecrets(*self.replies, self.stderr.getvalue(), self.disk())
+
+    def test_login_in_the_url_is_accepted(self):
+        self.replies = []
+        url = 'https://alice:s3cr%40t-p%C3%A4ssword@repo.example/fdroid/repo'
+        job = self.job(self.request('POST', '/api/sources', {'action': 'add', 'url': url, 'fingerprint': PIN})[1])
+        self.assertIsNone(job['error'])
+        self.assertEqual(fdroid.user_repos()[0]['url'], REPO)
+        self.assertNoSecrets(*self.replies, self.stderr.getvalue())
+
+
+if __name__ == '__main__':
+    unittest.main()

tests/test_telemetry.py

@@ -138,7 +138,7 @@ class Gates(Base):
         tm.diagnostic("POST /api/android install", RuntimeError("boom"))
         tm.diagnostic("POST /api/comfort status", RuntimeError("ssh exited 255"))  # the command's own exit code?
         self.assertEqual([e["properties"]["error_category"] for e in self.queued()],
-                         ["frame_unreachable", "internal_error", "other", "other"])
+                         ["frame_unreachable", "other", "other", "other"])
 
     def test_page_events_are_checked(self):
         self.assertTrue(tm.page_event({"event": "tab_viewed", "properties": {"tab": "android", "extra": "x"}})["queued"])
@@ -164,38 +164,6 @@ class Lifecycle(Base):
         tm.frame_seen("20260922.1", "3.8")
         self.assertEqual(len(self.queued()), 1)
 
-    def test_frame_once_per_headset_with_saved_count(self):
-        tm.frame_seen("20260922.1", "3.8", "0123456789abcdef", 2)
-        tm.frame_seen("20260922.1", "3.8", "0123456789abcdef", 2)
-        tm.frame_seen("20260922.1", "3.8", "fedcba9876543210", 2)
-        props = [e["properties"] for e in self.queued()]
-        self.assertEqual([(p["headset_id"], p["headsets_saved"]) for p in props],
-                         [("0123456789abcdef", 2), ("fedcba9876543210", 2)])
-
-    def test_frame_many_headsets_are_not_sent_twice(self):
-        ids = ["%016x" % n for n in range(25)]
-        for _ in range(2):
-            for h in ids:
-                tm.frame_seen("20260922.1", "3.8", h, 25)
-        self.assertEqual(len(self.queued()), 25)
-
-    def test_frame_headset_id_must_be_the_hash(self):
-        tm.frame_seen("20260922.1", "3.8", "/etc/machine-id contents", None)
-        p = self.queued()[0]["properties"]
-        self.assertNotIn("headset_id", p)
-        self.assertNotIn("headsets_saved", p)
-
-
-class HeadsetId(unittest.TestCase):
-    def test_hash_of_machine_id_and_none_without_one(self):
-        import hashlib
-        import frame_status
-        with mock.patch.object(frame_status, "read", return_value="abc123"):
-            self.assertEqual(frame_status.headset_id(),
-                             hashlib.sha256(b"frame-control-headset:abc123").hexdigest()[:16])
-        with mock.patch.object(frame_status, "read", return_value=None):
-            self.assertIsNone(frame_status.headset_id())
-
 
 class Sending(Base):
     def serve(self, status=200):
@@ -295,72 +263,6 @@ class Scrub(unittest.TestCase):
         self.assertEqual(tm.categorize(FileNotFoundError(2, "No such file or directory", "ssh"))[0], "tool_missing")
         self.assertEqual(tm.categorize("[WinError 2] The system cannot find the file specified")[0], "tool_missing")
 
-    def test_apk_install_failures_that_were_other(self):
-        """What frame_android.install can raise, worded as it is, that 0.4.x sent as 'other'
-        (or as frame_unreachable, when it was the Steam client or a slow copy)."""
-        tail = 'Traceback (most recent call last):\n  File "<stdin>", line 19, in target_ws\n'
-        cases = {
-            "steam_not_running": [
-                "SharedJSContext not found: is the Steam client running?",
-                tail + "urllib.error.URLError: <urlopen error [Errno 111] Connection refused>",
-                tail + "ConnectionRefusedError: [Errno 111] Connection refused",
-            ],
-            "steam_not_responding": [
-                "websocket handshake failed",
-                tail + "TimeoutError: timed out",
-                tail + "socket.timeout: timed out",
-                tail + '  File "<stdin>", line 45, in _read\nEOFError',
-            ],
-            "steam_js_error": ['JS error: {"exceptionId": 1, "text": "Uncaught", "exception": '
-                               '{"description": "Error: Steam artwork API unavailable; installation is incomplete"}}'],
-            "artwork_failed": [
-                "Steam artwork renderer did not produce every slot",
-                tail + "ValueError: incomplete artwork render",
-                tail + "ValueError: hero render is 0 bytes of png",
-                "could not prepare artwork: unknown artwork slot",
-                tail + "ValueError: all five Steam artwork slots are required",
-            ],
-            "copy_timeout": ["copying Big Game (timed out).apk to the Frame timed out",
-                             "copying launch.sh to the Frame timed out"],
-            "copy_failed": [
-                "copying game.apk to the Frame failed: rsync: connection unexpectedly closed (0 bytes received "
-                "so far) [sender]\nrsync error: error in rsync protocol data stream (code 12) at io.c(231)",
-                "copying game.apk to the Frame failed: ",
-            ],
-            "apk_unreadable": [
-                "x.apk: not a readable APK: [Errno 2] No such file or directory: 'x.apk'",
-                "x.apk: could not read the APK manifest: Error -3 while decompressing data: invalid block type",
-                "x.apk: the APK manifest has no package name",
-                "x.apk: lib/arm64-v8a/libx.so in the APK uses a compression Android does not",
-                "x.apk: AndroidManifest.xml in the APK is 40 MB, more than a real one (16 MB)",
-            ],
-        }
-        for category, messages in cases.items():
-            for message in messages:
-                self.assertEqual(tm.categorize(message)[0], category, message)
-        # Still where they were: ssh's own failures, and a full disk during a copy.
-        self.assertEqual(tm.categorize("timed out talking to frame")[0], "frame_unreachable")
-        self.assertEqual(tm.categorize("copying a.apk to the Frame failed: ssh: connect to host 10.0.0.2 "
-                                       "port 22: Connection refused")[0], "frame_unreachable")
-        self.assertEqual(tm.categorize("copying a.apk to the Frame failed: rsync: write failed on "
-                                       "\"app.apk.part\": No space left on device (28)")[0], "frame_disk_full")
-        self.assertEqual(tm.categorize("download failed: timed out")[0], "download_failed")
-
-    def test_python_errors_are_internal_whatever_they_say(self):
-        import json as _json
-        for error in (KeyError("paths"), TypeError("'NoneType' object is not subscriptable"),
-                      AttributeError("x"), _json.JSONDecodeError("Expecting value", "", 0),
-                      UnicodeDecodeError("utf-8", b"\xff", 0, 1, "invalid start byte"),
-                      KeyError("timed out")):
-            self.assertEqual(tm.categorize(error)[0], "internal_error", repr(error))
-        self.assertEqual(tm.categorize("KeyError: 'paths'")[0], "other")  # only the exception itself
-        self.assertEqual(tm.categorize(OSError(28, "No space left on device"))[0], "frame_disk_full")
-
-    def test_privacy_doc_lists_every_category(self):
-        doc = (ROOT / "docs" / "privacy.md").read_text()
-        for name in {n for n, _ in tm.CATEGORIES} | {"internal_error", "other"}:
-            self.assertIn(f"`{name}`", doc)
-
 
 class Compat(Base):
     def test_reports_are_shared_only_after_opting_in_without_file_names(self):
@@ -474,17 +376,14 @@ class LinkFailureProvenance(Base):
         c.close()
         return result
 
-    def serve(self):
+    def test_routes_that_rewrap_a_link_failure_keep_its_mark(self):
+        # Review round 4: /api/android and /api/titles re-raise frame_android's FrameError as a
+        # Failure; after the status poll's first connection failure, theirs are held back too,
+        # also past the 10-minute window.
         self.httpd = self.server.ThreadingHTTPServer(("127.0.0.1", 0), self.server.Handler)
         threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
         self.addCleanup(self.httpd.server_close)
         self.addCleanup(self.httpd.shutdown)
-
-    def test_routes_that_rewrap_a_link_failure_keep_its_mark(self):
-        # Review round 4: /api/android and /api/titles re-raise frame_android's FrameError as a
-        # Failure; after the first connection failure, theirs are held back too, also past the
-        # 10-minute window.
-        self.serve()
         down = subprocess.CompletedProcess(["ssh"], 255, "", "ssh: connect to host 10.0.0.5 port 22: Connection timed out\r\n")
         meta = {"label": "Test App", "game_id": 5, "shortcut": None}
         with mock.patch.object(frame_host, "run_ssh", return_value=down), \
@@ -500,34 +399,7 @@ class LinkFailureProvenance(Base):
                 self.assertEqual(self.request("/api/android", {"action": "launch", "package": "org.test"})[0], 502)
         sent = [(e["properties"]["where"], e["properties"]["error_category"]) for e in self.queued()
                 if e["event"] == "$exception"]
-        # The comfort poll's own failure isn't one (see the next test).
-        self.assertEqual(sent, [("POST /api/android launch", "frame_unreachable"), ("POST /api/android launch", "other")])
-
-    def test_the_comfort_poll_meeting_an_unreachable_frame_is_not_sent(self):
-        # 0.4.2: most reports were this poll (every 5 s) finding the headset asleep, away or
-        # refusing ssh. Like GET /api/status, it isn't reported; a session started or cancelled
-        # by hand still is, once, and a real failure of the poll is too.
-        self.serve()
-        for stderr in ("ssh: connect to host 10.0.0.5 port 22: Connection timed out\r\n",
-                       "banner exchange: Connection to UNKNOWN port -1: Connection refused\r\n",
-                       "ssh: Could not resolve hostname frame: nodename nor servname provided\r\n",
-                       "@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @\r\nHost key verification failed.\r\n",
-                       "No ED25519 host key is known for frame-control-f7b4e9ac and you have requested strict "
-                       "checking.\r\nHost key verification failed.\r\n",
-                       "steamos@192.168.1.9: Permission denied (publickey,password).\r\n"):
-            down = subprocess.CompletedProcess(["ssh"], 255, "", stderr)
-            with mock.patch.object(frame_host, "run_ssh", return_value=down):
-                self.assertGreaterEqual(self.request("/api/comfort", {"action": "status"})[0], 500, stderr)
-        self.assertEqual(self.queued(), [])
-        down = subprocess.CompletedProcess(["ssh"], 255, "", "ssh: connect to host 10.0.0.5 port 22: Host is down\r\n")
-        with mock.patch.object(frame_host, "run_ssh", return_value=down):
-            self.request("/api/comfort", {"action": "cancel"})
-        broken = subprocess.CompletedProcess(["ssh"], 1, "", "Traceback (most recent call last):\nOSError: boom\n")
-        with mock.patch.object(frame_host, "run_ssh", return_value=broken):
-            self.assertEqual(self.request("/api/comfort", {"action": "status"})[0], 502)
-        sent = [(e["properties"]["where"], e["properties"]["error_category"]) for e in self.queued()
-                if e["event"] == "$exception"]
-        self.assertEqual(sent, [("POST /api/comfort cancel", "frame_unreachable"), ("POST /api/comfort status", "other")])
+        self.assertEqual(sent, [("POST /api/comfort status", "frame_unreachable"), ("POST /api/android launch", "other")])
 
     def test_download_failures_after_a_poll_failure_are_still_sent(self):
         # Review round 3: these file names, in the real checksum message through the web-link worker,

ui/apk_sources/__init__.py

@@ -45,6 +45,10 @@ class SourceError(Exception):
     """User-readable failure from a source (network, format, verification)."""
 
 
+class SourceAuthError(SourceError):
+    """The server refused our login (HTTP 401, or 403 when a login was sent)."""
+
+
 class SourceLimited(SourceError):
     """The source's host asked us to slow down; retry_after is in seconds."""
 

ui/apk_sources/_credentials.py

@@ -0,0 +1,270 @@
+"""Logins for sources behind one: HTTP basic auth and access tokens.
+
+Saved apart from the source settings, in frame_host.data_dir('source-credentials.json')
+(mode 0600), keyed by a caller-chosen name: an F-Droid source id, 'github'. Each
+credential belongs to one exact origin (https, host, port) and is only ever sent
+to a request for that origin: Handler adds it to each request, redirects
+included, and removes any Authorization header from every other request.
+
+Scoped use (for_url, and Handler(..., scoped=True), which F-Droid repositories and
+artwork use) is stricter: the URL must also be under the credential's scope by whole
+path segments, with no dot segments, encoded slashes or ';' a server could normalise
+into a path outside it. Origin-only matching suits a host-wide token (GitHub's API).
+
+Secrets leave this module only as an Authorization header value. Credential's
+repr hides them, and nothing here puts them in an exception message.
+
+Each change is a read-modify-write under an inter-process lock (the CLI and the
+app may both save), so one process can't undo another's save or removal.
+"""
+import base64
+import contextlib
+import errno
+import json
+import os
+import tempfile
+import threading
+import urllib.parse
+import urllib.request
+
+import frame_host
+
+_lock = threading.RLock()
+MAX = 4096  # longest username, password or token accepted
+
+
+def _path():
+    return frame_host.data_dir('source-credentials.json')
+
+
+@contextlib.contextmanager
+def _changing():
+    """This thread, then this process, holds the credentials file for a read-modify-write."""
+    path = _path().with_name('source-credentials.lock')
+    with _lock:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with open(str(path), 'a+b') as f:
+            if os.name == 'nt':
+                import msvcrt
+                while True:
+                    try:
+                        f.seek(0)
+                        msvcrt.locking(f.fileno(), msvcrt.LK_LOCK, 1)
+                        break
+                    except OSError as e:  # LK_LOCK gives up after ~10 s of contention; keep waiting
+                        if e.errno not in (errno.EACCES, errno.EDEADLK):
+                            raise
+                try:
+                    yield
+                finally:
+                    f.seek(0)
+                    msvcrt.locking(f.fileno(), msvcrt.LK_UNLCK, 1)
+            else:
+                import fcntl
+                fcntl.flock(f.fileno(), fcntl.LOCK_EX)
+                try:
+                    yield
+                finally:
+                    fcntl.flock(f.fileno(), fcntl.LOCK_UN)
+
+
+def path_segments(path):
+    """A URL path's decoded segments, or None if a server might read it as another path:
+    '.' or '..' (plain or percent-encoded), encoded '/' or '\\', '%' left after decoding
+    (double encoding), or ';' path parameters."""
+    if ';' in path or '\\' in path:
+        return None
+    parts = [urllib.parse.unquote(part) for part in path.split('/')]
+    if any(part in ('.', '..') or '/' in part or '\\' in part or '%' in part or ';' in part for part in parts):
+        return None
+    return parts
+
+
+def in_scope(credential, url):
+    """True if url is on credential's origin and under its scope by whole, plain path segments."""
+    if credential is None or not credential.matches(url):
+        return False
+    try:
+        path, scope = urllib.parse.urlsplit(url).path, urllib.parse.urlsplit(credential.scope).path
+    except ValueError:
+        return False
+    parts, base = path_segments(path), path_segments(scope)
+    if parts is None or base is None:
+        return False
+    if base and base[-1] == '':
+        base = base[:-1]  # 'https://h/repo/' covers '/repo/x', not '/repository'
+    return parts[:len(base)] == base
+
+
+def origin(url):
+    """'https://host:port' for an https URL; ValueError for anything else."""
+    p = urllib.parse.urlsplit(url)
+    port = p.port  # raises ValueError on a bad port
+    if p.scheme.lower() != 'https' or not p.hostname:
+        raise ValueError('credentials are only sent over HTTPS')
+    host = p.hostname.lower()
+    return 'https://%s:%d' % ('[%s]' % host if ':' in host else host, port or 443)
+
+
+def _check(value, what, allow_empty=False):
+    if not isinstance(value, str) or (not value and not allow_empty) or len(value) > MAX:
+        raise ValueError('invalid ' + what)
+    if any(ord(c) < 32 or ord(c) == 127 for c in value):
+        raise ValueError(what + ' must not contain control characters')
+    return value
+
+
+class Credential:
+    """A username and password (basic) or a token (bearer) for one origin.
+
+    scope is the URL prefix it belongs under: in_scope checks it (by path segments),
+    matches() checks only the origin."""
+    __slots__ = ('origin', 'scope', 'username', '_secret')
+
+    def __init__(self, scope, secret, username=None):
+        self.origin = origin(scope)
+        self.scope = scope
+        if username is not None:
+            _check(username, 'username')
+            if ':' in username:
+                raise ValueError('username must not contain ":"')
+            _check(secret, 'password', allow_empty=True)
+        else:
+            _check(secret, 'token')
+            if any(c.isspace() for c in secret):
+                raise ValueError('token must not contain spaces')
+        self.username, self._secret = username, secret
+
+    @property
+    def kind(self):
+        return 'bearer' if self.username is None else 'basic'
+
+    def header(self):
+        if self.username is None:
+            return 'Bearer ' + self._secret
+        return 'Basic ' + base64.b64encode((self.username + ':' + self._secret).encode('utf-8')).decode('ascii')
+
+    def matches(self, url):
+        """True only for a URL on exactly this credential's origin."""
+        try:
+            return origin(url) == self.origin
+        except ValueError:
+            return False
+
+    def same(self, other):
+        return isinstance(other, Credential) and (self.origin, self.username, self._secret) == (
+            other.origin, other.username, other._secret)
+
+    def __repr__(self):
+        return '<Credential %s for %s>' % (self.kind, self.origin)
+
+    def _json(self):
+        return {'scope': self.scope, 'username': self.username, 'secret': self._secret}
+
+
+def basic(scope, username, password):
+    """Basic auth for scope's origin, e.g. basic('https://example.org/fdroid/repo/', 'me', 'pw')."""
+    return Credential(scope, password, username)
+
+
+def bearer(scope, token):
+    """An access token for scope's origin, sent as 'Authorization: Bearer <token>'."""
+    return Credential(scope, token)
+
+
+def _read():
+    try:
+        with open(str(_path()), encoding='utf-8') as f:
+            data = json.load(f)
+        if not isinstance(data, dict):
+            raise ValueError
+        return data
+    except FileNotFoundError:
+        return {}
+    except (OSError, ValueError):
+        return {}  # unreadable: as if none were saved; the next save rewrites it
+
+
+def _save(data):
+    path = _path()
+    if not data:
+        try:
+            os.remove(str(path))
+        except FileNotFoundError:
+            pass
+        return
+    path.parent.mkdir(parents=True, exist_ok=True)
+    fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix='.credentials-', suffix='.part')  # created 0600
+    try:
+        with os.fdopen(fd, 'w', encoding='utf-8') as f:
+            json.dump(data, f, separators=(',', ':'))
+        os.chmod(tmp, 0o600)
+        os.replace(tmp, str(path))
+    finally:
+        if os.path.exists(tmp):
+            os.unlink(tmp)
+
+
+def get(key):
+    """The saved Credential for key, or None."""
+    with _lock:
+        item = _read().get(key)
+    try:
+        return Credential(item['scope'], item['secret'], item.get('username'))
+    except (TypeError, KeyError, ValueError):
+        return None
+
+
+def has(key):
+    return get(key) is not None
+
+
+def put(key, credential):
+    if not isinstance(credential, Credential):
+        raise TypeError('put() needs a Credential')
+    with _changing():
+        data = _read()
+        data[key] = credential._json()
+        _save(data)
+
+
+def remove(key):
+    """Delete key's credential from disk; True if there was one."""
+    with _changing():
+        data = _read()
+        if key not in data:
+            return False
+        del data[key]
+        _save(data)
+        return True
+
+
+def for_url(url):
+    """A saved credential url is in the scope of (see in_scope), or None. Call it again for
+    each redirect hop: the answer depends only on the URL being requested."""
+    with _lock:
+        keys = list(_read())
+    for key in keys:
+        credential = get(key)
+        if in_scope(credential, url):
+            return credential
+    return None
+
+
+class Handler(urllib.request.BaseHandler):
+    """Adds the credential to requests for its origin (redirects included) and strips
+    Authorization from every other request, so a redirect elsewhere never gets it.
+    scoped=True also requires each request, redirects included, to be in_scope."""
+    handler_order = 900  # after the default processors
+
+    def __init__(self, credential, scoped=False):
+        self.credential, self.scoped = credential, scoped
+
+    def _request(self, req):
+        req.remove_header('Authorization')
+        allowed = in_scope if self.scoped else lambda c, url: c is not None and c.matches(url)
+        if allowed(self.credential, req.full_url):
+            req.add_unredirected_header('Authorization', self.credential.header())
+        return req
+
+    http_request = https_request = _request

ui/apk_sources/_images.py

@@ -9,7 +9,7 @@ import threading
 import time
 from urllib.parse import urljoin, urlsplit
 
-from apk_sources import SourceError
+from apk_sources import SourceError, _credentials
 
 _lock = threading.Lock()
 _urls = OrderedDict()
@@ -141,7 +141,11 @@ def get(url, headers=None, redirects=3, deadline=None, limit=MAX_IMAGE):
         path = p.path or '/'
         if p.query:
             path += '?' + p.query
-        conn.request('GET', path, headers={'User-Agent': 'FrameControl/0.3.1', **(headers or {})})
+        sent = {'User-Agent': 'FrameControl/0.3.1', **(headers or {})}
+        login = _credentials.for_url(url)  # a repository behind a login: its own artwork only, per hop
+        if login:
+            sent['Authorization'] = login.header()
+        conn.request('GET', path, headers=sent)
         live[0].settimeout(min(10, left()))
         response = conn.getresponse()
         if response.status in (301, 302, 303, 307, 308) and redirects:

ui/apk_sources/_web.py

@@ -3,7 +3,7 @@ import hashlib, os, shutil, tempfile, threading, time, urllib.error, urllib.pars
 from email.utils import parsedate_to_datetime
 
 import frame_host
-from . import SourceError, SourceLimited
+from . import SourceAuthError, SourceError, SourceLimited, _credentials
 
 UA = 'FrameControl/0.1'
 APK_CAP = 2 * 1024 ** 3  # cached APKs across all sources, least recently used go first
@@ -38,7 +38,7 @@ def release(path):
         else:
             _in_use.pop(path, None)
 BACKOFF = 600  # seconds to leave a host alone after 403/429 without Retry-After
-_limited = {}  # host -> time.time() before which we don't contact it
+_limited = {}  # host (+ login identity) -> time.time() before which we don't ask it with that login
 _limited_lock = threading.Lock()
 
 
@@ -46,8 +46,16 @@ def _host(url):
     return (urllib.parse.urlsplit(url).hostname or '').lower()
 
 
-def throttle(url, headers=None):
-    """Remember that url's host asked us to back off; return the delay in seconds."""
+def _who(url, auth=None):
+    """Backoff key: the host, per login. A token has its own quota, so anonymous
+    rate limiting mustn't hold back requests made with one (or the reverse)."""
+    if auth is None:
+        return _host(url)
+    return _host(url) + ' ' + hashlib.sha256(auth.header().encode()).hexdigest()[:16]
+
+
+def throttle(url, headers=None, auth=None):
+    """Remember that url's host asked us (with this login) to back off; return the delay in seconds."""
     headers = headers or {}
     now, delay = time.time(), None
     value = (headers.get('Retry-After') or '').strip()
@@ -62,14 +70,15 @@ def throttle(url, headers=None):
     if delay is None and headers.get('X-RateLimit-Remaining') == '0' and reset.isdigit():
         delay = int(reset) - now  # GitHub
     delay = min(max(delay if delay is not None else BACKOFF, 1), 6 * 3600)
+    key = _who(url, auth)
     with _limited_lock:
-        _limited[_host(url)] = max(_limited.get(_host(url), 0), now + delay)
+        _limited[key] = max(_limited.get(key, 0), now + delay)
     return delay
 
 
-def wait_time(url):
+def wait_time(url, auth=None):
     with _limited_lock:
-        return max(0, _limited.get(_host(url), 0) - time.time())
+        return max(0, _limited.get(_who(url, auth), 0) - time.time())
 
 
 def limited_error(name, seconds, hint=''):
@@ -136,6 +145,8 @@ def checked_url(url, hosts):
 
 
 class Redirect(urllib.request.HTTPRedirectHandler):
+    """Only to the allowed hosts, and never carrying an Authorization header along:
+    a credential passed to open_url() is added again, by its Handler, only on its origin."""
     def __init__(self, hosts):
         self.hosts = hosts
 
@@ -147,13 +158,15 @@ class Redirect(urllib.request.HTTPRedirectHandler):
         return result
 
 
-def open_url(url, hosts, headers=None):
+def open_url(url, hosts, headers=None, auth=None):
+    """auth: an _credentials.Credential, sent only to requests on its own origin."""
     checked_url(url, hosts)
-    return urllib.request.build_opener(Redirect(hosts)).open(
+    handlers = [Redirect(hosts)] + ([_credentials.Handler(auth)] if auth else [])
+    return urllib.request.build_opener(*handlers).open(
         urllib.request.Request(url, headers={'User-Agent': UA, **(headers or {})}), timeout=60)
 
 
-def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
+def read(url, hosts, headers=None, ttl=3600, name=None, hint='', auth=None):
     path = os.path.join(cache(), hashlib.sha256(url.encode()).hexdigest() + '.data')
     name = name or _host(url) or 'The source'
 
@@ -165,13 +178,13 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
         if os.path.isfile(path) and time.time() - os.path.getmtime(path) < ttl:
             with open(path, 'rb') as f:
                 return f.read()
-        wait = wait_time(url)
+        wait = wait_time(url, auth)
         if wait:
             data = cached()
             if data is None:
                 raise limited_error(name, wait, hint)
             return data
-        with open_url(url, hosts, headers) as r:
+        with open_url(url, hosts, headers, auth) as r:
             data = r.read(8 * 1024 * 1024 + 1)
         if len(data) > 8 * 1024 * 1024:
             raise SourceError('Source index is too large')
@@ -185,8 +198,10 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
                 os.remove(tmp)
         return data
     except urllib.error.HTTPError as e:
+        if e.code == 401:  # not 403: GitHub also uses that for rate limits, token or not
+            raise SourceAuthError('%s refused the login (HTTP 401)' % name) from e
         if e.code in (403, 429):
-            delay = throttle(url, e.headers)
+            delay = throttle(url, e.headers, auth)
             data = cached()
             if data is None:
                 raise limited_error(name, delay, hint) from e

ui/apk_sources/fdroid.py

@@ -20,7 +20,7 @@ import zipfile
 
 if __package__ in (None, ''):
     sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
-from apk_sources import SourceError, SourceLimited, _web
+from apk_sources import SourceAuthError, SourceError, SourceLimited, _credentials, _web
 import frame_host
 from frame_apk_sign import _der_parts, _cert_key, der
 from frame_catalog import _IndexReader, _reduce_index, _sha256
@@ -71,6 +71,39 @@ def _url(url, fingerprint=None):
     return urllib.parse.urlunsplit(('https', p.netloc.lower(), p.path.rstrip('/') + '/', '', '')), pin
 
 
+def _split_login(url, username=None, password=None):
+    """(url without user:pass, credential or None). The address may carry the login, as
+    F-Droid's client accepts (https://user:pass@host/fdroid/repo), or it may be given apart."""
+    try:
+        p = urllib.parse.urlsplit(url)
+        p.port
+    except ValueError as e:
+        raise SourceError('invalid repository URL') from e
+    if '@' in p.netloc:
+        if p.username is None:
+            raise SourceError('invalid repository URL')
+        linked = urllib.parse.unquote(p.username), None if p.password is None else urllib.parse.unquote(p.password)
+        if username not in (None, '', linked[0]) or (linked[1] is not None and password not in (None, '', linked[1])):
+            raise SourceError('the address and the username/password fields disagree')
+        username = linked[0]
+        password = linked[1] if linked[1] is not None else password
+        url = urllib.parse.urlunsplit((p.scheme, p.netloc.rpartition('@')[2], p.path, p.query, p.fragment))
+    if username in (None, ''):
+        if password:
+            raise SourceError('a password needs a username')
+        return url, None
+    clean, _ = _url(url)
+    try:
+        return url, _credentials.basic(clean, username, password or '')
+    except ValueError as e:
+        raise SourceError('cannot use that login: ' + str(e)) from None
+
+
+def _login(source):
+    """The saved login for a user repository, or None."""
+    return None if source.get('builtin') else _credentials.get(source['id'])
+
+
 def _child(base, name):
     name = str(name).lstrip('/')
     decoded = urllib.parse.unquote(name)
@@ -81,6 +114,9 @@ def _child(base, name):
     url = urllib.parse.urljoin(base, urllib.parse.quote(decoded, safe='/'))  # names may contain '#' or spaces
     if not url.startswith(base) or urllib.parse.urlsplit(url).query or urllib.parse.urlsplit(url).fragment:
         raise SourceError('repository file is outside its repository')
+    # Nothing a server could normalise elsewhere: double-encoded dots, ';' parameters, encoded slashes.
+    if _credentials.path_segments(urllib.parse.urlsplit(url).path) is None:
+        raise SourceError('unsafe repository file name')
     return url
 
 
@@ -91,14 +127,17 @@ class _HTTPSRedirect(urllib.request.HTTPRedirectHandler):
         return super().redirect_request(req, fp, code, msg, headers, newurl)
 
 
-def _fetch(url, path, maximum):
+def _fetch(url, path, maximum, auth=None):
+    """auth: the repository's login, sent only inside the repository (its origin and directory,
+    by whole plain path segments), on every request including redirects."""
     host = urllib.parse.urlsplit(url).hostname
-    wait = _web.wait_time(url)
+    wait = _web.wait_time(url, auth)
     if wait:
         raise _web.limited_error(host, wait)
     request = urllib.request.Request(url, headers={'User-Agent': 'FrameControl/1.0'})
+    handlers = [_HTTPSRedirect()] + ([_credentials.Handler(auth, scoped=True)] if auth else [])
     try:
-        with urllib.request.build_opener(_HTTPSRedirect()).open(request, timeout=60) as r, open(path, 'wb') as f:
+        with urllib.request.build_opener(*handlers).open(request, timeout=60) as r, open(path, 'wb') as f:
             total = 0
             while True:
                 chunk = r.read(1 << 20)
@@ -109,8 +148,11 @@ def _fetch(url, path, maximum):
                     raise SourceError('repository file exceeds size limit')
                 f.write(chunk)
     except urllib.error.HTTPError as e:
+        if e.code == 401 or (e.code == 403 and auth):
+            raise SourceAuthError('The repository refused the login (HTTP %d); check the username and password'
+                                  % e.code if auth else 'This repository needs a username and password') from None
         if e.code in (403, 429):
-            raise _web.limited_error(host, _web.throttle(url, e.headers)) from e
+            raise _web.limited_error(host, _web.throttle(url, e.headers, auth)) from e
         raise
 
 
@@ -342,7 +384,7 @@ def sources():
     return [dict(id=i, kind=KIND, name=n, url=u, fingerprint=p, builtin=True,
                  # The archive only holds superseded versions; it clutters search unless asked for.
                  enabled=settings['enabled'].get(i, i != 'fdroid-archive'), trust='community')
-            for i, n, u, p in builtins] + settings['repos']
+            for i, n, u, p in builtins] + [dict(s, login=_credentials.has(s['id'])) for s in settings['repos']]
 
 
 def _text(value):
@@ -514,10 +556,12 @@ def _source_lock(source_id):
         return _load_locks.setdefault(source_id, threading.Lock())
 
 
-def _load(source, force=False):
+def _load(source, force=False, auth=None):
+    """auth: a login to use instead of the saved one (while adding, before anything is saved)."""
     if not re.fullmatch(r'[a-z0-9-]+', source['id']):
         raise SourceError('invalid source id')
     _url(source['url'], source.get('fingerprint'))
+    auth = auth or _login(source)
     cache = frame_host.cache_dir('apk-sources', source['id'] + '.json')
     found = None if force else _cached(source, cache)
     if found:
@@ -535,7 +579,7 @@ def _load(source, force=False):
                 jar, raw = Path(tmp) / 'index.jar', Path(tmp) / 'index.json'
                 v2 = True
                 try:
-                    _fetch(source['url'] + 'entry.jar', jar, 8 * 1024 * 1024)
+                    _fetch(source['url'] + 'entry.jar', jar, 8 * 1024 * 1024, auth=auth)
                 except urllib.error.HTTPError as e:
                     if e.code not in (404, 410):
                         raise
@@ -543,7 +587,7 @@ def _load(source, force=False):
                         raise SourceError('repository no longer serves its signed v2 index; '
                                           'refusing to fall back to the older v1 index') from e
                     v2 = False
-                    _fetch(source['url'] + 'index-v1.jar', jar, 256 * 1024 * 1024)
+                    _fetch(source['url'] + 'index-v1.jar', jar, 256 * 1024 * 1024, auth=auth)
                     content, pin = _jar(jar, 'index-v1.json', source.get('fingerprint'))
                     timestamp = _v1(content, raw)
                     _check_timestamp(source, timestamp)
@@ -552,7 +596,7 @@ def _load(source, force=False):
                     signed = json.loads(content)
                     timestamp, entry = signed.get('timestamp'), signed['index']
                     _check_timestamp(source, timestamp)
-                    _fetch(_child(source['url'], entry['name']), raw, 512 * 1024 * 1024)
+                    _fetch(_child(source['url'], entry['name']), raw, 512 * 1024 * 1024, auth=auth)
                     if _sha256(raw) != entry['sha256'] or (entry.get('size') is not None and raw.stat().st_size != entry['size']):
                         raise SourceError('index SHA-256 or size mismatch')
                 apps = _reduce(raw, source)
@@ -572,7 +616,10 @@ def _load(source, force=False):
             raise SourceError('cannot load repository: ' + str(e)) from e
 
 
-def add_repo(url, fingerprint=None, name=None):
+def add_repo(url, fingerprint=None, name=None, username=None, password=None):
+    """Check and save a repository. A login (in the URL or apart) is kept out of the
+    saved URL, in _credentials, and saved only once the repository's index verified."""
+    url, login = _split_login(url, username, password)
     url, pin = _url(url, fingerprint)
     with _LOCK:
         existing = next((s for s in _read()['repos'] if s['url'] == url), None)
@@ -583,15 +630,26 @@ def add_repo(url, fingerprint=None, name=None):
     source = dict(id='fdroid-user-' + hashlib.sha256(url.encode()).hexdigest()[:20], kind=KIND,
                   name=name or (existing or {}).get('name') or urllib.parse.urlsplit(url).hostname,
                   url=url, builtin=False, enabled=True, trust='user', fingerprint=pin)
-    _, source['fingerprint'] = _load(source, force=True)  # network work outside the settings lock
+    # Network work outside the settings lock; without a new login, a saved one (re-adding) is used.
+    _, source['fingerprint'] = _load(source, force=True, auth=login)
     source['trust_on_first_use'] = existing.get('trust_on_first_use', False) if existing else pin is None
     with _LOCK:
         settings = _read()
         current = next((s for s in settings['repos'] if s['id'] == source['id']), None)
         if current and current['fingerprint'] != source['fingerprint']:
             raise SourceError('repository already has a different pinned fingerprint; remove it first')
-        settings['repos'] = [s for s in settings['repos'] if s['id'] != source['id']] + [source]
-        _write(_storage(), settings)
+        previous = _credentials.get(source['id'])
+        if login:
+            _credentials.put(source['id'], login)
+        try:
+            settings['repos'] = [s for s in settings['repos'] if s['id'] != source['id']] + [source]
+            _write(_storage(), settings)
+        except BaseException:
+            if login and previous:  # not saved after all: leave the login as it was
+                _credentials.put(source['id'], previous)
+            elif login:
+                _credentials.remove(source['id'])
+            raise
     return source
 
 
@@ -602,6 +660,7 @@ def remove_repo(source_id):
             raise SourceError('unknown user repository')
         settings['repos'] = [s for s in settings['repos'] if s['id'] != source_id]
         _write(_storage(), settings)
+        _credentials.remove(source_id)
     with _state_file_lock(), _LOCK:  # lock order: state file, then _LOCK
         states = _states()
         if states.pop(source_id, None) is not None:  # re-adding is a deliberate new trust decision
@@ -663,7 +722,7 @@ def download(source, entry_id, version_code=None):
             fd, tmp = tempfile.mkstemp(dir=str(path.parent), suffix='.part')
             os.close(fd)
             try:
-                _fetch(_child(source['url'], version['name']), tmp, 4 * 1024 ** 3)
+                _fetch(_child(source['url'], version['name']), tmp, 4 * 1024 ** 3, auth=_login(source))
                 if _sha256(tmp) != sha:
                     raise SourceError('APK SHA-256 mismatch; download discarded')
                 os.replace(tmp, path)
@@ -684,6 +743,8 @@ def main():
     add.add_argument('url')
     add.add_argument('--fingerprint')
     add.add_argument('--name')
+    add.add_argument('--username', help='for a repository behind a login (HTTP basic auth)')
+    add.add_argument('--password-stdin', action='store_true', help='read the password from the first line of stdin')
     sub.add_parser('list')
     remove = sub.add_parser('remove')
     remove.add_argument('source')
@@ -702,7 +763,12 @@ def main():
 def _cli(parser, args):
     try:
         if args.command == 'add':
-            result = add_repo(args.url, args.fingerprint, args.name)
+            password = None
+            if args.password_stdin:
+                if not args.username:
+                    raise SourceError('--password-stdin needs --username')
+                password = sys.stdin.readline().rstrip('\r\n')
+            result = add_repo(args.url, args.fingerprint, args.name, args.username, password)
         elif args.command == 'list':
             result = sources()
         elif args.command == 'remove':

ui/apk_sources/github.py

@@ -1,17 +1,77 @@
 """Curated publisher releases; optional topic discovery is page-only."""
-import fnmatch, json, os, re, urllib.parse
+import fnmatch, hashlib, json, os, re, sys, threading, urllib.parse
 
-from . import SourceError, _web
+from . import SourceAuthError, SourceError, _credentials, _web
 
 KIND = 'github'
 API = 'https://api.github.com'
+ENV = 'FRAME_GITHUB_TOKEN'  # wins over a token saved in source settings
+CREDENTIAL = 'github'  # _credentials key; only ever sent to api.github.com
+_rejected = set()  # SHA-256 of tokens GitHub answered 401 to (this run only)
+_rejected_lock = threading.Lock()
 TOPICS = ('oculus-quest', 'openxr', 'quest')
 HOSTS = ('github.com', 'release-assets.githubusercontent.com', 'objects.githubusercontent.com')
 
 
 def sources():
     return [{'id': KIND, 'kind': KIND, 'name': 'GitHub releases', 'url': 'https://github.com',
-             'builtin': True, 'enabled': True, 'trust': 'community'}]
+             'builtin': True, 'enabled': True, 'trust': 'community', 'token': token_status()}]
+
+
+def _token():
+    """(token, where it came from) or (None, None). The environment variable wins."""
+    env = os.environ.get(ENV)
+    if env:
+        return env, 'env'
+    saved = _credentials.get(CREDENTIAL)
+    if saved and saved.kind == 'bearer':
+        return saved, 'saved'
+    return None, None
+
+
+def _credential(token):
+    if isinstance(token, _credentials.Credential):
+        return token
+    try:
+        return _credentials.bearer(API + '/', token)
+    except ValueError as e:
+        raise SourceError('%s is not a usable GitHub token: %s' % (ENV, e)) from None
+
+
+def _digest(credential):
+    return hashlib.sha256(credential.header().encode()).hexdigest()
+
+
+def token_status():
+    """Whether a token is set, where from, and whether GitHub rejected it; never the token."""
+    token, where = _token()
+    rejected = False
+    if token is not None:
+        try:
+            with _rejected_lock:
+                rejected = _digest(_credential(token)) in _rejected
+        except SourceError:
+            rejected = True
+    return {'set': token is not None, 'from': where, 'rejected': rejected}
+
+
+def set_token(token):
+    """Save a personal access token for api.github.com (FRAME_GITHUB_TOKEN still wins)."""
+    if not isinstance(token, str) or not token.strip():
+        raise SourceError('Paste a GitHub token')
+    try:
+        credential = _credentials.bearer(API + '/', token.strip())
+    except ValueError as e:
+        raise SourceError('That is not a usable GitHub token: ' + str(e)) from None
+    _credentials.put(CREDENTIAL, credential)
+    with _rejected_lock:
+        _rejected.discard(_digest(credential))  # a fresh chance
+    return {'message': 'GitHub token saved' + (' (%s is set and is used instead)' % ENV if os.environ.get(ENV) else '')}
+
+
+def clear_token():
+    _credentials.remove(CREDENTIAL)
+    return {'message': 'GitHub token removed'}
 
 
 def _curated():
@@ -20,13 +80,33 @@ def _curated():
 
 
 def _api(path):
+    """GET from the GitHub API, with the user's token if there is one. If GitHub rejects
+    the token (401), retry once without it, like Obtainium, and remember the rejection."""
     headers = {'Accept': 'application/vnd.github+json', 'X-GitHub-Api-Version': '2022-11-28'}
-    token = os.environ.get('FRAME_GITHUB_TOKEN')
-    if token:
-        headers['Authorization'] = 'Bearer ' + token
+    token, _ = _token()
+    auth = _credential(token) if token is not None else None
+    with _rejected_lock:
+        if auth and _digest(auth) in _rejected:
+            auth = None  # already refused this run: don't spend a request on it again
+    hint = ' (add a GitHub token in the store\'s source settings, or set %s, to raise the limit)' % ENV
+    if token is not None and not auth:
+        hint = ' (GitHub rejected your token; replace it in the store\'s source settings)'
+
+    def get(auth):
+        return _web.read(API + path, ('api.github.com',), headers, name='GitHub',
+                         hint='' if auth else hint, auth=auth)
     try:
-        return json.loads(_web.read(API + path, ('api.github.com',), headers, name='GitHub',
-                                    hint='' if token else ' (set FRAME_GITHUB_TOKEN to raise the limit)'))
+        try:
+            data = get(auth)
+        except SourceAuthError:
+            if not auth:
+                raise
+            with _rejected_lock:
+                _rejected.add(_digest(auth))
+            print('GitHub rejected the access token (HTTP 401); continuing without it', file=sys.stderr)
+            hint = ' (GitHub rejected your token; replace it in the store\'s source settings)'
+            data = get(None)
+        return json.loads(data)
     except (ValueError, TypeError) as e:
         raise SourceError('Invalid GitHub response') from e
 

ui/apk_sources/search.py

@@ -113,6 +113,14 @@ def manage_repo(action, **kwargs):
     return getattr(module, action)(**kwargs)
 
 
+def set_token(source_id, token):
+    """Save (token) or remove (None) a source's access token; status is in sources()."""
+    module, _ = resolve(source_id)
+    if not hasattr(module, 'set_token'):
+        raise SourceError('This source does not use an access token')
+    return module.set_token(token) if token is not None else module.clear_token()
+
+
 def normalise(name):
     return ' '.join(re.findall(r'\w+', unicodedata.normalize('NFKC', name or '').casefold()))
 

ui/frame_android.py

@@ -7,7 +7,7 @@ in the Steam library and gets its own SteamVR panel. Nothing goes through
 Lepton Development, which wipes its apps on exit. See docs/apks.md.
 
 Python stdlib only. CLI: python3 ui/frame_android.py
-  install APK [SPLIT.apk ...] [--vr|--flat] [--no-xr-compat] | info APK | versions APK-or-PKG
+  install APK [--vr|--flat] [--no-xr-compat] | info APK | versions APK-or-PKG
   install-obb PKG OBB [OBB ...] | backup-data PKG ARCHIVE | restore-data PKG ARCHIVE
   refresh-art PKG|--all | patch SRC DST [--add NAME=PATH ...] | list | launch PKG | stop PKG | remove PKG | probe PKG
 """
@@ -18,7 +18,7 @@ import frame_artwork
 import frame_host
 import tempfile
 import zipfile
-from frame_apk_vr import add_launcher_category, drop_split_required
+from frame_apk_vr import add_launcher_category
 from frame_apk_sign import repack
 
 ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -61,20 +61,13 @@ def ssh(cmd, input=None, timeout=120):
     try:
         # No inherited stdin (see server.ssh): Windows' ssh.exe would wait on it.
         feed = {'input': input} if input is not None else {'stdin': subprocess.DEVNULL}
-        # The Frame writes UTF-8, whatever this computer's code page (Windows would decode it as
-        # cp1252). errors='replace': stray bytes (a file name) mustn't turn a failed command, or
-        # one that worked, into a UnicodeDecodeError.
-        text = ({'text': True, 'encoding': 'utf-8', 'errors': 'replace'}
-                if isinstance(input, str) or input is None else {})
         p = frame_host.run_ssh(['ssh', *SSH_OPTS, FRAME, cmd], capture_output=True, **feed,
-                               timeout=timeout, **text)
+                               timeout=timeout, text=isinstance(input, str) or input is None)
     except subprocess.TimeoutExpired:
         raise frame_host.link_failure(FrameError(f'timed out talking to {FRAME}'))
     if p.returncode != 0:
-        # Bytes input gives bytes output: decode it, so the message isn't "b'...'".
-        stderr, stdout = (v if isinstance(v, str) else (v or b'').decode(errors='replace')
-                          for v in (p.stderr, p.stdout))
-        error = FrameError((stderr or stdout or f'ssh exited {p.returncode}').strip()[-600:])
+        error = FrameError((p.stderr or p.stdout or f'ssh exited {p.returncode}').strip()[-600:])
+        stderr = p.stderr if isinstance(p.stderr, str) else (p.stderr or b'').decode(errors='replace')
         raise frame_host.link_failure(error) if frame_host.ssh_link_failed(p.returncode, stderr) else error
     return p.stdout
 
@@ -142,22 +135,12 @@ def check_installable(info):
                          "(or arm64, or universal) and install that instead")
 
 
-# Installs are rare; one at a time avoids every race. Reentrant so a caller (a Quest import's
-# final checks and data copy) can hold it around install_obb and install, which take it too.
-_install_lock = threading.RLock()
-# The slowest transfer a copy is allowed before it times out, in bytes a second (about 4 Mbit/s,
-# a slow or busy Wi-Fi link). A fixed 10 minutes failed any APK over about 300 MB at that rate.
-COPY_MIN_RATE = 512 * 1024
+_install_lock = threading.Lock()  # installs are rare; one at a time avoids every race
 
 
 def _copy(src, dest, executable=False, timeout=600):
-    """Copy a local file to the Frame: rsync where installed (not on Windows), else scp.
-    timeout is the least time allowed; a large file gets longer (COPY_MIN_RATE)."""
+    """Copy a local file to the Frame: rsync where installed (not on Windows), else scp."""
     name = os.path.basename(src)
-    try:
-        timeout = max(timeout, os.path.getsize(src) / COPY_MIN_RATE)
-    except OSError:
-        pass  # rsync or scp says what's wrong with the file
     rsync = None if frame_host.WINDOWS else shutil.which('rsync')  # see server.push_file
     if rsync:
         cmd = ['rsync', '-a', *(['--chmod=u+x'] if executable else []),
@@ -165,8 +148,7 @@ def _copy(src, dest, executable=False, timeout=600):
     else:
         cmd = ['scp', *SSH_OPTS, src, f'{FRAME}:{dest}']
     try:
-        frame_host.run_ssh(cmd, check=True, capture_output=True, stdin=subprocess.DEVNULL, text=True,
-                           errors='replace', timeout=timeout)
+        frame_host.run_ssh(cmd, check=True, capture_output=True, stdin=subprocess.DEVNULL, text=True, timeout=timeout)
     except subprocess.TimeoutExpired:
         raise FrameError(f'copying {name} to the Frame timed out')
     except subprocess.CalledProcessError as e:
@@ -192,303 +174,7 @@ def _write_meta(d, meta):
 install_hooks = []
 
 
-# Split APK members that belong to the split itself, not the app: never merged.
-SPLIT_OWN = re.compile(r'^(AndroidManifest\.xml|stamp-cert-sha256|META-INF/.*)$')
-# What a mergeable split may add: native libraries and assets, nothing else.
-SPLIT_ADDS = re.compile(r'^(lib/[^/]+/[^/]+\.so|assets/.*[^/])$')
-# Manifest elements a config or install-time asset split carries. Anything else (an
-# activity, service, uses-library...) belongs to a feature split, which needs its manifest.
-SPLIT_TAGS = {'manifest', 'application', 'uses-sdk', 'module', 'delivery', 'install-time', 'fusing'}
-# Every merged byte is held in memory while the APK is repacked and signed (as for every
-# patched APK), so splits are bounded by what's actually inflated, not only the headers.
-MAX_SPLIT_MEMBER = 512 * 1024**2
-MAX_SPLIT_TOTAL = 1024**3
-
-
-def _split_manifest(path, name):
-    """(package, split name) from a split APK's manifest, refusing splits that declare
-    anything the merged app would lose with that manifest."""
-    with zipfile.ZipFile(path) as z:
-        elements = frame_apk.manifest_elements(frame_apk._read(z, 'AndroidManifest.xml', frame_apk.MAX_MANIFEST))
-    attrs = next((a for tag, a in elements if tag == 'manifest'), {})
-    split = (attrs.get('split') or (None, None, None))[2]
-    feature = attrs.get('isFeatureSplit')
-    if feature and feature[1]:
-        raise FrameError(f'{name} ({split or "split"}) is a feature split with its own code and manifest; '
-                         'Frame Control can only merge splits that add native libraries or assets')
-    extra = sorted({tag for tag, _ in elements if tag not in SPLIT_TAGS})
-    if extra:
-        raise FrameError(f'{name} ({split or "split"}) declares <{">, <".join(extra)}> in its manifest; '
-                         'Frame Control can only merge splits that add native libraries or assets')
-    return (attrs.get('package') or (None, None, None))[2], split
-
-
-def split_entries(base, splits):
-    """What merging splits into base adds: {entry name: (split path, entry name, size)}.
-
-    Lepton runs one APK per instance, so a split app is merged into one APK. Only splits
-    that add native libraries (the arm64 ABI split) or install-time assets can be: a split
-    with code, components or resources (screen density, language) needs its own manifest
-    or resource table, which Frame Control doesn't merge, so those are refused."""
-    try:
-        with zipfile.ZipFile(base) as z:
-            taken = set(z.namelist())
-        package = apk_info(base)['package']
-        plan, total = {}, 0
-        for path in splits:
-            name = os.path.basename(path)
-            split_pkg, split_name = _split_manifest(path, name)
-            label = f'{name} ({split_name or "split"})'
-            if split_pkg != package:
-                raise FrameError(f'{name} is not part of {package} (it says {split_pkg or "nothing"})')
-            with zipfile.ZipFile(path) as z:
-                for member in z.infolist():
-                    entry = member.filename
-                    if entry.endswith('/') or SPLIT_OWN.match(entry):
-                        continue
-                    if entry == 'resources.arsc':
-                        # A config split for native code may carry an empty table; any resource needs merging.
-                        if not resources_empty(frame_apk._read(z, entry, frame_apk.MAX_ARSC)):
-                            raise FrameError(f'{label} holds screen-density or language resources; Frame Control '
-                                             'can only merge splits that add native libraries or assets')
-                        continue
-                    if entry.startswith('res/'):
-                        raise FrameError(f'{label} holds screen-density or language resources ({entry}); Frame '
-                                         'Control can only merge splits that add native libraries or assets')
-                    if re.match(r'^classes\d*\.dex$', entry):
-                        raise FrameError(f'{label} contains code ({entry}); Frame Control can only merge splits '
-                                         'that add native libraries or assets')
-                    if not SPLIT_ADDS.match(entry):
-                        raise FrameError(f'{label} contains {entry}; Frame Control can only merge splits that '
-                                         'add native libraries or assets')
-                    if entry in taken:
-                        raise FrameError(f'{name} and another part of the app both contain {entry}')
-                    if member.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED):
-                        raise FrameError(f'{entry} in {name} uses a compression Android does not')
-                    if member.file_size > MAX_SPLIT_MEMBER:
-                        raise FrameError(f'{entry} in {name} is {member.file_size / 1024**2:.0f} MB, more than '
-                                         f'Frame Control merges ({MAX_SPLIT_MEMBER // 1024**2} MB)')
-                    total += member.file_size
-                    if total > MAX_SPLIT_TOTAL:
-                        raise FrameError(f'the split APKs add more than {MAX_SPLIT_TOTAL // 1024**3} GB, more than '
-                                         'Frame Control merges')
-                    taken.add(entry)
-                    plan[entry] = (path, entry, member.file_size)
-        return plan
-    except (OSError, zipfile.BadZipFile, struct.error, IndexError, frame_apk.ApkError) as e:
-        raise FrameError(f'could not read the split APKs: {e}') from e
-
-
-def resources_empty(data):
-    """True only for a resource table that provably defines nothing: every type chunk's
-    entries absent. Bags (styles, plurals) count, and anything not understood counts too."""
-    def covered(start, end):
-        chunks, at = [], start
-        for chunk in frame_apk._chunks(data, start, end):
-            if chunk[2] != at:
-                return None
-            chunks.append(chunk)
-            at = chunk[2] + chunk[3]
-        return chunks if at == end else None
-
-    try:
-        if len(data) < 12 or struct.unpack_from('<H', data, 0)[0] != 0x0002:
-            return False
-        top = covered(struct.unpack_from('<H', data, 2)[0], len(data))
-        if top is None:
-            return False
-        for ctype, _, off, size in top:
-            if ctype == 0x0001:  # the global string pool
-                continue
-            if ctype != 0x0200:
-                return False
-            inner = covered(off + struct.unpack_from('<H', data, off + 2)[0], off + size)
-            if inner is None:
-                return False
-            for kind, hs, t, _ in inner:
-                if kind in (0x0001, 0x0202):  # type and key names, type specs
-                    continue
-                if kind != 0x0201:
-                    return False
-                flags, count = struct.unpack_from('<xBxxI', data, t + 8)
-                if flags & 0x01:  # sparse: only present entries are listed
-                    if count:
-                        return False
-                elif flags & 0x02:
-                    if any(o != 0xffff for o in struct.unpack_from(f'<{count}H', data, t + hs)):
-                        return False
-                elif any(o != 0xffffffff for o in struct.unpack_from(f'<{count}I', data, t + hs)):
-                    return False
-        return True
-    except (struct.error, IndexError):
-        return False
-
-
-def _zip64_sizes(extra, usize, csize):
-    """(uncompressed, compressed) with 0xffffffff fields taken from a ZIP64 extra field."""
-    at = 0
-    while at + 4 <= len(extra):
-        kind, length = struct.unpack_from('<HH', extra, at)
-        body = extra[at + 4:at + 4 + length]
-        if kind == 0x0001:
-            values, pos = [], 0
-            for field in (usize, csize):
-                if field == 0xffffffff:
-                    if pos + 8 > len(body):
-                        raise FrameError('truncated ZIP64 extra field')
-                    values.append(struct.unpack_from('<Q', body, pos)[0])
-                    pos += 8
-                else:
-                    values.append(field)
-            return tuple(values)
-        at += 4 + length
-    if 0xffffffff in (usize, csize):
-        raise FrameError('ZIP64 sizes without a ZIP64 extra field')
-    return usize, csize
-
-
-def _has_zip64_extra(extra):
-    at = 0
-    while at + 4 <= len(extra):
-        kind, length = struct.unpack_from('<HH', extra, at)
-        if kind == 0x0001:
-            return True
-        at += 4 + length
-    return False
-
-
-def read_member(path, member):
-    """A split member's bytes, read from its raw data and checked against both headers.
-
-    The local header must name the same file, with the same method and (unless a data
-    descriptor carries them, which is then checked) the same sizes and checksum as the
-    central directory; the data must end before the next member; a stored member must be
-    exactly its size and a deflated stream must end exactly at the end of its data, with
-    nothing after it. Output is bounded by the declared size: never trusts zipfile, which
-    stops early at a smaller declared size."""
-    with zipfile.ZipFile(path) as z:
-        info = z.getinfo(member)
-        starts = sorted({i.header_offset for i in z.infolist()} | {z.start_dir})
-    name, size = f'{member} in {os.path.basename(path)}', info.file_size
-    if info.flag_bits & 1 or info.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED):
-        raise FrameError(f'{name} is encrypted or uses a compression Android does not')
-    if size > MAX_SPLIT_MEMBER or info.compress_size > MAX_SPLIT_MEMBER:
-        raise FrameError(f'{name} is larger than Frame Control merges ({MAX_SPLIT_MEMBER // 1024**2} MB)')
-    following = next((o for o in starts if o > info.header_offset), None)
-    with open(path, 'rb') as raw:
-        raw.seek(info.header_offset)
-        header = raw.read(30)
-        if len(header) != 30 or header[:4] != b'PK\3\4':
-            raise FrameError(f'{name}: invalid local ZIP header')
-        flags, method, crc, csize, usize, nl, el = struct.unpack_from('<2xHH4xIIIHH', header, 4)
-        local_name, extra = raw.read(nl), raw.read(el)
-        if local_name != info.orig_filename.encode('utf-8' if flags & 0x800 else 'cp437', 'surrogateescape'):
-            raise FrameError(f'{name}: the local header names {local_name.decode("utf-8", "replace")!r}')
-        if method != info.compress_type or (flags & 0x08) != (info.flag_bits & 0x08):
-            raise FrameError(f'{name} does not match its header (local and central entries differ)')
-        start = info.header_offset + 30 + nl + el
-        compressed = raw.read(info.compress_size)
-        if len(compressed) != info.compress_size:
-            raise FrameError(f'{name} is truncated')
-        if flags & 0x08:
-            # Sizes and checksum follow the data: (signature,) crc, then sizes of 4 bytes each,
-            # or 8 for ZIP64. Streaming writers use the ZIP64 form even for small files when
-            # sizes weren't known up front, so the width isn't implied by the final sizes: take
-            # the layout that matches the central directory and fits before the next entry,
-            # preferring one that ends exactly there, then the width the local header implies.
-            zip64 = 0xffffffff in (csize, usize) or _has_zip64_extra(extra)
-            tail = raw.read(24)
-            if len(tail) < 12:
-                raise FrameError(f'{name}: truncated data descriptor')
-            end = start + info.compress_size
-            fits = []
-            for signed in (True, False):
-                if signed and tail[:4] != b'PK\7\x08':
-                    continue
-                for width in (20, 12):
-                    body = tail[4:4 + width] if signed else tail[:width]
-                    if len(body) < width:
-                        continue
-                    values = struct.unpack_from('<IQQ' if width == 20 else '<III', body)
-                    length = width + (4 if signed else 0)
-                    if values == (info.CRC, info.compress_size, size) and \
-                            (following is None or end + length <= following):
-                        exact = following is not None and end + length == following
-                        fits.append((not exact, (width == 20) != zip64, length))
-            if not fits:
-                raise FrameError(f'{name} does not match its header (local and central entries differ, '
-                                 'or its data descriptor is truncated)')
-            crc, csize, usize = info.CRC, info.compress_size, size
-            descriptor = min(fits)[2]
-        else:
-            usize, csize = _zip64_sizes(extra, usize, csize)
-            descriptor = 0
-    if (crc, csize, usize) != (info.CRC, info.compress_size, size):
-        raise FrameError(f'{name} does not match its header (local and central entries differ)')
-    if following is not None and start + info.compress_size + descriptor > following:
-        raise FrameError(f'{name} overlaps the next ZIP entry')
-    if info.compress_type == zipfile.ZIP_STORED:
-        if info.compress_size != size:
-            raise FrameError(f'{name} does not match its header (stored {info.compress_size} bytes, says {size})')
-        data = compressed
-    else:
-        inflate = zlib.decompressobj(-15)
-        try:
-            data = inflate.decompress(compressed, size + 1)
-        except zlib.error as e:
-            raise FrameError(f'{name} is corrupt: {e}') from e
-        if len(data) != size or not inflate.eof or inflate.unconsumed_tail or inflate.unused_data:
-            raise FrameError(f'{name} does not match its header (its compressed data is not exactly '
-                             f'{size} bytes with nothing after it)')
-    if zlib.crc32(data) != info.CRC:
-        raise FrameError(f'{name} does not match its checksum')
-    return data
-
-
-def merge_splits(base, splits, dst):
-    """One installable APK from a base APK and its splits (see split_entries), re-signed
-    with Frame Control's key, without the base's isSplitRequired flag."""
-    plan = split_entries(base, splits)
-    add, total = {}, 0
-    try:
-        for entry, (path, member, size) in plan.items():
-            data = read_member(path, member)
-            total += len(data)
-            if total > MAX_SPLIT_TOTAL:
-                raise FrameError(f'the split APKs add more than {MAX_SPLIT_TOTAL // 1024**3} GB')
-            add[entry] = data
-        with zipfile.ZipFile(base) as z:
-            original = frame_apk._read(z, 'AndroidManifest.xml', frame_apk.MAX_MANIFEST)
-        manifest = drop_split_required(original)
-        repack(base, dst, replace={'AndroidManifest.xml': manifest} if manifest != original else None, add=add)
-    except (OSError, ValueError, IndexError, struct.error, zipfile.BadZipFile, frame_apk.ApkError) as e:
-        raise FrameError(f'could not merge the split APKs: {e}') from e
-    return {'merged': sorted(os.path.basename(p) for p in splits), 'entries': len(plan)}
-
-
-def split_size(splits):
-    """Bytes a merge adds, from the split headers (merge_splits enforces the real sizes)."""
-    total = 0
-    for path in splits:
-        with zipfile.ZipFile(path) as z:
-            total += sum(m.file_size for m in z.infolist() if SPLIT_ADDS.match(m.filename))
-    return total
-
-
-def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None,
-            splits=None):
-    """splits: the rest of a split APK (base first, as apk_path); merged into one APK first."""
-    if splits:
-        start = time.time()
-        with tempfile.TemporaryDirectory(prefix=f'{TMP_PREFIX}{os.getpid()}-') as tmp:
-            merged = os.path.join(tmp, 'merged.apk')
-            try:
-                merge_splits(apk_path, list(splits), merged)
-            except Exception as e:  # reported like any other failed install
-                _after_install(None, None, e, start)
-                raise
-            meta = install(merged, flatscreen, name, source or os.path.basename(apk_path), icon_png, xr_compat, artwork)
-            return meta
+def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None):
     start, info = time.time(), None
     try:
         info = apk_info(apk_path)
@@ -571,7 +257,6 @@ def _install(apk_path, info, pkg, flatscreen, name, source, artwork=None):
                                               'source': source or os.path.basename(apk_path)})
         presentation['warnings'] = art_warnings + presentation.get('warnings', [])
         meta = {'package': pkg, 'label': name or info['label'], 'version': info['version'],
-                'version_code': info.get('version_code'),
                 'instance': iid, 'shortcut': shortcut, 'game_id': game_id(shortcut),
                 'vr': info.get('vr', False), 'vr_issues': info.get('vr_issues', []),
                 'launchable': info.get('launchable', False), 'patched': info.get('patched', []),
@@ -594,13 +279,6 @@ def _install(apk_path, info, pkg, flatscreen, name, source, artwork=None):
                 ssh(f'rm -rf {d}', timeout=30)
             except FrameError:
                 pass
-        elif not ok:
-            # An update that failed: the app still works, but a partial copy of the new APK
-            # (scp leaves one; it can be gigabytes) mustn't stay beside it.
-            try:
-                ssh(f'rm -f {d}/app.apk.part', timeout=30)
-            except FrameError:
-                pass
 
 
 
@@ -658,34 +336,6 @@ except (OSError, ValueError, TypeError, AttributeError):
 """
 
 
-def _apk_parser_script():
-    """Frame-side Python preamble that loads this copy's stdlib APK parser (frame_apk)."""
-    script = 'import sys, types, json, base64\n'
-    for module in ('frame_apk', 'frame_apk_vr'):
-        with open(os.path.join(ROOT, 'ui', module + '.py')) as f:
-            source = f.read()
-        script += f'm = types.ModuleType({module!r}); sys.modules[{module!r}] = m; exec({source!r}, m.__dict__)\n'
-    return script
-
-
-def installed_version_code(pkg):
-    """The version code of pkg's installed app.apk, read on the Frame (only its manifest
-    crosses the link), or None if there is no readable APK. Connection failures raise."""
-    if not PKG_RE.match(pkg or ''):
-        raise FrameError(f'bad package name {pkg!r}')
-    script = _apk_parser_script() + f"""try:
-    code = sys.modules['frame_apk'].apk_info({(APPS_DIR + '/' + pkg + '/app.apk')!r}).get('version_code')
-except Exception:
-    code = None
-print(json.dumps(code if isinstance(code, int) else None))
-"""
-    try:
-        code = json.loads(ssh('python3 -', input=script, timeout=120).strip().splitlines()[-1])
-    except (ValueError, IndexError):
-        return None
-    return code if isinstance(code, int) and not isinstance(code, bool) else None
-
-
 def art_missing(m):
     """True when a Frame Control install has no complete Steam artwork on record."""
     return set((m or {}).get('artwork') or {}) != set(frame_artwork.SLOTS)
@@ -707,7 +357,13 @@ def refresh_art(pkg=None, artwork=None, fill_only=False):
         m = _meta_or_fail(pkg)
         d = f'{APPS_DIR}/{pkg}'
         # Parse the APK on the Frame, transferring only its icon/label, not the APK.
-        script = _apk_parser_script()
+        modules = {}
+        for module in ('frame_apk', 'frame_apk_vr'):
+            with open(os.path.join(ROOT, 'ui', module + '.py')) as f:
+                modules[module] = f.read()
+        script = 'import sys, types, json, base64\n'
+        for module, source in modules.items():
+            script += f'm = types.ModuleType({module!r}); sys.modules[{module!r}] = m; exec({source!r}, m.__dict__)\n'
         script += f"info = sys.modules['frame_apk'].apk_info({(d + '/app.apk')!r})\n"
         script += "info['icon_png'] = base64.b64encode(info.get('icon_png') or b'').decode()\n"
         script += cached_art_script(d) + "info['artwork'] = cached\nprint(json.dumps(info))\n"
@@ -768,22 +424,10 @@ def _clean_meta(m):
     return m
 
 
-def running_instances(strict=False):
-    """Lepton container name -> adb port, for the instances that are running now.
-
-    strict: a failed `podman ps` raises FrameError instead of reading as "nothing running",
-    and containers without an adb port label count too (port None); for callers that must
-    not mistake an error for a stopped instance."""
-    if not strict:
-        out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}" 2>/dev/null || true')
-        return dict(line.split()[:2] for line in out.splitlines() if len(line.split()) >= 2)
-    out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}"')
-    found = {}
-    for line in out.splitlines():
-        cols = line.split()
-        if cols:
-            found[cols[0]] = cols[1] if len(cols) > 1 and cols[1] != '<no' else None
-    return found
+def running_instances():
+    """Lepton container name -> adb port, for the instances that are running now."""
+    out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}" 2>/dev/null || true')
+    return dict(line.split()[:2] for line in out.splitlines() if len(line.split()) >= 2)
 
 
 def list_apps():
@@ -932,11 +576,8 @@ def main():
             r = frame_apk_versions.alternatives(
                 info['package'] if info else args[0], info.get('version_code') if info else None)
         elif cmd == 'install':
-            files = [a for a in args if not a.startswith('--')]
-            if not files:
-                raise FrameError('install requires APK [SPLIT.apk ...]')
-            r = install(files[0], flatscreen=False if '--vr' in args else True if '--flat' in args else None,
-                        xr_compat=False if '--no-xr-compat' in args else None, splits=files[1:])
+            r = install(args[0], flatscreen=False if '--vr' in args else True if '--flat' in args else None,
+                        xr_compat=False if '--no-xr-compat' in args else None)
         elif cmd == 'refresh-art':
             if len(args) != 1:
                 raise FrameError('usage: refresh-art PACKAGE or refresh-art --all')

ui/frame_apk.py

@@ -7,7 +7,6 @@ the resource table in resources.arsc.
 """
 import struct
 import zipfile
-import zlib
 
 # android: attribute resource ids; names can be stripped by shrinkers, ids can't.
 ATTR = {0x01010001: 'label', 0x01010002: 'icon', 0x01010003: 'name',
@@ -222,9 +221,7 @@ def apk_info(path):
             res = Resources(_read(z, 'resources.arsc', MAX_ARSC) if 'resources.arsc' in names else b'')
             from frame_apk_vr import detection
             vr_info = detection(manifest_data, names)
-        # A damaged download: corrupt deflate data (zlib.error), a member cut short (EOFError),
-        # undecodable text (UnicodeDecodeError) or an encrypted entry (RuntimeError).
-        except (struct.error, IndexError, zipfile.BadZipFile, zlib.error, EOFError, ValueError, RuntimeError) as e:
+        except (struct.error, IndexError, zipfile.BadZipFile) as e:
             raise ApkError(f'could not read the APK manifest: {e}')
         tags = {}
         for tag, attrs in elements:

ui/frame_apk_vr.py

@@ -126,46 +126,3 @@ def detection(data, names):
         issues.append('Uses OpenXR (good).')
     info['vr_issues'] = issues
     return info
-
-
-IS_SPLIT_REQUIRED = 0x01010591  # android:isSplitRequired
-
-
-def drop_split_required(axml_bytes):
-    """The manifest without <application android:isSplitRequired>, for a base APK whose
-    splits were merged into it: Android refuses a lone base that still says it needs them."""
-    chunks = list(frame_apk._chunks(axml_bytes, 8, len(axml_bytes)))
-    pool = next((c for c in chunks if c[0] == 1), None)
-    if pool is None:
-        raise frame_apk.ApkError('AndroidManifest.xml has no string pool')
-    strings = frame_apk._string_pool(axml_bytes, pool[2])
-    resmap = ()
-    for kind, header_size, offset, chunk_size in chunks:
-        if kind == 0x180:
-            resmap = struct.unpack_from('<%dI' % ((chunk_size - header_size) // 4), axml_bytes, offset + header_size)
-    result, changed = bytearray(axml_bytes[:8]), False
-    for kind, hs, off, size in chunks:
-        chunk = bytearray(axml_bytes[off:off + size])
-        if kind == 0x0102 and not changed:
-            name, astart, asize, count = struct.unpack_from('<4xIHHH', chunk, hs)
-            if name < len(strings) and strings[name] == 'application':
-                for i in range(count):
-                    a = hs + astart + i * asize
-                    attr = struct.unpack_from('<I', chunk, a + 4)[0]
-                    known = resmap[attr] if attr < len(resmap) else None
-                    if known == IS_SPLIT_REQUIRED or (known is None and attr < len(strings)
-                                                      and strings[attr] == 'isSplitRequired'):
-                        del chunk[a:a + asize]
-                        struct.pack_into('<I', chunk, 4, len(chunk))
-                        struct.pack_into('<H', chunk, hs + 12, count - 1)
-                        # id/class/style attribute indexes are 1-based positions in the list.
-                        for at in (hs + 14, hs + 16, hs + 18):
-                            index = struct.unpack_from('<H', chunk, at)[0]
-                            struct.pack_into('<H', chunk, at, 0 if index == i + 1 else index - 1 if index > i + 1 else index)
-                        changed = True
-                        break
-        result += chunk
-    if not changed:
-        return axml_bytes
-    struct.pack_into('<I', result, 4, len(result))
-    return bytes(result)

ui/frame_link.py

@@ -238,13 +238,6 @@ def scrub_failure(text, hosts=()):
     return frame_telemetry.scrub(hide_operands(hide_hosts(str(text or ""), hosts)), 600)
 
 
-def failure_detail(error):
-    """What an address that answered but failed SSH shows: the short reason when there is one
-    (the whole message is the banner's headline already), else the message."""
-    error = error or {}
-    return error.get("short") or error.get("message") or "SSH failed"
-
-
 def failure_category(message, raw=""):
     """The telemetry error category (frame_unreachable, frame_auth, ...) for a failure: a fixed
     word, never its text."""
@@ -817,9 +810,7 @@ class Link:
         return "Connecting"
 
     def fail(self, sid, message, raw="", **more):
-        """A stage failed. `more`: what the page can offer to fix it (hostkey, auth), and `short`:
-        the reason in a few words, for the places a whole sentence doesn't fit (the header pill,
-        the address that failed)."""
+        """A stage failed. `more`: what the page can offer to fix it (hostkey, auth)."""
         self.stage(sid, "failed", message)
         with self.cond:
             self.state["error"] = dict({"stage": sid, "message": message, "raw": raw}, **more)
@@ -905,7 +896,7 @@ class Link:
                         self.learn(device, a["host"], net, None)
                         return True
                     with self.cond:
-                        why_not = failure_detail(self.state["error"])
+                        why_not = (self.state["error"] or {}).get("message") or "SSH failed"
                     self.probe_update(i, state="sshfailed", detail=why_not)
                     if outcome != "next":
                         return False
@@ -955,7 +946,7 @@ class Link:
                 self.learn(device, a["host"], net, results[pick]["rtt_ms"])
                 return True
             with self.cond:
-                why_not = failure_detail(self.state["error"])
+                why_not = (self.state["error"] or {}).get("message") or "SSH failed"
             self.probe_update(pick, state="sshfailed", detail=why_not)
             if outcome != "next":
                 return False
@@ -1278,7 +1269,6 @@ class Link:
                            "If SteamOS was reinstalled or reset, or this is a new headset, check its new key and trust "
                            "it on the Devices tab. If not, a different device may be answering here.")
             self.fail("identity", message, text,
-                      short="Saved SSH identity missing" if lost else "SSH identity changed",
                       hostkey={"state": "missing" if lost else "changed", "type": shown[0] if shown else None,
                                "fingerprint": shown[1] if shown else None, "host": host})
             return "next"
@@ -1289,8 +1279,7 @@ class Link:
         if re.search(r"Permission denied", text) and not forward:
             # Not something a retry fixes: the headset needs this computer's key (Set Up Connection).
             self.fail("login", self.explain(text) or "The headset didn't accept this computer's key. Run Set Up "
-                      "Connection: it pairs this computer with the headset again.", text, auth="denied",
-                      short="This computer's key refused")
+                      "Connection: it pairs this computer with the headset again.", text, auth="denied")
             return "stop"
         if step == "connected":
             self.fail("login", "Logged in, but the shared SSH connection didn't start.", text)

ui/frame_quest.py

@@ -1,630 +0,0 @@
-"""Import from my Quest: copy games you own off your own Meta Quest onto the Frame.
-
-The Quest is plugged into this computer by USB with developer mode on; its own
-adb copies the APK (and split APKs) into a per-user cache. The APK is checked
-before anything goes to the Frame, then installed through frame_android.install
-like any other APK. Game data (the OBB folder and, if asked, the external
-app-data folder) is copied by a second import while the person has the game
-open, since only a running Lepton instance can take it; Frame Control never
-starts the game itself. The cache is always removed. Games that check their Meta store licence
-(libovrplatformloader.so) are refused: that check needs Meta's services, and
-Frame Control does not remove it or work around it. See docs/quest-import.md.
-
-Python stdlib only. CLI: python3 ui/frame_quest.py
-  devices | list [--serial SERIAL] | import PACKAGE [--serial SERIAL] [--data] [--allow-downgrade]
-"""
-import json
-import os
-from pathlib import Path
-import re
-import secrets
-import shlex
-import shutil
-import subprocess
-import sys
-import tarfile
-import threading
-import time
-import zipfile
-
-import frame_android as android
-import frame_host
-
-OBB = '/sdcard/Android/obb'
-DATA = '/sdcard/Android/data'
-OBB_NAME = r'(main|patch)\.[0-9]+\.{}\.obb'
-# Serials of adb connections that aren't a USB headset: Frame Control's own SSH tunnels
-# to the Frame's Lepton instances (127.0.0.1:<port>, server.AdbTunnel) and emulators.
-TUNNEL_RE = re.compile(r'^(127\.0\.0\.1|localhost|\[::1\]):\d+$')
-QUEST_MAKERS = {'oculus', 'meta'}
-META_APPS = ('com.oculus.', 'com.meta.', 'com.facebook.')
-CACHE_TTL = 6 * 3600  # an import left behind by a crash is swept after this
-ADB_TIMEOUT = 30
-PULL_TIMEOUT = 3 * 3600
-READY_TIMEOUT = 150
-BATCH = 15  # packages per adb shell command while listing
-
-ENTITLEMENT = ("{label} checks your Meta Quest store licence with Meta's Platform SDK "
-               "(libovrplatformloader.so). That check needs Meta's services, which only exist on Meta "
-               "headsets, so the game would quit on the Frame. Frame Control won't remove or work around "
-               "the check. Ask the developer for a build without it, or look for the game on Steam or itch.io.")
-NO_DEVICE = ("No Quest found on this computer's USB. Connect the Quest with a cable that carries data, "
-             "put the headset on and allow USB debugging. Developer mode must be on first: it needs a Meta "
-             "developer organisation on a verified Meta account (developers.meta.com), then Developer Mode "
-             "switched on for the headset in the Meta Horizon phone app.")
-UNAUTHORISED = ("The Quest hasn't allowed this computer yet. Put the headset on and accept the \"Allow USB "
-                "debugging\" prompt (tick Always allow from this computer), then try again. No prompt? Unplug "
-                "and replug the cable, or switch developer mode off and on.")
-NO_PERMISSIONS = ("This computer sees the Quest but isn't allowed to open it over USB. On Linux, add a udev "
-                  "rule for Meta's USB vendor id 2833 (see docs/quest-import.md), then replug it.")
-OFFLINE = "adb reports the Quest offline. Unplug and replug the cable, or restart the headset."
-GONE = "The Quest was disconnected. Plug it back in, put it on, and try again."
-
-
-class QuestError(android.FrameError):
-    pass
-
-
-def _adb_cmd():
-    try:
-        return [frame_host.adb()]
-    except frame_host.HostError as e:
-        raise QuestError(str(e)) from e
-
-
-def _friendly(out):
-    low = out.lower()
-    if 'unauthorized' in low:
-        return UNAUTHORISED
-    if 'no permissions' in low or 'insufficient permissions' in low:
-        return NO_PERMISSIONS
-    if 'device offline' in low:
-        return OFFLINE
-    if 'no devices/emulators found' in low or re.search(r"device '[^']*' not found", low):
-        return GONE
-    return None
-
-
-def adb(*args, serial=None, timeout=ADB_TIMEOUT):
-    """Run this computer's adb (the one Frame Control already finds) and return stdout."""
-    cmd = _adb_cmd() + (['-s', serial] if serial else []) + list(args)
-    try:
-        r = subprocess.run(cmd, capture_output=True, stdin=subprocess.DEVNULL, text=True, errors='replace',
-                           timeout=timeout)
-    except subprocess.TimeoutExpired:
-        raise QuestError(f'adb {args[0]} took more than {timeout} s; check the Quest is still connected')
-    except OSError as e:
-        raise QuestError(f'could not run adb: {e}')
-    if r.returncode != 0:
-        out = (r.stderr or '').strip() or (r.stdout or '').strip()
-        raise QuestError(_friendly(out) or out[-600:] or f'adb exited {r.returncode}')
-    return r.stdout
-
-
-# ---- Finding the Quest ------------------------------------------------------
-
-def _parse_devices(out):
-    found = []
-    for line in out.splitlines():
-        line = line.strip()
-        if not line or line.startswith('List of devices') or line.startswith('*'):
-            continue
-        parts = line.split(None, 1)
-        serial, rest = parts[0], (parts[1] if len(parts) > 1 else '')
-        state = 'no permissions' if rest.startswith('no permissions') else (rest.split() or ['unknown'])[0]
-        fields = dict(t.split(':', 1) for t in rest.split() if re.match(r'^[a-z_]+:', t))
-        found.append({'serial': serial, 'state': state, 'usb': 'usb' in fields,
-                      'model': fields.get('model', '').replace('_', ' '), 'manufacturer': ''})
-    return found
-
-
-def devices():
-    """Every adb device on this computer, each with a kind: quest, frame, android (another
-    Android device), frame-tunnel (Frame Control's own SSH-tunnelled connection to a Lepton
-    instance, never touched here), emulator, or unknown (not authorised, offline...)."""
-    found = _parse_devices(adb('devices', '-l', timeout=15))
-    for d in found:
-        if TUNNEL_RE.match(d['serial']):
-            d['kind'] = 'frame-tunnel'
-        elif d['serial'].startswith('emulator-'):
-            d['kind'] = 'emulator'
-        elif d['state'] != 'device':
-            d['kind'] = 'unknown'
-        else:
-            try:
-                props = adb('shell', 'getprop ro.product.manufacturer; getprop ro.product.model',
-                            serial=d['serial'], timeout=15).splitlines() + ['', '']
-            except QuestError:
-                d['kind'] = 'unknown'
-                continue
-            d['manufacturer'], d['model'] = props[0].strip(), props[1].strip() or d['model']
-            maker = d['manufacturer'].lower()
-            d['kind'] = ('quest' if maker in QUEST_MAKERS or 'quest' in d['model'].lower()
-                         else 'frame' if maker == 'valve' else 'android')
-    return found
-
-
-def find_quest(serial=None):
-    """The one connected, authorised Quest, or a QuestError saying what to do."""
-    found = devices()
-    if serial:
-        d = next((d for d in found if d['serial'] == serial), None)
-        if not d:
-            raise QuestError(f'No adb device {serial} on this computer. ' + NO_DEVICE)
-        if d['kind'] == 'quest':
-            return d
-        state = {'unauthorized': UNAUTHORISED, 'no permissions': NO_PERMISSIONS, 'offline': OFFLINE}.get(d['state'])
-        what = ' '.join(x for x in (d['manufacturer'] or d['kind'], d['model']) if x)
-        raise QuestError(state or f"{serial} isn't a Meta Quest ({what}).")
-    quests = [d for d in found if d['kind'] == 'quest']
-    if len(quests) == 1:
-        return quests[0]
-    if quests:
-        raise QuestError('Several Quests are connected: ' + ', '.join(f"{d['model']} ({d['serial']})" for d in quests)
-                         + '. Choose one, or unplug the others.')
-    states = {d['state'] for d in found if d['kind'] == 'unknown'}
-    for state, message in (('unauthorized', UNAUTHORISED), ('no permissions', NO_PERMISSIONS), ('offline', OFFLINE)):
-        if state in states:
-            raise QuestError(message)
-    others = [d for d in found if d['kind'] in ('android', 'frame')]
-    if others:
-        raise QuestError(NO_DEVICE + ' (adb sees ' + ', '.join(
-            f"{d['manufacturer']} {d['model']}".strip() for d in others) + ", which isn't a Quest.)")
-    raise QuestError(NO_DEVICE)
-
-
-# ---- What's on it -------------------------------------------------------------
-
-def _package_script(packages):
-    parts = []
-    for p in packages:
-        parts += [f'echo @@{p}', f'pm path {p}', f'du -sk {OBB}/{p} 2>/dev/null', f'du -sk {DATA}/{p} 2>/dev/null']
-    return '; '.join(parts + ['true'])
-
-
-def _parse_packages(out, packages):
-    apps, current = {}, None
-    for line in out.splitlines():
-        line = line.strip()
-        if line.startswith('@@') and line[2:] in packages:
-            current = apps[line[2:]] = {'package': line[2:], 'apks': [], 'obb': None, 'data': None}
-        elif current is None or not line:
-            continue
-        elif line.startswith('package:'):
-            current['apks'].append(line[len('package:'):])
-        else:
-            size, _, where = line.partition('\t') if '\t' in line else line.partition(' ')
-            if size.isdigit():
-                key = 'obb' if where.strip().startswith(OBB + '/') else 'data' if where.strip().startswith(DATA + '/') else None
-                if key:
-                    current[key] = int(size) * 1024
-    result = []
-    for p in packages:
-        app = apps.get(p)
-        if not app or not app['apks']:
-            continue  # removed while listing
-        app['split'] = len(app['apks']) > 1
-        app['meta'] = p.startswith(META_APPS)
-        result.append(app)
-    return result
-
-
-def packages(serial):
-    """Third-party apps on the Quest: APK paths (several means a split APK), and the size
-    in bytes of the OBB and external data folders (None when there is none)."""
-    out = adb('shell', 'pm list packages -3', serial=serial)
-    names = sorted({l.split(':', 1)[1].strip() for l in out.splitlines() if l.startswith('package:')})
-    names = [n for n in names if android.PKG_RE.match(n)]
-    apps = []
-    for i in range(0, len(names), BATCH):
-        batch = names[i:i + BATCH]
-        apps += _parse_packages(adb('shell', _package_script(batch), serial=serial, timeout=120), batch)
-    return apps
-
-
-def package(serial, pkg):
-    apps = _parse_packages(adb('shell', _package_script([pkg]), serial=serial, timeout=60), [pkg])
-    if not apps:
-        raise QuestError(f"{pkg} isn't installed on the Quest")
-    return apps[0]
-
-
-def status(serial=None, with_apps=True):
-    """For the page: the Quest (or why there isn't one), what else adb sees, and its apps."""
-    try:
-        found = devices()
-    except QuestError as e:
-        return {'quest': None, 'devices': [], 'guidance': str(e), 'apps': []}
-    try:
-        quest = find_quest(serial)
-    except QuestError as e:
-        return {'quest': None, 'devices': found, 'guidance': str(e), 'apps': []}
-    return {'quest': quest, 'devices': found, 'guidance': None,
-            'apps': packages(quest['serial']) if with_apps else []}
-
-
-# ---- Checking what was copied ---------------------------------------------------
-
-def inspect(package_name, base, splits=()):
-    """Refuse what can't or mustn't run on the Frame, before anything is copied to it."""
-    names = set()
-    for path in [base, *splits]:
-        try:
-            with zipfile.ZipFile(path) as z:
-                names |= set(z.namelist())
-        except (zipfile.BadZipFile, OSError) as e:
-            raise QuestError(f'{os.path.basename(path)} copied from the Quest is not a readable APK: {e}')
-    info = android.apk_info(base)
-    label = info['label']
-    if info['package'] != package_name:
-        raise QuestError(f"The APK copied for {package_name} says it is {info['package']}; not importing it")
-    if any(n.rsplit('/', 1)[-1] == 'libovrplatformloader.so' for n in names):
-        raise QuestError(ENTITLEMENT.format(label=label))
-    abis = sorted({n.split('/')[1] for n in names if n.startswith('lib/') and n.count('/') >= 2})
-    if abis and 'arm64-v8a' not in abis:
-        raise QuestError(f"{label} is a 32-bit-only build ({', '.join(abis)}). The Frame runs 64-bit ARM "
-                         "(arm64-v8a) apps only; older Quest games are often 32-bit. Ask the developer "
-                         "whether a 64-bit build exists.")
-    android.check_installable(dict(info, abis=abis))  # Android version
-    if splits:
-        android.split_entries(base, splits)  # a split APK Frame Control can't merge fails now
-    warnings = []
-    openxr = 'lib/arm64-v8a/libopenxr_loader.so' in names
-    if 'lib/arm64-v8a/libvrapi.so' in names and not openxr:
-        warnings.append("Uses Meta's legacy VrApi and no OpenXR. The Frame doesn't have VrApi, so it "
-                        "will install but won't run.")
-    return {'package': info['package'], 'label': label, 'version': info['version'],
-            'version_code': info.get('version_code'), 'abis': abis,
-            'vr': bool(info.get('vr') or openxr), 'warnings': warnings}
-
-
-# ---- Importing ------------------------------------------------------------------
-
-_import_lock = threading.Lock()
-_sleep = time.sleep
-
-
-def cache_root():
-    return Path(frame_host.cache_dir('quest-import'))
-
-
-def sweep_cache(now=None):
-    """Remove imports a crash left behind (older than CACHE_TTL)."""
-    root, now = cache_root(), now or time.time()
-    if root.is_dir():
-        for d in root.iterdir():
-            try:
-                if now - d.stat().st_mtime > CACHE_TTL:
-                    shutil.rmtree(d, ignore_errors=True)
-            except OSError:
-                pass
-
-
-def _safe(name):
-    return re.sub(r'[^A-Za-z0-9._-]', '_', name)[:120] or 'part.apk'
-
-
-def _remote_sizes(serial, paths):
-    out = adb('shell', 'stat -c %s ' + ' '.join(shlex.quote(p) for p in paths), serial=serial)
-    return sum(int(x) for x in out.split() if x.isdigit())
-
-
-def _pull(serial, remote, local):
-    adb('pull', remote, str(local), serial=serial, timeout=PULL_TIMEOUT)
-
-
-def _wait_ready(meta):
-    """Until the app's Lepton instance has finished booting Android."""
-    container = f"lepton-steamlaunch-{int(meta['instance'])}"
-    deadline = time.monotonic() + READY_TIMEOUT
-    while time.monotonic() < deadline:
-        out = android.ssh(f"podman exec {container} /system/bin/sh -c 'getprop sys.boot_completed' "
-                          "2>/dev/null || true", timeout=30)
-        if out.strip() == '1':
-            return
-        _sleep(3)
-    raise QuestError(f"{meta['label']} didn't start on the Frame within {READY_TIMEOUT // 60} minutes, so its "
-                     "game data couldn't be added. Check it opens from the Steam library, then import again.")
-
-
-def push_data(meta, folder):
-    """Stream a pulled /sdcard/Android/data/<package> folder into the running instance as a
-    tar, written straight into the SSH transfer: nothing is staged on this computer."""
-    import frame_android_data
-    pkg = meta['package']
-    container = f"lepton-steamlaunch-{int(meta['instance'])}"
-    script = f'set -eu; mkdir -p {DATA}/{pkg}; cd {DATA}/{pkg}; tar -x -o -f -'
-    read_end, write_end = os.pipe()
-    state = {'count': 0, 'error': None}
-
-    def write():
-        try:
-            with os.fdopen(write_end, 'wb') as out, \
-                    tarfile.open(fileobj=out, mode='w|', format=tarfile.GNU_FORMAT) as tar:
-                for path in sorted(Path(folder).rglob('*')):
-                    if path.is_symlink() or not (path.is_file() or path.is_dir()):
-                        continue  # regular files and folders only
-                    info = tar.gettarinfo(str(path), arcname=path.relative_to(folder).as_posix())
-                    info.uid = info.gid = 0
-                    info.uname = info.gname = ''
-                    if path.is_file():
-                        with path.open('rb') as f:
-                            tar.addfile(info, f)
-                        state['count'] += 1
-                    else:
-                        tar.addfile(info)
-        except (OSError, tarfile.TarError, ValueError) as e:  # ValueError: the reader closed first
-            state['error'] = e
-
-    writer = threading.Thread(target=write, daemon=True)
-    writer.start()
-    try:
-        with os.fdopen(read_end, 'rb') as src:
-            frame_android_data._stream(shlex.join(['podman', 'exec', '-i', container, '/system/bin/sh', '-c', script]),
-                                       src=src)
-    finally:
-        writer.join(60)
-    if state['error']:
-        raise QuestError(f'could not read the app files copied from the Quest: {state["error"]}')
-    return state['count']
-
-
-def space_needed(apk_bytes, merged_bytes, obb_bytes, data_bytes):
-    """Bytes an import needs free on this computer: the APKs as copied, a merged APK and the
-    install's patched copy of it (each up to merged_bytes), the OBB folder and the
-    Android/data folder (streamed to the Frame, never staged twice)."""
-    return apk_bytes + 2 * merged_bytes + obb_bytes + data_bytes
-
-
-def _check_space(work, need, pkg):
-    free = shutil.disk_usage(str(work)).free
-    if need > free:
-        raise QuestError(f'Importing {pkg} needs about {need / 1024**3:.1f} GB free on this computer '
-                         f'({free / 1024**3:.1f} GB available in {work.parent})')
-
-
-def import_game(package_name, serial=None, data=False, progress=None, allow_downgrade=False):
-    """Copy package_name from the Quest to the Frame. progress(stage, percent).
-
-    Frame Control never starts the game itself. Its game data (OBB, and Android/data if
-    asked) can only be written into a running Lepton instance, and there is no way to
-    boot one without its game, so that's a second import: the person starts the game,
-    imports again, and Frame Control copies the data in and closes the game.
-
-    allow_downgrade: replace a newer version on the Frame with the Quest's older one."""
-    if not android.PKG_RE.match(package_name or ''):
-        raise QuestError('invalid package name')
-    report = progress or (lambda stage, percent=None: None)
-    if not _import_lock.acquire(blocking=False):
-        raise QuestError('Another Quest import is running; wait for it to finish')
-    try:
-        report('Looking for your Quest', 1)
-        quest = find_quest(serial)
-        app = package(quest['serial'], package_name)
-        sweep_cache()
-        work = cache_root() / f'{package_name}-{secrets.token_hex(4)}'
-        work.mkdir(parents=True)
-        try:
-            return _import(quest['serial'], app, data, work, report, allow_downgrade)
-        finally:
-            shutil.rmtree(work, ignore_errors=True)
-    finally:
-        _import_lock.release()
-
-
-def _running(pkg):
-    # strict: a failed `podman ps` raises, never reads as "not running".
-    return f'lepton-steamlaunch-{android.instance_id(pkg)}' in android.running_instances(strict=True)
-
-
-def _installed_code(pkg, meta):
-    """The installed version code: from the install record, else (installs made before
-    records kept it) read from the installed app.apk on the Frame; None if unknown."""
-    code = (meta or {}).get('version_code')
-    if isinstance(code, int) and not isinstance(code, bool):
-        return code
-    return android.installed_version_code(pkg) if meta else None
-
-
-def _version(name, code):
-    return ' '.join(x for x in (f'version {name}' if name else '', f'code {code}' if code is not None else 'unknown code') if x)
-
-
-def _check_downgrade(existing, checked, allow):
-    """Refuse to replace a newer install on the Frame with the Quest's older copy."""
-    if not existing or allow:
-        return
-    have, new = _installed_code(existing['package'], existing), checked['version_code']
-    if have is None or new is None:
-        raise QuestError(f"Frame Control can't tell which version of {checked['label']} is on the Frame, so it "
-                         "won't replace it in case that one is newer. Allow replacing a newer version (or "
-                         "--allow-downgrade) to install your Quest's copy anyway.")
-    if int(new) < int(have):
-        raise QuestError(f"The Frame has a newer {checked['label']} ({_version(existing.get('version'), have)}) than "
-                         f"your Quest ({_version(checked['version'], new)}). Frame Control won't replace it with "
-                         "an older version unless you allow it (Allow replacing a newer version, or "
-                         "--allow-downgrade).")
-
-
-def _check_target(pkg, meta, checked):
-    """The install record of the game open on the Frame, if it is this package, in its own
-    instance, at the Quest's exact version code; else a QuestError."""
-    label = checked['label']
-    if (not meta or meta.get('package') != pkg or android._int(meta.get('instance')) != android.instance_id(pkg)):
-        raise QuestError(f"{label} is open on the Frame, but Frame Control has no matching install record for "
-                         'it. Close it, then import again to install it.')
-    have, new = _installed_code(pkg, meta), checked['version_code']
-    if have is None or new is None or int(have) != int(new):
-        raise QuestError(f"{label} is open on the Frame with a different version (Frame: "
-                         f"{_version(meta.get('version'), have)}; Quest: {_version(checked['version'], new)}). "
-                         "Close it, then import again to install the Quest's version.")
-    return meta
-
-
-def _check_android_package(meta, code):
-    """The package Android itself has installed in the instance is this one, at this version."""
-    pkg = meta['package']
-    container = f"lepton-steamlaunch-{int(meta['instance'])}"
-    out = android.ssh(f"podman exec {container} /system/bin/sh -c 'dumpsys package {pkg}' 2>/dev/null || true",
-                      timeout=60)
-    # Only this package's own block (dumpsys can list other packages too).
-    block = re.search(r'^\s*Package \[' + re.escape(pkg) + r'\] \(.*?(?=^\s*Package \[|\Z)', out, re.M | re.S)
-    found = re.search(r'\bversionCode=(\d+)', block.group(0)) if block else None
-    if not found:
-        raise QuestError(f'Android in {container} does not have {pkg} installed')
-    if int(found.group(1)) != int(code):
-        raise QuestError(f'Android in {container} has {pkg} version code {found.group(1)}, not the '
-                         f"Quest's {code}")
-
-
-def _close(pkg):
-    """Stop the game and confirm its instance is gone: (closed, why not)."""
-    try:
-        android.stop(pkg)
-        for _ in range(6):
-            if not _running(pkg):
-                return True, None
-            _sleep(2)
-        return False, 'it is still running'
-    except Exception as e:  # never replaces the error that led here; unconfirmed counts as not closed
-        return False, str(e) or type(e).__name__
-
-
-def _import(serial, app, data, work, report, allow_downgrade=False):
-    pkg = app['package']
-    obb_bytes = app['obb'] or 0
-    data_bytes = (app['data'] or 0) if data else 0
-    apk_bytes = _remote_sizes(serial, app['apks'])
-    _check_space(work, space_needed(apk_bytes, apk_bytes, obb_bytes, data_bytes), pkg)
-    # 1. The APK (and its splits), checked before anything else is copied.
-    local = []
-    for i, remote in enumerate(app['apks']):
-        report(f'Copying the app from your Quest ({i + 1} of {len(app["apks"])})', 5 + 20 * i // len(app['apks']))
-        name = 'base.apk' if os.path.basename(remote) == 'base.apk' else f'{i:02d}-{_safe(os.path.basename(remote))}'
-        _pull(serial, remote, work / name)
-        if not (work / name).is_file():
-            raise QuestError(f'adb did not copy {remote} from the Quest')
-        local.append(work / name)
-    base = next((p for p in local if p.name == 'base.apk'), local[0])
-    splits = [p for p in local if p != base]
-    report('Checking it can run on the Frame', 27)
-    checked = inspect(pkg, str(base), [str(p) for p in splits])
-    warnings = list(checked['warnings'])
-    label = checked['label']
-    # A merge stores what splits compressed: count it uncompressed, now the files are here.
-    merged = base.stat().st_size + android.split_size([str(p) for p in splits])
-    _check_space(work, space_needed(0, merged, obb_bytes, data_bytes), pkg)
-    wants_data = app['obb'] is not None or bool(data and app['data'] is not None)
-    if not _running(pkg):
-        # 2a. Install. The game is never started here: see import_game. Checked and installed
-        # under the shared install lock, so nothing installs or removes it in between.
-        report('Installing on the Frame', 60)
-        with android._install_lock:
-            if _running(pkg):
-                raise QuestError(f'{label} was started on the Frame during the import. Close it, then import again.')
-            _check_downgrade(android.read_meta(pkg), checked, allow_downgrade)
-            meta = android.install(str(base), splits=[str(p) for p in splits], source='Meta Quest (USB)')
-        report('Done', 100)
-        message = f"Imported {meta['label']} from your Quest. It's in the Steam library."
-        if wants_data:
-            what = ' and '.join(x for x in ('its OBB game data' if app['obb'] is not None else '',
-                                            'its Android/data files' if data and app['data'] is not None else '') if x)
-            message += (f" To add {what}, start {meta['label']} from the Steam library, leave it on its first "
-                        "screen, then import it again here: Frame Control copies the data in and closes the game.")
-        note = android.layer_note(meta)
-        return {'message': ' '.join([message, *warnings, *([note] if note else [])]), 'app': meta, 'obb': [],
-                'files': 0, 'warnings': warnings, 'split': bool(splits), 'needs_game_data': wants_data,
-                'closed': None}
-    # 2b. The game is open on the Frame: add its data to exactly this install, never replacing it.
-    _check_target(pkg, android.read_meta(pkg), checked)  # early, before copying the data off the Quest
-    if not wants_data:
-        raise QuestError(f"{label} is already installed and open on the Frame, and has no game data to add. "
-                         'Close it first to install it again.')
-    obb_files = []
-    if app['obb'] is not None:
-        report('Copying game data (OBB) from your Quest', 30)
-        (work / 'obb').mkdir()
-        _pull(serial, f'{OBB}/{pkg}', work / 'obb')
-        folder = work / 'obb' / pkg
-        files = sorted(p for p in folder.iterdir() if p.is_file()) if folder.is_dir() else []
-        obb_files = [p for p in files if re.fullmatch(OBB_NAME.format(re.escape(pkg)), p.name)]
-        skipped = [p.name for p in files if p not in obb_files]
-        if skipped:
-            warnings.append('Not copied, not standard OBB file names: ' + ', '.join(skipped))
-    data_folder = None
-    if data and app['data'] is not None:
-        report('Copying app files from your Quest', 50)
-        (work / 'data').mkdir()
-        _pull(serial, f'{DATA}/{pkg}', work / 'data')
-        data_folder = work / 'data' / pkg if (work / 'data' / pkg).is_dir() else None
-    copied_obb, copied_files, failure = [], 0, None
-    # The pulls can take a long time: check the target again, and hold the shared install
-    # lock (install, remove and install_obb take it too) until the game is closed.
-    with android._install_lock:
-        if not _running(pkg):
-            raise QuestError(f'{label} was closed on the Frame during the import. Start it, then import again.')
-        meta = _check_target(pkg, android.read_meta(pkg), checked)
-        closed, why = None, None
-        try:
-            report('Waiting for Android on the Frame', 70)
-            _wait_ready(meta)
-            _check_android_package(meta, checked['version_code'])
-            if obb_files:
-                report('Copying game data (OBB) to the Frame', 80)
-                copied_obb = android.install_obb(pkg, [str(p) for p in obb_files])['obb']
-            if data_folder:
-                report('Copying app files to the Frame', 92)
-                copied_files = push_data(meta, data_folder)
-        except Exception as e:  # any failure, local (OSError) or on the Frame, is reported below
-            failure = e
-        finally:
-            # Whatever happened, even Ctrl-C part-way through a write, the game is closed so it
-            # never runs on half-copied data. A failed close never hides the original error.
-            interrupted = sys.exc_info()[0] is not None
-            closed, why = _close(pkg)
-            if interrupted and not closed:
-                print(f"Frame Control couldn't close {meta['label']} on the Frame ({why}); close it from Steam "
-                      'before playing.', file=sys.stderr)
-    if failure:
-        message = f"Copying {meta['label']}'s game data failed: {failure}."
-        message += (' The game was closed; start it and import again to retry.' if closed else
-                    f" Frame Control also couldn't close the game ({why}): close it from Steam, then start it "
-                    'and import again.')
-        raise QuestError(message) from failure
-    report('Done', 100)
-    message = f"Added {meta['label']}'s game data from your Quest"
-    message += ('; the game is closed, start it again from the Steam library.' if closed else
-                f", but Frame Control couldn't close the game ({why}). Close it from Steam before playing, so it "
-                'starts with the new data.')
-    if copied_obb:
-        message += f' Added {len(copied_obb)} OBB file{"s" if len(copied_obb) != 1 else ""}.'
-    if copied_files:
-        message += f' Copied {copied_files} app file{"s" if copied_files != 1 else ""}.'
-    return {'message': ' '.join([message, *warnings]), 'app': meta, 'obb': copied_obb, 'files': copied_files,
-            'warnings': warnings, 'split': bool(splits), 'needs_game_data': False, 'closed': closed}
-
-
-def main():
-    cmd, *args = sys.argv[1:] or ['help']
-    serial = args[args.index('--serial') + 1] if '--serial' in args and args.index('--serial') + 1 < len(args) else None
-    try:
-        if cmd == 'devices':
-            r = devices()
-        elif cmd == 'list':
-            r = packages(find_quest(serial)['serial'])
-        elif cmd == 'import':
-            names = [a for a in args if not a.startswith('--') and a != serial]
-            if len(names) != 1:
-                raise QuestError('usage: import PACKAGE [--serial SERIAL] [--data] [--allow-downgrade]')
-            r = import_game(names[0], serial, data='--data' in args, allow_downgrade='--allow-downgrade' in args,
-                            progress=lambda stage, percent=None: print(f'{percent:>3}% {stage}' if percent is not None
-                                                                       else stage, file=sys.stderr))
-        else:
-            sys.exit(__doc__)
-    except (android.FrameError, OSError) as e:
-        sys.exit(f'error: {e}')
-    print(json.dumps(r, indent=1))
-
-
-if __name__ == '__main__':
-    main()

ui/frame_status.py

@@ -6,7 +6,6 @@ error. Sensor paths verified on SteamOS 0.3.0; OpenVR timing on 0.4.1
 """
 import ctypes as C
 import glob
-import hashlib
 import math
 import json
 import os
@@ -122,16 +121,7 @@ def os_release():
         k, _, v = line.partition("=")
         out[k] = v.strip('"')
     return {"version": out.get("VERSION_ID"), "build": out.get("BUILD_ID"),
-            "variant": out.get("VARIANT_ID"), "headset": headset_id()}
-
-
-def headset_id():
-    """A one-way hash of the headset's /etc/machine-id: the same for this headset on every
-    computer, and the machine-id itself never leaves the headset. None without one."""
-    mid = read("/etc/machine-id")
-    if not mid:
-        return None
-    return hashlib.sha256(("frame-control-headset:" + mid).encode()).hexdigest()[:16]
+            "variant": out.get("VARIANT_ID")}
 
 
 def volume():

ui/frame_telemetry.py

@@ -50,7 +50,6 @@ SETTINGS = STATE / 'settings.json'
 OUTBOX = STATE / 'outbox.jsonl'
 SENT = STATE / 'sent.jsonl'
 SENT_KEEP = 200
-FRAMES_SEEN_KEEP = 100  # build/headset pairs already reported, enough for several headsets' updates
 OUTBOX_MAX = 2000  # events kept while offline; the oldest go first
 FLUSH_EVERY = 60
 REPEAT_WINDOW = 600  # the same diagnostic error is sent at most once in this many seconds
@@ -267,23 +266,9 @@ CATEGORIES = [
     ('layer_missing', re.compile(r'OpenXR compatibility layer')),
     ('apk_repack_failed', re.compile(r'could not prepare the APK for the Frame')),
     ('tool_missing', re.compile(r"\[WinError 2\]|No such file or directory: '(?:ssh|scp|rsync|adb)")),
-    ('apk_unreadable', re.compile(r'(?i)not a zip|bad apk|AndroidManifest|ApkError|unexpected package name|'
-                                  r'not a readable APK|not an APK|APK manifest|in the APK (?:uses a compression|is )')),
+    ('apk_unreadable', re.compile(r'(?i)not a zip|bad apk|AndroidManifest|ApkError|unexpected package name')),
     ('cant_run_on_frame', re.compile(r"can't run on the Frame")),
     ('steam_shortcut', re.compile(r'(?i)steam did not return a shortcut|shortcut list|no Steam shortcut')),
-    # The Frame-side scripts that talk to the Steam client's UI (frame/android/steam_shortcuts.py), which
-    # fail with their own words or a Python traceback, its last lines kept. Before frame_unreachable:
-    # "Connection refused" and "timed out" here are Steam's debug port on the Frame, not ssh.
-    ('steam_not_running', re.compile(r'SharedJSContext not found|'
-                                     r'URLError: <urlopen error \[Errno 111\] Connection refused>|'
-                                     r'ConnectionRefusedError: \[Errno 111\]')),
-    ('steam_not_responding', re.compile(r'websocket handshake failed|(?:socket\.timeout|TimeoutError): timed out|'
-                                        r'(?:\A|\n)EOFError\Z')),
-    ('steam_js_error', re.compile(r'JS error: ')),
-    ('artwork_failed', re.compile(r'incomplete artwork render|artwork renderer did not produce|'
-                                  r'render is \d+ bytes of|could not prepare artwork|all five Steam artwork slots')),
-    # Copying the APK (or its launcher) to the Frame: rsync or scp ran out of time.
-    ('copy_timeout', re.compile(r'to the Frame timed out\Z')),
     ('frame_not_set_up', re.compile(r'(?i)Could not resolve hostname|no "?frame"? (?:SSH )?alias')),
     ('frame_auth', re.compile(r'(?i)Permission denied|Host key verification failed|'
                               r'REMOTE HOST IDENTIFICATION HAS CHANGED|host key is known for')),
@@ -295,8 +280,6 @@ CATEGORIES = [
                                      r'Network is unreachable|Host is down|asleep|kex_exchange|banner exchange|'
                                      r'ssh: connect to host |mux_client_|client_loop: ')),
     ('frame_disk_full', re.compile(r'(?i)No space left|disk full|ENOSPC')),
-    # rsync or scp failed for another reason ("connection unexpectedly closed", or no words at all).
-    ('copy_failed', re.compile(r'\Acopying .+ to the Frame failed: ')),
     ('download_failed', re.compile(r'(?i)HTTP (?:Error )?\d{3}|URLError|download|certificate verify failed')),
     ('flatpak', re.compile(r'(?i)flatpak|flathub')),
     ('cancelled', re.compile(r'(?i)cancel')),
@@ -304,16 +287,8 @@ CATEGORIES = [
 ]
 
 
-# Python's own errors for a mistake in Frame Control's code, whatever their message says.
-BUG_ERRORS = (KeyError, IndexError, TypeError, AttributeError, NameError, AssertionError,
-              ZeroDivisionError, RecursionError, UnicodeError, json.JSONDecodeError)
-
-
 def categorize(message):
-    """(category, detail): a fixed category name, plus an Android installer code when there is one.
-    message may be the exception itself: one of BUG_ERRORS is internal_error."""
-    if isinstance(message, BUG_ERRORS):
-        return 'internal_error', None
+    """(category, detail): a fixed category name, plus an Android installer code when there is one."""
     text = str(message or '')
     for name, pattern in CATEGORIES:
         m = pattern.search(text)
@@ -391,24 +366,16 @@ def app_started():
         _save(s)
 
 
-def frame_seen(build, version, headset=None, saved=None):
-    """The Frame's SteamOS build (public build numbers), once per build for each headset.
-    headset: the anonymous hash from frame_status.headset_id; saved: how many headsets
-    this copy of the app has set up."""
-    headset = headset if isinstance(headset, str) and re.fullmatch(r'[0-9a-f]{16}', headset) else None
-    key = f'{build}/{version}' + (f'/{headset}' if headset else '')
+def frame_seen(build, version):
+    """The Frame's SteamOS build, once per build (public build numbers)."""
+    key = f'{build}/{version}'
     with _lock:
         s = settings()
         if not build or key in s['frames_seen']:
             return
-        s['frames_seen'] = (s['frames_seen'] + [key])[-FRAMES_SEEN_KEEP:]
+        s['frames_seen'] = (s['frames_seen'] + [key])[-20:]
         _save(s)
-    p = {'steamos_build': str(build)[:40], 'steamos_version': str(version or '')[:40]}
-    if headset:
-        p['headset_id'] = headset
-    if isinstance(saved, int) and saved >= 0:
-        p['headsets_saved'] = saved
-    capture('frame_connected', p)
+    capture('frame_connected', {'steamos_build': str(build)[:40], 'steamos_version': str(version or '')[:40]})
 
 
 def install_finished(kind, ok, seconds=None, error=None, diagnose=True, **props):

ui/index.html

@@ -51,10 +51,6 @@
   nav a:hover { color: var(--bright); }
   nav a.on { color: var(--blue); border-bottom-color: var(--blue); }
   .spacer { flex: 1; }
-  /* The connection pill and the buttons after it, pushed to the right: closer together than the tabs, and the
-     pill (not the buttons) gives way when the window is narrow, so nothing is pushed past the right edge. */
-  .hdr-end { display: flex; align-items: center; gap: 10px; min-width: 0; flex: 0 1 auto; margin-left: auto; }
-  .hdr-end > :not(.pill, .devsel) { flex: none; }
   .chip { display: inline-flex; align-items: center; gap: 8px; height: 32px; padding: 0 12px; border-radius: 3px;
           background: var(--btn); color: var(--text); font-size: 13px; white-space: nowrap; }
   .dot { width: 8px; height: 8px; border-radius: 50%; background: var(--dim); flex: none; }
@@ -64,7 +60,6 @@
   main { max-width: 1480px; margin: 0 auto; padding: 22px calc(28px + env(safe-area-inset-right)) 30px calc(28px + env(safe-area-inset-left)); display: flex; flex-direction: column; gap: 26px; }
   .grid-top { display: grid; grid-template-columns: minmax(0, 1.7fr) minmax(330px, 1fr); gap: 22px; }
   .grid-3 { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 22px; }
-  @media (max-width: 1366px) { header { gap: 20px; } nav a { padding: 0 11px; } }
   @media (max-width: 1150px) { .grid-top, .grid-3 { grid-template-columns: 1fr; } header { gap: 16px; padding: env(safe-area-inset-top) calc(18px + env(safe-area-inset-right)) 0 calc(18px + env(safe-area-inset-left)); } nav a { padding: 0 10px; } }
   @media (max-width: 820px) { .brand b { display: none; } }
   nav svg { display: none; }
@@ -79,7 +74,7 @@
   .page { display: flex; flex-direction: column; gap: 26px; }
   .page:not(.on) { display: none; }
   nav kbd { font: inherit; font-size: 10.5px; color: var(--dim); margin-left: 7px; letter-spacing: 0; }
-  @media (max-width: 1366px) { nav kbd { display: none; } }  /* the tabs' titles still name the key */
+  @media (max-width: 1150px) { nav kbd { display: none; } }
 
   /* ---- offline banner ---- */
   .banner { display: flex; align-items: center; gap: 14px; padding: 12px 16px; border-radius: 4px;
@@ -92,14 +87,14 @@
   .wait::before { content: ""; width: 7px; height: 7px; border-radius: 50%; background: var(--dim); flex: none; }
 
   /* ---- connection pill, its details dialog, and the Devices tab (frame_link.py) ---- */
-  .pill { height: 40px; padding: 0 12px; gap: 9px; max-width: 420px; min-width: 150px; flex: 0 1 auto; background: var(--btn); }
+  .pill { height: 40px; padding: 0 12px; gap: 9px; max-width: 420px; min-width: 190px; flex: 0 1 auto; background: var(--btn); }
   .pill .dot { flex: none; }
   .pill .dot.wait { background: var(--warn); box-shadow: 0 0 6px rgba(217,162,58,.7); animation: pulse 1s ease-in-out infinite; }
   @keyframes pulse { 50% { opacity: .35; } }
   .pill .pt { display: flex; flex-direction: column; align-items: flex-start; min-width: 0; line-height: 1.2; text-align: left; }
   .pill .pt b { font-weight: 500; font-size: 13px; color: var(--bright); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
   .pill .pt span { font-size: 11.5px; color: var(--muted); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
-  .devsel { background: var(--btn); color: var(--text); border: 0; border-radius: 3px; height: 40px; padding: 0 8px; font: inherit; font-size: 13px; max-width: 160px; min-width: 90px; }
+  .devsel { background: var(--btn); color: var(--text); border: 0; border-radius: 3px; height: 40px; padding: 0 8px; font: inherit; font-size: 13px; max-width: 160px; }
   .dlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px;
          padding: 22px; width: min(640px, 94vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); }
   .dlg::backdrop { background: rgba(0,0,0,.55); }
@@ -137,7 +132,7 @@
   .dev-panel select, .dlg select { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px;
                                    padding: 8px 8px; font: inherit; font-size: 13px; }
   .addr .t { overflow-wrap: anywhere; }
-  .addr .s, .dev-problem .s { white-space: normal; }
+  .addr .s { white-space: normal; }
   .addr-edit { display: grid; grid-template-columns: 2fr 1fr 1.4fr auto auto; gap: 8px; align-items: center; width: 100%; }
   .addr-add { display: grid; grid-template-columns: 2fr 1fr 1.4fr auto; gap: 8px; margin-top: 12px; }
   @media (max-width: 700px) { .dev-form, .addr-edit, .addr-add { grid-template-columns: 1fr; } }
@@ -616,6 +611,8 @@
   .source-repo-form input { margin-top:7px; display:block; border-radius:6px; }
   .source-repo-form details { font-size:11px; color:var(--muted); }
   .source-repo-form button { justify-self:start; border-radius:6px; }
+  .source-token { display:flex; flex-wrap:wrap; align-items:center; gap:7px; margin:8px 0 4px; }
+  .source-token input { flex:1; min-width:150px; border-radius:6px; font-size:12px; }
   #appStore :focus-visible, .store-dialog :focus-visible, .store-tabs :focus-visible { outline:2px solid var(--link); outline-offset:4px; }
   .store-search input:focus-visible { outline:0!important; }
   @media(min-width:1400px) { .store-grid { grid-template-columns:repeat(5,minmax(0,1fr)); } }
@@ -660,7 +657,7 @@
     <a href="#tools" title="Files, clipboard, Linux apps, remote and power (4)"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M14.7 6.3a4 4 0 0 0-5.2 5.2L3.5 17.5a2.1 2.1 0 0 0 3 3l6-6a4 4 0 0 0 5.2-5.2l-2.5 2.5-2.5-2.5z"/></svg>Tools<kbd>4</kbd></a>
     <a href="#devices" class="desk-only" title="Headsets, their addresses and the connection (5)"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M5 12.5a10 10 0 0 1 14 0M8.5 16a5 5 0 0 1 7 0"/><circle cx="12" cy="19.5" r="1"/><path d="M2 9a15 15 0 0 1 20 0"/></svg>Devices<kbd>5</kbd></a>
   </nav>
-  <div class="hdr-end">
+  <div class="spacer"></div>
   <button class="pill" id="conn" role="status" aria-live="polite" title="Connection details"><span class="dot wait"></span><span class="pt"><b>Connecting…</b><span></span></span></button>
   <select class="devsel desk-only" id="devSel" aria-label="Headset" title="Switch headset" hidden></select>
   <span class="chip" id="battChip" title="Battery">—</span>
@@ -670,7 +667,6 @@
   <button id="refreshAll" title="Refresh (R)" aria-label="Refresh">
     <svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" aria-hidden="true"><path d="M21 12a9 9 0 1 1-3-6.7"/><path d="M21 3v6h-6"/></svg>
   </button>
-  </div>
 </header>
 
 <main>
@@ -683,8 +679,7 @@
   </div>
   <div class="notice" id="privacyNotice" hidden>
     <div class="grow">Frame Control sends anonymous usage statistics: that it was installed and opened, its version,
-      your operating system, which tabs you use, whether installs on the Frame worked, and how many headsets you use
-      (a one-way hash of each headset's machine ID, the same on every computer, so they're counted once). Never file names, paths,
+      your operating system, which tabs you use, and whether installs on the Frame worked. Never file names, paths,
       addresses or anything you've typed, and it isn't linked to you. You can also share whether Android apps
       worked and the details of errors, which helps fix problems faster.</div>
     <button class="small" id="noticeSettings">Privacy settings</button>
@@ -958,20 +953,6 @@
           library under its own name and opens in its own panel. Test launches it and records
           whether it stays up; Report records how it worked for you.</div>
       </div>
-      <section class="panel" id="quest">
-        <div class="shelf-head"><h2>Import from my Quest</h2><span class="count" id="questCount"></span><span class="spacer"></span>
-          <button class="small" id="questScan" title="Look for a Meta Quest plugged into this computer by USB">Look for my Quest</button></div>
-        <div class="sub" id="questState" role="status" aria-live="polite">Plug your Quest into this computer by USB, put it on, then Look for my Quest.</div>
-        <label class="sub" style="display:block;margin-top:8px"><input type="checkbox" id="questData"> Also copy each game's files in Android/data (some games keep saves there)</label>
-        <label class="sub" style="display:block;margin-top:4px"><input type="checkbox" id="questOlder"> Allow replacing a newer version on the Frame with the Quest's older one</label>
-        <progress id="questProgress" max="100" hidden aria-label="Import progress" style="width:100%;margin-top:8px"></progress>
-        <div class="list" id="questApps" style="margin-top:8px"></div>
-        <div class="hint">Copies games you own from your own Quest (developer mode on) to the Frame. A game with
-          OBB data takes two imports: the first installs it; start it from the Steam library, then import again to
-          copy its data in. Frame Control never starts the game itself. Games that
-          check their Meta store licence need Meta's services, which only Meta headsets have, so they are refused:
-          Frame Control won't remove that check. <a href="https://github.com/saphid/frame-control/blob/main/docs/quest-import.md" target="_blank" rel="noopener noreferrer">How it works</a></div>
-      </section>
       <section class="panel disp" id="display">
       <div class="shelf-head"><h2>Android display</h2><span class="count">Lepton</span><span class="spacer"></span>
         <button class="small" id="dispReload" title="Read the current values again">Refresh</button></div>
@@ -1163,8 +1144,7 @@
       <div class="shelf-head"><h2>Privacy &amp; updates</h2><span class="spacer"></span><span class="sub" id="appVersion"></span></div>
       <label class="popt"><input type="checkbox" id="tUsage"><b>Anonymous usage statistics</b>
         <span class="sub">Installs, opens, version, operating system, tabs used, and whether installs on the Frame
-          worked (with an error category, never the message). F-Droid package names only. Each headset's SteamOS
-          build, a one-way hash of its machine ID (to count headsets, not just installs), and how many you've set up.</span></label>
+          worked (with an error category, never the message). F-Droid package names only.</span></label>
       <label class="popt"><input type="checkbox" id="tCompat"><b>Share compatibility results</b>
         <span class="sub">Your APK reports and tests (package, version, result, your notes) go to the shared
           compatibility database, so the verdicts get better for everyone.</span></label>
@@ -1229,7 +1209,7 @@
   <button class="store-close" data-close="sourcesDialog" aria-label="Close source settings">×</button>
   <div class="sources-intro"><div class="store-eyebrow">Your app shelves</div><h1 id="sourcesTitle">A world of apps.</h1><p>Choose where you discover games and apps. We bring them together for you.</p></div>
   <div id="sourcesList"></div>
-  <form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><button class="action">Add source</button></form>
+  <form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><details><summary>This repository needs a login (optional)</summary><label>Username<input name="username" type="text" autocomplete="off" autocapitalize="none" spellcheck="false"></label><label>Password<input name="password" type="password" autocomplete="new-password"></label><p>Kept on this computer, apart from the address, and only sent to this repository over HTTPS.</p></details><button class="action">Add source</button></form>
   <p id="sourceError" class="store-notice" role="status" aria-live="polite"></p>
 </dialog>
 
@@ -1449,7 +1429,6 @@ function applyHostWording(h) {
   // The iPhone app: the server runs on the Frame, and saving goes to the phone's Photos.
   document.body.classList.toggle("mobile", !!h.mobile);
   $("shotsFolder").hidden = !!h.mobile;
-  $("quest").hidden = !!h.mobile;  // the Quest plugs into a computer; the phone app's server runs on the Frame
   if (!h.mobile) $("shotsFolder").textContent = h.fileManager === "your file manager" ? "Open folder" : `Show in ${h.fileManager}`;
   $("shotsSaveNew").textContent = h.mobile ? `Save all to ${h.computer}` : `Save new to ${h.computer}`;
   $("clipMac").textContent = `Send ${h.computer} clipboard`;
@@ -1610,7 +1589,7 @@ $("vrUtilities").onclick = async e => {
   await loadUtilities();
 };
 
-// The Frame owns the clock. Poll independently of the selected tab, from the first status check on.
+// The Frame owns the clock. Poll independently of status and the selected tab.
 let comfortBusy = false, comfortSeen = new Set(), comfortSession = null, comfortHydrated = false;
 async function localNotification(message, request = false) {
   try {
@@ -1669,12 +1648,7 @@ function renderComfort(s) {
   if ($("comfortEvents").textContent !== history) $("comfortEvents").textContent = history;
 }
 async function pollComfort() {
-  // Not while the Frame can't be reached: each poll would be another ssh attempt waiting to
-  // time out, and the banner already says why. Status checks bring it back (setOnline).
-  if (online === false) {
-    const message = "Session status appears when the Frame is connected.";
-    if ($("comfortStatus").textContent !== message) $("comfortStatus").textContent = message;
-  } else if (!comfortBusy) {
+  if (!comfortBusy) {
     comfortBusy = true;
     try { renderComfort(await api("/api/comfort", {action: "status"})); }
     catch (e) {
@@ -1708,6 +1682,7 @@ $("castBtn").onclick = () => {
   // Fullscreen is a direct user gesture; iPhone falls back to its inline viewer.
   $("viewer").requestFullscreen?.().catch(() => {});
 };
+pollComfort();
 
 // ---- battery ----
 function battery(b, power) {
@@ -3166,12 +3141,18 @@ function storeIcon(o) { return `<span class="store-icon" style="--hue:${storeHue
 document.addEventListener("error", e => { if (e.target.matches?.(".store-art img, .store-icon img")) e.target.remove(); }, true);
 function sourceMark(s) { const name=storeName(s); return `<span class="offer-mark" style="--hue:${storeHue(name)}" aria-hidden="true">${esc(name === "SideQuest" ? "SQ" : name === "F-Droid" ? "Fd" : name.slice(0,2))}</span>`; }
 function sourceDescription(s) { return s.description || ({fdroid:"Independent, open-source apps",sidequest:"Discover something new in VR",itch:"Games from independent creators",github:"Direct from the people who make it"})[s.kind] || (s.name.includes("GitHub") ? "Direct from the people who make it" : s.name.includes("Droid") ? "Independent, open-source apps" : s.name.includes("SideQuest") ? "Discover something new in VR" : s.name.includes("itch") ? "Games from independent creators" : "Apps shared by this community"); }
+// GitHub's optional access token: only whether one is set is ever shown, never its value.
+function sourceToken(s) {
+  const t=s.token;if(!t)return "";
+  const note=t.rejected ? "GitHub rejected this token, so searches continue without it. Replace or clear it." : t.from==="env" ? "Using the token from FRAME_GITHUB_TOKEN." : t.set ? "A token is saved." : "Optional: a token raises GitHub's limit of 60 requests an hour.";
+  return `<form class="source-token" data-token="${esc(s.id)}"><input name="token" type="password" autocomplete="off" spellcheck="false" aria-label="${esc(storeName(s.name))} access token" placeholder="${t.set ? "Replace access token" : "Access token"}"><button class="small">Save</button>${t.from==="saved" ? `<button type="button" class="small" data-clear-token="${esc(s.id)}">Clear</button>` : ""}</form><small>${esc(note)}</small>`;
+}
 async function loadSources() {
   try {
     const {sources} = await api("/api/sources"); sourceState.sources = sources;
     $("sourceFilter").innerHTML = `<option value="">All sources</option>` + sources.filter(s=>s.enabled).map(s=>`<option value="${esc(s.id)}">${esc(storeName(s.name))}</option>`).join("");
     $("sourceFilter").value = sourceState.source;
-    $("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}</small>${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
+    $("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}${s.login ? " · Uses a login" : ""}</small>${sourceToken(s)}${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
   } catch (e) { $("sourceError").textContent = e.message; }
 }
 function storeCard(app) {
@@ -3298,7 +3279,8 @@ $("detailBody").onclick=e=>{if(e.target.closest('[data-close]'))return $("appDet
 $("detailBody").onchange=e=>{if(e.target.name==='storeOffer'){sourceState.detail.choice=+e.target.value;renderAppDetail();loadOfferDetails();}};
 $("sourcesOpen").onclick=()=>{loadSources();$('sourcesDialog').showModal();};
 $("sourcesList").onchange=e=>{if(e.target.matches('[data-enable]'))changeSource({action:'enable',source:e.target.dataset.enable,enabled:e.target.checked});};
-$("sourcesList").onclick=e=>{const b=e.target.closest('[data-remove]');if(b)changeSource({action:'remove',source:b.dataset.remove});};
+$("sourcesList").onclick=e=>{const b=e.target.closest('[data-remove]');if(b)changeSource({action:'remove',source:b.dataset.remove});const c=e.target.closest('[data-clear-token]');if(c)changeSource({action:'clear-token',source:c.dataset.clearToken});};
+$("sourcesList").onsubmit=e=>{const f=e.target.closest('[data-token]');if(!f)return;e.preventDefault();const token=f.elements.token.value.trim();f.reset();if(token)changeSource({action:'set-token',source:f.dataset.token,token});};
 async function addSource(body) {
   $("sourceError").textContent="Checking the repository. Large ones can take a minute…";
   let error="";
@@ -3306,7 +3288,49 @@ async function addSource(body) {
   $("sourceError").textContent=result ? result.message : error || "We couldn't add that repository.";
   if(result){$("addSource").reset();sourceState.source='';await loadSources();await searchSources();}
 }
-$("addSource").onsubmit=e=>{e.preventDefault();const f=new FormData(e.target);addSource({action:'add',url:f.get('url'),fingerprint:f.get('fingerprint')});};
+// A login typed into the address moves into the login fields, on the page too, before anything
+// is sent: the address is shown and logged, the login never. It stays there if adding fails.
+// Plain text, not the URL parser: an address it rejects (or a login it can't decode) must
+// still lose its login first. Decoding falls back to the text as typed.
+// Percent-decode what decodes; keep any byte sequence that isn't UTF-8 as typed.
+// One forward pass: each lead byte gives its sequence length (at most 4 bytes).
+function decodeLenient(s) {
+  return s.replace(/(?:%[0-9a-f]{2})+/gi, run => {
+    const b=run.match(/%[0-9a-f]{2}/gi); let out='', i=0;
+    while(i<b.length){
+      const v=parseInt(b[i].slice(1),16);
+      const n=v<0x80 ? 1 : v>=0xC2&&v<0xE0 ? 2 : v>=0xE0&&v<0xF0 ? 3 : v>=0xF0&&v<0xF5 ? 4 : 0;
+      if(n){ try { out+=decodeURIComponent(b.slice(i,i+n).join('')); i+=n; continue; } catch {} }
+      out+=b[i]; i++;
+    }
+    return out;
+  });
+}
+// Moves a login typed into the address into the login fields. Returns false for an
+// address the URL parser rejects that may hold a login: it is refused, not sent.
+function moveLogin(form) {
+  // Normalise the way URL parsers do: trim C0 controls and spaces, drop tabs and newlines.
+  const el=form.elements;
+  el.url.value=el.url.value.replace(/^[\u0000- ]+|[\u0000- ]+$/g,'').replace(/[\t\n\r]/g,'');
+  let u;
+  // A backslash and an '@' together are read differently by browsers and by the
+  // server, so the address can't be split safely: refuse it rather than guess.
+  if(el.url.value.includes('@') && el.url.value.includes('\\'))return false;
+  try { u=new URL(el.url.value); } catch { return !el.url.value.includes('@'); }
+  if(!u.username && !u.password)return true;
+  const user=decodeLenient(u.username), pass=decodeLenient(u.password);
+  u.username=''; u.password=''; el.url.value=u.href;
+  if(!el.username.value)el.username.value=user;
+  if(!el.password.value)el.password.value=pass;
+  el.username.closest('details').open=true;
+  return true;
+}
+$("addSource").onsubmit=e=>{
+  e.preventDefault();
+  if(!moveLogin(e.target)){ $("sourceError").textContent="That address isn't valid. Put the username and password in the login fields below, not in the address."; return; }
+  const f=new FormData(e.target), url=f.get('url'), username=f.get('username'), password=f.get('password');
+  addSource({action:'add',url,fingerprint:f.get('fingerprint'),...(username ? {username,password} : {})});
+};
 document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>$(b.dataset.close).close());
 $('appDetail').addEventListener('close',()=>{sourceState.detailRequest++;});
 document.querySelectorAll('[data-store-tab]').forEach(b=>b.onclick=()=>{const browse=b.dataset.storeTab==='browse';$('appStore').hidden=!browse;$('androidLibrary').hidden=browse;$('repNew').hidden=browse;$('sourcesOpen').hidden=!browse;document.querySelectorAll('[data-store-tab]').forEach(t=>{t.classList.toggle('on',t===b);t.setAttribute('aria-pressed',t===b);});if(!browse)loadAndroid();});
@@ -3344,51 +3368,6 @@ async function loadAndroid() {
   });
 }
 
-// ---- Import from my Quest (frame_quest.py): adb on this computer, never on the Frame ----
-const quest = { serial: null };
-function questSize(bytes) {
-  if (bytes == null) return "";
-  if (bytes >= 1073741824) return `${(bytes / 1073741824).toFixed(1)} GB`;
-  if (bytes >= 1048576) return `${Math.round(bytes / 1048576)} MB`;
-  return `${Math.max(1, Math.round(bytes / 1024))} KB`;
-}
-async function loadQuest() {
-  $("questScan").disabled = true; $("questState").textContent = "Looking for your Quest…"; $("questApps").innerHTML = "";
-  let d;
-  try { d = await api("/api/quest"); }
-  catch (e) { $("questState").textContent = e.message; return; }
-  finally { $("questScan").disabled = false; }
-  quest.serial = d.quest ? d.quest.serial : null;
-  $("questCount").textContent = d.quest ? `${d.apps.length} app${d.apps.length === 1 ? "" : "s"}` : "";
-  if (!d.quest) { $("questState").textContent = d.guidance || "No Quest found."; return; }
-  $("questState").textContent = `${d.quest.model || "Quest"} connected (${d.quest.serial}). Games that check their Meta licence are refused when you import them.`;
-  const installed = new Set(androidApps.map(a => a.package));
-  $("questApps").innerHTML = d.apps.length ? d.apps.map(a => {
-    const bits = [a.split ? `split APK (${a.apks.length} files)` : "single APK"];
-    if (a.obb != null) bits.push(`OBB ${questSize(a.obb)}`);
-    if (a.data != null) bits.push(`Android/data ${questSize(a.data)}`);
-    if (a.meta) bits.push("Meta app: needs Meta's services");
-    return `<div class="item"><div class="grow"><div class="t">${esc(a.package)}</div><div class="s">${esc(bits.join(" · "))}</div></div>
-      ${installed.has(a.package) ? `<span class="tag">On the Frame</span>` : ""}
-      <button class="small" data-quest-import="${esc(a.package)}"${installing.has("quest:" + a.package) ? " disabled" : ""}>${installed.has(a.package) ? "Import again" : "Import"}</button></div>`;
-  }).join("") : `<div class="sub">No games or apps you installed are on this Quest.</div>`;
-}
-$("questScan").onclick = loadQuest;
-$("questApps").addEventListener("click", e => {
-  const b = e.target.closest("[data-quest-import]");
-  if (!b) return;
-  const pkg = b.dataset.questImport, key = "quest:" + pkg;
-  if (installing.has(key)) return;
-  b.disabled = true;
-  const bar = $("questProgress");
-  bar.hidden = false; bar.value = 0;
-  runJob(`Import ${pkg} from Quest`, key,
-         () => api("/api/quest", { action: "import", package: pkg, serial: quest.serial, data: $("questData").checked, allowDowngrade: $("questOlder").checked }),
-         j => { if (j.stage) $("questState").textContent = `${pkg}: ${j.stage}`; if (j.percent != null) bar.value = j.percent; })
-    .then(r => { bar.hidden = true; if (r && r.app) { $("questState").textContent = r.message || `Imported ${pkg}`; loadAndroid(); }
-                  else { b.disabled = false; $("questState").textContent = `${pkg} wasn't imported; the reason is in the message log below.`; } });
-});
-
 async function loadCatalog() {
   try { cat.apps = (await api("/api/android/catalog")).apps; }
   catch (e) { $("catGrid").innerHTML = `<div class="sub">${esc(e.message)}</div>`; return; }
@@ -4567,7 +4546,7 @@ if (window.frameApp && window.frameApp.onInstallLink) {
 }
 
 setView("headset");
-refresh().then(() => { loadShots(); pollComfort(); });  // after status, so app names resolve and comfort knows if the Frame answers
+refresh().then(() => loadShots());  // after status, so app names resolve
 document.addEventListener("visibilitychange", () => { if (!document.hidden && online === false) refresh(); });
 </script>
 <script>
@@ -4596,20 +4575,6 @@ function netLabel(s) {
 }
 function viaText(v) { return v ? v.host + (v.ip && v.ip !== v.host ? ` (${v.ip})` : "") : ""; }
 
-// Why the last attempt failed, in a few words: the server's short reason, else the stage that failed.
-function failureReason(s) {
-  const err = s.error || {};
-  if (err.short) return err.short;
-  const st = (s.stages || []).find(x => x.id === err.stage);
-  return st ? `${st.label} failed` : "Not connected";
-}
-// A sentence or a label as a clause inside another sentence: no closing full stop (the sentence
-// adds its own) and a lower-case first letter, unless the first word is an abbreviation ("SSH").
-function asClause(text) {
-  const t = String(text || "").trim().replace(/[.\s]+$/, "");
-  return /^[A-Z][a-z]/.test(t) ? t[0].toLowerCase() + t.slice(1) : t;
-}
-
 // The pill in the header: always there, and says what's happening right now.
 function renderPill() {
   const s = link.s;
@@ -4621,8 +4586,9 @@ function renderPill() {
     line = `${netLabel(s)} → ${viaText(s.via)}` + (s.via && s.via.rtt_ms != null ? ` · ${s.via.rtt_ms} ms` : "");
   } else if (s.phase === "failed") {
     dot = "off";
-    // The reason first, in a few words, so a narrow pill cuts the countdown rather than the reason.
-    line = failureReason(s) + (s.retry_at ? ` · retry in ${Math.max(0, Math.ceil(s.retry_at - serverNow()))} s` : "");
+    const left = s.retry_at ? Math.max(0, Math.ceil(s.retry_at - serverNow())) : null;
+    const st = (s.stages || []).find(x => x.id === (s.error || {}).stage);
+    line = `Offline: ${st ? st.label.toLowerCase() : "not connected"} failed` + (left != null ? ` · retry in ${left} s` : "");
   } else {
     const st = activeStage(s);
     const probing = (s.probes || []).find(p => p.state === "trying" || p.state === "resolving");
@@ -4630,7 +4596,7 @@ function renderPill() {
               : "Connecting…";
   }
   $("conn").innerHTML = `<span class="dot ${dot}"></span><span class="pt"><b>${esc(dev)}</b><span>${esc(line)}</span></span>`;
-  $("conn").title = `${dev}${s.phase === "failed" ? " is offline" : ""}: ${line}. Click for details.`;
+  $("conn").title = `${dev}: ${line}. Click for details.`;
   return true;
 }
 window.connPill = renderPill;
@@ -4647,13 +4613,13 @@ function renderBanner() {
   if (s.phase === "failed" && s.error) {
     $("offline").hidden = false;
     $("offMsg").textContent = s.error.message;
-    const tried = (s.probes || []).map(p => `${p.host}: ${asClause(p.detail)}`).join("; ");
+    const tried = (s.probes || []).map(p => `${p.host}: ${p.detail.toLowerCase()}`).join("; ");
     const left = s.retry_at ? Math.max(0, Math.ceil(s.retry_at - serverNow())) : null;
     $("offDetail").textContent = `${s.device ? s.device.name : "Frame"} on ${netLabel(s)}. ${tried ? "Tried " + tried + ". " : ""}` +
       (left != null ? `Trying again in ${left} s.` : "");
   } else if (s.phase === "connecting" && online === false) {
     const st = activeStage(s);
-    $("offDetail").textContent = `Trying again: ${st ? asClause(st.label) + (st.detail ? " (" + st.detail + ")" : "") : "connecting"}…`;
+    $("offDetail").textContent = `Trying again: ${st ? st.label.toLowerCase() + (st.detail ? " (" + st.detail + ")" : "") : "connecting"}…`;
   }
 }
 window.connBanner = renderBanner;
@@ -4831,10 +4797,7 @@ async function useDevice(id) {
   if (!res) {  // refused (an install is running): show the headset still in use
     const current = dv.list.find(x => x.active);
     if (current) $("devSel").value = current.id;
-    return;
   }
-  // The Devices tab shows the headset now in use, not the one it was showing before.
-  if (dv.sel !== id) { dv.sel = id; dv.found = null; dv.editing = null; renderDevices(); }
 }
 async function devAction(label, body, btn) {
   const res = await act(label, () => api("/api/devices", body), btn);
@@ -4846,8 +4809,7 @@ function deviceStatus(d) {
   const s = link.s;
   if (d.active && s) {
     return s.phase === "connected" ? ["on", `Connected via ${viaText(s.via)}`]
-         : s.phase === "failed" ? ["off", s.error ? s.error.short || s.error.message : "Offline", s.error && s.error.message]
-         : ["", "Connecting…"];
+         : s.phase === "failed" ? ["off", s.error ? s.error.message : "Offline"] : ["", "Connecting…"];
   }
   const last = Math.max(0, ...d.addresses.map(a => a.last_ok || 0));
   return ["", d.transient ? "Not set up in Frame Control" : last ? `Last connected ${ago(last)}` : "Not connected yet"];
@@ -4857,10 +4819,10 @@ function renderDevices() {
   if (!dv.data) return;
   $("devCount").textContent = dv.list.length;
   $("devList").innerHTML = dv.list.map(d => {
-    const [dot, text, full] = deviceStatus(d);
+    const [dot, text] = deviceStatus(d);
     return `<div class="item dev-item${d.id === dv.sel ? " sel" : ""}" data-dev="${esc(d.id)}">
       <span class="dot ${dot}"></span><div class="grow"><div class="t">${esc(d.name)} ${d.active ? '<span class="tag">In use</span>' : ""}</div>
-      <div class="s"${full ? ` title="${esc(full)}"` : ""}>ssh ${esc(d.alias)} · ${esc(text)}</div></div>
+      <div class="s">ssh ${esc(d.alias)} · ${esc(text)}</div></div>
       ${d.active ? "" : `<button class="small" data-use="${esc(d.id)}">Use this headset</button>`}</div>`;
   }).join("") || `<div class="sub">No headsets yet. Add one to get started.</div>`;
   $("devList").querySelectorAll("[data-dev]").forEach(el => el.onclick = e => {
@@ -4911,10 +4873,10 @@ function renderDetail() {
       <button class="small" data-mv="1" title="Try later"${i < d.addresses.length - 1 ? "" : " disabled"}>↓</button>
       <button class="small" data-ed>Edit</button><button class="small danger" data-rm>Remove</button></div>`).join("");
   const err = d.active && link.s && link.s.phase === "failed" ? link.s.error : null;
-  const problem = err && err.hostkey ? `<div class="item dev-problem" role="alert"><div class="grow"><div class="t">${err.hostkey.state === "missing"
+  const problem = err && err.hostkey ? `<div class="item" role="alert"><div class="grow"><div class="t">${err.hostkey.state === "missing"
         ? "Its saved SSH identity is missing" : "Its SSH identity has changed"}</div>
         <div class="s">${esc(err.message)}</div></div><button class="small action" id="dvTrustNow">Check the new key…</button></div>`
-    : err && err.auth === "denied" ? `<div class="item dev-problem" role="alert"><div class="grow"><div class="t">The headset didn't accept this computer's key</div>
+    : err && err.auth === "denied" ? `<div class="item" role="alert"><div class="grow"><div class="t">The headset didn't accept this computer's key</div>
         <div class="s">Set Up Connection pairs this computer with it again. On the headset, Developer Mode must be on.</div></div>
         <button class="small action" id="dvPair">Set Up Connection…</button></div>` : "";
   el.innerHTML = `<div class="shelf-head"><h2>${esc(d.name)}</h2>${d.active ? '<span class="count">In use</span>' : ""}<span class="spacer"></span>

ui/server.py

@@ -57,7 +57,6 @@ import frame_link  # noqa: E402
 import frame_macview  # noqa: E402
 import frame_media  # noqa: E402
 import frame_panels  # noqa: E402
-import frame_quest  # noqa: E402
 import frame_report  # noqa: E402
 import frame_store  # noqa: E402
 import frame_telemetry  # noqa: E402
@@ -404,38 +403,11 @@ def status(_body):
     s = json.loads(ssh("python3 -", stdin=(HERE / "frame_status.py").read_text(), timeout=20))
     osr = s.get("os") if isinstance(s, dict) else None
     if isinstance(osr, dict):
-        # The headset's hash is for usage statistics only; the page and MCP clients don't get it.
-        frame_telemetry.frame_seen(osr.get("build"), osr.get("version"), osr.pop("headset", None),
-                                   len(LINK.reg.devices()) if LINK else None)
+        frame_telemetry.frame_seen(osr.get("build"), osr.get("version"))
         frame_report.frame.update(build=osr.get("build"), version=osr.get("version"))
     return s
 
 
-COMFORT_FILES = ("frame_comfort.py", "frame_status.py", "frame_steam.py")
-COMFORT_MISSING = {"comfort": "missing"}  # what COMFORT_RUN prints when the cached copy isn't complete
-# Runs frame_comfort.py from ~/.cache/frame-control/comfort/<version>, first writing any of
-# `files` that aren't there (none: just check they all are). Private, and each file written
-# whole or not at all, so a copy cut short is finished by the next call.
-COMFORT_RUN = """import json, os, pathlib, subprocess, sys
-os.umask(0o077)
-files = %r
-root = pathlib.Path.home() / '.cache/frame-control/comfort' / %r
-if files:
-    root.mkdir(parents=True, exist_ok=True)
-for name, source in files.items():
-    path = root / name
-    if not path.exists():
-        tmp = root / (name + '.' + str(os.getpid()))
-        tmp.write_text(source)
-        tmp.replace(path)
-if not all((root / name).is_file() for name in %r):
-    print('{"comfort": "missing"}')
-    sys.exit()
-r = subprocess.run([sys.executable, str(root / 'frame_comfort.py'), %r], capture_output=True, text=True)
-print(r.stdout, end='')
-"""
-
-
 def comfort(body):
     try:
         frame_comfort.validate(body)
@@ -443,14 +415,25 @@ def comfort(body):
         raise Failure(str(e), 400)
     # Content-addressed, user-only helper bundle. Desktop and phone use the same
     # on-headset state/lock; no listener, service registration or third-party app.
-    # The page polls this every few seconds, so run the copy already on the headset and
-    # send the sources only when that version isn't there yet.
-    files = {name: (HERE / name).read_text() for name in COMFORT_FILES}
+    import hashlib
+    files = {name: (HERE / name).read_text() for name in
+             ("frame_comfort.py", "frame_status.py", "frame_steam.py")}
     version = hashlib.sha256(json.dumps(files, sort_keys=True).encode()).hexdigest()[:16]
-    out = json.loads(ssh("python3 -", stdin=COMFORT_RUN % ({}, version, COMFORT_FILES, json.dumps(body)), timeout=65))
-    if out == COMFORT_MISSING:
-        out = json.loads(ssh("python3 -", stdin=COMFORT_RUN % (files, version, COMFORT_FILES, json.dumps(body)),
-                             timeout=65))
+    script = """import json, os, pathlib, subprocess, sys
+os.umask(0o077)
+files = %r
+root = pathlib.Path.home() / '.cache/frame-control/comfort' / %r
+root.mkdir(parents=True, exist_ok=True)
+for name, source in files.items():
+    path = root / name
+    if not path.exists():
+        tmp = root / (name + '.' + str(os.getpid()))
+        tmp.write_text(source)
+        tmp.replace(path)
+r = subprocess.run([sys.executable, str(root / 'frame_comfort.py'), %r], capture_output=True, text=True)
+print(r.stdout, end='')
+""" % (files, version, json.dumps(body))
+    out = json.loads(ssh("python3 -", stdin=script, timeout=65))
     if out.get("error") and "active" not in out:
         raise Failure(out["error"], 409)
     return out
@@ -1326,48 +1309,6 @@ def android(body):
     raise Failure("unknown action", 400)
 
 
-# ---- Import from my Quest (frame_quest.py, docs/quest-import.md) --------------
-
-QUEST_ON_FRAME = "Import from my Quest runs on your computer, with the Quest plugged into it by USB."
-QUEST_SERIAL = re.compile(r"^[A-Za-z0-9._:\[\]-]{1,128}$")
-
-
-def quest_serial(value):
-    if value in (None, ""):
-        return None
-    if not isinstance(value, str) or not QUEST_SERIAL.match(value):
-        raise Failure("invalid Quest serial", 400)
-    return value
-
-
-def quest_status(query):
-    """The Quest on this computer's USB (or what to do), and the apps on it."""
-    if LOCAL:
-        return {"quest": None, "devices": [], "guidance": QUEST_ON_FRAME, "apps": []}
-    try:
-        return frame_quest.status(quest_serial((parse_qs(query).get("serial") or [None])[0]))
-    except frame_android.FrameError as e:
-        raise Failure(str(e)) from e
-
-
-def quest(body):
-    """{"action": "import", "package", "serial"?, "data"?}: a background job with progress."""
-    if LOCAL:
-        raise Failure(QUEST_ON_FRAME, 400)
-    if body.get("action") != "import":
-        raise Failure("unknown action", 400)
-    pkg = str(body.get("package", ""))
-    if not frame_android.PKG_RE.match(pkg):
-        raise Failure("invalid Android package id", 400)
-    serial, data, older = quest_serial(body.get("serial")), body.get("data", False), body.get("allowDowngrade", False)
-    if type(data) is not bool or type(older) is not bool:
-        raise Failure("data and allowDowngrade must be true or false", 400)
-    ensure_master()
-    return start_job(f"Import {pkg} from Quest",
-                     lambda report: frame_quest.import_game(pkg, serial, data=data, progress=report,
-                                                            allow_downgrade=older), progress=True)
-
-
 # Errors that are the APK's own fault, so they belong in the compatibility
 # database as install_failed. Connection trouble and the like don't.
 APK_FAULTS = {"android_installer", "apk_needs_newer_android", "apk_wrong_abi"}
@@ -2212,6 +2153,16 @@ def source_text(body, key, optional=False):
     return value.strip()
 
 
+def login_text(body, key):
+    """A username, password or token: kept exactly (no stripping) and never echoed back."""
+    value = body.get(key)
+    if value in (None, ''):
+        return None
+    if not isinstance(value, str) or len(value) > 4096:
+        raise Failure('Provide a valid ' + key, 400)
+    return value
+
+
 def source_search(query):
     args = parse_qs(query)
     q = args.get('q', [''])[0]
@@ -2250,19 +2201,31 @@ def source_manage(body):
         if action == 'add':
             url = source_text(body, 'url')
             fingerprint, name = source_text(body, 'fingerprint', True), source_text(body, 'name', True)
-            parts = urlparse(url)
-            if parts.scheme not in ('https', 'fdroidrepos') or not parts.hostname or parts.username:
-                raise Failure('Use an HTTPS or fdroidrepos:// repository URL without credentials', 400)
+            username, password = login_text(body, 'username'), login_text(body, 'password')
+            try:
+                parts = urlparse(url)
+                parts.port
+            except ValueError:
+                raise Failure('Use an HTTPS or fdroidrepos:// repository URL', 400) from None
+            if parts.scheme not in ('https', 'fdroidrepos') or not parts.hostname:
+                raise Failure('Use an HTTPS or fdroidrepos:// repository URL', 400)
             apk_search.repo_module()  # fail now if this build can't manage repositories
 
             def add():  # downloads and verifies the whole index: a job, not a request
-                source = apk_search.manage_repo('add_repo', url=url, fingerprint=fingerprint, name=name)
+                # A login (here or in the URL) is stored apart and never comes back in a response.
+                login = {'username': username, 'password': password} if username or password else {}
+                source = apk_search.manage_repo('add_repo', url=url, fingerprint=fingerprint, name=name, **login)
                 message = 'Added ' + source['name']
                 if source.get('trust_on_first_use'):
                     message += '. Trusted on first use: ' + source['fingerprint'].upper()
                 return {'message': message, 'source': {k: source.get(k) for k in
                                                        ('id', 'name', 'fingerprint', 'trust_on_first_use')}}
             return start_job('Add repository', add)
+        if action in ('set-token', 'clear-token'):
+            token = login_text(body, 'token') if action == 'set-token' else None
+            if action == 'set-token' and not token:
+                raise Failure('Paste a token', 400)
+            return apk_search.set_token(source_text(body, 'source'), token)
         if action == 'game-data':
             package = source_text(body, 'package')
             return start_job('Add game data', lambda: apk_search.add_game_data(package))
@@ -2280,7 +2243,7 @@ POST = {
     "/api/agent/approval": agent_approval, "/api/assistant/chat": assistant_chat,
     "/api/input": remote_input, "/api/touch": remote_touch,
     "/api/settings/artwork": frame_steamgriddb.save_settings,
-    "/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/quest": quest, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
+    "/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
         "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
         "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
         "/api/webinstall/cancel": webinstall_cancel,
@@ -2339,10 +2302,6 @@ def connection_state():
 
 # ---- HTTP ------------------------------------------------------------------
 
-# POST reads the page repeats on a timer, as (path, action).
-POLLS = {("/api/comfort", "status")}
-
-
 def action_of(body):
     """The action a request asked for, for diagnostics: a short word, never user data."""
     a = body.get("action") if isinstance(body, dict) else None
@@ -2499,8 +2458,6 @@ class Handler(BaseHTTPRequestHandler):
                 apps = frame_android.list_apps()
                 backfill_art(apps=apps)
                 self.send_json({"apps": apps})
-            elif path == "/api/quest":
-                self.send_json(quest_status(url.query))
             elif path == "/api/titles":
                 ensure_master()
                 titles_list = frame_titles.list_titles()
@@ -2598,10 +2555,7 @@ class Handler(BaseHTTPRequestHandler):
         except ClientGone:
             raise
         except Failure as e:
-            # A poll that couldn't reach the headset is the headset asleep or away, which the page
-            # already shows; like GET /api/status, not an error to report.
-            polled = (path, action_of(body)) in POLLS and frame_telemetry.link_failed(e)
-            if e.status >= 500 and not polled:
+            if e.status >= 500:
                 frame_telemetry.diagnostic(f"POST {path} {action_of(body)}", e)
             self.send_error_json(str(e), e.status, e.apk)
         except (ValueError, TypeError) as e: