Merge remote-tracking branch 'origin/main' into fix/direct-local-transfer

test: avoid unread body in Windows auth guard probe

docs/evidence/direct-local-transfer-2026-10-28.md

@@ -0,0 +1,31 @@
+# Large Send to Frame files: source and offline verification
+
+[Issue #373](https://github.com/saphid/frame-control/issues/373) reports that
+a 26.5 GB file selected from external USB media failed with “empty or
+too-large upload” in Frame Control 0.4.8. Current main still imposed an
+8 GiB `POST /api/upload` body cap for **Send to Frame**, even in the desktop
+app. The desktop already knows the selected local file path, but previously
+sent the whole file through HTTP into another temporary copy before SSH.
+
+The desktop page now passes that path to `POST /api/transfer`, which is
+enabled in the app-started server. It checks for an absolute, nonempty
+regular file and copies it directly by the
+existing rsync/scp route to `~/Downloads`. This avoids both the 8 GiB HTTP
+cap and a duplicate temporary file on the computer. The endpoint preserves
+the selected-headset work guard. Large copies have a finite timeout that
+scales with source size. Plain browser uploads still have an 8 GiB cap and
+now state it before sending bytes; the page says which route applies.
+
+Offline checks use a mocked 27 GiB file size without allocating that space.
+They verify the direct server copy path, a timeout longer than one hour, the
+page's desktop-versus-browser choice, and early rejection of an oversized
+browser upload. Loopback tests confirm the browser cannot call the direct
+route, reject a mismatched selected headset with HTTP 409, and accept the
+matching desktop request with HTTP 200. Unit tests reject missing, relative,
+directory, hidden and empty source paths before any SSH. The full Python
+suite on the final branch passed **1,690 tests, 16 skipped**. Hosted CI is a
+separate merge gate.
+
+An actual 26.5 GB transfer from USB media to a Frame, end-to-end throughput,
+resume after interruption and free-space behavior remain unverified. No real
+headset or user file was touched for this change.

docs/evidence/main-windows-denied-post-2026-10-28.md

@@ -0,0 +1,18 @@
+# Main Windows test gate, 28 October 2026
+
+Exact main commit `b5bbcec1a0cc3f8b3d25ef2686fd8315e209f167`
+([checks run 40723931638](https://github.com/saphid/frame-control/actions/runs/40723931638))
+failed in `server-tests (windows-latest, 3.12)`. Its server test step had one
+error: `test_settings_need_and_accept_the_ui_key` received Windows
+`ConnectionAbortedError [WinError 10053]` while reading an unauthorized
+POST response. The test sent a JSON body, but the missing-key guard rejects
+the request before reading it. Closing a connection with an unread body can
+reset it on Windows; that is the likely cause, rather than a settings write
+failure. The other Windows tests completed, and the exact-main native Decky
+runtime run passed.
+
+The test now sends an empty POST when checking that a missing key is refused.
+Its subsequent authorized POST still sends a settings body and verifies the
+saved value. The local `tests/test_server.py` module passed 45 tests after
+this change. Hosted Windows CI on the repair commit and green exact-main CI
+remain required before further merges.

tests/page/large_transfer.mjs

@@ -0,0 +1,33 @@
+// Exercise the real page functions for desktop direct copy and browser size guard.
+import { readFileSync } from "fs";
+import assert from "node:assert/strict";
+
+const src = readFileSync(new URL("../../ui/index.html", import.meta.url), "utf8");
+const lines = src.split("\n");
+function grab(name) {
+  const start = lines.findIndex(line => line.startsWith(`function ${name}(`));
+  assert.notEqual(start, -1, `${name} exists`);
+  let depth = 0;
+  for (let i = start; i < lines.length; i++) {
+    for (const c of lines[i]) depth += c === "{" ? 1 : c === "}" ? -1 : 0;
+    if (depth === 0) return lines.slice(start, i + 1).join("\n");
+  }
+  throw new Error(`${name} is unbalanced`);
+}
+
+const calls = [];
+const copy = new Function("api", "upload", grab("copyToDownloads") + "\nreturn copyToDownloads;")(
+  async (...args) => { calls.push(["api", ...args]); return { message: "sent" }; },
+  async (...args) => { calls.push(["upload", ...args]); return { message: "uploaded" }; });
+const large = { name: "usb-file.bin", size: 27 * 1024 ** 3 };
+assert.deepEqual(await copy(large, "/media/usb/usb-file.bin", "frame-a"), { message: "sent" });
+assert.deepEqual(calls.pop(), ["api", "/api/transfer", { action: "copy-local", path: "/media/usb/usb-file.bin" }, "frame-a"]);
+assert.deepEqual(await copy({ name: "small.bin", size: 100 }, "", "frame-a"), { message: "uploaded" });
+assert.deepEqual(calls.pop(), ["upload", { name: "small.bin", size: 100 }, "push", "frame-a"]);
+
+let opened = false;
+const browserUpload = new Function("XMLHttpRequest", grab("upload") + "\nreturn upload;")(
+  class { constructor() { opened = true; } });
+await assert.rejects(browserUpload(large, "push", "frame-a"), /limited to 8 GiB/);
+assert.equal(opened, false, "oversized browser file is refused before creating a request");
+console.log("ok");

tests/test_large_transfer.py

@@ -0,0 +1,86 @@
+"""Large desktop file copies use the selected local path, not an 8 GiB HTTP upload."""
+import sandbox  # noqa: F401  (first: keeps tests off real data and services)
+import http.client
+import json
+import os
+import stat
+import subprocess
+import sys
+import tempfile
+import threading
+import unittest
+from pathlib import Path
+from types import SimpleNamespace
+from unittest import mock
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "ui"))
+import server
+
+
+class LargeTransfer(unittest.TestCase):
+    def test_desktop_http_copy_stays_on_selected_headset(self):
+        with tempfile.TemporaryDirectory() as tmp, server.LoopbackServer(("127.0.0.1", 0), server.Handler) as httpd:
+            source = Path(tmp) / "usb-file.bin"
+            source.write_bytes(b"a small stand-in for a large file")
+            worker = threading.Thread(target=httpd.serve_forever, daemon=True)
+            worker.start()
+            link = mock.Mock()
+            link.active_device.return_value = {"id": "frame-a"}
+            body = json.dumps({"action": "copy-local", "path": str(source)})
+
+            def copy(device):
+                conn = http.client.HTTPConnection("127.0.0.1", httpd.server_port, timeout=5)
+                try:
+                    conn.request("POST", "/api/transfer", body, {"X-Frame-UI": server.UI_KEY,
+                                 "X-Frame-Device": device, "Content-Type": "application/json"})
+                    response = conn.getresponse()
+                    return response.status, json.loads(response.read())
+                finally:
+                    conn.close()
+
+            try:
+                with mock.patch.dict(os.environ, {"FRAME_CONTROL_APP": "1"}), \
+                     mock.patch.object(server, "LINK", link), \
+                     mock.patch.object(server, "push_file", return_value="Sent usb-file.bin to ~/Downloads/") as push:
+                    self.assertEqual(copy("frame-b")[0], 409)
+                    push.assert_not_called()
+                    self.assertEqual(copy("frame-a"), (200, {"message": "Sent usb-file.bin to ~/Downloads/"}))
+                    push.assert_called_once_with(source)
+            finally:
+                httpd.shutdown()
+                worker.join(timeout=5)
+
+    def test_27_gib_local_file_reaches_direct_copy(self):
+        source = Path(os.path.abspath("large-file.bin"))
+        size = 27 * 1024 ** 3
+        info = SimpleNamespace(st_mode=stat.S_IFREG | 0o644, st_size=size)
+        with mock.patch.dict(os.environ, {"FRAME_CONTROL_APP": "1"}), \
+             mock.patch.object(Path, "stat", return_value=info), \
+             mock.patch.object(server, "push_file", return_value="Sent large-file.bin to ~/Downloads/") as push:
+            self.assertEqual(server.transfer_local({"action": "copy-local", "path": str(source)}),
+                             {"message": "Sent large-file.bin to ~/Downloads/"})
+        push.assert_called_once_with(source)
+
+    def test_direct_copy_validates_desktop_file_before_ssh(self):
+        with tempfile.TemporaryDirectory() as tmp, \
+             mock.patch.dict(os.environ, {"FRAME_CONTROL_APP": "1"}), \
+             mock.patch.object(server, "push_file") as push:
+            empty = Path(tmp) / "empty.bin"
+            empty.touch()
+            hidden = Path(tmp) / ".hidden"
+            hidden.write_bytes(b"x")
+            for path in ("relative.bin", str(Path(tmp) / "missing.bin"), tmp,
+                         str(empty), str(hidden)):
+                with self.subTest(path=path), self.assertRaises(server.Failure):
+                    server.transfer_local({"action": "copy-local", "path": path})
+        push.assert_not_called()
+
+    def test_large_copy_gets_more_than_one_hour_without_loading_file(self):
+        info = SimpleNamespace(st_size=27 * 1024 ** 3)
+        result = subprocess.CompletedProcess([], 0, "", "")
+        with mock.patch.object(Path, "stat", return_value=info), \
+             mock.patch.object(server.shutil, "which", return_value=None), \
+             mock.patch.object(server.frame_host, "run_ssh", return_value=result) as run:
+            self.assertEqual(server.push_file("/tmp/large-file.bin"),
+                             "Sent large-file.bin to ~/Downloads/")
+        self.assertGreater(run.call_args.kwargs["timeout"], 3600)

tests/test_server.py

@@ -267,6 +267,17 @@ class ServerGuards(unittest.TestCase):
         self.assertEqual(self.request("GET", "/api/titles/job?token=nope", headers={"X-Frame-UI": "1"})[0], 404)
         self.assertEqual(self.request("POST", "/api/titles", {"action": "list"})[0], 403)
 
+    def test_direct_local_transfer_is_desktop_only(self):
+        status, payload = self.post("/api/transfer", {"action": "copy-local", "path": "/tmp/a.bin"})
+        self.assertEqual(status, 403, payload)
+
+    def test_oversized_browser_upload_rejected_without_reading_body(self):
+        status, _, payload = self.request("POST", "/api/upload", b"",
+                                          {"X-Frame-UI": "1", "X-Filename": "large.bin",
+                                           "Content-Length": str(27 * 1024 ** 3)})
+        self.assertEqual(status, 413, payload)
+        self.assertIn(b"limited to 8 GiB", payload)
+
     def test_web_install_needs_the_app_page(self):
         # A website can only open frame-control:// links; it can't call these itself.
         link = {"url": "https://cdn.example.com/game.apk"}
@@ -497,7 +508,10 @@ class ArtworkSettings(unittest.TestCase):
                         time.sleep(0.05)
                 key = {"X-Frame-UI": "1", "Content-Type": "application/json"}
                 self.assertEqual(request("GET", "/api/settings/artwork")[0], 403)
-                self.assertEqual(request("POST", "/api/settings/artwork", {"steamgriddb_api_key": "abc"})[0], 403)
+                # The missing-key guard runs before reading a body. Leaving an
+                # unread body on Windows can reset this test connection before
+                # http.client receives the 403 response.
+                self.assertEqual(request("POST", "/api/settings/artwork")[0], 403)
                 status, payload = request("GET", "/api/settings/artwork", headers=key)
                 self.assertEqual((status, json.loads(payload)["steamgriddb_configured"]), (200, False))
                 status, payload = request("POST", "/api/settings/artwork", {"steamgriddb_api_key": "abc_1"}, key)
@@ -575,6 +589,10 @@ class PageScripts(unittest.TestCase):
         r = subprocess.run(["node", str(ROOT / "tests" / "page" / "meta_rows.mjs")], capture_output=True, text=True)
         self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
 
+    def test_large_local_file_uses_direct_copy(self):
+        r = subprocess.run(["node", str(ROOT / "tests" / "page" / "large_transfer.mjs")], capture_output=True, text=True)
+        self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
+
 
 @unittest.skipIf(os.name == "nt", "runs on the Frame (Linux); local-bin/ssh is a POSIX shell script")
 class LocalMode(unittest.TestCase):

ui/index.html

@@ -1686,6 +1686,7 @@
         <option value="vr">VR app</option>
       </select>
       <div class="progress" id="prog"><i></i></div>
+      <div class="hint">The desktop app copies files from this computer directly, including files larger than 8 GiB. Browser uploads are limited to 8 GiB.</div>
       <div class="hint">Clipboard text, both ways, is in <a href="#pad">Type on the headset</a>.</div>
     </section>
 
@@ -3843,6 +3844,7 @@ window.addEventListener("drop", e => {
 function upload(file, mode, dev = window.connDevice, backupToken = null, progress = () => {}, signal = null) {
   return new Promise((resolve, reject) => {
     if (signal?.aborted) return reject(new Error("Import cancelled"));
+    if (file.size > 8 * 1024 ** 3) return reject(new Error("Browser uploads are limited to 8 GiB; use the desktop app to copy larger files"));
     const xhr = new XMLHttpRequest();
     const abort = () => xhr.abort();
     const cleanup = () => signal?.removeEventListener("abort", abort);
@@ -3875,6 +3877,9 @@ function upload(file, mode, dev = window.connDevice, backupToken = null, progres
     xhr.send(file);
   });
 }
+function copyToDownloads(file, path, dev) {
+  return path ? api("/api/transfer", { action: "copy-local", path }, dev) : upload(file, "push", dev);
+}
 async function refreshMedia(selectId) {
   const data = await api("/api/media", { action: "list" });
   const selected = selectId || $("mediaFile").value;
@@ -4001,7 +4006,7 @@ async function sendFilesOne(files, dirs, dev) {
     if (TITLE_EXT.test(f.name)) { await sideload(f, path); continue; }
     const apk = f.name.toLowerCase().endsWith(".apk");
     if (apk) await importAndroidFiles([f], dev);
-    else await act(`Copy ${f.name} to ~/Downloads`, () => upload(f, "push", dev));
+    else await act(`Copy ${f.name} to ~/Downloads`, () => copyToDownloads(f, path, dev));
   }
   $("fileInput").value = "";
   refresh();
@@ -4137,7 +4142,7 @@ async function sideloadOne(file, path, isDir = false, dev = window.connDevice) {
   catch (e) {
     log(`${file.name}: ${e.message}`, "e");
     if (canPush && confirm(`${file.name}: ${e.message}\n\nCopy it to ~/Downloads instead?`))
-      await act(`Copy ${file.name} to ~/Downloads`, () => upload(file, "push", dev));
+      await act(`Copy ${file.name} to ~/Downloads`, () => copyToDownloads(file, path, dev));
     return;
   }
   // Its own fresh list, so the dialog can say whether this replaces an installed title.
@@ -4146,7 +4151,7 @@ async function sideloadOne(file, path, isDir = false, dev = window.connDevice) {
   const choice = await confirmTitle(r.plan, canPush, installed);
   if (choice === "push") {
     api("/api/titles", { action: "discard", token: r.token }).catch(() => {});
-    await act(`Copy ${file.name} to ~/Downloads`, () => upload(file, "push", dev));
+    await act(`Copy ${file.name} to ~/Downloads`, () => copyToDownloads(file, path, dev));
     return;
   }
   if (!choice) { api("/api/titles", { action: "discard", token: r.token }).catch(() => {}); return; }

ui/server.py

@@ -28,6 +28,7 @@ import shutil
 import signal
 import socket
 import socketserver
+import stat
 import subprocess
 import sys
 import tempfile
@@ -3494,8 +3495,27 @@ def power_action(body):
             raise Failure(str(exc), 409) from exc
 
 
+def transfer_local(body):
+    """Copy a desktop-selected file without a second, size-limited HTTP upload to /tmp."""
+    if os.environ.get("FRAME_CONTROL_APP") != "1":
+        raise Failure("Direct file copy needs the Frame Control desktop app", 403)
+    if body.get("action") != "copy-local" or not isinstance(body.get("path"), str):
+        raise Failure("Choose a file on this computer", 400)
+    path = Path(body["path"])
+    if not path.is_absolute() or not path.name or path.name.startswith("."):
+        raise Failure("Choose a regular file with a visible name", 400)
+    try:
+        info = path.stat()
+    except OSError:
+        raise Failure("That file is no longer available on this computer", 400) from None
+    if not stat.S_ISREG(info.st_mode) or info.st_size <= 0:
+        raise Failure("Choose a nonempty regular file", 400)
+    return {"message": push_file(path)}
+
+
 POST = {
     "/api/steam-pipewire": steam_pipewire_action,
+    "/api/transfer": transfer_local,
     "/api/power": power_action,
     "/api/android/backup": android_backup, "/api/backup": backup_action,
     "/api/decky": decky,
@@ -3611,6 +3631,12 @@ def _next_chunk(chunks, timeout):
 def push_file(path, dest="Downloads/"):
     """Copy a file to the Frame (as scripts/push.sh): rsync where both ends have it, else scp."""
     name = Path(path).name
+    # A large file can legitimately take more than an hour over Wi-Fi. Keep a
+    # finite deadline, but allow roughly one MiB/s plus startup time.
+    try:
+        timeout = max(3600, min(24 * 3600, int(Path(path).stat().st_size / (1 << 20)) + 600))
+    except OSError:
+        raise Failure(f"{name} is no longer available on this computer", 400) from None
     try:
         # Not on Windows: a Windows rsync (cwRsync, MSYS2) wouldn't take our POSIX -e quoting.
         if not frame_host.WINDOWS and shutil.which("rsync") and ssh("command -v rsync >/dev/null && echo yes || true").strip() == "yes":
@@ -3618,7 +3644,7 @@ def push_file(path, dest="Downloads/"):
         else:
             # Modern scp uses SFTP, so the remote path isn't parsed by a shell.
             cmd = ["scp", *SSH[1:], "-r", str(path), f"{FRAME}:{dest}"]
-        r = frame_host.run_ssh(cmd, capture_output=True, stdin=subprocess.DEVNULL, text=True, errors="replace", timeout=3600)
+        r = frame_host.run_ssh(cmd, capture_output=True, stdin=subprocess.DEVNULL, text=True, errors="replace", timeout=timeout)
     except subprocess.TimeoutExpired:
         raise Failure(f"Copying {name} timed out")
     if r.returncode != 0:
@@ -4070,8 +4096,10 @@ class Handler(BaseHTTPRequestHandler):
         length = int(self.headers.get("Content-Length") or 0)
         if not name or name.startswith("."):
             raise Failure("missing filename", 400)
-        if length <= 0 or length > MAX_UPLOAD:
-            raise Failure("empty or too-large upload", 400)
+        if length <= 0:
+            raise Failure("empty upload", 400)
+        if length > MAX_UPLOAD:
+            raise Failure("Browser uploads are limited to 8 GiB; use the desktop app to copy larger files", 413)
         if mode in ("apk", "apkinfo") and not name.lower().endswith(".apk"):
             raise Failure("APK install needs a .apk file", 400)
         backup = None