mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-18 02:01:20 +02:00
Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 SAPHID
|
62b3cbf705 | ||
|
saphidandClaude Sonnet 5.5
|
024ec593f9 | ||
|
saphidandClaude Sonnet 5.5
|
0373555519 | ||
|
saphidandClaude Sonnet 5.5
|
3539215517 | ||
|
saphidandClaude Sonnet 5.5
|
9f3fb89e30 | ||
|
saphidandClaude Sonnet 5.5
|
dc7854d10b | ||
|
saphidandClaude Sonnet 5.5
|
5a6b925448 | ||
|
saphidandClaude Sonnet 5.5
|
60ade8c2e8 | ||
|
saphidandClaude Opus 5.5
|
aa1ac7368c | ||
|
saphidandClaude Opus 5.5
|
46b6966cf7 | ||
|
saphidandClaude Opus 5.5
|
be2e770e66 | ||
|
saphidandClaude Opus 5.5
|
597f98af6d | ||
|
saphidandClaude Opus 5.5
|
ac07efa8f1 | ||
|
saphidandClaude Opus 5.5
|
0f4f0b863f | ||
|
saphidandClaude Opus 5.5
|
e7c81733b1 |
No files matched your search
@@ -73,7 +73,7 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a
|
||||
Browse the shots you take in the headset and save them to your Pictures folder.
|
||||
|
||||
**⌨️ Keyboard and trackpad**<br>
|
||||
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Nothing else to install, anywhere.
|
||||
Type and point in the headset from your computer or phone, through Valve's own input path, with nothing to install. Accents and emoji use KDE Connect, which Frame Control brings along and sets up the first time you type one.
|
||||
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
+1
-54
@@ -58,61 +58,8 @@ form shows the pinned one ("Trusted on first use: …") so you can compare it.
|
||||
Re-adding an existing URL preserves its pin; changing it requires deliberately
|
||||
removing and re-adding it.
|
||||
|
||||
### Repositories behind a login
|
||||
|
||||
Some self-hosted repositories sit behind HTTP basic auth. As in F-Droid's own
|
||||
client, the login can be part of the address
|
||||
(`https://user:pass@example.org/fdroid/repo`), or given separately: in the
|
||||
store's **Add a source** form under "This repository needs a login", in the
|
||||
server API (`username` and `password` beside `url`), or on the command line,
|
||||
where the password is read from standard input so it stays out of shell history
|
||||
and `ps`:
|
||||
|
||||
```sh
|
||||
printf '%s\n' "$PASSWORD" | python3 ui/apk_sources/fdroid.py add 'https://example.org/fdroid/repo' --username me --password-stdin
|
||||
```
|
||||
|
||||
How the login is kept and used:
|
||||
|
||||
- **Kept out of the address.** The `user:pass@` part is removed before anything
|
||||
is saved, so the saved repository URL, its id, the source list, job results
|
||||
and the page never contain it. The store only says "Uses a login".
|
||||
- **Stored separately.** Logins live in
|
||||
`frame_host.data_dir('source-credentials.json')`
|
||||
(`~/Library/Application Support/Frame Control/source-credentials.json` on
|
||||
macOS), written with mode 0600 and keyed by source id. They are stored in
|
||||
plain text there, protected by file permissions only (on Windows,
|
||||
the folder's normal ACLs). Removing the repository deletes its login, and
|
||||
the file goes when the last login is removed. Each change holds a lock
|
||||
file (`source-credentials.lock`), so the app and the command line can't undo
|
||||
each other's saves or removals.
|
||||
- **HTTPS only.** An `http://` address with a login is refused before any request.
|
||||
- **Inside the repository only.** The `Authorization` header is decided
|
||||
separately for every request, redirects included: index, `entry.jar`,
|
||||
APKs and the store's artwork proxy. It is sent only when the request's
|
||||
scheme, host and port match the repository's and its path is inside the
|
||||
repository's directory by whole path segments. A redirect to another host
|
||||
or port, or to another path on the same server, gets no login, and that
|
||||
request's own `Authorization` header is removed. Paths with `.` or `..`
|
||||
segments (plain or percent-encoded), encoded `/` or `\`, double encoding
|
||||
or `;` get no login, since a server could normalise them to a path
|
||||
outside the repository. Index file names like that are refused outright.
|
||||
- **Never echoed.** Logins are not written to logs, error messages, API
|
||||
responses or the cache. The CLI prints only the saved source.
|
||||
- **Checked before saving.** As with any repository, the whole index must
|
||||
download and verify before anything is saved. HTTP 401 (or 403 when a login
|
||||
was sent) stops the add with "check the username and password" and saves
|
||||
nothing: no repository, no login, no rollback state. A 403 with a login is
|
||||
not treated as rate limiting.
|
||||
- **Same verification.** The login changes only who may download. Signature,
|
||||
fingerprint pin or TOFU, index hash and rollback checks apply exactly as for
|
||||
any other repository.
|
||||
|
||||
Re-adding the same address with a new login replaces the saved one. Re-adding
|
||||
it without a login keeps the saved one.
|
||||
|
||||
The API for the search/server integration is in `ui/apk_sources/fdroid.py`:
|
||||
`add_repo(url, fingerprint=None, name=None, username=None, password=None)`, `remove_repo(source_id)`,
|
||||
`add_repo(url, fingerprint=None, name=None)`, `remove_repo(source_id)`,
|
||||
`set_enabled(source_id, enabled)`, and `user_repos()`. The module also exposes
|
||||
`sources`, `search`, `details`, and `download` from the shared source contract.
|
||||
This change supplies the CLI and API; the integrated source-management UI is
|
||||
|
||||
+4
-17
@@ -45,23 +45,10 @@ search. Results outside the curated list stay page-only, even if a repository
|
||||
claims an open-source license. This prevents an arbitrary tagged mirror from
|
||||
becoming a trusted downloader. Extend the curated JSON after provenance review.
|
||||
|
||||
For a higher API quota, add a GitHub access token in the store's source
|
||||
settings (the GitHub row's token field) or through the server API
|
||||
(`POST /api/sources` with `action: "set-token"` or `"clear-token"` and
|
||||
`source: "github"`). `FRAME_GITHUB_TOKEN` in the process environment still
|
||||
works and wins over a saved token. A saved token is kept with repository
|
||||
logins in `source-credentials.json` (mode 0600, see
|
||||
[APK repositories](apk-repos.md#repositories-behind-a-login)). The source list
|
||||
reports only whether a token is set, where it came from and whether GitHub
|
||||
rejected it, never its value. Tokens are sent as `Authorization: Bearer` only to
|
||||
`api.github.com`, never written to the cache, never sent to asset hosts (public
|
||||
release downloads don't need them), and removed on redirects. If GitHub answers
|
||||
401 to the token, the request is retried once without it, as Obtainium does.
|
||||
The rejection is logged without the token and shown in source settings. The
|
||||
token is not tried again until it is saved again or the app restarts. Rate-limit
|
||||
backoff is kept per login (anonymous, or each token), so saving a token after
|
||||
anonymous requests were limited lets the next request go out with it. The
|
||||
adapter does not read `gh` credentials automatically. Metadata is cached for one hour under
|
||||
Set optional `FRAME_GITHUB_TOKEN` in the process environment for a higher API
|
||||
quota. Tokens are sent only to `api.github.com`, never written to the cache,
|
||||
never sent to asset hosts, and removed on redirects. The adapter does not
|
||||
read `gh` credentials automatically. Metadata is cached for one hour under
|
||||
`frame_host.cache_dir('apk-sources', 'publisher')`. A cold details request
|
||||
fetches at most ten releases. Rate-limit errors are surfaced without retry
|
||||
loops. Asset IDs and release tags are not Android version codes: metadata
|
||||
|
||||
+7
-4
@@ -151,10 +151,13 @@ two fingers to scroll, two-finger tap to right-click) plus a text field that
|
||||
types on the Frame. On a computer, clicking the pad passes your mouse and
|
||||
keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac).
|
||||
|
||||
It goes through **KDE Connect**, the first-party route (KDE makes the Frame's
|
||||
desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py)
|
||||
on the Frame, which talks KDE Connect's own LAN protocol to the Frame's
|
||||
`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking.
|
||||
Since Control (below), the pad goes through **gamescope's own input socket**
|
||||
([`ui/frame_touch.py`](../ui/frame_touch.py)), so it reaches whichever panel has
|
||||
focus, Steam's own menus included, and needs nothing installed on the Frame.
|
||||
Only characters that aren't on a US keyboard (accents, emoji) go through
|
||||
**KDE Connect** as below; it starts the first time you type one. (Before
|
||||
Control, everything went through KDE Connect, which is why the notes below
|
||||
describe it in detail.)
|
||||
|
||||
**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901):
|
||||
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
// The store's "Add a source" login handling and GitHub token field, run in node against
|
||||
// the real functions from ui/index.html. A login typed into the address must leave the
|
||||
// address field (into the login fields) before anything is sent, and stay out of it.
|
||||
import { readFileSync } from "fs";
|
||||
const src = readFileSync(new URL("../../ui/index.html", import.meta.url), "utf8");
|
||||
const lines = src.split("\n");
|
||||
const fail = msg => { console.log("FAIL " + msg); process.exit(1); };
|
||||
|
||||
function grab(name) {
|
||||
const start = lines.findIndex(l => new RegExp(`^(?:(?:async )?function ${name}\\(|const ${name} = )`).test(l));
|
||||
if (start < 0) fail(`no top-level ${name} in index.html`);
|
||||
let depth = 0;
|
||||
for (let i = start; i < lines.length; i++) {
|
||||
for (const c of lines[i]) depth += c === "{" ? 1 : c === "}" ? -1 : 0;
|
||||
if (depth <= 0) return lines.slice(start, i + 1).join("\n") + "\n";
|
||||
}
|
||||
fail(`unbalanced ${name}`);
|
||||
}
|
||||
const { moveLogin, sourceToken } = new Function(
|
||||
"const esc = s => String(s).replace(/[&<>\"']/g, c => `&#${c.charCodeAt(0)};`); const storeName = s => s;\n" +
|
||||
grab("decodeLenient") + grab("moveLogin") + grab("sourceToken") + "return { moveLogin, sourceToken };")();
|
||||
|
||||
function form(url, username = "", password = "") {
|
||||
const details = { open: false };
|
||||
const field = value => ({ value, closest: () => details });
|
||||
return { details, elements: { url: field(url), username: field(username), password: field(password) } };
|
||||
}
|
||||
|
||||
let f = form("https://alice:s3cr%40t@repo.example/fdroid/repo");
|
||||
moveLogin(f);
|
||||
if (f.elements.url.value !== "https://repo.example/fdroid/repo") fail("address still holds the login: " + f.elements.url.value);
|
||||
if (f.elements.username.value !== "alice" || f.elements.password.value !== "s3cr@t") fail("login not moved into its fields");
|
||||
if (!f.details.open) fail("login fields not shown");
|
||||
|
||||
f = form("https://alice:fromurl@repo.example/repo", "typed", "typedpw");
|
||||
moveLogin(f);
|
||||
if (f.elements.url.value.includes("fromurl") || f.elements.username.value !== "typed" || f.elements.password.value !== "typedpw")
|
||||
fail("typed login replaced, or address kept the login");
|
||||
|
||||
// Undecodable logins and unusual spellings the URL parser accepts still lose their login; undecodable bytes stay as typed.
|
||||
for (const [url, address, user, pass] of [
|
||||
["https://alice:100%done@repo.example/private/repo", "https://repo.example/private/repo", "alice", "100%done"],
|
||||
["https:\t//alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
|
||||
["ht\ntps://alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
|
||||
["https:alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
|
||||
["\u0000https://alice:s3cret@repo.example/repo", "https://repo.example/repo", "alice", "s3cret"],
|
||||
["https://alice:%FF@repo.example/private/repo", "https://repo.example/private/repo", "alice", "%FF"],
|
||||
["https://al%20ice:p%40ss:word@repo.example/r", "https://repo.example/r", "al ice", "p@ss:word"],
|
||||
["https://alice:p%FF\u00e4@repo.example/repo", "https://repo.example/repo", "alice", "p%FF\u00e4"],
|
||||
["fdroidrepos://alice:pw@repo.example/repo?fingerprint=AB", "fdroidrepos://repo.example/repo?fingerprint=AB", "alice", "pw"],
|
||||
["https://alice@repo.example/repo", "https://repo.example/repo", "alice", ""],
|
||||
]) {
|
||||
f = form(url);
|
||||
moveLogin(f);
|
||||
if (f.elements.url.value !== address) fail(`address kept the login: ${url} -> ${f.elements.url.value}`);
|
||||
if (f.elements.username.value !== user || f.elements.password.value !== pass)
|
||||
fail(`login for ${url}: ${f.elements.username.value} / ${f.elements.password.value}`);
|
||||
}
|
||||
|
||||
// An address the parser rejects that may hold a login is refused and left as typed, never sent.
|
||||
for (const url of ["https://alice\\name:pw@repo.example/repo", "https:\\\\alice:s3cret@repo.example/repo", "fdroidrepos://alice:p\\w@repo.example/repo", "https://alice:s3cret@repo example/repo", "https:alice:s3cret@repo example/repo", "https://repo.example:bad\\a@b"]) {
|
||||
f = form(url);
|
||||
if (moveLogin(f) !== false) fail("accepted an invalid address with a login: " + url);
|
||||
if (f.elements.username.value || f.elements.password.value) fail("guessed a login from an invalid address: " + url);
|
||||
}
|
||||
|
||||
for (const url of ["https://repo.example/repo", "not a url", "https://repo.example/a@b", "fdroidrepos:///a@b", "fdroidrepos:/a@b", "https://repo.example/r?x=a@b#c@d"]) {
|
||||
f = form(url);
|
||||
moveLogin(f);
|
||||
if (f.elements.url.value !== url || f.elements.username.value) fail("changed an address without a login: " + url);
|
||||
}
|
||||
|
||||
const html = sourceToken({ id: "github", name: "GitHub releases", token: { set: true, from: "saved", rejected: true } });
|
||||
if (!html.includes("rejected") || !html.includes('data-clear-token="github"')) fail("token row: " + html);
|
||||
if (sourceToken({ id: "fdroid", name: "F-Droid" }) !== "") fail("token field on a source without one");
|
||||
const slow = "https://a:" + "%FF".repeat(4000) + "@repo.example/r", t0 = Date.now();
|
||||
f = form(slow); moveLogin(f);
|
||||
if (Date.now() - t0 > 200 || f.elements.password.value !== "%FF".repeat(4000)) fail("decoding malformed escapes is slow or wrong");
|
||||
console.log("ok");
|
||||
@@ -325,7 +325,7 @@ class EndpointTests(SettingsTest):
|
||||
mod.add_repo.side_effect = SourceError('repository fingerprint mismatch')
|
||||
job = self.wait(self.request('POST', '/api/sources', {'action': 'add', 'url': link})[1]['job'])
|
||||
self.assertEqual(job['error'], 'repository fingerprint mismatch') # no "SourceError:" prefix
|
||||
for url in ('http://repo.example/repo', 'fdroidrepo://repo.example/repo', 'https://repo.example:x/'):
|
||||
for url in ('http://repo.example/repo', 'fdroidrepo://repo.example/repo', 'https://u@repo.example/'):
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'add', 'url': url})[0], 400)
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'remove', 'source': 'fdroid'})[0], 200)
|
||||
mod.remove_repo.assert_called_once_with(source_id='fdroid')
|
||||
|
||||
@@ -81,7 +81,7 @@ class Repositories(unittest.TestCase):
|
||||
state.clear()
|
||||
self.addCleanup(state.clear)
|
||||
|
||||
def fetch(self, url, path, maximum, auth=None):
|
||||
def fetch(self, url, path, maximum):
|
||||
name = url.rsplit('/', 1)[-1]
|
||||
if self.v1 and name == 'entry.jar':
|
||||
raise urllib.error.HTTPError(url, 404, 'missing', None, None)
|
||||
@@ -311,7 +311,7 @@ class Repositories(unittest.TestCase):
|
||||
source = self.add()
|
||||
other = dict(source, id='other-repo')
|
||||
started, release = threading.Event(), threading.Event()
|
||||
def fetch(url, path, maximum, auth=None):
|
||||
def fetch(url, path, maximum):
|
||||
if threading.current_thread().name == 'slow':
|
||||
started.set()
|
||||
release.wait(5)
|
||||
@@ -355,7 +355,7 @@ class Repositories(unittest.TestCase):
|
||||
self.files['entry.jar'], _ = entry_jar(50)
|
||||
source = fdroid.add_repo(URL)
|
||||
jars = {'older': entry_jar(100)[0], 'newer': entry_jar(200)[0]}
|
||||
def fetch(url, path, maximum, auth=None):
|
||||
def fetch(url, path, maximum):
|
||||
name = threading.current_thread().name
|
||||
if name in jars and url.endswith('entry.jar'):
|
||||
Path(path).write_bytes(jars[name])
|
||||
@@ -401,7 +401,7 @@ class Repositories(unittest.TestCase):
|
||||
source = dict(id='overlap', name='Overlap', url=URL, fingerprint=None)
|
||||
self.v1 = True
|
||||
inner = []
|
||||
def fetch(url, path, maximum, auth=None):
|
||||
def fetch(url, path, maximum):
|
||||
if url.endswith('index-v1.jar') and not inner:
|
||||
inner.append(1) # the v1 load passed its fallback check; a v2 load finishes now
|
||||
self.v1 = False
|
||||
@@ -471,7 +471,7 @@ class Repositories(unittest.TestCase):
|
||||
source = self.add()
|
||||
self.expire(source)
|
||||
release = threading.Event()
|
||||
def slow(url, path, maximum, auth=None):
|
||||
def slow(url, path, maximum):
|
||||
release.wait(5)
|
||||
self.fetch(url, path, maximum)
|
||||
self.fetch_mock.side_effect = slow
|
||||
@@ -528,7 +528,7 @@ class Repositories(unittest.TestCase):
|
||||
self.expire(source)
|
||||
before = self.fetch_mock.call_count
|
||||
release = __import__('threading').Event()
|
||||
def slow(url, path, maximum, auth=None):
|
||||
def slow(url, path, maximum):
|
||||
release.wait(5)
|
||||
self.fetch(url, path, maximum)
|
||||
self.fetch_mock.side_effect = slow
|
||||
|
||||
@@ -18,6 +18,7 @@ import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest import mock
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
@@ -631,3 +632,257 @@ class PageQueue(unittest.TestCase):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class PadEvents(unittest.TestCase):
|
||||
"""The trackpad and key row go through gamescope; accents and emoji go the KDE Connect way."""
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
import server
|
||||
cls.pad = staticmethod(server.pad_events)
|
||||
|
||||
def test_moves_clicks_and_scroll(self):
|
||||
out, extra = self.pad([{"dx": 3, "dy": -2}, {"singleclick": True}, {"singlehold": True},
|
||||
{"singlerelease": True}, {"scroll": True, "dy": 1}, {"rightclick": True}])
|
||||
self.assertEqual(out[0], {"dx": 3, "dy": -2})
|
||||
self.assertEqual(out[1:3], [{"button": "left", "down": True}, {"button": "left", "down": False}])
|
||||
self.assertEqual(out[3:5], [{"button": "left", "down": True}, {"button": "left", "down": False}])
|
||||
self.assertEqual(out[5], {"scroll": [0, -15]}) # up = content follows the finger
|
||||
self.assertEqual(out[6:], [{"button": "right", "down": True}, {"button": "right", "down": False}])
|
||||
self.assertEqual(extra, [])
|
||||
|
||||
def test_keys_and_modifiers(self):
|
||||
out, _ = self.pad([{"specialKey": 12}, {"specialKey": 1, "ctrl": True}, {"key": "c", "ctrl": True}])
|
||||
self.assertEqual(out[:2], [{"key": 28, "down": True}, {"key": 28, "down": False}])
|
||||
self.assertEqual([e["key"] for e in out[2:6]], [29, 14, 14, 29]) # Ctrl+Backspace
|
||||
self.assertEqual([(e["key"], e["down"]) for e in out[6:]],
|
||||
[(29, True), (46, True), (46, False), (29, False)]) # Ctrl+C
|
||||
|
||||
def test_text_stays_ascii_through_gamescope(self):
|
||||
out, slow = self.pad([{"key": "Hi "}, {"key": "there"}])
|
||||
self.assertEqual(out, [{"text": "Hi there"}])
|
||||
self.assertEqual(slow, [])
|
||||
|
||||
def test_keyboard_goes_to_kde_whole_and_in_order(self):
|
||||
events = [{"key": "a"}, {"key": "é"}, {"specialKey": 12}, {"key": "c", "ctrl": True}, {"dx": 2, "dy": 1}]
|
||||
out, slow = self.pad(events, kde=True)
|
||||
self.assertEqual(out, [{"dx": 2, "dy": 1}]) # the pointer still goes through gamescope
|
||||
self.assertEqual(slow, events[:4]) # the keyboard as sent, in order
|
||||
|
||||
def test_shifted_key_with_modifier(self):
|
||||
out, _ = self.pad([{"key": "A", "ctrl": True}])
|
||||
self.assertEqual([e["key"] for e in out], [29, 42, 30, 30, 42, 29])
|
||||
|
||||
|
||||
class PadDelivery(unittest.TestCase):
|
||||
"""remote_input's promises: "sent" means the whole batch is taken; accents go once, in order."""
|
||||
|
||||
class Agent:
|
||||
def __init__(self, sent=True, state="ready"):
|
||||
self.sent, self.state, self.got = sent, state, []
|
||||
|
||||
def send(self, events):
|
||||
if events and self.sent:
|
||||
self.got.append(events)
|
||||
return {"state": self.state, "sent": self.sent and bool(events)}
|
||||
|
||||
def setUp(self):
|
||||
import server
|
||||
self.s = server
|
||||
self.saved = (server._touch, server._input, server.KDE_WAIT)
|
||||
server._touch, server._input = self.Agent(), self.Agent()
|
||||
self.reset()
|
||||
|
||||
def tearDown(self):
|
||||
self.s._touch, self.s._input, self.s.KDE_WAIT = self.saved
|
||||
self.settle()
|
||||
self.reset()
|
||||
|
||||
def reset(self):
|
||||
self.s._kde_queue.clear()
|
||||
self.s._kde_until = self.s._typed_until = 0.0
|
||||
self.s._kde_worker[0] = None
|
||||
|
||||
def settle(self):
|
||||
worker = self.s._kde_worker[0]
|
||||
if worker:
|
||||
worker.join(5)
|
||||
|
||||
def typed(self):
|
||||
return [e for batch in self.s._input.got for e in batch]
|
||||
|
||||
def test_accent_only_batch_is_acknowledged_and_typed_once(self):
|
||||
status = self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.assertTrue(status["sent"]) # so the page doesn't send it again
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
|
||||
def test_mixed_text_stays_in_one_ordered_stream(self):
|
||||
self.s.remote_input({"events": [{"key": "aéb"}, {"specialKey": 12}]})
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "aéb"}, {"specialKey": 12}])
|
||||
self.assertEqual(self.s._touch.got, []) # none of it split off through gamescope
|
||||
|
||||
def test_batch_gamescope_didnt_take_is_not_queued(self):
|
||||
self.s._touch = self.Agent(sent=False, state="starting")
|
||||
status = self.s.remote_input({"events": [{"dx": 1, "dy": 1}, {"key": "é"}]})
|
||||
self.assertFalse(status["sent"]) # the page tries the whole batch again...
|
||||
self.assertEqual(self.s._kde_queue, []) # ...so the accent isn't queued twice
|
||||
|
||||
def test_waits_for_kde_connect_then_sends_without_more_input(self):
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
time.sleep(0.2)
|
||||
self.assertEqual(self.s._kde_queue, [{"key": "é"}])
|
||||
self.s._input.sent = True
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
|
||||
def test_later_ascii_waits_behind_a_pending_accent(self):
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.s.remote_input({"events": [{"key": "x"}]})
|
||||
self.assertEqual(self.s._touch.got, []) # "x" can't overtake the "é"
|
||||
self.assertEqual(self.s._kde_queue, [{"key": "é"}, {"key": "x"}])
|
||||
self.s._input.sent = True
|
||||
self.settle()
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
|
||||
|
||||
def test_queue_is_bounded_and_dropped_on_error(self):
|
||||
self.s._input = self.Agent(sent=False, state="error")
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.s._input = self.Agent(sent=False, state="starting")
|
||||
self.s.KDE_WAIT = 0.3
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
self.assertLessEqual(len(self.s._kde_queue), self.s.KDE_QUEUE_LIMIT)
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
|
||||
def test_ascii_just_after_an_accent_stays_behind_it(self):
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.settle()
|
||||
self.s.remote_input({"events": [{"key": "x"}]}) # KDE Connect may still be typing the é
|
||||
self.settle()
|
||||
self.assertEqual(self.s._touch.got, [])
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
|
||||
|
||||
def test_accent_just_after_gamescope_text_waits_for_it(self):
|
||||
self.s.remote_input({"events": [{"key": "x" * 50}]})
|
||||
self.assertEqual(self.s._touch.got, [[{"text": "x" * 50}]])
|
||||
start = time.time()
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
self.settle()
|
||||
self.assertGreater(time.time() - start, 0.6) # 50 keys at 16 ms, plus the margin
|
||||
self.assertEqual(self.typed(), [{"key": "é"}])
|
||||
|
||||
def test_typing_time_counts_every_key_transition(self):
|
||||
t = self.s.typing_seconds
|
||||
self.assertAlmostEqual(t([{"text": "ab"}]), 4 * 0.008)
|
||||
self.assertAlmostEqual(t([{"text": "A"}]), 4 * 0.008) # shift down, key down, key up, shift up
|
||||
self.assertAlmostEqual(t([{"text": "x" * 500}]), 8.0) # a long paste
|
||||
self.assertAlmostEqual(t([{"key": 29, "down": True}, {"dx": 3, "dy": 0}]), 0.008)
|
||||
|
||||
def test_the_margin_is_added_once_not_per_request(self):
|
||||
for _ in range(10):
|
||||
self.s.remote_input({"events": [{"key": "ab"}]})
|
||||
# ten requests of 4 transitions each: the work queues up, with no margin added per request
|
||||
self.assertLess(self.s._typed_until - time.time(), 10 * 4 * 0.008 + 0.05)
|
||||
|
||||
def test_trimming_while_sending_doesnt_drop_unsent_events(self):
|
||||
slow = self.Agent()
|
||||
gate = threading.Event()
|
||||
send = slow.send
|
||||
|
||||
def held(events):
|
||||
gate.wait(5)
|
||||
return send(events)
|
||||
slow.send = held
|
||||
self.s._input = slow
|
||||
self.s.remote_input({"events": [{"key": "é"}] * 150})
|
||||
time.sleep(0.2) # the worker has taken its 150 and is mid-send
|
||||
self.s.remote_input({"events": [{"key": "ü"}] * 150})
|
||||
gate.set()
|
||||
self.settle()
|
||||
sent = self.typed()
|
||||
self.assertEqual(sent[:150], [{"key": "é"}] * 150)
|
||||
self.assertEqual(sent[150:], [{"key": "ü"}] * 150) # none of the new ones went missing
|
||||
|
||||
def test_input_queued_as_the_worker_finishes_is_still_sent(self):
|
||||
for _ in range(40):
|
||||
self.reset()
|
||||
self.s._input.got.clear()
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
time.sleep(0.002)
|
||||
self.s.remote_input({"events": [{"key": "ü"}]})
|
||||
self.settle()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "ü"}])
|
||||
|
||||
|
||||
class PadRoute(unittest.TestCase):
|
||||
def setUp(self):
|
||||
import server
|
||||
self.s = server
|
||||
self.agents = mock.patch.multiple(server, _touch=mock.Mock(), _input=mock.Mock())
|
||||
self.agents.start()
|
||||
self.addCleanup(self.agents.stop)
|
||||
self.s._touch.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s._input.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s.reset_pad_route()
|
||||
self.s._touch.reset_mock(); self.s._input.reset_mock()
|
||||
self.addCleanup(self.s.reset_pad_route)
|
||||
|
||||
def switch(self):
|
||||
with mock.patch.multiple(self.s, FRAME="frame-a", HOST_OPTS=[], MUX=[], SSH=[], macview=None), \
|
||||
mock.patch.multiple(self.s.frame_android, FRAME="frame-a", SSH_OPTS=[]):
|
||||
self.s.route("frame-b", [])
|
||||
|
||||
def test_switch_stops_both_input_agents_and_discards_old_queue(self):
|
||||
self.s._kde_queue.append({"key": "old"})
|
||||
self.s._kde_until = self.s._typed_until = time.time() + 60
|
||||
cancel = self.s._kde_cancel
|
||||
self.switch()
|
||||
self.s._input.stop.assert_called_once()
|
||||
self.s._touch.stop.assert_called_once()
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
self.assertEqual((self.s._kde_until, self.s._typed_until), (0, 0))
|
||||
self.assertTrue(cancel.is_set())
|
||||
|
||||
def test_switch_cancels_delayed_and_retrying_text_before_new_input(self):
|
||||
for delaying in (True, False):
|
||||
with self.subTest(delaying=delaying):
|
||||
waiting, cancel = threading.Event(), threading.Event()
|
||||
class ObservedCancel:
|
||||
def wait(self, timeout):
|
||||
waiting.set()
|
||||
return cancel.wait(timeout)
|
||||
def set(self):
|
||||
cancel.set()
|
||||
self.s.reset_pad_route()
|
||||
self.s._kde_cancel = ObservedCancel()
|
||||
self.s._typed_until = time.time() + 60 if delaying else 0
|
||||
self.s._input.reset_mock()
|
||||
self.s._input.send.return_value = {"state": "starting", "sent": False}
|
||||
self.s.remote_input({"events": [{"key": "é"}]})
|
||||
worker = self.s._kde_worker[0]
|
||||
self.assertTrue(waiting.wait(1), "worker never reached its cancellable delay")
|
||||
self.switch()
|
||||
worker.join(1)
|
||||
self.assertFalse(worker.is_alive())
|
||||
attempts = self.s._input.send.call_count
|
||||
self.assertEqual(attempts, 0 if delaying else 1)
|
||||
self.s._input.send.return_value = {"state": "ready", "sent": True}
|
||||
self.s.remote_input({"events": [{"key": "ü"}]})
|
||||
next_worker = self.s._kde_worker[0]
|
||||
if next_worker:
|
||||
next_worker.join(1)
|
||||
self.assertFalse(next_worker.is_alive())
|
||||
self.assertEqual(self.s._input.send.call_count, attempts + 1)
|
||||
self.assertEqual(self.s._input.send.call_args.args[0], [{"key": "ü"}])
|
||||
self.assertEqual(self.s._kde_queue, [])
|
||||
@@ -1,566 +0,0 @@
|
||||
"""Repositories and sources behind a login: F-Droid basic auth and GitHub tokens.
|
||||
|
||||
A fake HTTPS layer stands in for the network (urllib's real redirect, error and
|
||||
request processing still run), so these tests see every request and its headers.
|
||||
"""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import base64
|
||||
import contextlib
|
||||
import http.client
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
import urllib.request
|
||||
import urllib.response
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
|
||||
from apk_sources import SourceAuthError, SourceError, SourceLimited, _credentials, _images, _web, fdroid, github, search
|
||||
import server
|
||||
|
||||
FIXTURES = Path(__file__).parent / 'fixtures' / 'fdroid'
|
||||
PIN = (FIXTURES / 'fingerprint.txt').read_text().strip()
|
||||
REPO = 'https://repo.example/fdroid/repo/'
|
||||
USER, PASSWORD = 'alice', 's3cr@t-pässword'
|
||||
TOKEN = 'ghp_frameControlTestToken0123456789'
|
||||
SECRETS = (PASSWORD, TOKEN, base64.b64encode((USER + ':' + PASSWORD).encode()).decode())
|
||||
REAL_BUILD_OPENER = urllib.request.build_opener
|
||||
|
||||
|
||||
def basic(user, password):
|
||||
return 'Basic ' + base64.b64encode((user + ':' + password).encode()).decode()
|
||||
|
||||
|
||||
class FakeHTTPS(urllib.request.HTTPSHandler):
|
||||
"""Answers https requests from route(url, authorization) -> (status, headers, body)."""
|
||||
|
||||
def __init__(self, route):
|
||||
super().__init__()
|
||||
self.route, self.seen = route, []
|
||||
|
||||
def https_open(self, req):
|
||||
auth = req.get_header('Authorization')
|
||||
self.seen.append((req.full_url, auth))
|
||||
status, headers, body = self.route(req.full_url, auth)
|
||||
message = http.client.HTTPMessage()
|
||||
for key, value in headers.items():
|
||||
message[key] = value
|
||||
response = urllib.response.addinfourl(io.BytesIO(body), message, req.full_url, status)
|
||||
response.msg = 'fake'
|
||||
return response
|
||||
|
||||
|
||||
class Base(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.root = Path(tmp.name)
|
||||
for name, value in [('data_dir', lambda *p: self.root.joinpath('data', *p)),
|
||||
('cache_dir', lambda *p: self.root.joinpath('cache', *p))]:
|
||||
mock = patch.object(fdroid.frame_host, name, value)
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
self.web = FakeHTTPS(self.route)
|
||||
mock = patch('urllib.request.build_opener', lambda *handlers: REAL_BUILD_OPENER(*handlers, self.web))
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
mock = patch.dict(os.environ, {github.ENV: ''})
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
for state in (_web._limited, fdroid._stale, fdroid._retry_at, fdroid._refreshing, github._rejected):
|
||||
state.clear()
|
||||
self.addCleanup(state.clear)
|
||||
self.stderr = io.StringIO()
|
||||
mock = patch('sys.stderr', self.stderr)
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
|
||||
def route(self, url, auth):
|
||||
raise AssertionError('unexpected request ' + url)
|
||||
|
||||
def assertNoSecrets(self, *texts):
|
||||
for text in texts:
|
||||
for secret in SECRETS + (USER + ':',):
|
||||
self.assertNotIn(secret, text)
|
||||
|
||||
def disk(self):
|
||||
"""Everything saved under the data and cache folders except the credentials file."""
|
||||
out = []
|
||||
for path in self.root.rglob('*'):
|
||||
if path.is_file() and path.name != 'source-credentials.json':
|
||||
out.append(path.read_bytes().decode('utf-8', 'replace'))
|
||||
return '\n'.join(out)
|
||||
|
||||
def credentials_file(self):
|
||||
return self.root / 'data' / 'source-credentials.json'
|
||||
|
||||
|
||||
class Repository(Base):
|
||||
"""A repository at REPO that wants USER and PASSWORD."""
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.login = basic(USER, PASSWORD)
|
||||
self.redirect = None # file name -> absolute URL it redirects to
|
||||
self.forbidden = 401
|
||||
|
||||
def serve(self, name):
|
||||
return (FIXTURES / ('example.apk' if name.endswith('.apk') else name)).read_bytes()
|
||||
|
||||
def route(self, url, auth):
|
||||
for mirror in ('https://mirror.example/fdroid/repo/', 'https://repo.example/elsewhere/'):
|
||||
if url.startswith(mirror): # serves anyone; a login must never get here unless same-origin
|
||||
return 200, {}, self.serve(url[len(mirror):])
|
||||
self.assertTrue(url.startswith(REPO), url)
|
||||
if auth != self.login:
|
||||
return self.forbidden, {'WWW-Authenticate': 'Basic realm="repo"'}, b'no'
|
||||
name = url[len(REPO):]
|
||||
if self.redirect and name in self.redirect:
|
||||
return 302, {'Location': self.redirect[name]}, b''
|
||||
return 200, {}, self.serve(name)
|
||||
|
||||
def add(self, **kw):
|
||||
return fdroid.add_repo('https://alice:s3cr%40t-p%C3%A4ssword@repo.example/fdroid/repo', PIN, **kw)
|
||||
|
||||
|
||||
class RepositoryLogin(Repository):
|
||||
def test_add_search_download_with_login_in_the_url(self):
|
||||
source = self.add()
|
||||
self.assertEqual(source['url'], REPO)
|
||||
self.assertEqual(source['fingerprint'], PIN)
|
||||
saved = fdroid.user_repos()[0]
|
||||
self.assertEqual(saved['url'], REPO)
|
||||
self.assertNotIn('login', saved)
|
||||
self.assertEqual(len(fdroid.search(source, 'example offline')), 1)
|
||||
downloaded = fdroid.download(source, 'org.example.app', 1)
|
||||
self.assertTrue(downloaded['verified'])
|
||||
self.assertTrue(self.web.seen)
|
||||
self.assertTrue(all(auth == self.login for url, auth in self.web.seen))
|
||||
self.assertTrue(all('@' not in url for url, auth in self.web.seen)) # never in the URL
|
||||
self.assertTrue(next(s for s in fdroid.sources() if s['id'] == source['id'])['login'])
|
||||
self.assertNoSecrets(json.dumps(source), json.dumps(fdroid.sources()), self.disk(), self.stderr.getvalue())
|
||||
# The settings file holds no part of the login; the credentials file holds it, privately.
|
||||
settings = (self.root / 'data' / 'apk-repos.json').read_text()
|
||||
self.assertNotIn(USER, settings)
|
||||
self.assertTrue(self.credentials_file().is_file())
|
||||
if os.name != 'nt':
|
||||
self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
|
||||
|
||||
def test_login_given_apart_and_saved_login_reused_on_refresh(self):
|
||||
source = fdroid.add_repo(REPO, PIN, username=USER, password=PASSWORD)
|
||||
self.assertEqual(source['url'], REPO)
|
||||
self.web.seen.clear()
|
||||
fdroid._load(fdroid.user_repos()[0], force=True) # as a background refresh would
|
||||
self.assertTrue(self.web.seen)
|
||||
self.assertTrue(all(auth == self.login for url, auth in self.web.seen))
|
||||
fdroid.add_repo(REPO) # re-adding without a login keeps the saved one
|
||||
self.assertTrue(_credentials.has(source['id']))
|
||||
|
||||
def test_wrong_or_missing_login_saves_nothing(self):
|
||||
for kw, message in [({'username': USER, 'password': 'wrong'}, 'check the username and password'),
|
||||
({}, 'needs a username and password')]:
|
||||
with self.subTest(kw=kw):
|
||||
with self.assertRaisesRegex(SourceAuthError, message) as caught:
|
||||
fdroid.add_repo(REPO, PIN, **kw)
|
||||
self.assertNoSecrets(str(caught.exception), repr(caught.exception))
|
||||
self.assertNotIn('wrong', str(caught.exception))
|
||||
self.assertEqual(fdroid.user_repos(), [])
|
||||
self.assertFalse(self.credentials_file().exists())
|
||||
self.assertFalse((self.root / 'data' / 'apk-repo-state.json').exists())
|
||||
self.assertEqual(_web._limited, {}) # an auth failure isn't a rate limit
|
||||
|
||||
def test_403_with_a_login_is_a_login_error_not_a_rate_limit(self):
|
||||
self.forbidden = 403
|
||||
with self.assertRaisesRegex(SourceAuthError, 'check the username and password'):
|
||||
fdroid.add_repo(REPO, PIN, username=USER, password='wrong')
|
||||
self.assertEqual(_web._limited, {})
|
||||
self.assertEqual(fdroid.user_repos(), [])
|
||||
|
||||
def test_signature_checks_still_apply(self):
|
||||
with self.assertRaisesRegex(SourceError, 'fingerprint mismatch'):
|
||||
fdroid.add_repo(REPO, '0' * 64, username=USER, password=PASSWORD)
|
||||
self.assertEqual(fdroid.user_repos(), [])
|
||||
self.assertFalse(self.credentials_file().exists())
|
||||
|
||||
def test_cross_origin_redirect_never_gets_the_login(self):
|
||||
self.redirect = {'entry.jar': 'https://mirror.example/fdroid/repo/entry.jar',
|
||||
'index-v2.json': 'https://repo.example/elsewhere/index-v2.json'}
|
||||
source = self.add()
|
||||
self.assertEqual(source['fingerprint'], PIN) # the mirror's copy still had to verify
|
||||
mirror = [auth for url, auth in self.web.seen if url.startswith('https://mirror.example/')]
|
||||
same_origin = [auth for url, auth in self.web.seen if url.startswith('https://repo.example/elsewhere/')]
|
||||
self.assertEqual(mirror, [None])
|
||||
self.assertEqual(same_origin, [None]) # same origin but outside the repository: no login either
|
||||
|
||||
def test_traversal_names_are_refused_and_never_get_the_login(self):
|
||||
for name in ('%252e%252e/%252e%252e/evil/collect.apk', '..;/..;/evil/collect.apk', 'a%2f..%2f..%2fevil.apk',
|
||||
'a%5c..%5cevil.apk', '%2e%2e/evil.apk', 'x;y.apk', '100%25.apk'):
|
||||
with self.subTest(name=name), self.assertRaisesRegex(SourceError, 'unsafe|outside'):
|
||||
fdroid._child(REPO, name)
|
||||
self.assertEqual(fdroid._child(REPO, '/app/en-US/phoneScreenshots/#0 a.png'),
|
||||
REPO + 'app/en-US/phoneScreenshots/%230%20a.png')
|
||||
# Even if such a URL reached the downloader directly, the login stays home.
|
||||
login = _credentials.basic(REPO, USER, PASSWORD)
|
||||
for url in (REPO + '%252e%252e/%252e%252e/evil/collect.apk', REPO + '..;/..;/evil/collect.apk'):
|
||||
with self.subTest(url=url), tempfile.TemporaryDirectory() as tmp:
|
||||
self.web.seen.clear()
|
||||
with contextlib.suppress(SourceError): # the repository answers 401 without the login
|
||||
fdroid._fetch(url, Path(tmp) / 'out', 1 << 20, auth=login)
|
||||
self.assertEqual(self.web.seen, [(url, None)])
|
||||
|
||||
def test_same_origin_redirect_outside_the_repository_gets_no_login(self):
|
||||
self.redirect = {'example1.apk': 'https://repo.example/elsewhere/example1.apk'}
|
||||
source = self.add()
|
||||
self.web.seen.clear()
|
||||
self.assertTrue(fdroid.download(source, 'org.example.app', 1)['verified'])
|
||||
self.assertEqual(self.web.seen, [(REPO + 'example1.apk', self.login),
|
||||
('https://repo.example/elsewhere/example1.apk', None)])
|
||||
|
||||
def test_other_ports_and_http_never_get_the_login(self):
|
||||
login = _credentials.basic(REPO, USER, PASSWORD)
|
||||
for url in ('https://repo.example:8443/fdroid/repo/x', 'http://repo.example/fdroid/repo/x',
|
||||
'https://other.example/fdroid/repo/x', 'https://repo.example.evil/x'):
|
||||
self.assertFalse(login.matches(url), url)
|
||||
self.assertTrue(login.matches('https://REPO.example:443/anything'))
|
||||
self.assertNoSecrets(repr(login))
|
||||
handler = _credentials.Handler(login)
|
||||
req = urllib.request.Request('https://other.example/x', headers={'Authorization': 'Basic stale'})
|
||||
self.assertIsNone(handler.https_request(req).get_header('Authorization'))
|
||||
|
||||
def test_http_and_bad_logins_are_refused_before_any_request(self):
|
||||
for url, kw in [('http://alice:pw@repo.example/fdroid/repo', {}),
|
||||
('http://repo.example/fdroid/repo', {'username': USER, 'password': PASSWORD}),
|
||||
(REPO, {'username': 'a:b', 'password': PASSWORD}),
|
||||
(REPO, {'username': USER, 'password': 'x\r\nX-Evil: 1'}),
|
||||
(REPO, {'password': PASSWORD}),
|
||||
('https://alice:pw@repo.example/fdroid/repo', {'username': 'bob'})]:
|
||||
with self.subTest(url=url, kw=kw), self.assertRaises(SourceError) as caught:
|
||||
fdroid.add_repo(url, PIN, **kw)
|
||||
self.assertNotIn('pw', str(caught.exception).replace('password', ''))
|
||||
self.assertNoSecrets(str(caught.exception))
|
||||
self.assertEqual(self.web.seen, [])
|
||||
|
||||
def test_remove_deletes_the_login(self):
|
||||
source = self.add()
|
||||
other = _credentials.bearer('https://api.github.com/', TOKEN)
|
||||
_credentials.put('github', other)
|
||||
fdroid.remove_repo(source['id'])
|
||||
self.assertFalse(_credentials.has(source['id']))
|
||||
self.assertNotIn(PASSWORD, self.credentials_file().read_text())
|
||||
_credentials.remove('github')
|
||||
self.assertFalse(self.credentials_file().exists()) # nothing left: the file goes
|
||||
|
||||
def test_cli_reads_the_password_from_stdin(self):
|
||||
out = io.StringIO()
|
||||
argv = ['fdroid.py', 'add', REPO, '--fingerprint', PIN, '--username', USER, '--password-stdin']
|
||||
with patch.object(sys, 'argv', argv), patch.object(sys, 'stdin', io.StringIO(PASSWORD + '\n')), \
|
||||
contextlib.redirect_stdout(out):
|
||||
fdroid.main()
|
||||
self.assertEqual(json.loads(out.getvalue())['url'], REPO)
|
||||
self.assertNoSecrets(out.getvalue())
|
||||
self.assertTrue(_credentials.has(fdroid.user_repos()[0]['id']))
|
||||
|
||||
def test_artwork_from_the_repository_gets_the_login_and_only_there(self):
|
||||
source = self.add()
|
||||
self.assertEqual(_credentials.for_url(REPO + 'icons/a.png').header(), self.login)
|
||||
self.assertTrue(_credentials.has(source['id']))
|
||||
self.assertIsNone(_credentials.for_url('https://repo.example/other/a.png')) # outside the repository
|
||||
self.assertIsNone(_credentials.for_url('https://cdn.example/fdroid/repo/a.png'))
|
||||
|
||||
|
||||
class GitHubToken(Base):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.accept = {TOKEN}
|
||||
self.releases = json.dumps([]).encode()
|
||||
self.anonymous_limited = False
|
||||
|
||||
def route(self, url, auth):
|
||||
if url.startswith('https://api.github.com/'):
|
||||
if auth is not None and auth not in {'Bearer ' + t for t in self.accept}:
|
||||
return 401, {}, b'{"message": "Bad credentials"}'
|
||||
if auth is None and self.anonymous_limited:
|
||||
return 403, {'X-RateLimit-Remaining': '0', 'X-RateLimit-Reset': str(int(time.time()) + 1800)}, b'{}'
|
||||
if url.endswith('/redirected'):
|
||||
return 302, {'Location': 'https://github.com/landed'}, b''
|
||||
return 200, {}, self.releases
|
||||
if url == 'https://github.com/landed':
|
||||
return 200, {}, b'{"ok": true}'
|
||||
raise AssertionError('unexpected request ' + url)
|
||||
|
||||
def test_saved_token_goes_only_to_the_api(self):
|
||||
github.set_token(TOKEN)
|
||||
self.assertEqual(github._api('/repos/a/b/releases'), [])
|
||||
self.assertEqual(self.web.seen, [('https://api.github.com/repos/a/b/releases', 'Bearer ' + TOKEN)])
|
||||
auth = github._credential(github._token()[0])
|
||||
self.assertEqual(_web.read('https://api.github.com/redirected', ('api.github.com', 'github.com'), auth=auth),
|
||||
b'{"ok": true}')
|
||||
self.assertEqual(self.web.seen[-2:], [('https://api.github.com/redirected', 'Bearer ' + TOKEN),
|
||||
('https://github.com/landed', None)]) # redirect: no token
|
||||
status = github.sources()[0]['token']
|
||||
self.assertEqual(status, {'set': True, 'from': 'saved', 'rejected': False})
|
||||
self.assertNoSecrets(json.dumps(github.sources()), self.disk(), self.stderr.getvalue())
|
||||
if os.name != 'nt':
|
||||
self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
|
||||
|
||||
def test_rejected_token_retries_once_without_it(self):
|
||||
github.set_token(TOKEN)
|
||||
self.accept = set()
|
||||
self.assertEqual(github._api('/repos/a/b/releases'), [])
|
||||
self.assertEqual([auth for _, auth in self.web.seen], ['Bearer ' + TOKEN, None])
|
||||
self.assertTrue(github.token_status()['rejected'])
|
||||
self.assertIn('rejected', self.stderr.getvalue())
|
||||
self.assertNoSecrets(self.stderr.getvalue())
|
||||
self.web.seen.clear()
|
||||
github._api('/repos/a/b/other')
|
||||
self.assertEqual([auth for _, auth in self.web.seen], [None]) # not tried again this run
|
||||
github.set_token(TOKEN) # saving it again gives it a fresh chance
|
||||
self.assertFalse(github.token_status()['rejected'])
|
||||
|
||||
def test_environment_variable_wins(self):
|
||||
github.set_token(TOKEN)
|
||||
env = 'ghp_fromTheEnvironment'
|
||||
self.accept = {env}
|
||||
with patch.dict(os.environ, {github.ENV: env}):
|
||||
github._api('/repos/a/b/releases')
|
||||
self.assertEqual(self.web.seen[-1][1], 'Bearer ' + env)
|
||||
self.assertEqual(github.token_status(), {'set': True, 'from': 'env', 'rejected': False})
|
||||
self.assertEqual(github.token_status()['from'], 'saved')
|
||||
|
||||
def test_clear_removes_it_from_disk(self):
|
||||
github.set_token(TOKEN)
|
||||
github.clear_token()
|
||||
self.assertFalse(self.credentials_file().exists())
|
||||
self.assertEqual(github.token_status(), {'set': False, 'from': None, 'rejected': False})
|
||||
github._api('/repos/a/b/releases')
|
||||
self.assertEqual(self.web.seen[-1][1], None)
|
||||
|
||||
def test_saving_a_token_escapes_the_anonymous_rate_limit(self):
|
||||
self.anonymous_limited = True
|
||||
with self.assertRaisesRegex(SourceLimited, 'GitHub is limiting requests'):
|
||||
github._api('/repos/a/b/releases')
|
||||
self.assertEqual(self.web.seen[-1][1], None)
|
||||
github.set_token(TOKEN)
|
||||
self.assertEqual(github._api('/repos/a/b/other'), []) # tried, with the token's own quota
|
||||
self.assertEqual(self.web.seen[-1], ('https://api.github.com/repos/a/b/other', 'Bearer ' + TOKEN))
|
||||
github.clear_token()
|
||||
count = len(self.web.seen)
|
||||
with self.assertRaisesRegex(SourceLimited, 'GitHub is limiting requests'):
|
||||
github._api('/repos/a/b/third')
|
||||
self.assertEqual(len(self.web.seen), count) # anonymous is still backing off: no request
|
||||
|
||||
def test_bad_tokens_are_refused(self):
|
||||
for token in ('', ' ', 'two words', 'x\nAuthorization: y'):
|
||||
with self.subTest(token=token), self.assertRaises(SourceError):
|
||||
github.set_token(token)
|
||||
self.assertFalse(self.credentials_file().exists())
|
||||
|
||||
|
||||
PNG = b'\x89PNG\r\n\x1a\nfixture'
|
||||
PRIVATE = 'https://repo.example/private/repo/'
|
||||
TRAVERSALS = [PRIVATE + '%2e%2e/%2e%2e/evil/collect.png', PRIVATE + '%2E%2E/x.png', PRIVATE + '../../evil/x.png',
|
||||
PRIVATE + './icons/a.png', PRIVATE + '..%2f..%2fevil/x.png', PRIVATE + 'a%5c..%5c..%5cevil.png',
|
||||
PRIVATE + 'a;b/../../evil.png', PRIVATE + '%252e%252e/%252e%252e/evil.png',
|
||||
'https://repo.example/private/repository/x.png', 'https://repo.example/private/repo%2fx.png']
|
||||
|
||||
|
||||
class ArtworkScope(Base):
|
||||
"""The artwork proxy: actual outgoing headers, through _images.get's own HTTP code."""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.sent, self.redirects = [], {}
|
||||
_credentials.put('fdroid-user-private', _credentials.basic(PRIVATE, USER, PASSWORD))
|
||||
test = self
|
||||
|
||||
class Response:
|
||||
def __init__(self, status, location=''):
|
||||
self.status, self.location, self.body = status, location, PNG if status == 200 else b''
|
||||
|
||||
def getheader(self, name, default=''):
|
||||
return self.location if name == 'Location' else default
|
||||
|
||||
def read1(self, n):
|
||||
chunk, self.body = self.body[:n], self.body[n:]
|
||||
return chunk
|
||||
|
||||
class Connection:
|
||||
def __init__(self, host, port, timeout=None):
|
||||
self.host, self.port = host, port
|
||||
|
||||
def request(self, method, path, headers):
|
||||
url = 'https://%s%s' % (self.host, path)
|
||||
test.sent.append((url, headers.get('Authorization')))
|
||||
self.url = url
|
||||
|
||||
def getresponse(self):
|
||||
if self.url in test.redirects:
|
||||
return Response(302, test.redirects[self.url])
|
||||
return Response(200)
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
# A public address, no real socket or TLS, and Connection in place of http.client's.
|
||||
for target, name, value in [(_images, '_resolve', lambda host, port, timeout: [(2, 1, 6, '', ('93.184.216.34', port))]),
|
||||
(_images.socket, 'create_connection', lambda *a, **k: Mock()),
|
||||
(_images.ssl, 'create_default_context', lambda: Mock()),
|
||||
(_images.http.client, 'HTTPConnection', Connection)]:
|
||||
mock = patch.object(target, name, value)
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
|
||||
def test_artwork_in_the_repository_gets_the_login(self):
|
||||
self.assertEqual(_images.get(PRIVATE + 'icons/a%20b.png'), PNG)
|
||||
self.assertEqual(self.sent, [(PRIVATE + 'icons/a%20b.png', basic(USER, PASSWORD))])
|
||||
|
||||
def test_paths_a_server_could_normalise_outside_the_repository_get_nothing(self):
|
||||
for url in TRAVERSALS:
|
||||
with self.subTest(url=url):
|
||||
self.sent.clear()
|
||||
self.assertEqual(_images.get(url), PNG)
|
||||
self.assertEqual(self.sent, [(url, None)])
|
||||
self.assertIsNone(_credentials.for_url(url))
|
||||
|
||||
def test_redirect_to_a_traversal_gets_nothing(self):
|
||||
start = PRIVATE + 'icons/a.png'
|
||||
self.redirects[start] = PRIVATE + '%2e%2e/%2e%2e/evil/collect.png'
|
||||
self.assertEqual(_images.get(start), PNG)
|
||||
self.assertEqual(self.sent, [(start, basic(USER, PASSWORD)), (self.redirects[start], None)])
|
||||
|
||||
|
||||
class ScopedHandler(Base):
|
||||
"""Handler(scoped=True), for callers (artwork-like) that must stay inside a scope, redirects included."""
|
||||
|
||||
def route(self, url, auth):
|
||||
if url == PRIVATE + 'icons/a.png':
|
||||
return 302, {'Location': PRIVATE + '%2e%2e/%2e%2e/evil/collect.png'}, b''
|
||||
return 200, {}, PNG
|
||||
|
||||
def test_redirect_into_a_traversal_is_stripped(self):
|
||||
login = _credentials.basic(PRIVATE, USER, PASSWORD)
|
||||
opener = urllib.request.build_opener(_credentials.Handler(login, scoped=True))
|
||||
with opener.open(PRIVATE + 'icons/a.png') as r:
|
||||
self.assertEqual(r.read(), PNG)
|
||||
self.assertEqual(self.web.seen, [(PRIVATE + 'icons/a.png', basic(USER, PASSWORD)),
|
||||
(PRIVATE + '%2e%2e/%2e%2e/evil/collect.png', None)])
|
||||
for url in TRAVERSALS:
|
||||
self.assertFalse(_credentials.in_scope(login, url), url)
|
||||
self.assertTrue(_credentials.in_scope(login, PRIVATE + 'a/b%20c.png?x=1'))
|
||||
self.assertTrue(_credentials.in_scope(login, PRIVATE.rstrip('/')))
|
||||
|
||||
|
||||
CHURN = '''
|
||||
import sys
|
||||
sys.path.insert(0, sys.argv[1])
|
||||
from apk_sources import _credentials
|
||||
for i in range(int(sys.argv[3])):
|
||||
key = sys.argv[2] + str(i)
|
||||
_credentials.put(key, _credentials.basic('https://h.example/', 'u', 'p'))
|
||||
if i % 2:
|
||||
_credentials.remove(key)
|
||||
'''
|
||||
|
||||
|
||||
class TwoProcesses(Base):
|
||||
def test_interleaved_saves_and_removals_are_not_lost(self):
|
||||
ui = str(Path(__file__).resolve().parents[1] / 'ui')
|
||||
env = dict(os.environ, FRAME_CONTROL_DATA_DIR=str(self.root / 'data'))
|
||||
count = 60
|
||||
procs = [subprocess.Popen([sys.executable, '-c', CHURN, ui, prefix, str(count)], env=env,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE) for prefix in ('a', 'b')]
|
||||
for p in procs:
|
||||
out, err = p.communicate(timeout=120)
|
||||
self.assertEqual(p.returncode, 0, err.decode())
|
||||
saved = set(json.loads(self.credentials_file().read_text()))
|
||||
self.assertEqual(saved, {p + str(i) for p in 'ab' for i in range(0, count, 2)})
|
||||
if os.name != 'nt':
|
||||
self.assertEqual(stat.S_IMODE(self.credentials_file().stat().st_mode), 0o600)
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which('node'), 'needs node')
|
||||
class PageLogin(unittest.TestCase):
|
||||
def test_login_leaves_the_address_field_before_sending(self):
|
||||
"""tests/page/source_login.mjs runs the real page functions from index.html."""
|
||||
r = subprocess.run(['node', str(Path(__file__).parent / 'page' / 'source_login.mjs')],
|
||||
capture_output=True, text=True)
|
||||
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
|
||||
self.assertIn('ok', r.stdout)
|
||||
|
||||
|
||||
class Endpoints(Repository):
|
||||
"""The server API: logins in, never out, in responses or the server log."""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
mock = patch.object(search, 'modules', return_value=([fdroid, github], []))
|
||||
mock.start()
|
||||
self.addCleanup(mock.stop)
|
||||
self.httpd = server.ThreadingHTTPServer(('127.0.0.1', 0), server.Handler)
|
||||
threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
|
||||
self.addCleanup(self.httpd.server_close)
|
||||
self.addCleanup(self.httpd.shutdown)
|
||||
|
||||
def request(self, method, path, body=None):
|
||||
c = http.client.HTTPConnection('127.0.0.1', self.httpd.server_port)
|
||||
c.request(method, path, json.dumps(body) if body is not None else None,
|
||||
{'X-Frame-UI': '1', 'Content-Type': 'application/json'})
|
||||
r = c.getresponse()
|
||||
raw = r.read().decode()
|
||||
c.close()
|
||||
self.replies.append(raw)
|
||||
return r.status, json.loads(raw)
|
||||
|
||||
def job(self, reply):
|
||||
for _ in range(500):
|
||||
job = self.request('GET', '/api/job?id=' + reply['job'])[1]
|
||||
if job['done']:
|
||||
return job
|
||||
time.sleep(.01)
|
||||
self.fail('job did not finish')
|
||||
|
||||
def test_add_with_login_and_token_settings(self):
|
||||
self.replies = []
|
||||
bad = self.job(self.request('POST', '/api/sources', {'action': 'add', 'url': REPO, 'fingerprint': PIN,
|
||||
'username': USER, 'password': 'wrong'})[1])
|
||||
self.assertIn('check the username and password', bad['error'])
|
||||
self.assertEqual(fdroid.user_repos(), [])
|
||||
code, reply = self.request('POST', '/api/sources', {'action': 'add', 'url': REPO, 'fingerprint': PIN,
|
||||
'username': USER, 'password': PASSWORD})
|
||||
self.assertEqual(code, 200)
|
||||
job = self.job(reply)
|
||||
self.assertIsNone(job['error'])
|
||||
listed = self.request('GET', '/api/sources')[1]['sources']
|
||||
repo = next(s for s in listed if s['id'] == job['result']['source']['id'])
|
||||
self.assertEqual((repo['url'], repo['login']), (REPO, True))
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'set-token', 'source': 'github',
|
||||
'token': TOKEN})[0], 200)
|
||||
github_row = next(s for s in self.request('GET', '/api/sources')[1]['sources'] if s['id'] == 'github')
|
||||
self.assertEqual(github_row['token'], {'set': True, 'from': 'saved', 'rejected': False})
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'set-token', 'source': 'fdroid',
|
||||
'token': TOKEN})[0], 400)
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'clear-token', 'source': 'github'})[0], 200)
|
||||
self.assertFalse(_credentials.has('github'))
|
||||
self.assertEqual(self.request('POST', '/api/sources', {'action': 'remove', 'source': repo['id']})[0], 200)
|
||||
self.assertFalse(self.credentials_file().exists())
|
||||
self.assertNoSecrets(*self.replies, self.stderr.getvalue(), self.disk())
|
||||
|
||||
def test_login_in_the_url_is_accepted(self):
|
||||
self.replies = []
|
||||
url = 'https://alice:s3cr%40t-p%C3%A4ssword@repo.example/fdroid/repo'
|
||||
job = self.job(self.request('POST', '/api/sources', {'action': 'add', 'url': url, 'fingerprint': PIN})[1])
|
||||
self.assertIsNone(job['error'])
|
||||
self.assertEqual(fdroid.user_repos()[0]['url'], REPO)
|
||||
self.assertNoSecrets(*self.replies, self.stderr.getvalue())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -150,6 +150,14 @@ class Apply(unittest.TestCase):
|
||||
self.assertFalse(self.t.STALE[0])
|
||||
self.assertEqual(self.said[-1][1].get("stale"), None)
|
||||
|
||||
def test_stale_survives_a_bare_release(self):
|
||||
gs = FakeGamescope()
|
||||
panel = self.t.apply(gs, {"button": "left", "down": True, "window": 99, "display": ":1"}, None)
|
||||
self.assertTrue(self.t.STALE[0])
|
||||
self.t.apply(gs, {"button": "left", "down": False}, panel) # the page's release names no panel
|
||||
self.assertTrue(self.t.STALE[0])
|
||||
self.assertEqual(self.said[-1][1].get("stale"), True)
|
||||
|
||||
def test_same_window_id_on_the_other_display_is_another_panel(self):
|
||||
gs = FakeGamescope()
|
||||
self.t.apply(gs, {"fx": 0.5, "fy": 0.5, "window": 7, "display": ":0"}, None)
|
||||
|
||||
@@ -45,10 +45,6 @@ class SourceError(Exception):
|
||||
"""User-readable failure from a source (network, format, verification)."""
|
||||
|
||||
|
||||
class SourceAuthError(SourceError):
|
||||
"""The server refused our login (HTTP 401, or 403 when a login was sent)."""
|
||||
|
||||
|
||||
class SourceLimited(SourceError):
|
||||
"""The source's host asked us to slow down; retry_after is in seconds."""
|
||||
|
||||
|
||||
@@ -1,270 +0,0 @@
|
||||
"""Logins for sources behind one: HTTP basic auth and access tokens.
|
||||
|
||||
Saved apart from the source settings, in frame_host.data_dir('source-credentials.json')
|
||||
(mode 0600), keyed by a caller-chosen name: an F-Droid source id, 'github'. Each
|
||||
credential belongs to one exact origin (https, host, port) and is only ever sent
|
||||
to a request for that origin: Handler adds it to each request, redirects
|
||||
included, and removes any Authorization header from every other request.
|
||||
|
||||
Scoped use (for_url, and Handler(..., scoped=True), which F-Droid repositories and
|
||||
artwork use) is stricter: the URL must also be under the credential's scope by whole
|
||||
path segments, with no dot segments, encoded slashes or ';' a server could normalise
|
||||
into a path outside it. Origin-only matching suits a host-wide token (GitHub's API).
|
||||
|
||||
Secrets leave this module only as an Authorization header value. Credential's
|
||||
repr hides them, and nothing here puts them in an exception message.
|
||||
|
||||
Each change is a read-modify-write under an inter-process lock (the CLI and the
|
||||
app may both save), so one process can't undo another's save or removal.
|
||||
"""
|
||||
import base64
|
||||
import contextlib
|
||||
import errno
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
import frame_host
|
||||
|
||||
_lock = threading.RLock()
|
||||
MAX = 4096 # longest username, password or token accepted
|
||||
|
||||
|
||||
def _path():
|
||||
return frame_host.data_dir('source-credentials.json')
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _changing():
|
||||
"""This thread, then this process, holds the credentials file for a read-modify-write."""
|
||||
path = _path().with_name('source-credentials.lock')
|
||||
with _lock:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(str(path), 'a+b') as f:
|
||||
if os.name == 'nt':
|
||||
import msvcrt
|
||||
while True:
|
||||
try:
|
||||
f.seek(0)
|
||||
msvcrt.locking(f.fileno(), msvcrt.LK_LOCK, 1)
|
||||
break
|
||||
except OSError as e: # LK_LOCK gives up after ~10 s of contention; keep waiting
|
||||
if e.errno not in (errno.EACCES, errno.EDEADLK):
|
||||
raise
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
f.seek(0)
|
||||
msvcrt.locking(f.fileno(), msvcrt.LK_UNLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
fcntl.flock(f.fileno(), fcntl.LOCK_EX)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
|
||||
|
||||
|
||||
def path_segments(path):
|
||||
"""A URL path's decoded segments, or None if a server might read it as another path:
|
||||
'.' or '..' (plain or percent-encoded), encoded '/' or '\\', '%' left after decoding
|
||||
(double encoding), or ';' path parameters."""
|
||||
if ';' in path or '\\' in path:
|
||||
return None
|
||||
parts = [urllib.parse.unquote(part) for part in path.split('/')]
|
||||
if any(part in ('.', '..') or '/' in part or '\\' in part or '%' in part or ';' in part for part in parts):
|
||||
return None
|
||||
return parts
|
||||
|
||||
|
||||
def in_scope(credential, url):
|
||||
"""True if url is on credential's origin and under its scope by whole, plain path segments."""
|
||||
if credential is None or not credential.matches(url):
|
||||
return False
|
||||
try:
|
||||
path, scope = urllib.parse.urlsplit(url).path, urllib.parse.urlsplit(credential.scope).path
|
||||
except ValueError:
|
||||
return False
|
||||
parts, base = path_segments(path), path_segments(scope)
|
||||
if parts is None or base is None:
|
||||
return False
|
||||
if base and base[-1] == '':
|
||||
base = base[:-1] # 'https://h/repo/' covers '/repo/x', not '/repository'
|
||||
return parts[:len(base)] == base
|
||||
|
||||
|
||||
def origin(url):
|
||||
"""'https://host:port' for an https URL; ValueError for anything else."""
|
||||
p = urllib.parse.urlsplit(url)
|
||||
port = p.port # raises ValueError on a bad port
|
||||
if p.scheme.lower() != 'https' or not p.hostname:
|
||||
raise ValueError('credentials are only sent over HTTPS')
|
||||
host = p.hostname.lower()
|
||||
return 'https://%s:%d' % ('[%s]' % host if ':' in host else host, port or 443)
|
||||
|
||||
|
||||
def _check(value, what, allow_empty=False):
|
||||
if not isinstance(value, str) or (not value and not allow_empty) or len(value) > MAX:
|
||||
raise ValueError('invalid ' + what)
|
||||
if any(ord(c) < 32 or ord(c) == 127 for c in value):
|
||||
raise ValueError(what + ' must not contain control characters')
|
||||
return value
|
||||
|
||||
|
||||
class Credential:
|
||||
"""A username and password (basic) or a token (bearer) for one origin.
|
||||
|
||||
scope is the URL prefix it belongs under: in_scope checks it (by path segments),
|
||||
matches() checks only the origin."""
|
||||
__slots__ = ('origin', 'scope', 'username', '_secret')
|
||||
|
||||
def __init__(self, scope, secret, username=None):
|
||||
self.origin = origin(scope)
|
||||
self.scope = scope
|
||||
if username is not None:
|
||||
_check(username, 'username')
|
||||
if ':' in username:
|
||||
raise ValueError('username must not contain ":"')
|
||||
_check(secret, 'password', allow_empty=True)
|
||||
else:
|
||||
_check(secret, 'token')
|
||||
if any(c.isspace() for c in secret):
|
||||
raise ValueError('token must not contain spaces')
|
||||
self.username, self._secret = username, secret
|
||||
|
||||
@property
|
||||
def kind(self):
|
||||
return 'bearer' if self.username is None else 'basic'
|
||||
|
||||
def header(self):
|
||||
if self.username is None:
|
||||
return 'Bearer ' + self._secret
|
||||
return 'Basic ' + base64.b64encode((self.username + ':' + self._secret).encode('utf-8')).decode('ascii')
|
||||
|
||||
def matches(self, url):
|
||||
"""True only for a URL on exactly this credential's origin."""
|
||||
try:
|
||||
return origin(url) == self.origin
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
def same(self, other):
|
||||
return isinstance(other, Credential) and (self.origin, self.username, self._secret) == (
|
||||
other.origin, other.username, other._secret)
|
||||
|
||||
def __repr__(self):
|
||||
return '<Credential %s for %s>' % (self.kind, self.origin)
|
||||
|
||||
def _json(self):
|
||||
return {'scope': self.scope, 'username': self.username, 'secret': self._secret}
|
||||
|
||||
|
||||
def basic(scope, username, password):
|
||||
"""Basic auth for scope's origin, e.g. basic('https://example.org/fdroid/repo/', 'me', 'pw')."""
|
||||
return Credential(scope, password, username)
|
||||
|
||||
|
||||
def bearer(scope, token):
|
||||
"""An access token for scope's origin, sent as 'Authorization: Bearer <token>'."""
|
||||
return Credential(scope, token)
|
||||
|
||||
|
||||
def _read():
|
||||
try:
|
||||
with open(str(_path()), encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError
|
||||
return data
|
||||
except FileNotFoundError:
|
||||
return {}
|
||||
except (OSError, ValueError):
|
||||
return {} # unreadable: as if none were saved; the next save rewrites it
|
||||
|
||||
|
||||
def _save(data):
|
||||
path = _path()
|
||||
if not data:
|
||||
try:
|
||||
os.remove(str(path))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
return
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix='.credentials-', suffix='.part') # created 0600
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, separators=(',', ':'))
|
||||
os.chmod(tmp, 0o600)
|
||||
os.replace(tmp, str(path))
|
||||
finally:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
|
||||
|
||||
def get(key):
|
||||
"""The saved Credential for key, or None."""
|
||||
with _lock:
|
||||
item = _read().get(key)
|
||||
try:
|
||||
return Credential(item['scope'], item['secret'], item.get('username'))
|
||||
except (TypeError, KeyError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def has(key):
|
||||
return get(key) is not None
|
||||
|
||||
|
||||
def put(key, credential):
|
||||
if not isinstance(credential, Credential):
|
||||
raise TypeError('put() needs a Credential')
|
||||
with _changing():
|
||||
data = _read()
|
||||
data[key] = credential._json()
|
||||
_save(data)
|
||||
|
||||
|
||||
def remove(key):
|
||||
"""Delete key's credential from disk; True if there was one."""
|
||||
with _changing():
|
||||
data = _read()
|
||||
if key not in data:
|
||||
return False
|
||||
del data[key]
|
||||
_save(data)
|
||||
return True
|
||||
|
||||
|
||||
def for_url(url):
|
||||
"""A saved credential url is in the scope of (see in_scope), or None. Call it again for
|
||||
each redirect hop: the answer depends only on the URL being requested."""
|
||||
with _lock:
|
||||
keys = list(_read())
|
||||
for key in keys:
|
||||
credential = get(key)
|
||||
if in_scope(credential, url):
|
||||
return credential
|
||||
return None
|
||||
|
||||
|
||||
class Handler(urllib.request.BaseHandler):
|
||||
"""Adds the credential to requests for its origin (redirects included) and strips
|
||||
Authorization from every other request, so a redirect elsewhere never gets it.
|
||||
scoped=True also requires each request, redirects included, to be in_scope."""
|
||||
handler_order = 900 # after the default processors
|
||||
|
||||
def __init__(self, credential, scoped=False):
|
||||
self.credential, self.scoped = credential, scoped
|
||||
|
||||
def _request(self, req):
|
||||
req.remove_header('Authorization')
|
||||
allowed = in_scope if self.scoped else lambda c, url: c is not None and c.matches(url)
|
||||
if allowed(self.credential, req.full_url):
|
||||
req.add_unredirected_header('Authorization', self.credential.header())
|
||||
return req
|
||||
|
||||
http_request = https_request = _request
|
||||
@@ -9,7 +9,7 @@ import threading
|
||||
import time
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
|
||||
from apk_sources import SourceError, _credentials
|
||||
from apk_sources import SourceError
|
||||
|
||||
_lock = threading.Lock()
|
||||
_urls = OrderedDict()
|
||||
@@ -141,11 +141,7 @@ def get(url, headers=None, redirects=3, deadline=None, limit=MAX_IMAGE):
|
||||
path = p.path or '/'
|
||||
if p.query:
|
||||
path += '?' + p.query
|
||||
sent = {'User-Agent': 'FrameControl/0.3.1', **(headers or {})}
|
||||
login = _credentials.for_url(url) # a repository behind a login: its own artwork only, per hop
|
||||
if login:
|
||||
sent['Authorization'] = login.header()
|
||||
conn.request('GET', path, headers=sent)
|
||||
conn.request('GET', path, headers={'User-Agent': 'FrameControl/0.3.1', **(headers or {})})
|
||||
live[0].settimeout(min(10, left()))
|
||||
response = conn.getresponse()
|
||||
if response.status in (301, 302, 303, 307, 308) and redirects:
|
||||
|
||||
+13
-28
@@ -3,7 +3,7 @@ import hashlib, os, shutil, tempfile, threading, time, urllib.error, urllib.pars
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
import frame_host
|
||||
from . import SourceAuthError, SourceError, SourceLimited, _credentials
|
||||
from . import SourceError, SourceLimited
|
||||
|
||||
UA = 'FrameControl/0.1'
|
||||
APK_CAP = 2 * 1024 ** 3 # cached APKs across all sources, least recently used go first
|
||||
@@ -38,7 +38,7 @@ def release(path):
|
||||
else:
|
||||
_in_use.pop(path, None)
|
||||
BACKOFF = 600 # seconds to leave a host alone after 403/429 without Retry-After
|
||||
_limited = {} # host (+ login identity) -> time.time() before which we don't ask it with that login
|
||||
_limited = {} # host -> time.time() before which we don't contact it
|
||||
_limited_lock = threading.Lock()
|
||||
|
||||
|
||||
@@ -46,16 +46,8 @@ def _host(url):
|
||||
return (urllib.parse.urlsplit(url).hostname or '').lower()
|
||||
|
||||
|
||||
def _who(url, auth=None):
|
||||
"""Backoff key: the host, per login. A token has its own quota, so anonymous
|
||||
rate limiting mustn't hold back requests made with one (or the reverse)."""
|
||||
if auth is None:
|
||||
return _host(url)
|
||||
return _host(url) + ' ' + hashlib.sha256(auth.header().encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def throttle(url, headers=None, auth=None):
|
||||
"""Remember that url's host asked us (with this login) to back off; return the delay in seconds."""
|
||||
def throttle(url, headers=None):
|
||||
"""Remember that url's host asked us to back off; return the delay in seconds."""
|
||||
headers = headers or {}
|
||||
now, delay = time.time(), None
|
||||
value = (headers.get('Retry-After') or '').strip()
|
||||
@@ -70,15 +62,14 @@ def throttle(url, headers=None, auth=None):
|
||||
if delay is None and headers.get('X-RateLimit-Remaining') == '0' and reset.isdigit():
|
||||
delay = int(reset) - now # GitHub
|
||||
delay = min(max(delay if delay is not None else BACKOFF, 1), 6 * 3600)
|
||||
key = _who(url, auth)
|
||||
with _limited_lock:
|
||||
_limited[key] = max(_limited.get(key, 0), now + delay)
|
||||
_limited[_host(url)] = max(_limited.get(_host(url), 0), now + delay)
|
||||
return delay
|
||||
|
||||
|
||||
def wait_time(url, auth=None):
|
||||
def wait_time(url):
|
||||
with _limited_lock:
|
||||
return max(0, _limited.get(_who(url, auth), 0) - time.time())
|
||||
return max(0, _limited.get(_host(url), 0) - time.time())
|
||||
|
||||
|
||||
def limited_error(name, seconds, hint=''):
|
||||
@@ -145,8 +136,6 @@ def checked_url(url, hosts):
|
||||
|
||||
|
||||
class Redirect(urllib.request.HTTPRedirectHandler):
|
||||
"""Only to the allowed hosts, and never carrying an Authorization header along:
|
||||
a credential passed to open_url() is added again, by its Handler, only on its origin."""
|
||||
def __init__(self, hosts):
|
||||
self.hosts = hosts
|
||||
|
||||
@@ -158,15 +147,13 @@ class Redirect(urllib.request.HTTPRedirectHandler):
|
||||
return result
|
||||
|
||||
|
||||
def open_url(url, hosts, headers=None, auth=None):
|
||||
"""auth: an _credentials.Credential, sent only to requests on its own origin."""
|
||||
def open_url(url, hosts, headers=None):
|
||||
checked_url(url, hosts)
|
||||
handlers = [Redirect(hosts)] + ([_credentials.Handler(auth)] if auth else [])
|
||||
return urllib.request.build_opener(*handlers).open(
|
||||
return urllib.request.build_opener(Redirect(hosts)).open(
|
||||
urllib.request.Request(url, headers={'User-Agent': UA, **(headers or {})}), timeout=60)
|
||||
|
||||
|
||||
def read(url, hosts, headers=None, ttl=3600, name=None, hint='', auth=None):
|
||||
def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
|
||||
path = os.path.join(cache(), hashlib.sha256(url.encode()).hexdigest() + '.data')
|
||||
name = name or _host(url) or 'The source'
|
||||
|
||||
@@ -178,13 +165,13 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint='', auth=None):
|
||||
if os.path.isfile(path) and time.time() - os.path.getmtime(path) < ttl:
|
||||
with open(path, 'rb') as f:
|
||||
return f.read()
|
||||
wait = wait_time(url, auth)
|
||||
wait = wait_time(url)
|
||||
if wait:
|
||||
data = cached()
|
||||
if data is None:
|
||||
raise limited_error(name, wait, hint)
|
||||
return data
|
||||
with open_url(url, hosts, headers, auth) as r:
|
||||
with open_url(url, hosts, headers) as r:
|
||||
data = r.read(8 * 1024 * 1024 + 1)
|
||||
if len(data) > 8 * 1024 * 1024:
|
||||
raise SourceError('Source index is too large')
|
||||
@@ -198,10 +185,8 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint='', auth=None):
|
||||
os.remove(tmp)
|
||||
return data
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 401: # not 403: GitHub also uses that for rate limits, token or not
|
||||
raise SourceAuthError('%s refused the login (HTTP 401)' % name) from e
|
||||
if e.code in (403, 429):
|
||||
delay = throttle(url, e.headers, auth)
|
||||
delay = throttle(url, e.headers)
|
||||
data = cached()
|
||||
if data is None:
|
||||
raise limited_error(name, delay, hint) from e
|
||||
|
||||
+16
-82
@@ -20,7 +20,7 @@ import zipfile
|
||||
|
||||
if __package__ in (None, ''):
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from apk_sources import SourceAuthError, SourceError, SourceLimited, _credentials, _web
|
||||
from apk_sources import SourceError, SourceLimited, _web
|
||||
import frame_host
|
||||
from frame_apk_sign import _der_parts, _cert_key, der
|
||||
from frame_catalog import _IndexReader, _reduce_index, _sha256
|
||||
@@ -71,39 +71,6 @@ def _url(url, fingerprint=None):
|
||||
return urllib.parse.urlunsplit(('https', p.netloc.lower(), p.path.rstrip('/') + '/', '', '')), pin
|
||||
|
||||
|
||||
def _split_login(url, username=None, password=None):
|
||||
"""(url without user:pass, credential or None). The address may carry the login, as
|
||||
F-Droid's client accepts (https://user:pass@host/fdroid/repo), or it may be given apart."""
|
||||
try:
|
||||
p = urllib.parse.urlsplit(url)
|
||||
p.port
|
||||
except ValueError as e:
|
||||
raise SourceError('invalid repository URL') from e
|
||||
if '@' in p.netloc:
|
||||
if p.username is None:
|
||||
raise SourceError('invalid repository URL')
|
||||
linked = urllib.parse.unquote(p.username), None if p.password is None else urllib.parse.unquote(p.password)
|
||||
if username not in (None, '', linked[0]) or (linked[1] is not None and password not in (None, '', linked[1])):
|
||||
raise SourceError('the address and the username/password fields disagree')
|
||||
username = linked[0]
|
||||
password = linked[1] if linked[1] is not None else password
|
||||
url = urllib.parse.urlunsplit((p.scheme, p.netloc.rpartition('@')[2], p.path, p.query, p.fragment))
|
||||
if username in (None, ''):
|
||||
if password:
|
||||
raise SourceError('a password needs a username')
|
||||
return url, None
|
||||
clean, _ = _url(url)
|
||||
try:
|
||||
return url, _credentials.basic(clean, username, password or '')
|
||||
except ValueError as e:
|
||||
raise SourceError('cannot use that login: ' + str(e)) from None
|
||||
|
||||
|
||||
def _login(source):
|
||||
"""The saved login for a user repository, or None."""
|
||||
return None if source.get('builtin') else _credentials.get(source['id'])
|
||||
|
||||
|
||||
def _child(base, name):
|
||||
name = str(name).lstrip('/')
|
||||
decoded = urllib.parse.unquote(name)
|
||||
@@ -114,9 +81,6 @@ def _child(base, name):
|
||||
url = urllib.parse.urljoin(base, urllib.parse.quote(decoded, safe='/')) # names may contain '#' or spaces
|
||||
if not url.startswith(base) or urllib.parse.urlsplit(url).query or urllib.parse.urlsplit(url).fragment:
|
||||
raise SourceError('repository file is outside its repository')
|
||||
# Nothing a server could normalise elsewhere: double-encoded dots, ';' parameters, encoded slashes.
|
||||
if _credentials.path_segments(urllib.parse.urlsplit(url).path) is None:
|
||||
raise SourceError('unsafe repository file name')
|
||||
return url
|
||||
|
||||
|
||||
@@ -127,17 +91,14 @@ class _HTTPSRedirect(urllib.request.HTTPRedirectHandler):
|
||||
return super().redirect_request(req, fp, code, msg, headers, newurl)
|
||||
|
||||
|
||||
def _fetch(url, path, maximum, auth=None):
|
||||
"""auth: the repository's login, sent only inside the repository (its origin and directory,
|
||||
by whole plain path segments), on every request including redirects."""
|
||||
def _fetch(url, path, maximum):
|
||||
host = urllib.parse.urlsplit(url).hostname
|
||||
wait = _web.wait_time(url, auth)
|
||||
wait = _web.wait_time(url)
|
||||
if wait:
|
||||
raise _web.limited_error(host, wait)
|
||||
request = urllib.request.Request(url, headers={'User-Agent': 'FrameControl/1.0'})
|
||||
handlers = [_HTTPSRedirect()] + ([_credentials.Handler(auth, scoped=True)] if auth else [])
|
||||
try:
|
||||
with urllib.request.build_opener(*handlers).open(request, timeout=60) as r, open(path, 'wb') as f:
|
||||
with urllib.request.build_opener(_HTTPSRedirect()).open(request, timeout=60) as r, open(path, 'wb') as f:
|
||||
total = 0
|
||||
while True:
|
||||
chunk = r.read(1 << 20)
|
||||
@@ -148,11 +109,8 @@ def _fetch(url, path, maximum, auth=None):
|
||||
raise SourceError('repository file exceeds size limit')
|
||||
f.write(chunk)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 401 or (e.code == 403 and auth):
|
||||
raise SourceAuthError('The repository refused the login (HTTP %d); check the username and password'
|
||||
% e.code if auth else 'This repository needs a username and password') from None
|
||||
if e.code in (403, 429):
|
||||
raise _web.limited_error(host, _web.throttle(url, e.headers, auth)) from e
|
||||
raise _web.limited_error(host, _web.throttle(url, e.headers)) from e
|
||||
raise
|
||||
|
||||
|
||||
@@ -384,7 +342,7 @@ def sources():
|
||||
return [dict(id=i, kind=KIND, name=n, url=u, fingerprint=p, builtin=True,
|
||||
# The archive only holds superseded versions; it clutters search unless asked for.
|
||||
enabled=settings['enabled'].get(i, i != 'fdroid-archive'), trust='community')
|
||||
for i, n, u, p in builtins] + [dict(s, login=_credentials.has(s['id'])) for s in settings['repos']]
|
||||
for i, n, u, p in builtins] + settings['repos']
|
||||
|
||||
|
||||
def _text(value):
|
||||
@@ -556,12 +514,10 @@ def _source_lock(source_id):
|
||||
return _load_locks.setdefault(source_id, threading.Lock())
|
||||
|
||||
|
||||
def _load(source, force=False, auth=None):
|
||||
"""auth: a login to use instead of the saved one (while adding, before anything is saved)."""
|
||||
def _load(source, force=False):
|
||||
if not re.fullmatch(r'[a-z0-9-]+', source['id']):
|
||||
raise SourceError('invalid source id')
|
||||
_url(source['url'], source.get('fingerprint'))
|
||||
auth = auth or _login(source)
|
||||
cache = frame_host.cache_dir('apk-sources', source['id'] + '.json')
|
||||
found = None if force else _cached(source, cache)
|
||||
if found:
|
||||
@@ -579,7 +535,7 @@ def _load(source, force=False, auth=None):
|
||||
jar, raw = Path(tmp) / 'index.jar', Path(tmp) / 'index.json'
|
||||
v2 = True
|
||||
try:
|
||||
_fetch(source['url'] + 'entry.jar', jar, 8 * 1024 * 1024, auth=auth)
|
||||
_fetch(source['url'] + 'entry.jar', jar, 8 * 1024 * 1024)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code not in (404, 410):
|
||||
raise
|
||||
@@ -587,7 +543,7 @@ def _load(source, force=False, auth=None):
|
||||
raise SourceError('repository no longer serves its signed v2 index; '
|
||||
'refusing to fall back to the older v1 index') from e
|
||||
v2 = False
|
||||
_fetch(source['url'] + 'index-v1.jar', jar, 256 * 1024 * 1024, auth=auth)
|
||||
_fetch(source['url'] + 'index-v1.jar', jar, 256 * 1024 * 1024)
|
||||
content, pin = _jar(jar, 'index-v1.json', source.get('fingerprint'))
|
||||
timestamp = _v1(content, raw)
|
||||
_check_timestamp(source, timestamp)
|
||||
@@ -596,7 +552,7 @@ def _load(source, force=False, auth=None):
|
||||
signed = json.loads(content)
|
||||
timestamp, entry = signed.get('timestamp'), signed['index']
|
||||
_check_timestamp(source, timestamp)
|
||||
_fetch(_child(source['url'], entry['name']), raw, 512 * 1024 * 1024, auth=auth)
|
||||
_fetch(_child(source['url'], entry['name']), raw, 512 * 1024 * 1024)
|
||||
if _sha256(raw) != entry['sha256'] or (entry.get('size') is not None and raw.stat().st_size != entry['size']):
|
||||
raise SourceError('index SHA-256 or size mismatch')
|
||||
apps = _reduce(raw, source)
|
||||
@@ -616,10 +572,7 @@ def _load(source, force=False, auth=None):
|
||||
raise SourceError('cannot load repository: ' + str(e)) from e
|
||||
|
||||
|
||||
def add_repo(url, fingerprint=None, name=None, username=None, password=None):
|
||||
"""Check and save a repository. A login (in the URL or apart) is kept out of the
|
||||
saved URL, in _credentials, and saved only once the repository's index verified."""
|
||||
url, login = _split_login(url, username, password)
|
||||
def add_repo(url, fingerprint=None, name=None):
|
||||
url, pin = _url(url, fingerprint)
|
||||
with _LOCK:
|
||||
existing = next((s for s in _read()['repos'] if s['url'] == url), None)
|
||||
@@ -630,26 +583,15 @@ def add_repo(url, fingerprint=None, name=None, username=None, password=None):
|
||||
source = dict(id='fdroid-user-' + hashlib.sha256(url.encode()).hexdigest()[:20], kind=KIND,
|
||||
name=name or (existing or {}).get('name') or urllib.parse.urlsplit(url).hostname,
|
||||
url=url, builtin=False, enabled=True, trust='user', fingerprint=pin)
|
||||
# Network work outside the settings lock; without a new login, a saved one (re-adding) is used.
|
||||
_, source['fingerprint'] = _load(source, force=True, auth=login)
|
||||
_, source['fingerprint'] = _load(source, force=True) # network work outside the settings lock
|
||||
source['trust_on_first_use'] = existing.get('trust_on_first_use', False) if existing else pin is None
|
||||
with _LOCK:
|
||||
settings = _read()
|
||||
current = next((s for s in settings['repos'] if s['id'] == source['id']), None)
|
||||
if current and current['fingerprint'] != source['fingerprint']:
|
||||
raise SourceError('repository already has a different pinned fingerprint; remove it first')
|
||||
previous = _credentials.get(source['id'])
|
||||
if login:
|
||||
_credentials.put(source['id'], login)
|
||||
try:
|
||||
settings['repos'] = [s for s in settings['repos'] if s['id'] != source['id']] + [source]
|
||||
_write(_storage(), settings)
|
||||
except BaseException:
|
||||
if login and previous: # not saved after all: leave the login as it was
|
||||
_credentials.put(source['id'], previous)
|
||||
elif login:
|
||||
_credentials.remove(source['id'])
|
||||
raise
|
||||
settings['repos'] = [s for s in settings['repos'] if s['id'] != source['id']] + [source]
|
||||
_write(_storage(), settings)
|
||||
return source
|
||||
|
||||
|
||||
@@ -660,7 +602,6 @@ def remove_repo(source_id):
|
||||
raise SourceError('unknown user repository')
|
||||
settings['repos'] = [s for s in settings['repos'] if s['id'] != source_id]
|
||||
_write(_storage(), settings)
|
||||
_credentials.remove(source_id)
|
||||
with _state_file_lock(), _LOCK: # lock order: state file, then _LOCK
|
||||
states = _states()
|
||||
if states.pop(source_id, None) is not None: # re-adding is a deliberate new trust decision
|
||||
@@ -722,7 +663,7 @@ def download(source, entry_id, version_code=None):
|
||||
fd, tmp = tempfile.mkstemp(dir=str(path.parent), suffix='.part')
|
||||
os.close(fd)
|
||||
try:
|
||||
_fetch(_child(source['url'], version['name']), tmp, 4 * 1024 ** 3, auth=_login(source))
|
||||
_fetch(_child(source['url'], version['name']), tmp, 4 * 1024 ** 3)
|
||||
if _sha256(tmp) != sha:
|
||||
raise SourceError('APK SHA-256 mismatch; download discarded')
|
||||
os.replace(tmp, path)
|
||||
@@ -743,8 +684,6 @@ def main():
|
||||
add.add_argument('url')
|
||||
add.add_argument('--fingerprint')
|
||||
add.add_argument('--name')
|
||||
add.add_argument('--username', help='for a repository behind a login (HTTP basic auth)')
|
||||
add.add_argument('--password-stdin', action='store_true', help='read the password from the first line of stdin')
|
||||
sub.add_parser('list')
|
||||
remove = sub.add_parser('remove')
|
||||
remove.add_argument('source')
|
||||
@@ -763,12 +702,7 @@ def main():
|
||||
def _cli(parser, args):
|
||||
try:
|
||||
if args.command == 'add':
|
||||
password = None
|
||||
if args.password_stdin:
|
||||
if not args.username:
|
||||
raise SourceError('--password-stdin needs --username')
|
||||
password = sys.stdin.readline().rstrip('\r\n')
|
||||
result = add_repo(args.url, args.fingerprint, args.name, args.username, password)
|
||||
result = add_repo(args.url, args.fingerprint, args.name)
|
||||
elif args.command == 'list':
|
||||
result = sources()
|
||||
elif args.command == 'remove':
|
||||
|
||||
@@ -1,77 +1,17 @@
|
||||
"""Curated publisher releases; optional topic discovery is page-only."""
|
||||
import fnmatch, hashlib, json, os, re, sys, threading, urllib.parse
|
||||
import fnmatch, json, os, re, urllib.parse
|
||||
|
||||
from . import SourceAuthError, SourceError, _credentials, _web
|
||||
from . import SourceError, _web
|
||||
|
||||
KIND = 'github'
|
||||
API = 'https://api.github.com'
|
||||
ENV = 'FRAME_GITHUB_TOKEN' # wins over a token saved in source settings
|
||||
CREDENTIAL = 'github' # _credentials key; only ever sent to api.github.com
|
||||
_rejected = set() # SHA-256 of tokens GitHub answered 401 to (this run only)
|
||||
_rejected_lock = threading.Lock()
|
||||
TOPICS = ('oculus-quest', 'openxr', 'quest')
|
||||
HOSTS = ('github.com', 'release-assets.githubusercontent.com', 'objects.githubusercontent.com')
|
||||
|
||||
|
||||
def sources():
|
||||
return [{'id': KIND, 'kind': KIND, 'name': 'GitHub releases', 'url': 'https://github.com',
|
||||
'builtin': True, 'enabled': True, 'trust': 'community', 'token': token_status()}]
|
||||
|
||||
|
||||
def _token():
|
||||
"""(token, where it came from) or (None, None). The environment variable wins."""
|
||||
env = os.environ.get(ENV)
|
||||
if env:
|
||||
return env, 'env'
|
||||
saved = _credentials.get(CREDENTIAL)
|
||||
if saved and saved.kind == 'bearer':
|
||||
return saved, 'saved'
|
||||
return None, None
|
||||
|
||||
|
||||
def _credential(token):
|
||||
if isinstance(token, _credentials.Credential):
|
||||
return token
|
||||
try:
|
||||
return _credentials.bearer(API + '/', token)
|
||||
except ValueError as e:
|
||||
raise SourceError('%s is not a usable GitHub token: %s' % (ENV, e)) from None
|
||||
|
||||
|
||||
def _digest(credential):
|
||||
return hashlib.sha256(credential.header().encode()).hexdigest()
|
||||
|
||||
|
||||
def token_status():
|
||||
"""Whether a token is set, where from, and whether GitHub rejected it; never the token."""
|
||||
token, where = _token()
|
||||
rejected = False
|
||||
if token is not None:
|
||||
try:
|
||||
with _rejected_lock:
|
||||
rejected = _digest(_credential(token)) in _rejected
|
||||
except SourceError:
|
||||
rejected = True
|
||||
return {'set': token is not None, 'from': where, 'rejected': rejected}
|
||||
|
||||
|
||||
def set_token(token):
|
||||
"""Save a personal access token for api.github.com (FRAME_GITHUB_TOKEN still wins)."""
|
||||
if not isinstance(token, str) or not token.strip():
|
||||
raise SourceError('Paste a GitHub token')
|
||||
try:
|
||||
credential = _credentials.bearer(API + '/', token.strip())
|
||||
except ValueError as e:
|
||||
raise SourceError('That is not a usable GitHub token: ' + str(e)) from None
|
||||
_credentials.put(CREDENTIAL, credential)
|
||||
with _rejected_lock:
|
||||
_rejected.discard(_digest(credential)) # a fresh chance
|
||||
return {'message': 'GitHub token saved' + (' (%s is set and is used instead)' % ENV if os.environ.get(ENV) else '')}
|
||||
|
||||
|
||||
def clear_token():
|
||||
_credentials.remove(CREDENTIAL)
|
||||
return {'message': 'GitHub token removed'}
|
||||
'builtin': True, 'enabled': True, 'trust': 'community'}]
|
||||
|
||||
|
||||
def _curated():
|
||||
@@ -80,33 +20,13 @@ def _curated():
|
||||
|
||||
|
||||
def _api(path):
|
||||
"""GET from the GitHub API, with the user's token if there is one. If GitHub rejects
|
||||
the token (401), retry once without it, like Obtainium, and remember the rejection."""
|
||||
headers = {'Accept': 'application/vnd.github+json', 'X-GitHub-Api-Version': '2022-11-28'}
|
||||
token, _ = _token()
|
||||
auth = _credential(token) if token is not None else None
|
||||
with _rejected_lock:
|
||||
if auth and _digest(auth) in _rejected:
|
||||
auth = None # already refused this run: don't spend a request on it again
|
||||
hint = ' (add a GitHub token in the store\'s source settings, or set %s, to raise the limit)' % ENV
|
||||
if token is not None and not auth:
|
||||
hint = ' (GitHub rejected your token; replace it in the store\'s source settings)'
|
||||
|
||||
def get(auth):
|
||||
return _web.read(API + path, ('api.github.com',), headers, name='GitHub',
|
||||
hint='' if auth else hint, auth=auth)
|
||||
token = os.environ.get('FRAME_GITHUB_TOKEN')
|
||||
if token:
|
||||
headers['Authorization'] = 'Bearer ' + token
|
||||
try:
|
||||
try:
|
||||
data = get(auth)
|
||||
except SourceAuthError:
|
||||
if not auth:
|
||||
raise
|
||||
with _rejected_lock:
|
||||
_rejected.add(_digest(auth))
|
||||
print('GitHub rejected the access token (HTTP 401); continuing without it', file=sys.stderr)
|
||||
hint = ' (GitHub rejected your token; replace it in the store\'s source settings)'
|
||||
data = get(None)
|
||||
return json.loads(data)
|
||||
return json.loads(_web.read(API + path, ('api.github.com',), headers, name='GitHub',
|
||||
hint='' if token else ' (set FRAME_GITHUB_TOKEN to raise the limit)'))
|
||||
except (ValueError, TypeError) as e:
|
||||
raise SourceError('Invalid GitHub response') from e
|
||||
|
||||
|
||||
@@ -113,14 +113,6 @@ def manage_repo(action, **kwargs):
|
||||
return getattr(module, action)(**kwargs)
|
||||
|
||||
|
||||
def set_token(source_id, token):
|
||||
"""Save (token) or remove (None) a source's access token; status is in sources()."""
|
||||
module, _ = resolve(source_id)
|
||||
if not hasattr(module, 'set_token'):
|
||||
raise SourceError('This source does not use an access token')
|
||||
return module.set_token(token) if token is not None else module.clear_token()
|
||||
|
||||
|
||||
def normalise(name):
|
||||
return ' '.join(re.findall(r'\w+', unicodedata.normalize('NFKC', name or '').casefold()))
|
||||
|
||||
|
||||
+3
-1
@@ -506,8 +506,10 @@ def apply(gs, event, panel):
|
||||
say("ready", focus=panel.get("window"), display=panel.get("display"), stale=True) # the page re-syncs
|
||||
STALE[0] = True
|
||||
return panel
|
||||
if STALE[0] and not stale:
|
||||
release = event.get("down") is False and ("button" in event or "key" in event)
|
||||
if STALE[0] and not stale and not (release and "window" not in event):
|
||||
# Anything that goes through (a trackpad move names no panel) means caught up: stop re-syncing.
|
||||
# A bare release doesn't: the page leaves the panel off releases, so it says nothing about focus.
|
||||
STALE[0] = False
|
||||
say("ready", focus=(panel or {}).get("window"), display=(panel or {}).get("display"))
|
||||
if "fx" in event and panel and panel.get("window"):
|
||||
|
||||
+10
-62
@@ -626,8 +626,6 @@
|
||||
.source-repo-form input { margin-top:7px; display:block; border-radius:6px; }
|
||||
.source-repo-form details { font-size:11px; color:var(--muted); }
|
||||
.source-repo-form button { justify-self:start; border-radius:6px; }
|
||||
.source-token { display:flex; flex-wrap:wrap; align-items:center; gap:7px; margin:8px 0 4px; }
|
||||
.source-token input { flex:1; min-width:150px; border-radius:6px; font-size:12px; }
|
||||
#appStore :focus-visible, .store-dialog :focus-visible, .store-tabs :focus-visible { outline:2px solid var(--link); outline-offset:4px; }
|
||||
.store-search input:focus-visible { outline:0!important; }
|
||||
@media(min-width:1400px) { .store-grid { grid-template-columns:repeat(5,minmax(0,1fr)); } }
|
||||
@@ -934,10 +932,9 @@
|
||||
<button data-pad-key="12" title="Enter">Enter</button>
|
||||
</span>
|
||||
</div>
|
||||
<div class="hint">Types and points in apps on the Frame, such as Chromium or the desktop's Linux apps; Steam's own VR menus
|
||||
don't take it. It works through KDE Connect, which comes with Frame Control: the first time, it copies Valve's build
|
||||
for the Frame (3.6 MB, 18 MB unpacked) into your home folder there and pairs with it. No internet needed, and nothing
|
||||
else to install. <a href="#" data-about>Licences</a></div>
|
||||
<div class="hint">Types and points in whichever panel has focus in the headset, through Valve's own input path: nothing to
|
||||
install and no internet needed. Accents and emoji go through KDE Connect, which comes with Frame Control and starts
|
||||
the first time you type one (it copies Valve's build for the Frame, 3.6 MB, into your home folder there). <a href="#" data-about>Licences</a></div>
|
||||
</section>
|
||||
|
||||
<section id="shots">
|
||||
@@ -1320,7 +1317,7 @@
|
||||
<button class="store-close" data-close="sourcesDialog" aria-label="Close source settings">×</button>
|
||||
<div class="sources-intro"><div class="store-eyebrow">Your app shelves</div><h1 id="sourcesTitle">A world of apps.</h1><p>Choose where you discover games and apps. We bring them together for you.</p></div>
|
||||
<div id="sourcesList"></div>
|
||||
<form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><details><summary>This repository needs a login (optional)</summary><label>Username<input name="username" type="text" autocomplete="off" autocapitalize="none" spellcheck="false"></label><label>Password<input name="password" type="password" autocomplete="new-password"></label><p>Kept on this computer, apart from the address, and only sent to this repository over HTTPS.</p></details><button class="action">Add source</button></form>
|
||||
<form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><button class="action">Add source</button></form>
|
||||
<p id="sourceError" class="store-notice" role="status" aria-live="polite"></p>
|
||||
</dialog>
|
||||
|
||||
@@ -3308,18 +3305,12 @@ function storeIcon(o) { return `<span class="store-icon" style="--hue:${storeHue
|
||||
document.addEventListener("error", e => { if (e.target.matches?.(".store-art img, .store-icon img")) e.target.remove(); }, true);
|
||||
function sourceMark(s) { const name=storeName(s); return `<span class="offer-mark" style="--hue:${storeHue(name)}" aria-hidden="true">${esc(name === "SideQuest" ? "SQ" : name === "F-Droid" ? "Fd" : name.slice(0,2))}</span>`; }
|
||||
function sourceDescription(s) { return s.description || ({fdroid:"Independent, open-source apps",sidequest:"Discover something new in VR",itch:"Games from independent creators",github:"Direct from the people who make it"})[s.kind] || (s.name.includes("GitHub") ? "Direct from the people who make it" : s.name.includes("Droid") ? "Independent, open-source apps" : s.name.includes("SideQuest") ? "Discover something new in VR" : s.name.includes("itch") ? "Games from independent creators" : "Apps shared by this community"); }
|
||||
// GitHub's optional access token: only whether one is set is ever shown, never its value.
|
||||
function sourceToken(s) {
|
||||
const t=s.token;if(!t)return "";
|
||||
const note=t.rejected ? "GitHub rejected this token, so searches continue without it. Replace or clear it." : t.from==="env" ? "Using the token from FRAME_GITHUB_TOKEN." : t.set ? "A token is saved." : "Optional: a token raises GitHub's limit of 60 requests an hour.";
|
||||
return `<form class="source-token" data-token="${esc(s.id)}"><input name="token" type="password" autocomplete="off" spellcheck="false" aria-label="${esc(storeName(s.name))} access token" placeholder="${t.set ? "Replace access token" : "Access token"}"><button class="small">Save</button>${t.from==="saved" ? `<button type="button" class="small" data-clear-token="${esc(s.id)}">Clear</button>` : ""}</form><small>${esc(note)}</small>`;
|
||||
}
|
||||
async function loadSources() {
|
||||
try {
|
||||
const {sources} = await api("/api/sources"); sourceState.sources = sources;
|
||||
$("sourceFilter").innerHTML = `<option value="">All sources</option>` + sources.filter(s=>s.enabled).map(s=>`<option value="${esc(s.id)}">${esc(storeName(s.name))}</option>`).join("");
|
||||
$("sourceFilter").value = sourceState.source;
|
||||
$("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}${s.login ? " · Uses a login" : ""}</small>${sourceToken(s)}${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
|
||||
$("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}</small>${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
|
||||
} catch (e) { $("sourceError").textContent = e.message; }
|
||||
}
|
||||
function storeCard(app) {
|
||||
@@ -3447,8 +3438,7 @@ $("detailBody").onclick=e=>{if(e.target.closest('[data-close]'))return $("appDet
|
||||
$("detailBody").onchange=e=>{if(e.target.name==='storeOffer'){sourceState.detail.choice=+e.target.value;renderAppDetail();loadOfferDetails();}};
|
||||
$("sourcesOpen").onclick=()=>{loadSources();$('sourcesDialog').showModal();};
|
||||
$("sourcesList").onchange=e=>{if(e.target.matches('[data-enable]'))changeSource({action:'enable',source:e.target.dataset.enable,enabled:e.target.checked});};
|
||||
$("sourcesList").onclick=e=>{const b=e.target.closest('[data-remove]');if(b)changeSource({action:'remove',source:b.dataset.remove});const c=e.target.closest('[data-clear-token]');if(c)changeSource({action:'clear-token',source:c.dataset.clearToken});};
|
||||
$("sourcesList").onsubmit=e=>{const f=e.target.closest('[data-token]');if(!f)return;e.preventDefault();const token=f.elements.token.value.trim();f.reset();if(token)changeSource({action:'set-token',source:f.dataset.token,token});};
|
||||
$("sourcesList").onclick=e=>{const b=e.target.closest('[data-remove]');if(b)changeSource({action:'remove',source:b.dataset.remove});};
|
||||
async function addSource(body) {
|
||||
$("sourceError").textContent="Checking the repository. Large ones can take a minute…";
|
||||
let error="";
|
||||
@@ -3456,49 +3446,7 @@ async function addSource(body) {
|
||||
$("sourceError").textContent=result ? result.message : error || "We couldn't add that repository.";
|
||||
if(result){$("addSource").reset();sourceState.source='';await loadSources();await searchSources();}
|
||||
}
|
||||
// A login typed into the address moves into the login fields, on the page too, before anything
|
||||
// is sent: the address is shown and logged, the login never. It stays there if adding fails.
|
||||
// Plain text, not the URL parser: an address it rejects (or a login it can't decode) must
|
||||
// still lose its login first. Decoding falls back to the text as typed.
|
||||
// Percent-decode what decodes; keep any byte sequence that isn't UTF-8 as typed.
|
||||
// One forward pass: each lead byte gives its sequence length (at most 4 bytes).
|
||||
function decodeLenient(s) {
|
||||
return s.replace(/(?:%[0-9a-f]{2})+/gi, run => {
|
||||
const b=run.match(/%[0-9a-f]{2}/gi); let out='', i=0;
|
||||
while(i<b.length){
|
||||
const v=parseInt(b[i].slice(1),16);
|
||||
const n=v<0x80 ? 1 : v>=0xC2&&v<0xE0 ? 2 : v>=0xE0&&v<0xF0 ? 3 : v>=0xF0&&v<0xF5 ? 4 : 0;
|
||||
if(n){ try { out+=decodeURIComponent(b.slice(i,i+n).join('')); i+=n; continue; } catch {} }
|
||||
out+=b[i]; i++;
|
||||
}
|
||||
return out;
|
||||
});
|
||||
}
|
||||
// Moves a login typed into the address into the login fields. Returns false for an
|
||||
// address the URL parser rejects that may hold a login: it is refused, not sent.
|
||||
function moveLogin(form) {
|
||||
// Normalise the way URL parsers do: trim C0 controls and spaces, drop tabs and newlines.
|
||||
const el=form.elements;
|
||||
el.url.value=el.url.value.replace(/^[\u0000- ]+|[\u0000- ]+$/g,'').replace(/[\t\n\r]/g,'');
|
||||
let u;
|
||||
// A backslash and an '@' together are read differently by browsers and by the
|
||||
// server, so the address can't be split safely: refuse it rather than guess.
|
||||
if(el.url.value.includes('@') && el.url.value.includes('\\'))return false;
|
||||
try { u=new URL(el.url.value); } catch { return !el.url.value.includes('@'); }
|
||||
if(!u.username && !u.password)return true;
|
||||
const user=decodeLenient(u.username), pass=decodeLenient(u.password);
|
||||
u.username=''; u.password=''; el.url.value=u.href;
|
||||
if(!el.username.value)el.username.value=user;
|
||||
if(!el.password.value)el.password.value=pass;
|
||||
el.username.closest('details').open=true;
|
||||
return true;
|
||||
}
|
||||
$("addSource").onsubmit=e=>{
|
||||
e.preventDefault();
|
||||
if(!moveLogin(e.target)){ $("sourceError").textContent="That address isn't valid. Put the username and password in the login fields below, not in the address."; return; }
|
||||
const f=new FormData(e.target), url=f.get('url'), username=f.get('username'), password=f.get('password');
|
||||
addSource({action:'add',url,fingerprint:f.get('fingerprint'),...(username ? {username,password} : {})});
|
||||
};
|
||||
$("addSource").onsubmit=e=>{e.preventDefault();const f=new FormData(e.target);addSource({action:'add',url:f.get('url'),fingerprint:f.get('fingerprint')});};
|
||||
document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>$(b.dataset.close).close());
|
||||
$('appDetail').addEventListener('close',()=>{sourceState.detailRequest++;});
|
||||
function showAndroidTab(tab) {
|
||||
@@ -4374,10 +4322,10 @@ const typesText = el => el?.matches?.('textarea, input:not([type=range], [type=c
|
||||
document.addEventListener("focusin", e => document.body.classList.toggle("typing", !!typesText(e.target)));
|
||||
document.addEventListener("focusout", () => document.body.classList.remove("typing"));
|
||||
|
||||
// ---- keyboard and trackpad: events go to KDE Connect on the Frame (see ui/frame_input_agent.py) ----
|
||||
// ---- keyboard and trackpad: events go to the Frame through gamescope (ui/frame_touch.py); accents and emoji through KDE Connect ----
|
||||
const pad = { state: "off", queue: [], sending: false, poll: null, captured: false };
|
||||
const PAD_STATES = {
|
||||
off: "", starting: "Connecting…", installing: "Setting up KDE Connect on the Frame (first time only)…",
|
||||
off: "", starting: "Connecting…", installing: "Setting up (first time only)…",
|
||||
pairing: "Pairing with KDE Connect…", ready: "On",
|
||||
};
|
||||
// KDE Connect's specialKey numbers (plugins/mousepad in its source).
|
||||
@@ -4395,7 +4343,7 @@ function padShow(status) {
|
||||
$("padOn").hidden = ready || busy;
|
||||
$("padOn").textContent = pad.state === "error" ? "Try again" : "Turn on";
|
||||
$("padArea").classList.toggle("off", !ready);
|
||||
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach KDE Connect on the Frame."
|
||||
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach the Frame."
|
||||
: !ready ? (busy ? PAD_STATES[pad.state] : "Turn on to use this as a trackpad for the Frame.")
|
||||
: TOUCH ? "Drag to move the pointer · tap to click · two fingers to scroll · two-finger tap to right-click"
|
||||
: pad.captured ? "Your mouse and keyboard now control the Frame. Press Esc to stop."
|
||||
|
||||
+191
-32
@@ -63,6 +63,7 @@ import frame_report # noqa: E402
|
||||
import frame_store # noqa: E402
|
||||
import frame_telemetry # noqa: E402
|
||||
import frame_titles # noqa: E402
|
||||
import frame_touch # noqa: E402
|
||||
import frame_webinstall # noqa: E402
|
||||
import frame_vr # noqa: E402
|
||||
import frame_utilities # noqa: E402
|
||||
@@ -115,6 +116,9 @@ def route(alias, host_opts):
|
||||
with _route_lock:
|
||||
moved = alias != FRAME
|
||||
if moved:
|
||||
reset_pad = globals().get("reset_pad_route")
|
||||
if reset_pad:
|
||||
reset_pad() # cancel old input before any agent can start on the new route
|
||||
frame_catalog._env.clear() # the SteamOS and Lepton builds reports record are per headset
|
||||
FRAME = frame_android.FRAME = alias
|
||||
HOST_OPTS = list(host_opts)
|
||||
@@ -125,9 +129,6 @@ def route(alias, host_opts):
|
||||
mv = globals().get("macview")
|
||||
if mv:
|
||||
mv.retarget(alias, host_opts) # its tunnel is its own ssh: it must follow the headset too
|
||||
agent = globals().get("_input")
|
||||
if moved and agent:
|
||||
agent.stop() # typing and pointing mustn't go on reaching the headset switched away from
|
||||
|
||||
|
||||
LINK = None # the connector (frame_link.Link); None on the Frame itself
|
||||
@@ -1153,12 +1154,192 @@ def licenses():
|
||||
return notices
|
||||
|
||||
|
||||
# KDE Connect's specialKey numbers -> Linux key codes (KEY_BACKSPACE, KEY_TAB, ...).
|
||||
PAD_KEYS = {1: 14, 2: 15, 4: 105, 5: 103, 6: 106, 7: 108, 8: 104, 9: 109, 10: 102, 11: 107, 12: 28, 13: 111, 14: 1,
|
||||
**{21 + i: code for i, code in enumerate([59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 87, 88])}}
|
||||
PAD_MODS = (("ctrl", 29), ("alt", 56), ("shift", 42), ("super", 125))
|
||||
PAD_CLICKS = {"singleclick": "left", "rightclick": "right", "middleclick": "middle"}
|
||||
PAD_NOTCH = 15 # scroll units for one wheel notch
|
||||
|
||||
|
||||
def typed_in_kde(events):
|
||||
"""Whether these events must go the KDE Connect way: they hold an accent or emoji, or
|
||||
earlier ones are queued, being sent or still being typed there (typing must stay in
|
||||
order, so it can't be split between the two ways)."""
|
||||
with _kde_lock:
|
||||
waiting = bool(_kde_queue) or time.time() < _kde_until
|
||||
return waiting or any(ch not in frame_touch.ASCII for e in events for ch in e.get("key", ""))
|
||||
|
||||
|
||||
def pad_events(events, kde=False):
|
||||
"""The trackpad's and key row's events as (gamescope events, KDE Connect events).
|
||||
|
||||
Gamescope's own input reaches every panel and needs nothing installed, but types only
|
||||
US-keyboard characters. With kde, the keyboard goes to KDE Connect as it was sent, so
|
||||
text stays in order; the pointer always goes through gamescope.
|
||||
"""
|
||||
ascii_keys = frame_touch.ASCII
|
||||
out, slow = [], []
|
||||
|
||||
def tap(code, mods):
|
||||
down = [{"key": c, "down": True} for c in mods]
|
||||
out.extend(down + [{"key": code, "down": True}, {"key": code, "down": False}]
|
||||
+ [{"key": c, "down": False} for c in reversed(mods)])
|
||||
for e in events:
|
||||
mods = [code for name, code in PAD_MODS if e.get(name)]
|
||||
if "dx" in e or "dy" in e:
|
||||
if not e.get("scroll"):
|
||||
out.append({k: e[k] for k in ("dx", "dy") if k in e})
|
||||
if e.get("scroll"):
|
||||
out.append({"scroll": [0, -(e.get("dy") or 0) * PAD_NOTCH]})
|
||||
for flag, button in PAD_CLICKS.items():
|
||||
if e.get(flag):
|
||||
out += [{"button": button, "down": True}, {"button": button, "down": False}]
|
||||
if e.get("doubleclick"):
|
||||
out += [{"button": "left", "down": d} for d in (True, False, True, False)]
|
||||
if e.get("singlehold"):
|
||||
out.append({"button": "left", "down": True})
|
||||
if e.get("singlerelease"):
|
||||
out.append({"button": "left", "down": False})
|
||||
if kde and ("key" in e or "specialKey" in e):
|
||||
slow.append({k: e[k] for k in ("key", "specialKey", "ctrl", "alt", "shift", "super") if k in e})
|
||||
continue
|
||||
if "specialKey" in e and e["specialKey"] in PAD_KEYS:
|
||||
tap(PAD_KEYS[e["specialKey"]], mods)
|
||||
if "key" in e:
|
||||
for ch in e["key"]:
|
||||
if ch not in ascii_keys:
|
||||
continue # only reached when kde is wrong; the caller checks typed_in_kde first
|
||||
elif mods:
|
||||
code, shifted = ascii_keys[ch]
|
||||
tap(code, mods + ([42] if shifted and 42 not in mods else []))
|
||||
else:
|
||||
if out and "text" in out[-1] and len(out[-1]["text"]) < 100:
|
||||
out[-1]["text"] += ch
|
||||
else:
|
||||
out.append({"text": ch})
|
||||
return out, slow
|
||||
|
||||
|
||||
def remote_input(body):
|
||||
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent."""
|
||||
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent.
|
||||
|
||||
"sent" in the answer means the whole batch is taken (the page keeps it and tries again
|
||||
if not), so the keyboard's share is queued only once that's so.
|
||||
"""
|
||||
events = body.get("events", [])
|
||||
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
|
||||
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
|
||||
return _input.send([input_event(e) for e in events])
|
||||
events = [input_event(e) for e in events]
|
||||
touch, slow = pad_events(events, typed_in_kde(events))
|
||||
status = _touch.send(touch)
|
||||
if touch and not status.get("sent"):
|
||||
return status
|
||||
if touch:
|
||||
_note_typed(touch)
|
||||
if slow:
|
||||
_queue_for_kde(slow)
|
||||
return {**status, "sent": True}
|
||||
|
||||
|
||||
# Typing KDE Connect is to do: it starts the first time it's needed, so this waits for it.
|
||||
# Nothing on the Frame says when typed text has landed, so each way holds the other back
|
||||
# for as long as what it was given should take (a short margin on a time estimate).
|
||||
_kde_queue = []
|
||||
_kde_lock = threading.Lock()
|
||||
_pad_send_lock = threading.Lock()
|
||||
_kde_worker = [None]
|
||||
_kde_until = 0.0 # keys typed through KDE Connect should have landed by then
|
||||
_typed_until = 0.0 # when gamescope's agent will have typed everything it's been given
|
||||
_kde_route = object()
|
||||
_kde_cancel = threading.Event()
|
||||
KDE_QUEUE_LIMIT = 200
|
||||
KDE_WAIT = 90 # seconds to wait for KDE Connect before giving up on what's queued
|
||||
TYPING_SETTLE = 0.6 # seconds past the estimate, once
|
||||
|
||||
|
||||
def reset_pad_route():
|
||||
"""Discard input for the headset being left, including batches waiting for an agent."""
|
||||
global _kde_route, _kde_cancel, _kde_until, _typed_until
|
||||
with _pad_send_lock, _kde_lock:
|
||||
_kde_cancel.set()
|
||||
_kde_cancel = threading.Event()
|
||||
_kde_route = object()
|
||||
_kde_queue.clear()
|
||||
_kde_worker[0] = None
|
||||
_kde_until = _typed_until = 0.0
|
||||
_input.stop()
|
||||
_touch.stop()
|
||||
|
||||
|
||||
def typing_seconds(touch):
|
||||
"""How long gamescope's agent takes over these: it sleeps 8 ms after every key transition."""
|
||||
transitions = 0
|
||||
for e in touch:
|
||||
if "text" in e:
|
||||
transitions += sum(2 + 2 * frame_touch.ASCII[ch][1] for ch in e["text"] if ch in frame_touch.ASCII)
|
||||
elif "key" in e:
|
||||
transitions += 1
|
||||
return transitions * 0.008
|
||||
|
||||
|
||||
def _note_typed(touch):
|
||||
global _typed_until
|
||||
seconds = typing_seconds(touch)
|
||||
if seconds:
|
||||
with _kde_lock: # the end of the work, which queues up; the margin is added once, where it's checked
|
||||
_typed_until = max(_typed_until, time.time()) + seconds
|
||||
|
||||
|
||||
def _queue_for_kde(events):
|
||||
with _kde_lock:
|
||||
_kde_queue.extend(events)
|
||||
del _kde_queue[:-KDE_QUEUE_LIMIT]
|
||||
if _kde_worker[0] is None:
|
||||
_kde_worker[0] = threading.Thread(target=_drain_kde, args=(_kde_route, _kde_cancel), daemon=True)
|
||||
_kde_worker[0].start()
|
||||
|
||||
|
||||
def _drain_kde(route, cancel):
|
||||
"""The only sender, in order. A batch leaves the queue while it's being sent (the queue
|
||||
can be trimmed meanwhile) and goes back to the front if it didn't go."""
|
||||
global _kde_until
|
||||
deadline = time.time() + KDE_WAIT
|
||||
while True:
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
batch = _kde_queue[:INPUT_BATCH_LIMIT]
|
||||
del _kde_queue[:len(batch)]
|
||||
if not batch:
|
||||
_kde_worker[0] = None # retired under the lock, so a new request starts another
|
||||
return
|
||||
wait = _typed_until + TYPING_SETTLE - time.time()
|
||||
_kde_until = time.time() + 3600 # held while in flight
|
||||
if wait > 0 and cancel.wait(wait):
|
||||
return # a switch cancels the delay and its old-headset batch
|
||||
# Serialize sending with route resets without blocking callers from queuing input.
|
||||
with _pad_send_lock:
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
status = _input.send(batch)
|
||||
with _kde_lock:
|
||||
if route is not _kde_route:
|
||||
return
|
||||
if status.get("sent"):
|
||||
_kde_until = time.time() + len(batch) * 0.03 + TYPING_SETTLE
|
||||
deadline = time.time() + KDE_WAIT
|
||||
continue
|
||||
_kde_until = 0.0
|
||||
if status.get("state") == "error" or time.time() > deadline:
|
||||
_kde_queue.clear() # not coming; don't hold it forever
|
||||
_kde_worker[0] = None
|
||||
return
|
||||
_kde_queue[:0] = batch
|
||||
del _kde_queue[:-KDE_QUEUE_LIMIT] # as when queuing: the oldest go first
|
||||
if cancel.wait(0.4):
|
||||
return
|
||||
|
||||
|
||||
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
|
||||
@@ -2399,16 +2580,6 @@ def source_text(body, key, optional=False):
|
||||
return value.strip()
|
||||
|
||||
|
||||
def login_text(body, key):
|
||||
"""A username, password or token: kept exactly (no stripping) and never echoed back."""
|
||||
value = body.get(key)
|
||||
if value in (None, ''):
|
||||
return None
|
||||
if not isinstance(value, str) or len(value) > 4096:
|
||||
raise Failure('Provide a valid ' + key, 400)
|
||||
return value
|
||||
|
||||
|
||||
def source_search(query):
|
||||
args = parse_qs(query)
|
||||
q = args.get('q', [''])[0]
|
||||
@@ -2447,31 +2618,19 @@ def source_manage(body):
|
||||
if action == 'add':
|
||||
url = source_text(body, 'url')
|
||||
fingerprint, name = source_text(body, 'fingerprint', True), source_text(body, 'name', True)
|
||||
username, password = login_text(body, 'username'), login_text(body, 'password')
|
||||
try:
|
||||
parts = urlparse(url)
|
||||
parts.port
|
||||
except ValueError:
|
||||
raise Failure('Use an HTTPS or fdroidrepos:// repository URL', 400) from None
|
||||
if parts.scheme not in ('https', 'fdroidrepos') or not parts.hostname:
|
||||
raise Failure('Use an HTTPS or fdroidrepos:// repository URL', 400)
|
||||
parts = urlparse(url)
|
||||
if parts.scheme not in ('https', 'fdroidrepos') or not parts.hostname or parts.username:
|
||||
raise Failure('Use an HTTPS or fdroidrepos:// repository URL without credentials', 400)
|
||||
apk_search.repo_module() # fail now if this build can't manage repositories
|
||||
|
||||
def add(): # downloads and verifies the whole index: a job, not a request
|
||||
# A login (here or in the URL) is stored apart and never comes back in a response.
|
||||
login = {'username': username, 'password': password} if username or password else {}
|
||||
source = apk_search.manage_repo('add_repo', url=url, fingerprint=fingerprint, name=name, **login)
|
||||
source = apk_search.manage_repo('add_repo', url=url, fingerprint=fingerprint, name=name)
|
||||
message = 'Added ' + source['name']
|
||||
if source.get('trust_on_first_use'):
|
||||
message += '. Trusted on first use: ' + source['fingerprint'].upper()
|
||||
return {'message': message, 'source': {k: source.get(k) for k in
|
||||
('id', 'name', 'fingerprint', 'trust_on_first_use')}}
|
||||
return start_job('Add repository', add)
|
||||
if action in ('set-token', 'clear-token'):
|
||||
token = login_text(body, 'token') if action == 'set-token' else None
|
||||
if action == 'set-token' and not token:
|
||||
raise Failure('Paste a token', 400)
|
||||
return apk_search.set_token(source_text(body, 'source'), token)
|
||||
if action == 'game-data':
|
||||
package = source_text(body, 'package')
|
||||
return start_job('Add game data', lambda: apk_search.add_game_data(package))
|
||||
@@ -2730,7 +2889,7 @@ class Handler(BaseHTTPRequestHandler):
|
||||
elif path == "/api/touch":
|
||||
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
|
||||
elif path == "/api/input":
|
||||
self.send_json(_input.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_input.status))
|
||||
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
|
||||
elif path == "/api/job":
|
||||
self.send_json(job_status(url.query))
|
||||
elif path == "/api/queue":
|
||||
|
||||
Reference in new issue
Block a user