mirror of
https://github.com/Cogential/frametanium.git
synced 2026-10-17 13:00:27 +02:00
Discover new Steam Frame apps on GitHub
Searches GitHub (steam-frame topic, "steam frame" in names/descriptions) and the awesome-steam-frame list, checks each project's releases for a download the Frame can install, and lists them with New badges, in the headset app (X / Discover row) and on the web page. Cached, rechecked only when a repo changes, and limited per run so update checks keep enough of GitHub's unauthenticated rate limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VHDaz86aayqMULHsjByEmT
This commit is contained in:
CogentialandClaude Opus 5.5 committed
1 parent
c6b3edf252
commit
dca76585bc
4 files changed
+434
-19
No files matched your search
@@ -6,6 +6,12 @@ and Frametanium keeps it up to date. Works from the headset or a browser.
|
||||
- **Add any GitHub repo** by typing `owner/name` (or pasting a link) on the headset's on-screen keyboard, from the
|
||||
browser page, or by picking a suggestion made for the Frame. The built-in ports (Ship of Harkinian, 2 Ship 2
|
||||
Harkinian) are there from the start.
|
||||
- **Discover new Frame apps:** every 12 hours (or on demand) it searches GitHub for Steam Frame projects (the
|
||||
`steam-frame` topic, "steam frame" in names and descriptions, and the community's
|
||||
[awesome-steam-frame](https://github.com/kosantosbik/awesome-steam-frame) list) and shows the ones whose latest
|
||||
release has a download it can install, with ones you haven't seen marked **New**. Pick one to look at and add it,
|
||||
or hide it. Without a GitHub token, GitHub allows 60 lookups an hour, so each search checks a batch (keeping some
|
||||
for update checks) and the next search carries on; results are cached and only rechecked when a repo changes.
|
||||
- **Picks the right download:** ARM64 Linux builds (`aarch64`/`arm64`, "steamframe"); checksums, Windows, macOS,
|
||||
Android and x86 files are passed over. You can choose another one when adding. Zips, tarballs (gz/xz/bz2/zst),
|
||||
AppImages and single programs all work.
|
||||
@@ -23,7 +29,7 @@ and Frametanium keeps it up to date. Works from the headset or a browser.
|
||||
|
||||
## Use it
|
||||
- **In the headset:** open **Frametanium** in the library. D-pad to choose, **A** open an app (install/update,
|
||||
versions, previous version, pre-releases, remove), **Y** add an app, **Select** update all, **Start** check for
|
||||
versions, previous version, pre-releases, remove), **Y** add an app, **X** discover new apps, **Select** update all, **Start** check for
|
||||
updates, **B** back/close.
|
||||
- **From a phone or PC:** `http://<frame-ip>:8643` (add apps, edit name/install style/program, remove).
|
||||
|
||||
@@ -32,6 +38,8 @@ Updates are checked every 6 hours and on demand.
|
||||
## Files
|
||||
- `catalog.json`: the built-in ports.
|
||||
- `suggestions.json`: apps offered on the Add screen.
|
||||
- `~/.config/frametanium/discover.json`: discovered projects you've seen or hidden; `~/.cache/frametanium/discover.json`
|
||||
caches what each search found.
|
||||
- `~/.config/frametanium/catalog.json` on the Frame: the apps you added and changes to built-in ones
|
||||
(`{"id": "soh", "hidden": true}` hides one; adding its repo again brings it back).
|
||||
- `~/.config/frametanium/installed.json`: installed versions.
|
||||
|
||||
@@ -3,11 +3,11 @@
|
||||
* tab-separated replies.
|
||||
*
|
||||
* Screens: the app list; an app's menu (install/update, versions, previous version, pre-releases, remove); adding
|
||||
* an app (type a GitHub repo on an on-screen keyboard, paste a link, or pick a suggestion); and the found repo,
|
||||
* where you pick its download and add it.
|
||||
* an app (type a GitHub repo on an on-screen keyboard, paste a link, or pick a suggestion); discover (new Steam Frame
|
||||
* projects on GitHub with a release to install); and the found repo, where you pick its download and add it.
|
||||
*
|
||||
* Controls: D-pad/left stick to choose, A select, B back/close, Y add an app, Select update all, Start check for
|
||||
* updates. Keyboard: arrows, Enter, Esc, and typing on the on-screen keyboard screen. */
|
||||
* Controls: D-pad/left stick to choose, A select, B back/close, Y add an app, X discover, Select update all, Start
|
||||
* check for updates. Keyboard: arrows, Enter, Esc, and typing on the on-screen keyboard screen. */
|
||||
|
||||
#include <SDL.h>
|
||||
#include <arpa/inet.h>
|
||||
@@ -28,6 +28,7 @@
|
||||
#define MAX_RELEASES 40
|
||||
#define MAX_SUGGEST 32
|
||||
#define MAX_ASSETS 16
|
||||
#define MAX_DISC 64
|
||||
|
||||
typedef struct {
|
||||
char id[64], name[128], installed[64], latest[64], status[32], message[256], repo[128], mode[16], job[16];
|
||||
@@ -44,6 +45,11 @@ typedef struct {
|
||||
char repo[128], name[128], note[160];
|
||||
} Suggestion;
|
||||
|
||||
typedef struct {
|
||||
char repo[128], name[128], description[300], tag[64], date[16], asset[128];
|
||||
int stars, arm64, prerelease, is_new;
|
||||
} Disc;
|
||||
|
||||
typedef struct {
|
||||
char repo[128], name[128], description[300], tag[64], date[16], existing[64];
|
||||
int prereleases, nassets, asset;
|
||||
@@ -297,6 +303,41 @@ static int load_suggestions(Suggestion* sug) {
|
||||
return n;
|
||||
}
|
||||
|
||||
/* New Steam Frame projects found on GitHub; running says a search is going, msg is its progress. */
|
||||
static int load_discover(Disc* d, int* running, char* msg, size_t msz, int* nnew) {
|
||||
char* body = http("GET", "/api/discover?fmt=tsv", 5);
|
||||
*running = 0;
|
||||
*nnew = 0;
|
||||
msg[0] = '\0';
|
||||
if (!body) return -1;
|
||||
int n = 0;
|
||||
char* save;
|
||||
EACH_LINE(body, line, save) {
|
||||
char* f[11];
|
||||
int k = split(line, f, 11);
|
||||
if (k >= 4 && !strcmp(f[0], "#")) {
|
||||
*running = atoi(f[1]);
|
||||
copy(msg, msz, f[2]);
|
||||
*nnew = atoi(f[3]);
|
||||
continue;
|
||||
}
|
||||
if (k < 10 || n >= MAX_DISC) continue;
|
||||
Disc* x = &d[n++];
|
||||
copy(x->repo, sizeof x->repo, f[0]);
|
||||
copy(x->name, sizeof x->name, f[1]);
|
||||
copy(x->description, sizeof x->description, f[2]);
|
||||
x->stars = atoi(f[3]);
|
||||
copy(x->tag, sizeof x->tag, f[4]);
|
||||
copy(x->date, sizeof x->date, f[5]);
|
||||
copy(x->asset, sizeof x->asset, f[6]);
|
||||
x->arm64 = atoi(f[7]);
|
||||
x->prerelease = atoi(f[8]);
|
||||
x->is_new = atoi(f[9]);
|
||||
}
|
||||
free(body);
|
||||
return n;
|
||||
}
|
||||
|
||||
/* Looks a repo up; fills fd and returns 1, or writes the problem to err and returns 0. */
|
||||
static int inspect(const char* repo, Found* fd, char* err, size_t errsz) {
|
||||
char enc[600], path[700];
|
||||
@@ -433,7 +474,7 @@ static int scroll_first(int sel, int max_rows) {
|
||||
return sel >= max_rows ? sel - max_rows + 1 : 0;
|
||||
}
|
||||
|
||||
static void draw_list(App* apps, int n, int sel, const char* banner, Uint32 banner_rgb) {
|
||||
static void draw_list(App* apps, int n, int sel, const char* banner, Uint32 banner_rgb, int disc_new) {
|
||||
clear();
|
||||
int m = W / 20, y = title("Frametanium");
|
||||
if (banner && *banner) {
|
||||
@@ -450,13 +491,19 @@ static void draw_list(App* apps, int n, int sel, const char* banner, Uint32 bann
|
||||
int max_rows = (H - y - 40 * S) / row_h;
|
||||
if (max_rows < 1) max_rows = 1;
|
||||
int first = scroll_first(sel, max_rows);
|
||||
for (int i = first; i <= n && i < first + max_rows; i++) {
|
||||
for (int i = first; i <= n + 1 && i < first + max_rows; i++) {
|
||||
int x = m, w = W - 2 * m;
|
||||
if (i == n) { /* the "add" row */
|
||||
if (i >= n) { /* the "discover" and "add" rows */
|
||||
fill(x, y, w, 26 * S, i == sel ? ROW_SEL : ROW);
|
||||
if (i == sel) fill(x, y, 3 * S, 26 * S, ACCENT);
|
||||
text("+ Add an app from GitHub", x + 10 * S, y + 9 * S, S, ACCENT);
|
||||
break;
|
||||
char label[100];
|
||||
if (i == n && disc_new > 0)
|
||||
snprintf(label, sizeof label, "* Discover new apps for the Frame (%d new)", disc_new);
|
||||
else
|
||||
snprintf(label, sizeof label, "%s", i == n ? "* Discover new apps for the Frame" : "+ Add an app from GitHub");
|
||||
text(label, x + 10 * S, y + 9 * S, S, ACCENT);
|
||||
y += 32 * S;
|
||||
continue;
|
||||
}
|
||||
App* p = &apps[i];
|
||||
fill(x, y, w, row_h - 6 * S, i == sel ? ROW_SEL : ROW);
|
||||
@@ -481,7 +528,7 @@ static void draw_list(App* apps, int n, int sel, const char* banner, Uint32 bann
|
||||
}
|
||||
y += row_h;
|
||||
}
|
||||
legend("A Open Y Add app Select Update all Start Check B Close");
|
||||
legend("A Open Y Add X Discover Select Update all Start Check B Close");
|
||||
SDL_RenderPresent(ren);
|
||||
}
|
||||
|
||||
@@ -520,6 +567,49 @@ static void draw_wait(const char* heading, const char* what) {
|
||||
SDL_RenderPresent(ren);
|
||||
}
|
||||
|
||||
static void draw_discover(const Disc* d, int n, int sel, int running, const char* msg, const char* err) {
|
||||
clear();
|
||||
int m = W / 20, y = title("Discover new apps for the Frame");
|
||||
const char* sub = running ? (msg[0] ? msg : "Searching GitHub for Steam Frame projects...")
|
||||
: msg[0] ? msg : "Steam Frame projects on GitHub with a release you can install.";
|
||||
text_fit(sub, m, y, S, W - 2 * m, running ? ACCENT : MUTED);
|
||||
y += 16 * S;
|
||||
if (err && *err) {
|
||||
text_fit(err, m, y, S, W - 2 * m, BAD);
|
||||
y += 14 * S;
|
||||
}
|
||||
if (n <= 0) {
|
||||
text(n < 0 ? "The Frametanium service didn't answer." : running ? "" : "Nothing new right now. Start searches again.",
|
||||
m, y + 8 * S, S, MUTED);
|
||||
}
|
||||
int row_h = 50 * S;
|
||||
int max_rows = (H - y - 40 * S) / row_h;
|
||||
if (max_rows < 1) max_rows = 1;
|
||||
int first = scroll_first(sel, max_rows);
|
||||
for (int i = first; i < n && i < first + max_rows; i++) {
|
||||
const Disc* x = &d[i];
|
||||
int w = W - 2 * m, tx = m + 10 * S, ty = y + 8 * S;
|
||||
fill(m, y, w, row_h - 6 * S, i == sel ? ROW_SEL : ROW);
|
||||
if (i == sel) fill(m, y, 3 * S, row_h - 6 * S, ACCENT);
|
||||
int nw = 0;
|
||||
if (x->is_new) {
|
||||
nw = 6 * 8 * S;
|
||||
text("NEW", m + w - 10 * S - 3 * 8 * S, ty, S, ACCENT);
|
||||
}
|
||||
text_fit(x->name, tx, ty, S, w - 20 * S - nw, TXT);
|
||||
ty += 13 * S;
|
||||
if (x->description[0]) text_fit(x->description, tx, ty, S, w - 20 * S, MUTED);
|
||||
ty += 12 * S;
|
||||
char line[400];
|
||||
snprintf(line, sizeof line, "%s %s%s %s %d stars%s", x->repo, x->tag, x->prerelease ? " (pre)" : "", x->date,
|
||||
x->stars, x->arm64 ? "" : " not marked ARM64");
|
||||
text_fit(line, tx, ty, S, w - 20 * S, x->arm64 ? MUTED : WARN);
|
||||
y += row_h;
|
||||
}
|
||||
legend("A Look at it X Not interested Start Search again B Back");
|
||||
SDL_RenderPresent(ren);
|
||||
}
|
||||
|
||||
/* ---- on-screen keyboard */
|
||||
|
||||
static const char* KB_ROWS[] = { "1234567890", "qwertyuiop", "asdfghjkl-", "zxcvbnm_./" };
|
||||
@@ -605,7 +695,7 @@ static void draw_found(const Found* fd, const char* err) {
|
||||
/* ---------------------------------------------------------------- main */
|
||||
|
||||
enum { NONE, UP, DOWN, LEFT, RIGHT, OK, BACK, ADD, UPDATE_ALL, REFRESH, DELETE };
|
||||
enum { LIST, MENU, VERSIONS, CONFIRM, ADDING, KEYBOARD, FOUND };
|
||||
enum { LIST, MENU, VERSIONS, CONFIRM, ADDING, KEYBOARD, FOUND, DISCOVER };
|
||||
|
||||
static int map_button(int b) {
|
||||
switch (b) {
|
||||
@@ -639,13 +729,14 @@ static int map_key(SDL_Keycode k, int typing) {
|
||||
case SDLK_a: return ADD;
|
||||
case SDLK_u: return UPDATE_ALL;
|
||||
case SDLK_r: return REFRESH;
|
||||
case SDLK_x: return DELETE;
|
||||
default: return NONE;
|
||||
}
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
/* PM_SCREENSHOT=<file.bmp> draws one frame into memory, saves it and exits (testing without a display);
|
||||
* PM_SIZE=WxH sets its size, PM_SCREEN=menu|versions|add|keyboard|found picks the screen (for the first app;
|
||||
* PM_SIZE=WxH sets its size, PM_SCREEN=menu|versions|add|keyboard|found|discover picks the screen (for the first app;
|
||||
* PM_REPO is the repo for "found", PM_TYPED the text for "keyboard"). */
|
||||
const char* shot = getenv("PM_SCREENSHOT");
|
||||
SDL_Surface* shot_surface = NULL;
|
||||
@@ -675,7 +766,11 @@ int main(void) {
|
||||
static Release rels[MAX_RELEASES];
|
||||
static Suggestion sug[MAX_SUGGEST];
|
||||
static Found fd;
|
||||
static Disc disc[MAX_DISC];
|
||||
int napps = -1, sel = 0, nrels = 0, rsel = 0, msel = 0, asel = 0, nsug = 0, tried_start = 0;
|
||||
int ndisc = 0, dsel = 0, disc_running = 0, disc_new = 0, found_from = ADDING;
|
||||
char disc_msg[300] = "";
|
||||
Uint32 next_disc_poll = 0;
|
||||
int screen = LIST, kb_row = 1, kb_col = 0, notice_busy = 0;
|
||||
char err[300] = "", banner[300] = "", notice[300] = "", entry[256] = "", want_id[64] = "";
|
||||
char hint_name[128] = "";
|
||||
@@ -707,6 +802,8 @@ int main(void) {
|
||||
screen = FOUND;
|
||||
}
|
||||
if (!strcmp(sc, "confirm")) screen = CONFIRM;
|
||||
if (!strcmp(sc, "discover")) screen = DISCOVER;
|
||||
ndisc = load_discover(disc, &disc_running, disc_msg, sizeof disc_msg, &disc_new);
|
||||
next_poll = 0xffffffff;
|
||||
}
|
||||
|
||||
@@ -742,24 +839,34 @@ int main(void) {
|
||||
sel = i;
|
||||
want_id[0] = '\0';
|
||||
}
|
||||
if (sel > napps) sel = napps > 0 ? napps : 0;
|
||||
if (sel > napps + 1) sel = napps > 0 ? napps : 0;
|
||||
snprintf(banner, sizeof banner, "%s",
|
||||
napps < 0 && tried_start && SDL_GetTicks() > 8000
|
||||
? "The Frametanium service isn't running. Restart the Frame and try again."
|
||||
: notice);
|
||||
}
|
||||
if (SDL_GetTicks() >= next_disc_poll) {
|
||||
ndisc = load_discover(disc, &disc_running, disc_msg, sizeof disc_msg, &disc_new);
|
||||
next_disc_poll = SDL_GetTicks() + (screen == DISCOVER && disc_running ? 1500 : 30000);
|
||||
if (dsel >= ndisc) dsel = ndisc > 0 ? ndisc - 1 : 0;
|
||||
}
|
||||
App* cur = sel < napps ? &apps[sel] : NULL;
|
||||
|
||||
switch (screen) {
|
||||
case LIST:
|
||||
if (act == UP && sel > 0) sel--;
|
||||
if (act == DOWN && sel < napps) sel++;
|
||||
if (act == DOWN && sel < napps + 1) sel++;
|
||||
if (act == BACK) goto done;
|
||||
if (napps < 0) break;
|
||||
if (act == OK && cur) {
|
||||
screen = MENU;
|
||||
msel = 0;
|
||||
err[0] = '\0';
|
||||
} else if (act == DELETE || (act == OK && sel == napps)) {
|
||||
screen = DISCOVER;
|
||||
dsel = 0;
|
||||
err[0] = '\0';
|
||||
next_disc_poll = 0;
|
||||
} else if (act == ADD || (act == OK && !cur)) {
|
||||
draw_wait("Add an app from GitHub", "Loading suggestions...");
|
||||
nsug = load_suggestions(sug);
|
||||
@@ -877,6 +984,7 @@ int main(void) {
|
||||
if (inspect(repo, &fd, err, sizeof err)) {
|
||||
if (hint_name[0]) snprintf(fd.name, sizeof fd.name, "%s", hint_name);
|
||||
err[0] = '\0';
|
||||
found_from = ADDING;
|
||||
screen = FOUND;
|
||||
}
|
||||
}
|
||||
@@ -885,6 +993,36 @@ int main(void) {
|
||||
break;
|
||||
}
|
||||
|
||||
case DISCOVER:
|
||||
if (act == UP && dsel > 0) dsel--;
|
||||
if (act == DOWN && dsel < ndisc - 1) dsel++;
|
||||
if (act == BACK) {
|
||||
if (disc_new > 0) post("/api/discover/seen", NULL, 0); /* everything listed has now been seen */
|
||||
screen = LIST;
|
||||
next_disc_poll = SDL_GetTicks() + 300;
|
||||
}
|
||||
if (act == REFRESH && !disc_running) {
|
||||
post("/api/discover/refresh", NULL, 0);
|
||||
next_disc_poll = SDL_GetTicks() + 400;
|
||||
}
|
||||
if (act == DELETE && dsel < ndisc) {
|
||||
char enc[300], path[400];
|
||||
urlenc(enc, sizeof enc, disc[dsel].repo);
|
||||
snprintf(path, sizeof path, "/api/discover/hide?repo=%s", enc);
|
||||
post(path, NULL, 0);
|
||||
next_disc_poll = 0;
|
||||
}
|
||||
if (act == OK && dsel < ndisc) {
|
||||
draw_wait(disc[dsel].name, "Looking it up on GitHub...");
|
||||
if (inspect(disc[dsel].repo, &fd, err, sizeof err)) {
|
||||
snprintf(fd.name, sizeof fd.name, "%s", disc[dsel].name);
|
||||
err[0] = '\0';
|
||||
found_from = DISCOVER;
|
||||
screen = FOUND;
|
||||
}
|
||||
}
|
||||
break;
|
||||
|
||||
case KEYBOARD: {
|
||||
size_t len = strlen(entry);
|
||||
if (typed && len + strlen(typed) < sizeof entry - 1) strcat(entry, typed);
|
||||
@@ -923,6 +1061,7 @@ int main(void) {
|
||||
if (inspect(entry, &fd, err, sizeof err)) {
|
||||
err[0] = '\0';
|
||||
SDL_StopTextInput();
|
||||
found_from = ADDING;
|
||||
screen = FOUND;
|
||||
}
|
||||
}
|
||||
@@ -930,7 +1069,10 @@ int main(void) {
|
||||
}
|
||||
|
||||
case FOUND:
|
||||
if (act == BACK) screen = ADDING;
|
||||
if (act == BACK) {
|
||||
screen = found_from;
|
||||
err[0] = '\0';
|
||||
}
|
||||
if (fd.existing[0]) {
|
||||
if (act == OK) {
|
||||
snprintf(want_id, sizeof want_id, "%s", fd.existing);
|
||||
@@ -958,6 +1100,7 @@ int main(void) {
|
||||
screen = LIST;
|
||||
entry[0] = '\0';
|
||||
next_poll = 0;
|
||||
next_disc_poll = 0;
|
||||
}
|
||||
free(body);
|
||||
}
|
||||
@@ -1006,10 +1149,12 @@ int main(void) {
|
||||
draw_keyboard(entry, kb_row, kb_col, err);
|
||||
} else if (screen == FOUND) {
|
||||
draw_found(&fd, err);
|
||||
} else if (screen == DISCOVER) {
|
||||
draw_discover(disc, ndisc, dsel, disc_running, disc_msg, err);
|
||||
} else {
|
||||
if (screen != LIST) screen = LIST;
|
||||
int is_err = strstr(banner, "isn't running") != NULL;
|
||||
draw_list(apps, napps, sel, banner, is_err ? WARN : OK_C);
|
||||
draw_list(apps, napps, sel, banner, is_err ? WARN : OK_C, disc_new);
|
||||
}
|
||||
if (shot) {
|
||||
SDL_SaveBMP(shot_surface, shot);
|
||||
|
||||
+205
-1
@@ -130,9 +130,15 @@ def gh_request(url, accept="application/vnd.github+json"):
|
||||
return urllib.request.Request(url, headers=headers)
|
||||
|
||||
|
||||
_rate = {"core": None, "search": None} # requests GitHub still allows this hour (or minute, for search)
|
||||
|
||||
|
||||
def gh_json(url):
|
||||
try:
|
||||
with urllib.request.urlopen(gh_request(url), timeout=20) as r:
|
||||
left = r.headers.get("X-RateLimit-Remaining")
|
||||
if left is not None:
|
||||
_rate[r.headers.get("X-RateLimit-Resource") or "core"] = int(left)
|
||||
return json.load(r)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
@@ -329,6 +335,179 @@ def add_app(repo_text, asset=None, name=None, mode="auto", prereleases=None):
|
||||
return pid
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- discover
|
||||
|
||||
# Finding new Steam Frame projects: GitHub search plus the community's awesome-steam-frame list give the candidates,
|
||||
# and each candidate's releases are checked for a download the Frame can install. Without a token GitHub allows 60
|
||||
# requests an hour, so the verdicts are cached (and rechecked when the repo changes) and each run checks a batch,
|
||||
# leaving room for update checks; repeated runs work through the rest.
|
||||
|
||||
DISCOVER_FILE = CONFIG_DIR / "discover.json" # {"seen": [...], "hidden": [...]}: what the user has looked at
|
||||
DISCOVER_CACHE = CACHE_DIR / "discover.json" # {"repos": {repo: candidate + verdict}, "last_run": time}
|
||||
DISCOVER_QUERIES = ['topic:steam-frame', 'topic:steamframe', '"steam frame" in:name,description',
|
||||
'steamframe in:name,description', '"steam-frame" in:name']
|
||||
AWESOME_LISTS = ["kosantosbik/awesome-steam-frame"]
|
||||
DISCOVER_EVERY = 12 * 3600
|
||||
_discover = {"running": False, "message": "", "lock": threading.Lock()}
|
||||
|
||||
|
||||
def awesome_candidates():
|
||||
"""GitHub repos linked from the awesome lists, with the list's name and description for them."""
|
||||
out = {}
|
||||
for lst in AWESOME_LISTS:
|
||||
try:
|
||||
req = urllib.request.Request(f"https://raw.githubusercontent.com/{lst}/HEAD/README.md",
|
||||
headers={"User-Agent": "frametanium"})
|
||||
with urllib.request.urlopen(req, timeout=20) as r:
|
||||
text = r.read(512 * 1024).decode("utf-8", "replace")
|
||||
except OSError:
|
||||
continue
|
||||
for m in re.finditer(r"^\s*[-*] \[([^\]]+)\]\(https://github\.com/([\w.-]+/[\w.-]+?)/?\)\s*-?\s*(.*)$",
|
||||
text, re.M):
|
||||
out[m.group(2)] = {"repo": m.group(2), "name": m.group(1).strip(), "description": m.group(3).strip(),
|
||||
"stars": None, "pushed": None, "listed": True}
|
||||
return out
|
||||
|
||||
|
||||
def search_candidates():
|
||||
out = {}
|
||||
for q in DISCOVER_QUERIES:
|
||||
url = "https://api.github.com/search/repositories?" + urllib.parse.urlencode(
|
||||
{"q": q, "sort": "updated", "order": "desc", "per_page": 100})
|
||||
try:
|
||||
items = gh_json(url).get("items", [])
|
||||
except RuntimeError:
|
||||
continue
|
||||
for it in items:
|
||||
if it.get("archived") or it.get("fork"):
|
||||
continue
|
||||
out[it["full_name"]] = {
|
||||
"repo": it["full_name"], "name": nice_name(it["name"]),
|
||||
"description": re.sub(r"\s+", " ", it.get("description") or "").strip(),
|
||||
"stars": it.get("stargazers_count", 0), "pushed": it.get("pushed_at"),
|
||||
"topic": q.startswith("topic:"),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def check_candidate(c):
|
||||
"""Fills in the newest release with a download for the Frame, or "none"."""
|
||||
c["checked"] = int(time.time())
|
||||
c["checked_pushed"] = c.get("pushed")
|
||||
c.pop("release", None)
|
||||
for rel in gh_json(f"https://api.github.com/repos/{c['repo']}/releases?per_page=5"):
|
||||
if rel.get("draft"):
|
||||
continue
|
||||
asset = choose_asset({}, rel.get("assets", []))
|
||||
if asset:
|
||||
c["release"] = {"tag": rel["tag_name"], "date": (rel.get("published_at") or "")[:10],
|
||||
"asset": asset["name"], "arm64": bool(OUR_CPU.search(asset["name"])),
|
||||
"prerelease": bool(rel.get("prerelease"))}
|
||||
return
|
||||
|
||||
|
||||
def needs_check(c, now):
|
||||
if "checked" not in c:
|
||||
return True
|
||||
if c.get("pushed") and c.get("pushed") != c.get("checked_pushed"):
|
||||
return True
|
||||
return now - c["checked"] > (2 if c.get("release") else 7) * 86400
|
||||
|
||||
|
||||
def discover(force=False):
|
||||
"""One discovery run; returns False if one is already going or the last one was recent."""
|
||||
if not _discover["lock"].acquire(blocking=False):
|
||||
return False
|
||||
try:
|
||||
cache = load_json(DISCOVER_CACHE, {"repos": {}, "last_run": 0})
|
||||
if not force and time.time() - cache.get("last_run", 0) < DISCOVER_EVERY:
|
||||
return False
|
||||
_discover.update(running=True, message="Searching GitHub for Steam Frame projects...")
|
||||
found = awesome_candidates()
|
||||
for repo, c in search_candidates().items():
|
||||
listed = found.get(repo, {})
|
||||
found[repo] = {**listed, **c, "listed": bool(listed)}
|
||||
if listed: # the list's hand-written name reads better than one made from the repo name
|
||||
found[repo]["name"] = listed["name"]
|
||||
found[repo]["description"] = c["description"] or listed["description"]
|
||||
repos = cache["repos"]
|
||||
for repo, c in found.items():
|
||||
old = repos.get(repo, {})
|
||||
keep = {k: old[k] for k in ("checked", "checked_pushed", "release") if k in old}
|
||||
if found[repo].get("stars") is None and old.get("stars") is not None: # list-only entry seen in search before
|
||||
keep.update(stars=old["stars"], pushed=old.get("pushed"))
|
||||
repos[repo] = {**c, **keep}
|
||||
mine = {p["repo"].lower() for p in catalog().values()} | {"cogential/frametanium"}
|
||||
now = time.time()
|
||||
todo = [c for c in repos.values() if c["repo"].lower() not in mine and needs_check(c, now)]
|
||||
# Projects the community listed or tagged first, then the most recently changed.
|
||||
todo.sort(key=lambda c: (not c.get("listed"), not c.get("topic"), c.get("checked", 0) > 0,
|
||||
-(time.mktime(time.strptime(c["pushed"][:19], "%Y-%m-%dT%H:%M:%S"))
|
||||
if c.get("pushed") else 0)))
|
||||
budget = 150 if config().get("github_token") else 35
|
||||
checked = 0
|
||||
for c in todo[:budget]:
|
||||
if _rate["core"] is not None and _rate["core"] < (100 if config().get("github_token") else 15):
|
||||
break # leave the rest of the hour for update checks
|
||||
_discover["message"] = f"Checking {c['repo']} ({checked + 1} of {min(len(todo), budget)})..."
|
||||
try:
|
||||
check_candidate(c)
|
||||
except RuntimeError as e:
|
||||
if "not found" not in str(e):
|
||||
break # rate limit or offline: try again next run
|
||||
c["checked"], c["checked_pushed"] = int(time.time()), c.get("pushed") # deleted or renamed
|
||||
checked += 1
|
||||
left = len(todo) - checked
|
||||
cache["last_run"] = int(time.time())
|
||||
save_json(DISCOVER_CACHE, cache)
|
||||
if not left:
|
||||
_discover["message"] = ""
|
||||
elif checked:
|
||||
_discover["message"] = f"Checked {checked} projects; {left} more on the next search (GitHub's hourly limit)."
|
||||
else:
|
||||
_discover["message"] = f"GitHub's hourly limit is used up; {left} projects left to check on the next search."
|
||||
return True
|
||||
finally:
|
||||
_discover["running"] = False
|
||||
_discover["lock"].release()
|
||||
|
||||
|
||||
def discovered():
|
||||
"""Found projects with a download for the Frame that aren't on the list yet: unseen ones first, then newest."""
|
||||
cache = load_json(DISCOVER_CACHE, {"repos": {}})
|
||||
prefs = load_json(DISCOVER_FILE, {"seen": [], "hidden": []})
|
||||
seen, hidden = {r.lower() for r in prefs["seen"]}, {r.lower() for r in prefs["hidden"]}
|
||||
mine = {p["repo"].lower() for p in catalog().values()} | {"cogential/frametanium"}
|
||||
mine |= {x["repo"].lower() for x in suggestions()} # already offered as suggestions
|
||||
rows = []
|
||||
for c in cache["repos"].values():
|
||||
rel = c.get("release")
|
||||
if not rel or c["repo"].lower() in mine | hidden:
|
||||
continue
|
||||
rows.append({
|
||||
"repo": c["repo"], "name": c["name"], "description": c.get("description", "")[:300],
|
||||
"stars": c.get("stars") or 0, "tag": rel["tag"], "date": rel["date"], "asset": rel["asset"],
|
||||
"arm64": rel["arm64"], "prerelease": rel["prerelease"], "new": c["repo"].lower() not in seen,
|
||||
})
|
||||
rows.sort(key=lambda r: r["date"], reverse=True)
|
||||
rows.sort(key=lambda r: not r["new"])
|
||||
return rows
|
||||
|
||||
|
||||
def discover_mark(repos, key):
|
||||
prefs = load_json(DISCOVER_FILE, {"seen": [], "hidden": []})
|
||||
have = {r.lower() for r in prefs[key]}
|
||||
prefs[key] += [r for r in repos if r and r.lower() not in have]
|
||||
save_json(DISCOVER_FILE, prefs)
|
||||
|
||||
|
||||
def start_discover(force=True):
|
||||
if _discover["running"]:
|
||||
return False
|
||||
threading.Thread(target=discover, kwargs={"force": force}, daemon=True).start()
|
||||
return True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- Steam
|
||||
|
||||
|
||||
@@ -1045,7 +1224,7 @@ class Handler(BaseHTTPRequestHandler):
|
||||
"text/plain")
|
||||
else:
|
||||
self.reply({"apps": rows, "token_set": bool(config().get("github_token")), "busy": jobs.busy(),
|
||||
"notice": notice})
|
||||
"notice": notice, "discover_new": sum(r["new"] for r in discovered())})
|
||||
elif url.path == "/api/releases":
|
||||
port = catalog().get(arg("id"))
|
||||
if not port:
|
||||
@@ -1071,6 +1250,22 @@ class Handler(BaseHTTPRequestHandler):
|
||||
self.reply("=\t" + head + tsv(info["assets"], ["name", "size", "arm64"]), "text/plain")
|
||||
else:
|
||||
self.reply(info)
|
||||
elif url.path == "/api/discover":
|
||||
rows = discovered()
|
||||
cache = load_json(DISCOVER_CACHE, {})
|
||||
if not cache.get("last_run") and not _discover["running"]:
|
||||
start_discover()
|
||||
running = _discover["running"] or not cache.get("last_run")
|
||||
new = sum(r["new"] for r in rows)
|
||||
if fmt == "tsv":
|
||||
message = re.sub(r"[\t\n]+", " ", _discover["message"])
|
||||
head = f"#\t{int(running)}\t{message}\t{new}\n"
|
||||
self.reply(head + (tsv(rows, ["repo", "name", "description", "stars", "tag", "date", "asset",
|
||||
"arm64", "prerelease", "new"]) if rows else ""),
|
||||
"text/plain")
|
||||
else:
|
||||
self.reply({"projects": rows, "running": running, "message": _discover["message"], "new": new,
|
||||
"last_run": cache.get("last_run", 0)})
|
||||
elif url.path == "/api/suggestions":
|
||||
have = {p["repo"].lower() for p in catalog().values()}
|
||||
rows = [{**s, "added": s["repo"].lower() in have} for s in suggestions()]
|
||||
@@ -1121,6 +1316,14 @@ class Handler(BaseHTTPRequestHandler):
|
||||
start(install, new)
|
||||
self.reply(f"=\t{new}\n" if fmt == "tsv" else {"ok": True, "id": new},
|
||||
"text/plain" if fmt == "tsv" else "application/json")
|
||||
elif url.path == "/api/discover/refresh":
|
||||
self.reply({"ok": start_discover()})
|
||||
elif url.path in ("/api/discover/seen", "/api/discover/hide"):
|
||||
repos = [r for r in arg("repos").split(",") if r] or ([arg("repo")] if arg("repo") else [])
|
||||
if url.path.endswith("seen") and not repos:
|
||||
repos = [r["repo"] for r in discovered()]
|
||||
discover_mark(repos, "seen" if url.path.endswith("seen") else "hidden")
|
||||
self.reply({"ok": True})
|
||||
elif url.path == "/api/settings":
|
||||
port = catalog().get(pid)
|
||||
if not port:
|
||||
@@ -1158,6 +1361,7 @@ class Handler(BaseHTTPRequestHandler):
|
||||
def checker():
|
||||
while True:
|
||||
refresh(force=True)
|
||||
discover()
|
||||
time.sleep(max(1, float(config().get("check_hours", 6))) * 3600)
|
||||
|
||||
|
||||
|
||||
@@ -45,6 +45,11 @@
|
||||
.found { margin-top: 12px; border-top: 1px solid var(--line); padding-top: 12px; }
|
||||
.notice { color: var(--ok); margin: 0 0 12px; }
|
||||
label { display: flex; gap: 8px; align-items: center; }
|
||||
.disc { border-top: 1px solid var(--line); padding: 12px 0; }
|
||||
.disc:first-child { border-top: 0; padding-top: 4px; }
|
||||
.disc .desc { margin: 4px 0; font-size: .92rem; overflow-wrap: anywhere; }
|
||||
.badge.new { color: var(--accent); border-color: var(--accent); }
|
||||
.warn { color: var(--warn); }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -63,6 +68,15 @@
|
||||
<div class="chips" id="suggest"></div>
|
||||
</div>
|
||||
|
||||
<div class="panel" id="discover">
|
||||
<div class="top"><h2>New for the Frame on GitHub <span class="badge new" id="newcount" hidden></span></h2>
|
||||
<button id="research">Search again</button></div>
|
||||
<p class="hint" id="discstate">Projects made for the Steam Frame with a release you can install. Searched every
|
||||
12 hours.</p>
|
||||
<div id="disclist"></div>
|
||||
<button id="discmore" hidden></button>
|
||||
</div>
|
||||
|
||||
<div id="apps"></div>
|
||||
|
||||
<div class="panel">
|
||||
@@ -232,7 +246,7 @@ async function lookup(repo, name) {
|
||||
const add = async install => {
|
||||
const r = await post(`/api/add?repo=${q(info.repo)}&asset=${q(assetSel.value)}&name=${q(nameIn.value)}&mode=${q(modeSel.value)}&install=${install}`);
|
||||
if (r.error) { $("adderr").textContent = r.error; $("adderr").hidden = false; return; }
|
||||
f.hidden = true; $("repo").value = ""; last = ""; poll(); loadSuggestions();
|
||||
f.hidden = true; $("repo").value = ""; last = ""; poll(); loadSuggestions(); loadDiscover();
|
||||
};
|
||||
f.append(
|
||||
el("div", { className: "row" }, "Name", nameIn), el("p"),
|
||||
@@ -256,6 +270,49 @@ async function loadSuggestions() {
|
||||
if (box.children.length) box.prepend(el("div", { className: "hint", style: "width:100%", textContent: "Suggested for the Frame:" }));
|
||||
}
|
||||
|
||||
// ---- discover
|
||||
|
||||
const shownNew = new Set(); // repos that were new when this page opened keep their badge until reload
|
||||
let discAll = false, discTimer = null;
|
||||
async function loadDiscover() {
|
||||
const d = await get("/api/discover");
|
||||
for (const p of d.projects || []) if (p.new) shownNew.add(p.repo);
|
||||
const box = $("disclist");
|
||||
box.innerHTML = "";
|
||||
const rows = d.projects || [];
|
||||
const show = discAll ? rows : rows.slice(0, 8);
|
||||
for (const p of show) {
|
||||
const lookBtn = el("button", { className: "primary", textContent: "Look at it" });
|
||||
lookBtn.onclick = () => { $("repo").value = p.repo; lookup(p.repo, p.name); $("repo").scrollIntoView({ behavior: "smooth" }); };
|
||||
const hide = el("button", { textContent: "Not interested" });
|
||||
hide.onclick = async () => { await post(`/api/discover/hide?repo=${q(p.repo)}`); loadDiscover(); };
|
||||
const meta = `${p.tag}${p.prerelease ? " (pre-release)" : ""} · ${p.date} · ★ ${p.stars} · ${p.asset}`;
|
||||
box.append(el("div", { className: "disc" },
|
||||
el("div", { className: "top" },
|
||||
el("div", {}, el("span", { className: "name", textContent: p.name + " " }),
|
||||
shownNew.has(p.repo) ? el("span", { className: "badge new", textContent: "New" }) : "",
|
||||
el("br"), el("a", { className: "repo", href: `https://github.com/${p.repo}`, target: "_blank", rel: "noopener", textContent: p.repo }))),
|
||||
el("p", { className: "desc", textContent: p.description || "" }),
|
||||
el("p", { className: "hint", textContent: meta },
|
||||
p.arm64 ? "" : el("span", { className: "warn", textContent: " · doesn't say ARM64, check before installing" })),
|
||||
el("div", { className: "row" }, lookBtn, hide)));
|
||||
}
|
||||
if (!rows.length) box.append(el("p", { className: "hint", textContent: d.running ? "Searching…" : "Nothing new right now." }));
|
||||
$("discmore").hidden = rows.length <= 8;
|
||||
$("discmore").textContent = discAll ? "Show fewer" : `Show all ${rows.length}`;
|
||||
const n = [...shownNew].filter(r => rows.some(p => p.repo === r)).length;
|
||||
$("newcount").hidden = !n;
|
||||
$("newcount").textContent = `${n} new`;
|
||||
$("discstate").textContent = d.running ? (d.message || "Searching GitHub…")
|
||||
: (d.message || "Projects made for the Steam Frame with a release you can install. Searched every 12 hours.");
|
||||
$("research").disabled = d.running;
|
||||
clearTimeout(discTimer);
|
||||
if (d.running) discTimer = setTimeout(loadDiscover, 2000);
|
||||
else if (rows.some(p => p.new)) setTimeout(() => post("/api/discover/seen?repos=" + q(rows.map(p => p.repo).join(","))), 3000);
|
||||
}
|
||||
$("discmore").onclick = () => { discAll = !discAll; loadDiscover(); };
|
||||
$("research").onclick = async () => { await post("/api/discover/refresh"); setTimeout(loadDiscover, 500); };
|
||||
|
||||
let timer = null, last = "";
|
||||
async function poll() {
|
||||
const s = await get("/api/state");
|
||||
@@ -291,6 +348,7 @@ $("savetoken").onclick = async () => {
|
||||
};
|
||||
poll();
|
||||
loadSuggestions();
|
||||
loadDiscover();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Reference in new issue
Block a user