Compare commits

...
Author SHA1 Message Date
saphid e5f78e642d fix(android): import APK splits and external data from archives 2026-10-21 11:10:03 +11:00
Alex Southwell bcdd9d0830 Merge pull request #99: Queue Android and Quest installs
Queue Android and Quest installs; prepare Frame Control 0.4.7
2026-10-15 13:32:54 +11:00
13 changed files with 1172 additions and 30 deletions

No files matched your search

+19
View File
@@ -188,6 +188,17 @@
},
"deb": {
"depends": [
"libgtk-3-0",
"libnotify4",
"libnss3",
"libxss1",
"libxtst6",
"xdg-utils",
"libatspi2.0-0",
"libuuid1",
"libsecret-1-0",
"libgbm1",
"libasound2t64 | libasound2",
"openssh-client"
]
},
@@ -204,6 +215,14 @@
"perMachine": false,
"allowToChangeInstallationDirectory": true,
"artifactName": "Frame-Control-Setup-${arch}.${ext}"
},
"toolsets": {
"appimage": "1.0.3"
},
"appImage": {
"executableArgs": [
"--no-sandbox"
]
}
},
"homepage": "https://github.com/saphid/steam-frame",
+127 -1
View File
@@ -43,7 +43,9 @@ Split APKs (a base plus `split_*.apk` files) install as one app:
`python3 ui/frame_android.py install base.apk split1.apk …`. Lepton runs one
APK, so splits that add native libraries or assets are merged into the base
and re-signed; density and language splits are refused. See
[quest-import.md](quest-import.md#split-apks). Not yet tried on a real Frame.
[quest-import.md](quest-import.md#split-apks). A generated native app plus asset
split was installed and relaunched on a real Frame on 2026-10-18; this does not
verify arbitrary games or unsupported split types.
Verified with AntennaPod and Tabletop Tools: installed in about 7 s, launched
from the shortcut, stopped, and relaunched with their data intact. ADB and
@@ -58,6 +60,130 @@ Lepton Development must be installed once. Over SSH,
`ssh frame 'steam steam://install/3056000'` queues it, but the install still
needs to be confirmed or started in the headset.
## Games with extra data or split APKs
**Send to Frame accepts Android package sets, not just single APKs.** Select
matching files together or drop a ZIP-based Android bundle. Files are grouped
by their actual manifest package; the base is identified from its manifest,
not its filename or selection order. Imports wait in the existing install
queue and stay tied to the selected headset.
| Input | What happens |
|---|---|
| One `.apk` | Checks API/ABI/VR compatibility and installs as its own app, retaining its filename and install reporting. Replacing an installed app with an older standalone APK remains possible. A split without its base, or a base marked as requiring missing splits, is refused. |
| Base plus split `.apk` files selected together | Merges supported native-library and asset splits into one APK for Lepton. All must match the package and version. |
| `.xapk`, `.apks`, `.apkm` | Unpacks and checks one app's APK set, standard OBB files and `Android/data` files, then uses the same install/data paths. |
| Android `.zip` | Detected by its APK/OBB/data contents. PC-game ZIPs keep the title-import route. A ZIP containing PC executables takes that route even if it includes an Android build. |
| `.obb` with its APK, or after installing the APK | Associates standard main/patch filenames with their package and copies into that app's running Android instance, not `~/Downloads`. |
| `.aab` | A publishing bundle, not an installable app. Export a device APK/APKS set using the developer's build tools first. |
This grouped local-file importer is the source change for issue
[#102](https://github.com/saphid/frame-control/issues/102).
**Verified with generated fixtures, 2026-10-18:** a real arm64 app and asset
split installed on a Steam Frame. A second import copied matching OBB and
`Android/data` bytes, closed the app, and relaunch confirmed all three payloads
were readable inside it. RE4 gameplay and updates remain unverified.
### Games with companion data take two imports
An APK can install successfully but quit, hang or request a download when its
assets are missing. For a local bundle or APK/OBB selection:
1. Drop the files together. Frame Control checks the entire selection before
changing any app, installs the APK and says if game data still needs copying.
2. Start that game's Steam shortcut and leave it open. Drop the **same files**
again (or just the matching OBB). Frame Control checks the installed package
and exact APK version, copies the data and closes the game.
3. Relaunch from Steam. A successful copy verifies transfer integrity, not
whether those assets are the right ones for that game build.
Standard archive layouts are `base.apk` (or another base filename), split APKs,
`Android/obb/<package>/main|patch.<version>.<package>.obb` (or a standard OBB at
archive root or in `<package>/`), and `Android/data/<package>/...`. One shared
enclosing folder is accepted. OBB filenames must belong to
the app; use its matching assets, rather than renaming unrelated files. To
import an external data folder, ZIP its `Android/data/<package>/...` tree;
a loose folder drop still uses the PC-game workflow.
**Limits:** only native-library and asset splits can be merged. Resource,
density, language, code and feature splits fail with a specific reason; they
are never silently omitted. A universal APK or a compatible device-specific
set from the developer is needed instead. Bundles with multiple base APKs
(e.g. several device variants) are refused rather than choosing arbitrarily.
Repacking re-signs the APK, as the existing launcher fix does. See
[split limits](quest-import.md#split-apks).
Archives reject unsafe/duplicate paths, encrypted entries, links, special
files and unrecognised payload files. Recognised metadata (manifest/info JSON,
SAI v1/v2 JSON, icon, APKS table and APK incremental-signature files), Finder
`__MACOSX` sidecars, `.DS_Store` files and files named `._*` (AppleDouble) are
not installed, including inside `Android/data`. Metadata still
counts against archive limits and must pass the path/link/encryption checks.
Loose filenames follow the same portable path rules: rename files with colons,
trailing dots/spaces or Windows device names before importing them. Empty
companion files reject the selection before upload; they are never omitted
silently. Sets with splits or companion data cannot replace a newer or unknown
installed version. Close a running app before replacing it.
The import limit is 4,096 archive entries/files and 16 GiB in total; each
uploaded file is also subject to the server's 8 GiB upload limit. Staging must
have sufficient free disk space. Abandoned uploads expire after an hour;
finished, failed or removed queued imports clean up their uploaded files.
Data copying still needs the game's own Lepton instance to stay running.
**A game that exits immediately without its assets cannot use this data path
yet.** Frame Control does not start games implicitly or guess a host-side
`/sdcard` location. The generated fixture verified app-side readability after
relaunch; persistence through app or SteamOS updates and individual games'
storage requirements remain unverified.
If the game is already on your Quest, [Import from my Quest](quest-import.md)
uses the same two-step data workflow and can bring supported splits and
optional external app data. Quest import is on main/draft 0.4.7, absent from
public 0.4.6. The [local OBB CLI helper](sidequest.md#obb-files) remains available:
```sh
python3 ui/frame_android.py info /path/to/game.apk
python3 ui/frame_android.py install-obb org.example.game /path/to/main.42.org.example.game.obb
python3 ui/frame_android.py stop org.example.game
```
For that CLI path, install the APK and start its Steam shortcut first, wait
for a successful copy, then stop and relaunch. Retry the complete data set
after a partial failure before playing.
**Installation is not a compatibility verdict.** OBB files cannot fix an
unsupported ABI, a newer Android requirement, missing VrApi/OpenXR features,
or Meta services/licence checks. See [Quest compatibility limits](quest-import.md#what-wont-run-and-why).
## Diagnosing an APK game report (including RE4, issue [#102](https://github.com/saphid/frame-control/issues/102))
The RE4 screenshot says the APK "didn't work" and suggests missing OBBs. It
doesn't identify the game edition/build or show an error, so missing OBBs
remain a hypothesis. There is no verified RE4 compatibility result here.
1. Record the computer OS, Frame Control version, SteamOS/Lepton build and
exact game edition/platform. List filenames and sizes (APK, base/splits,
OBB, `Android/data`) without sharing game assets. From a source checkout,
`python3 ui/frame_android.py info /path/to/game.apk` reads local package,
version, minimum API, ABI and VR warnings without connecting to a headset.
2. Identify the first failure: upload/installation error, no Steam shortcut,
Lepton not starting, or the game opening then quitting/hanging/asking for
data. Save the exact Activity error and **Frame → Show Server Log** excerpt.
Remove credentials, account details and private paths before sharing.
3. If installation succeeded, establish whether that exact build needs
companion data. Use the matching data workflow above, then relaunch. A
successful transfer only proves the bytes reached Android. Don't uninstall
as a first diagnostic step: it deletes saved data.
4. If it still fails, **Test** in Android apps (or
`python3 ui/frame_android.py probe org.example.game`) launches the game and
reports instance-start failure, a running process or a crash summary.
"Runs" means the process survived a short check, not that gameplay works.
A missing-assets error supports the OBB hypothesis; a VrApi, entitlement,
missing-library or Android-service error points elsewhere. Review relevant
crash-log excerpts if the summary is empty; an empty crash buffer does not
establish compatibility.
## When an app needs a newer Android
Lepton is Android 11 (API 30), with arm64-v8a only. If Frame Control refuses
+393
View File
@@ -0,0 +1,393 @@
"""Local Android bundles and companion files, using synthetic APK/OBB payloads."""
import sandbox # noqa: F401
import io
from pathlib import Path
import sys
import tempfile
import unittest
import threading
from types import SimpleNamespace
from unittest.mock import patch
import zipfile
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
import frame_android_import as imports
from test_quest import apk_bytes, PKG
import server
from test_install_queue import wait
class AndroidImport(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name)
def file(self, name, data):
path = self.root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
return path
def bundle(self, name, files):
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w') as archive:
for entry, data in files.items():
archive.writestr(entry, data)
return self.file(name, buf.getvalue())
def prepare(self, paths):
work = Path(tempfile.mkdtemp(dir=self.root))
return imports.prepare(paths, work)
def test_xapk_keeps_apk_obb_and_external_data_together(self):
obb = 'main.7.' + PKG + '.obb'
path = self.bundle('game.xapk', {
'game.apk': apk_bytes(version_code=7), 'manifest.json': b'{}', 'icon.png': b'png',
'Android/obb/' + PKG + '/' + obb: b'owned assets',
'Android/data/' + PKG + '/files/levels.pak': b'owned levels',
})
plan, = self.prepare([path])
self.assertEqual(plan['package'], PKG)
self.assertEqual(plan['obb'][0].name, obb)
self.assertEqual((plan['data'] / 'files/levels.pak').read_bytes(), b'owned levels')
self.assertEqual(plan['checked']['version_code'], 7)
def test_wrapped_archives_and_package_obb_folders(self):
obb = 'main.7.' + PKG + '.obb'
for prefix in ('', 'Game/'):
for folder in ('', PKG + '/', 'Android/obb/' + PKG + '/'):
with self.subTest(prefix=prefix, folder=folder):
path = self.bundle('game.zip', {
prefix + 'base.apk': apk_bytes(version_code=7),
prefix + folder + obb: b'assets',
prefix + 'Android/data/' + PKG + '/files/setup.sh': b'opaque data',
})
self.assertTrue(imports.is_bundle(path))
plan, = self.prepare([path])
self.assertEqual(plan['obb'][0].name, obb)
self.assertEqual((plan['data'] / 'files/setup.sh').read_bytes(), b'opaque data')
def test_finder_and_sai_metadata_do_not_become_payloads(self):
path = self.bundle('game.apks', {
'Game/base.apk': apk_bytes(version_code=7),
'Game/meta.sai_v2.json': b'{}', 'Game/meta.sai_v1.json': b'{}',
'Game/icon.png': b'icon', 'Game/.DS_Store': b'finder',
'__MACOSX/Game/._base.apk': b'not an APK',
'Game/._base.apk': b'inline AppleDouble',
'Game/Android/data/' + PKG + '/files/.DS_Store': b'finder',
'Game/Android/data/' + PKG + '/files/._level': b'inline AppleDouble',
'Game/Android/data/' + PKG + '/files/level': b'data',
})
self.assertTrue(imports.is_bundle(path))
plan, = self.prepare([path])
self.assertEqual([p.name for p in plan['data'].rglob('*') if p.is_file()], ['level'])
self.assertFalse(imports.is_bundle(self.bundle('metadata.zip', {
'__MACOSX/._base.apk': b'finder', '._base.apk': b'finder', '.DS_Store': b'finder'})))
self.assertFalse(imports.is_bundle(self.bundle('pc-with-metadata.zip', {
'Game/game': b'\x7fELF binary', '__MACOSX/._base.apk': b'finder'})))
def test_ignored_metadata_still_obeys_archive_safety_and_budget(self):
for name in ('__MACOSX/../escape.apk', '__MACOSX/CON', 'Game/../.DS_Store', 'Game/../._base.apk'):
with self.subTest(name=name), self.assertRaises(imports.android.FrameError):
self.prepare([self.bundle('bad.zip', {'base.apk': apk_bytes(), name: b'x'})])
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w') as archive:
archive.writestr('base.apk', apk_bytes())
member = zipfile.ZipInfo('__MACOSX/._base.apk')
member.external_attr = 0o120777 << 16
archive.writestr(member, b'/etc/passwd')
with self.assertRaises(imports.android.FrameError):
self.prepare([self.file('link.zip', buf.getvalue())])
path = self.bundle('large.zip', {'base.apk': apk_bytes(), '.DS_Store': b'x' * 4096})
with patch.object(imports, 'MAX_BYTES', 2048), self.assertRaises(imports.android.FrameError):
self.prepare([path])
path = self.bundle('large-sidecar.zip', {'base.apk': apk_bytes(), '._base.apk': b'x' * 4096})
with patch.object(imports, 'MAX_BYTES', 2048), self.assertRaises(imports.android.FrameError):
self.prepare([path])
def test_apks_apkm_zip_and_loose_splits_use_manifest_not_filename_order(self):
for extension in ('apks', 'apkm', 'zip'):
with self.subTest(extension=extension):
path = self.bundle('game.' + extension, {
'splits/aaa.apk': apk_bytes(split='config.arm64_v8a', version_code=7,
files={'lib/arm64-v8a/libgame.so': b'native'}),
'splits/zzz.apk': apk_bytes(split_required=True, version_code=7), 'toc.pb': b'metadata'})
plan, = self.prepare([path])
self.assertEqual(plan['base'].name, 'zzz.apk')
self.assertEqual(len(plan['splits']), 1)
base = self.file('base.apk', apk_bytes(version_code=7))
split = self.file('split.apk', apk_bytes(split='assets', version_code=7, files={'assets/a': b'a'}))
plan, = self.prepare([split, base])
self.assertEqual(plan['base'], base)
def test_resource_code_and_mismatched_version_splits_fail_before_install(self):
base = self.file('base.apk', apk_bytes(version_code=7))
for version, files in ((8, {'assets/a': b'a'}), (7, {'classes.dex': b'code'}),
(7, {'res/drawable/a.xml': b'resource'})):
with self.subTest(version=version, files=files):
split = self.file('split.apk', apk_bytes(split='config', version_code=version, files=files))
with self.assertRaises(imports.android.FrameError):
self.prepare([base, split])
def test_missing_or_multiple_bases_and_mismatched_obb_are_refused(self):
for files in (
{'split.apk': apk_bytes(split='config', version_code=7)},
{'a.apk': apk_bytes(version_code=7), 'b.apk': apk_bytes(version_code=7)},
{'a.apk': apk_bytes(version_code=7), 'main.7.org.other.game.obb': b'assets'},
):
with self.subTest(files=list(files)):
with self.assertRaises(imports.android.FrameError):
self.prepare([self.bundle('bad.xapk', files)])
def test_traversal_duplicate_symlink_unknown_and_bomb_archives_are_refused(self):
for entry in ('../escape.apk', '/escape.apk', 'Android/../escape.apk', 'a\\b.apk', 'extra.pak',
'CON.apk', 'Android/data/' + PKG + '/files/NUL'):
with self.subTest(entry=entry):
with self.assertRaises(imports.android.FrameError):
self.prepare([self.bundle('bad.xapk', {'base.apk': apk_bytes(version_code=7), entry: b'x'})])
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w') as z:
member = zipfile.ZipInfo('base.apk')
member.create_system = 3
member.external_attr = 0o120777 << 16
z.writestr(member, b'/etc/passwd')
with self.assertRaises(imports.android.FrameError):
self.prepare([self.file('link.apks', buf.getvalue())])
for duplicate in ('base.apk', 'BASE.APK'):
with self.subTest(duplicate=duplicate):
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w') as z:
z.writestr('base.apk', apk_bytes(version_code=7))
# Avoid zipfile's duplicate-name warning while testing exact duplicates too.
import warnings
with warnings.catch_warnings():
warnings.simplefilter('ignore', UserWarning)
z.writestr(duplicate, apk_bytes(version_code=7))
with self.assertRaisesRegex(imports.android.FrameError, 'duplicate paths'):
self.prepare([self.file('duplicate.apkm', buf.getvalue())])
with patch.object(imports, 'MAX_BYTES', 16):
with self.assertRaises(imports.android.FrameError):
self.prepare([self.bundle('big.xapk', {'base.apk': b'x' * 100})])
def test_loose_obb_is_associated_by_package_and_plain_pc_zip_is_not_android(self):
name = 'main.7.' + PKG + '.obb'
plan, = self.prepare([self.file(name, b'assets')])
self.assertIsNone(plan['base'])
self.assertEqual(plan['package'], PKG)
self.assertFalse(imports.is_bundle(self.bundle('pc.zip', {'game.exe': b'MZ'})))
self.assertFalse(imports.is_bundle(self.bundle('mixed.zip', {'game.exe': b'MZ', 'android.apk': apk_bytes()})))
self.assertFalse(imports.is_bundle(self.bundle('linux.zip', {'game': b'\x7fELF binary', 'android.apk': apk_bytes()})))
self.assertTrue(imports.is_bundle(self.bundle('android.zip', {'base.apk': apk_bytes()})))
self.assertTrue(imports.is_bundle(self.bundle('data.zip', {'base.apk': apk_bytes(),
'Android/data/' + PKG + '/files/setup.sh': b'opaque game data'})))
def test_required_splits_and_wrong_abi_return_actionable_errors(self):
with self.assertRaisesRegex(imports.android.FrameError, 'requires splits'):
self.prepare([self.file('base.apk', apk_bytes(version_code=7, split_required=True))])
# Rebuild ZIP CRCs: only the binary manifest's attribute name is stripped.
with zipfile.ZipFile(io.BytesIO(apk_bytes(version_code=7, split_required=True, resmap=True))) as z:
manifest = z.read('AndroidManifest.xml').replace(
'isSplitRequired'.encode('utf-16-le'), ('x' * len('isSplitRequired')).encode('utf-16-le'))
with self.assertRaisesRegex(imports.android.FrameError, 'requires splits'):
self.prepare([self.file('shrunk.apk', self._manifest_apk(manifest))])
path = self.file('wrong.apk', apk_bytes(version_code=7, files={'lib/armeabi-v7a/libgame.so': b'arm32'}))
with self.assertRaises(imports.android.FrameError) as failure:
self.prepare([path])
self.assertEqual(failure.exception.apk['package'], PKG)
self.assertIn('no arm64-v8a build', failure.exception.apk['blocker'])
def test_refused_apks_keep_install_reporting(self):
unreadable = self.file('bad.apk', b'not an APK')
with patch.object(imports.android, '_after_install') as report:
with self.assertRaises(imports.android.FrameError):
self.prepare([unreadable])
self.assertIsNone(report.call_args.args[0])
self.assertIsInstance(report.call_args.args[2], imports.android.FrameError)
wrong = self.file('wrong.apk', apk_bytes(version_code=7, files={'lib/armeabi-v7a/lib.so': b'32bit'}))
with patch.object(imports.android, '_after_install') as report:
with self.assertRaises(imports.android.FrameError):
self.prepare([wrong])
self.assertEqual(report.call_args.args[0]['package'], PKG)
self.assertIn('no arm64-v8a build', str(report.call_args.args[2]))
def test_non_arm64_apk_error_matches_compatibility_report(self):
for abi in ('x86_64', 'x86', 'armeabi-v7a'):
with self.subTest(abi=abi):
wrong = self.file('wrong.apk', apk_bytes(version_code=7, files={'lib/' + abi + '/lib.so': b'native'}))
with patch.object(imports.android, '_after_install') as report:
with self.assertRaises(imports.android.FrameError) as failed:
self.prepare([wrong])
message = str(failed.exception)
self.assertIn('download the APK marked arm64-v8a', message)
self.assertNotIn('older Quest games', message)
self.assertEqual(str(report.call_args.args[2]), message)
self.assertEqual(failed.exception.apk['blocker'], message)
def test_single_apk_keeps_filename_and_can_replace_an_older_build(self):
path = self.file('Older Game.apk', apk_bytes(version_code=7))
meta = {'package': PKG, 'label': 'Game', 'version_code': 8, 'flatscreen': True}
with patch.object(imports.android, 'read_meta', return_value=meta), \
patch.object(imports.quest, '_running', return_value=False), \
patch.object(imports.android, 'install', return_value=meta) as install:
imports.import_files([path])
self.assertEqual(install.call_args.kwargs['source'], 'Older Game.apk')
bundle = self.bundle('older.zip', {'base.apk': apk_bytes(version_code=7),
'main.7.' + PKG + '.obb': b'assets'})
install.reset_mock()
with self.assertRaisesRegex(imports.android.FrameError, 'older or unknown'):
imports.import_files([bundle])
install.assert_not_called()
def _manifest_apk(self, manifest):
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w') as archive:
archive.writestr('AndroidManifest.xml', manifest)
return buf.getvalue()
def test_asset_apk_in_android_data_is_copied_as_data_not_installed(self):
path = self.bundle('game.xapk', {'base.apk': apk_bytes(version_code=7),
'Android/data/' + PKG + '/files/extension.apk': b'opaque game data'})
plan, = self.prepare([path])
self.assertEqual((plan['data'] / 'files/extension.apk').read_bytes(), b'opaque game data')
self.assertFalse(plan['splits'])
def test_archive_import_reaches_real_split_merge_and_signer(self):
path = self.bundle('game.apks', {
'base.apk': apk_bytes(version_code=7, split_required=True),
'config.apk': apk_bytes(split='config.arm64', version_code=7,
files={'lib/arm64-v8a/libgame.so': b'arm64 payload'})})
def installed(apk, info, package, flatscreen, name, source, artwork):
with zipfile.ZipFile(apk) as merged:
self.assertEqual(merged.read('lib/arm64-v8a/libgame.so'), b'arm64 payload')
self.assertIn(b'APK Sig Block 42', Path(apk).read_bytes())
return {'package': package, 'label': 'Game', 'version_code': 7, 'flatscreen': True}
with patch.object(imports.quest, '_running', return_value=False), \
patch.object(imports.android, 'read_meta', return_value=None), \
patch.object(imports.android, '_install', side_effect=installed) as install, \
patch.object(imports.android, '_after_install'):
result = imports.import_files([path])
self.assertFalse(result['needs_game_data'])
install.assert_called_once()
def test_two_step_bundle_import_reuses_existing_installer_and_data_copy(self):
path = self.bundle('game.xapk', {'base.apk': apk_bytes(version_code=7),
'main.7.' + PKG + '.obb': b'assets'})
meta = {'package': PKG, 'label': 'Game', 'version': '7', 'version_code': 7,
'instance': imports.android.instance_id(PKG), 'flatscreen': False}
with patch.object(imports.quest, '_running', return_value=False), \
patch.object(imports.android, 'read_meta', return_value=None), \
patch.object(imports.android, 'install', return_value=meta) as install, \
patch.object(imports.quest, 'add_game_data') as copy:
result = imports.import_files([path], display='vr')
self.assertTrue(result['needs_game_data'])
self.assertIn('drop', result['message'])
self.assertFalse(install.call_args.kwargs['flatscreen'])
copy.assert_not_called()
with patch.object(imports.quest, '_running', return_value=True), \
patch.object(imports.android, 'read_meta', return_value=meta), \
patch.object(imports.android, 'install') as install, \
patch.object(imports.quest, 'add_game_data', return_value={
'message': 'Added data; game closed', 'app': meta}) as copy:
result = imports.import_files([path])
self.assertFalse(result['needs_game_data'])
self.assertEqual(copy.call_args.args[0], PKG)
self.assertEqual(copy.call_args.kwargs['source'], 'your files')
install.assert_not_called()
def test_data_only_requires_existing_install_and_mismatch_never_writes(self):
path = self.file('main.7.' + PKG + '.obb', b'assets')
with patch.object(imports.android, 'read_meta', return_value=None), \
patch.object(imports.android, 'install') as install:
with self.assertRaisesRegex(imports.android.FrameError, 'APK first'):
imports.import_files([path])
install.assert_not_called()
path = self.bundle('game.xapk', {'base.apk': apk_bytes(version_code=7),
'main.7.' + PKG + '.obb': b'assets'})
meta = {'package': PKG, 'label': 'Game', 'version_code': 8, 'instance': imports.android.instance_id(PKG)}
with patch.object(imports.android, 'read_meta', return_value=meta), \
patch.object(imports.quest, '_running', return_value=True), \
patch.object(imports.quest, 'add_game_data') as copy:
with self.assertRaises(imports.android.FrameError):
imports.import_files([path])
copy.assert_not_called()
class StagedImports(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.addCleanup(server._purge_android_uploads, float('inf'))
self.master = patch.object(server, 'ensure_master').start()
self.link = patch.object(server, 'LINK', None).start()
self.addCleanup(patch.stopall)
def stage(self, name='base.apk'):
directory = Path(tempfile.mkdtemp(dir=self.tmp.name))
path = directory / name
path.write_bytes(apk_bytes(version_code=7))
return server.stage_android_file(path, str(directory)), directory
def test_one_use_tokens_pass_all_files_to_queued_import_and_clean_up(self):
a, da = self.stage()
b, db = self.stage('split.apk')
with patch.object(imports, 'import_files', return_value={'message': 'ok'}) as run:
job = server.android_import({'action': 'import', 'tokens': [a['token'], b['token']], 'display': 'flat'})
self.assertEqual(wait(job['job'])['message'], 'ok')
self.assertEqual(len(run.call_args.args[0]), 2)
self.assertEqual(run.call_args.kwargs['display'], 'flat')
# Queue completion publishes just before its cleanup callback.
import time
for _ in range(100):
if not da.exists() and not db.exists():
break
time.sleep(.01)
self.assertFalse(da.exists() or db.exists())
with self.assertRaisesRegex(server.Failure, 'already used'):
server.android_import({'action': 'import', 'tokens': [a['token']]})
def test_remove_queued_import_cleans_its_uploads(self):
release, started = threading.Event(), threading.Event()
first = server.queue_job('Block', lambda: (started.set(), release.wait(5), {'message': 'ok'})[-1])
self.addCleanup(release.set)
self.assertTrue(started.wait(5))
staged, directory = self.stage()
job = server.android_import({'action': 'import', 'tokens': [staged['token']]})
server.queue_action({'action': 'remove', 'job': job['job']})
self.assertFalse(directory.exists())
release.set()
wait(first['job'])
def test_headset_switch_and_bad_token_list_never_queue_an_import(self):
selected = ['frame-a']
with patch.object(server, 'LINK', SimpleNamespace(active_device=lambda: {'id': selected[0]})):
staged, directory = self.stage()
selected[0] = 'frame-b'
with self.assertRaises(server.Failure) as error:
server.android_import({'action': 'import', 'tokens': [staged['token']]})
self.assertEqual(error.exception.status, 409)
self.assertTrue(directory.exists())
server.android_import({'action': 'discard', 'tokens': [staged['token']]})
self.assertFalse(directory.exists())
selected[0] = 'frame-a'
staged, directory = self.stage()
with self.assertRaises(server.Failure):
server.android_import({'action': 'import', 'tokens': [staged['token'], staged['token']]})
server.android_import({'action': 'discard', 'tokens': [staged['token']]})
self.assertFalse(directory.exists())
def test_zip_detection_routes_android_bundles_and_preserves_local_original(self):
path = Path(self.tmp.name) / 'game.zip'
with zipfile.ZipFile(path, 'w') as archive:
archive.writestr('base.apk', apk_bytes(version_code=7))
staged = server.stage_title(str(path))
self.assertTrue(staged['android'])
server.android_import({'action': 'discard', 'tokens': [staged['token']]})
self.assertTrue(path.is_file())
def test_expired_uploads_are_deleted(self):
staged, directory = self.stage()
server._purge_android_uploads(float('inf'))
self.assertFalse(directory.exists())
with self.assertRaisesRegex(server.Failure, 'expired'):
server.android_import({'action': 'import', 'tokens': [staged['token']]})
+18
View File
@@ -1377,6 +1377,24 @@ class ServerConnection(unittest.TestCase):
self.assertIn("switched headsets", json.loads(r.read())["error"])
conn.close()
def test_android_discard_is_local_even_with_a_previous_headset_header(self):
conn = http.client.HTTPConnection('127.0.0.1', self.port, timeout=20)
conn.request('POST', '/api/upload', body=b'owned temporary bytes',
headers={'X-Frame-UI': '1', 'X-Mode': 'android', 'X-Filename': 'test.apk'})
response = conn.getresponse()
self.assertEqual(response.status, 200)
token = json.loads(response.read())['token']
conn.close()
for action, expected in [('import', 409), ('discard', 200), ('discard', 200)]:
conn = http.client.HTTPConnection('127.0.0.1', self.port, timeout=20)
conn.request('POST', '/api/android-import',
body=json.dumps({'action': action, 'tokens': [token]}),
headers={'X-Frame-UI': '1', 'X-Frame-Device': 'previous-headset'})
response = conn.getresponse()
payload = response.read()
self.assertEqual(response.status, expected, payload)
conn.close()
def test_guards_and_validation(self):
self.assertEqual(self.request("GET", "/api/connection", key="")[0], 403)
self.assertEqual(self.request("GET", "/api/devices", key="nope")[0], 403)
+1 -1
View File
@@ -811,7 +811,7 @@ class AndroidChecks(unittest.TestCase):
self.assertIn("podman exec lepton-steamlaunch-2800000001 /system/bin/sh -c 'dumpsys package "
f"{PKG}'", ssh.call_args[0][0])
with patch.object(android, 'ssh', return_value=dump):
with self.assertRaisesRegex(quest.QuestError, 'version code 100, not the Quest.s 999'):
with self.assertRaisesRegex(quest.QuestError, 'version code 100, expected 999'):
quest._check_android_package(self.META, 999)
with patch.object(android, 'ssh', return_value=''):
with self.assertRaisesRegex(quest.QuestError, 'does not have'):
+108
View File
@@ -0,0 +1,108 @@
"""Send to Frame groups Android files and keeps normal file/title routing (#102)."""
import sandbox # noqa: F401
from pathlib import Path
import shutil
import subprocess
import unittest
PAGE = (Path(__file__).resolve().parents[1] / 'ui/index.html').read_text(encoding='utf-8')
@unittest.skipUnless(shutil.which('node'), 'Node runs the page functions')
class SendFiles(unittest.TestCase):
def run_page(self, checks):
routing = PAGE[PAGE.index('const TITLE_EXT ='):PAGE.index('// ---- sideloaded titles')]
action = PAGE[PAGE.index('async function act('):PAGE.index('function setConn(')]
setup = r'''
const assert = require('node:assert/strict');
const calls = [], logs = [], toasts = [];
const window = {connDevice: 'frame-a'};
const elements = new Map();
const $ = id => {if (!elements.has(id)) elements.set(id, {value: id === 'apkDisplay' ? 'vr' : '', textContent: ''}); return elements.get(id);};
const log = (message, kind) => logs.push({message, kind});
const toast = (message, error) => toasts.push({message, error});
const refresh = () => calls.push(['refresh']);
let result = {message: 'Imported', apps: [{package: 'org.example.game'}], needs_game_data: false};
const api = async (path, body, dev) => {calls.push(['api', path, body, dev]); return result;};
let upload = async (file, mode, dev) => {
calls.push(['upload', file.name, mode, dev]); return {token: 'token-' + file.name};
};
const runJob = async (label, key, start) => start();
const offerTest = async app => calls.push(['test', app.package]);
const sideload = async file => calls.push(['title', file.name]);
const file = name => ({name, size: 10});
'''
script = setup + action + routing + '\n(async () => {\n' + checks + r'''
})().catch(e => {console.error(e); process.exitCode = 1;});
'''
result = subprocess.run(['node', '-e', script], capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
def test_companion_files_and_splits_are_one_import_not_separate_installs(self):
self.run_page(r'''
await sendFiles(['main.42.org.example.game.obb', 'base.apk', 'split.apk'].map(file));
assert.deepEqual(calls.filter(c => c[0] === 'upload').map(c => c.slice(1)), [
['main.42.org.example.game.obb', 'android', 'frame-a'],
['base.apk', 'android', 'frame-a'], ['split.apk', 'android', 'frame-a']]);
const imports = calls.filter(c => c[0] === 'api' && c[2].action === 'import');
assert.equal(imports.length, 1);
assert.deepEqual(imports[0][2], {action: 'import', display: 'vr',
tokens: ['token-main.42.org.example.game.obb', 'token-base.apk', 'token-split.apk']});
assert.equal(imports[0][3], 'frame-a');
assert.equal(calls.filter(c => c[0] === 'test').length, 1);
''')
def test_bundle_formats_use_android_import_and_pending_data_does_not_offer_test(self):
self.run_page(r'''
result.needs_game_data = true; result.message = 'Start the game, then drop again';
await sendFiles(['game.XAPK', 'game.APKS', 'game.apkm'].map(file));
assert.equal(calls.filter(c => c[0] === 'upload' && c[2] === 'android').length, 3);
assert.equal(calls.filter(c => c[0] === 'test').length, 0);
assert.equal($('androidImportState').textContent, result.message);
''')
def test_normal_file_and_pc_title_routes_survive(self):
self.run_page(r'''
await sendFiles(['game.apk', 'notes.txt', 'pc.zip'].map(file));
assert.ok(calls.some(c => c[0] === 'upload' && c[1] === 'notes.txt' && c[2] === 'push'));
assert.ok(calls.some(c => c[0] === 'title' && c[1] === 'pc.zip'));
assert.equal($('fileInput').value, '');
''')
def test_upload_failure_discards_previous_tokens_without_installing(self):
self.run_page(r'''
const originalUpload = upload;
upload = async (file, mode, dev) => {if (file.name === 'base.apk') throw new Error('transfer failed'); return originalUpload(file, mode, dev);};
await sendFiles(['main.42.org.example.game.obb', 'base.apk'].map(file));
assert.equal(calls.filter(c => c[0] === 'api' && c[2].action === 'import').length, 0);
assert.ok(calls.some(c => c[0] === 'api' && c[2].action === 'discard' && c[2].tokens.length === 1));
assert.match(toasts.at(-1).message, /transfer failed/);
''')
def test_empty_companion_is_rejected_before_any_upload_in_either_order(self):
self.run_page(r'''
for (const files of [[file('base.apk'), {name: 'main.7.org.example.game.obb', size: 0}],
[{name: 'main.7.org.example.game.obb', size: 0}, file('base.apk')]]) {
calls.length = 0;
await sendFiles(files);
assert.equal(calls.filter(c => c[0] === 'upload' || c[0] === 'api').length, 0);
assert.match(toasts.at(-1).message, /main\.7\.org\.example\.game\.obb.*empty/);
}
''')
def test_switch_during_upload_cannot_import_to_new_headset(self):
self.run_page(r'''
const originalUpload = upload;
upload = async (...args) => {const r = await originalUpload(...args); window.connDevice = 'frame-b'; return r;};
await sendFiles([file('game.xapk')]);
assert.equal(calls.filter(c => c[0] === 'api' && c[2].action === 'import').length, 0);
assert.match(toasts.at(-1).message, /Switched headsets/);
''')
def test_aab_is_explained_as_a_publishing_format(self):
self.run_page(r'''
await sendFiles([file('game.aab')]);
assert.deepEqual(calls, [['refresh']]);
assert.match(toasts.at(-1).message, /publishing file/);
assert.equal(toasts.at(-1).error, true);
''')
+18
View File
@@ -171,6 +171,24 @@ class ServerGuards(unittest.TestCase):
self.assertEqual(self.post("/api/titles", {"action": "discard", "token": r["token"]})[0], 200)
self.assertEqual(self.post("/api/titles", {"action": "install", "token": r["token"]})[0], 400)
def test_android_upload_and_zip_detection_stage_one_use_tokens(self):
from apk_fixtures import apk, manifest
payload = apk({'AndroidManifest.xml': manifest('org.example.game', 0, 0, 30)})
archive = io.BytesIO()
with zipfile.ZipFile(archive, 'w') as z:
z.writestr('base.apk', payload)
for mode, name in (('android', 'game.xapk'), ('title', 'game.zip')):
with self.subTest(mode=mode):
status, _, raw = self.request('POST', '/api/upload', archive.getvalue(),
{'X-Frame-UI': '1', 'X-Mode': mode, 'X-Filename': name})
staged = json.loads(raw)
self.assertEqual(status, 200, staged)
self.assertTrue(staged['android'])
body = {'action': 'discard', 'tokens': [staged['token']]}
self.assertEqual(self.post('/api/android-import', body)[0], 200)
body['action'] = 'import'
self.assertEqual(self.post('/api/android-import', body)[0], 400)
def test_title_input_validation(self):
status, _, _ = self.request("POST", "/api/upload", b"not a zip",
{"X-Frame-UI": "1", "X-Mode": "title", "X-Filename": "x.zip"})
+2 -2
View File
@@ -390,11 +390,11 @@ class ServerJobs(unittest.TestCase):
with tempfile.TemporaryDirectory() as cache, \
mock.patch.object(s.frame_host, "cache_dir", lambda *parts: Path(cache).joinpath(*parts)):
# Downloads, title staging, patched VR APKs, files staged for the
# assistant, and app-index downloads (in the cache folder).
# assistant, Android import staging, and app-index downloads (in the cache folder).
places = s._tmp_places()
self.assertEqual({p for _, p in places}, {s.WEB_TMP_PREFIX, s.frame_titles.TMP_PREFIX,
s.frame_android.TMP_PREFIX, s.frame_agent.TMP_PREFIX,
s.apk_fdroid.TMP_PREFIX})
s.apk_fdroid.TMP_PREFIX, s.frame_android_import.TMP_PREFIX})
for folder, prefix in places:
folder.mkdir(parents=True, exist_ok=True)
gone = tempfile.mkdtemp(prefix=f"{prefix}{dead.pid}-", dir=folder)
+309
View File
@@ -0,0 +1,309 @@
"""Local APK sets, ZIP-based Android bundles and companion data. Stdlib only.
Unpack into private staging, check every package before installing any, and use
the same installer and running-instance data copy as Quest import. No game is
started here. A data-bearing import takes two drops, as Quest takes two imports.
"""
from pathlib import Path, PurePosixPath
import os
import re
import shutil
import stat
import struct
import tempfile
import time
import zipfile
import frame_android as android
import frame_apk
import frame_quest as quest
ARCHIVES = {'.xapk', '.apks', '.apkm', '.zip'}
EXTENSIONS = ARCHIVES | {'.apk', '.obb'}
MAX_BYTES = 16 * 1024**3
MAX_FILES = 4096
TMP_PREFIX = 'frame-android-import-'
OBB = re.compile(r'(?:main|patch)\.[0-9]+\.([A-Za-z0-9_]+(?:\.[A-Za-z0-9_]+)+)\.obb')
METADATA = {'manifest.json', 'info.json', 'icon.png', 'toc.pb', 'meta.sai_v1.json', 'meta.sai_v2.json'}
WINDOWS_DEVICE = re.compile(r'(?:CON|PRN|AUX|NUL|COM[1-9¹²³]|LPT[1-9¹²³])(?:\..*)?', re.I)
def _finder_metadata(name):
parts = PurePosixPath(name).parts
return bool(parts) and (parts[0] == '__MACOSX' or parts[-1] == '.DS_Store' or parts[-1].startswith('._'))
def _members(archive):
"""Ignore Finder sidecars and strip at most one shared enclosing folder.
Original member names are still checked by _unpack before extraction.
Android itself is a payload root, never an enclosing folder.
"""
members = archive.infolist()
names = [m.filename for m in members if not m.is_dir() and not _finder_metadata(m.filename)]
first = names[0].split('/')[0] if names else ''
prefix = first + '/' if first and first != 'Android' and all(n.startswith(first + '/') for n in names) else ''
return [(m, m.filename[len(prefix):] if prefix and m.filename.startswith(prefix) else m.filename)
for m in members]
def is_bundle(path):
"""Sniff ZIPs so PC-game ZIPs retain their existing route. No extraction."""
try:
with zipfile.ZipFile(path) as archive:
members = [(m, n) for m, n in _members(archive)
if not m.is_dir() and not _finder_metadata(m.filename)]
# A PC distribution may include an optional Android build too.
for member, name in members:
# External app data is opaque, even if a file resembles a PC program.
if name.startswith(('Android/data/', 'Android/obb/')):
continue
if name.lower().endswith(('.exe', '.appimage', '.sh')):
return False
if not name.lower().endswith(('.apk', '.obb', '.apk.idsig')):
with archive.open(member) as src:
if src.read(4) == b'\x7fELF':
return False
return any(n.lower().endswith(('.apk', '.obb')) or n.startswith('Android/data/') for _, n in members)
except (OSError, ValueError, RuntimeError, NotImplementedError, zipfile.BadZipFile):
return False
def _path(name):
parts = PurePosixPath(name.rstrip('/')).parts
if (not parts or len(name) > 512 or len(parts) > 32 or name.startswith('/') or '\\' in name or
any(p in ('.', '..') or ':' in p or p.rstrip(' .') != p or WINDOWS_DEVICE.fullmatch(p)
for p in parts) or
'/'.join(parts) != name.rstrip('/')):
raise android.FrameError('unsafe path in Android bundle: ' + name[:200])
return parts
def _kind(name):
parts = _path(name)
if len(parts) >= 4 and parts[:2] == ('Android', 'data') and android.PKG_RE.fullmatch(parts[2]):
return 'data', parts[2], parts[3:]
if name.lower().endswith('.apk'):
return 'apk', None, None
if name.lower().endswith('.obb'):
match = OBB.fullmatch(parts[-1])
if not match or not (len(parts) == 1 or
len(parts) == 2 and parts[0] == match[1] or
len(parts) == 4 and parts[:2] == ('Android', 'obb') and parts[2] == match[1]):
raise android.FrameError('OBB needs its standard main/patch.<version>.<package>.obb name and Android/obb/<package> location')
return 'obb', match[1], None
if (len(parts) == 1 and name in METADATA) or name.lower().endswith('.apk.idsig'):
return 'metadata', None, None
raise android.FrameError('Unsupported Android bundle file: ' + name + '; no files were installed')
def _unpack(path, root, budget):
output, seen = [], set()
with zipfile.ZipFile(path) as archive:
members = _members(archive)
if len(members) + budget['files'] > MAX_FILES:
raise android.FrameError('Android import has too many files')
declared = sum(m.file_size for m, _ in members)
if declared + budget['bytes'] > MAX_BYTES or declared > max(0, shutil.disk_usage(root).free - 64 * 1024**2):
raise android.FrameError('Android bundle exceeds the size limit or available staging space')
# Validate the entire directory before creating any file.
for member, name in members:
_path(member.filename)
key = member.filename.rstrip('/').casefold()
mode = stat.S_IFMT(member.external_attr >> 16)
if key in seen or member.flag_bits & 1 or mode not in (0, stat.S_IFREG, stat.S_IFDIR):
raise android.FrameError('Android bundle contains duplicate paths, encrypted files, links or special files')
seen.add(key)
if not member.is_dir() and not _finder_metadata(member.filename):
_kind(name)
for member, name in members:
if member.is_dir():
continue
if _finder_metadata(member.filename):
budget['bytes'] += member.file_size
continue
kind, pkg, rel = _kind(name)
target = root.joinpath(*_path(member.filename))
target.parent.mkdir(parents=True, exist_ok=True)
size = 0
with archive.open(member) as src, target.open('xb') as dst:
while True:
chunk = src.read(1024 * 1024)
if not chunk:
break
size += len(chunk)
budget['bytes'] += len(chunk)
if budget['bytes'] > MAX_BYTES or size > member.file_size:
raise android.FrameError('Android bundle exceeds the size limit')
dst.write(chunk)
if size != member.file_size:
raise android.FrameError('Incomplete file in Android bundle: ' + member.filename)
if kind != 'metadata':
output.append((target, kind, pkg, rel))
budget['files'] += len(members)
return output
def prepare(paths, root):
"""Validate and group files by manifest package; return private install plans."""
groups, budget = {}, {'bytes': 0, 'files': 0}
if not paths or len(paths) > 128:
raise android.FrameError('Choose between 1 and 128 Android files')
for i, path in enumerate(paths):
path = Path(path)
if path.suffix.lower() not in EXTENSIONS or not path.is_file() or path.stat().st_size == 0:
raise android.FrameError('Choose a nonempty APK, OBB, XAPK, APKS, APKM or Android ZIP file')
if path.suffix.lower() in ARCHIVES:
directory = root / str(i)
directory.mkdir()
try:
entries = _unpack(path, directory, budget)
except (OSError, ValueError, RuntimeError, zipfile.BadZipFile, NotImplementedError) as e:
raise android.FrameError('Could not read Android bundle: ' + str(e)) from e
else:
budget['bytes'] += path.stat().st_size
budget['files'] += 1
entries = [(path, *_kind(path.name))]
if budget['bytes'] > MAX_BYTES or budget['files'] > MAX_FILES:
raise android.FrameError('Android import exceeds its size or file limit')
packages = set()
for local, kind, pkg, rel in entries:
split = None
if kind == 'apk':
started = time.time()
try:
with zipfile.ZipFile(local) as archive:
elements = frame_apk.manifest_elements(
frame_apk._read(archive, 'AndroidManifest.xml', frame_apk.MAX_MANIFEST))
attrs = next(a for tag, a in elements if tag == 'manifest')
pkg = attrs.get('package', (None, None, None))[2]
split = attrs.get('split', (None, None, None))[2]
except (OSError, ValueError, StopIteration, struct.error, IndexError, KeyError,
frame_apk.ApkError, zipfile.BadZipFile) as e:
error = android.FrameError('Could not read APK manifest: ' + str(e))
android._after_install(None, None, error, started)
raise error from e
if not pkg or not android.PKG_RE.fullmatch(pkg):
raise android.FrameError('Invalid Android package in import')
packages.add(pkg)
group = groups.setdefault(pkg, {'package': pkg, 'base': None, 'splits': [], 'obb': [],
'data': None, 'required': False,
'split_names': set(), 'data_names': set()})
if kind == 'apk':
if split:
if split in group['split_names']:
raise android.FrameError('Duplicate split APK: ' + split)
group['split_names'].add(split)
group['splits'].append(local)
elif group['base']:
raise android.FrameError('Several base APKs for ' + pkg + '; choose one build or device variant')
else:
group['base'] = local
group['required'] = any(bool(a.get('isSplitRequired', (None, 0, None))[1])
for tag, a in elements if tag == 'application')
elif kind == 'obb':
if local.stat().st_size == 0:
raise android.FrameError('OBB game data must not be empty')
if local.name in {p.name for p in group['obb']}:
raise android.FrameError('Duplicate OBB: ' + local.name)
group['obb'].append(local)
elif kind == 'data':
name = '/'.join(rel).casefold()
if name in group['data_names']:
raise android.FrameError('Duplicate Android/data file')
group['data_names'].add(name)
# Private destination, unrelated to archive paths or package-controlled symlinks.
group['data'] = root / ('data-' + pkg)
target = group['data'].joinpath(*rel)
target.parent.mkdir(parents=True, exist_ok=True)
local.rename(target)
if path.suffix.lower() in ARCHIVES and len(packages) != 1:
raise android.FrameError('An Android bundle must contain one matching app and its data')
for group in groups.values():
base, splits = group['base'], group['splits']
if splits and not base:
raise android.FrameError('Split APK set is missing its base APK')
if group['required'] and not splits:
raise android.FrameError('This base APK requires splits; select the complete APK set together')
if base:
started = time.time()
try:
checked = quest.inspect(group['package'], str(base), [str(p) for p in splits])
except android.FrameError as error:
# Retain the older-build lookup offered by the original single-APK uploader.
info = android.apk_info(base)
abis = set(info['abis'])
for split in splits:
abis.update(android.apk_info(split)['abis'])
try:
android.check_installable(dict(info, abis=sorted(abis)))
except android.FrameError as incompatible:
incompatible.apk = {'package': info['package'], 'version_code': info.get('version_code'),
'blocker': str(incompatible)}
android._after_install(info, None, incompatible, started)
raise incompatible from error
raise
if splits and (checked['version_code'] is None or any(
android.apk_info(p).get('version_code') != checked['version_code'] for p in splits)):
raise android.FrameError('Base and split APKs must have the same version code')
group['checked'] = checked
if not groups:
raise android.FrameError('No Android app or game data found')
return list(groups.values())
def import_files(paths, display='auto', progress=None):
if display not in ('auto', 'flat', 'vr'):
raise android.FrameError('invalid APK display mode')
report = progress or (lambda *args: None)
with tempfile.TemporaryDirectory(prefix=f'{TMP_PREFIX}{os.getpid()}-') as directory:
report('Checking Android files', 5)
plans = prepare(paths, Path(directory))
results, pending = [], False
with android._install_lock:
# Preflight every target before changing any of them.
for plan in plans:
pkg = plan['package']
meta = android.read_meta(pkg)
plan['existing'], plan['running'] = meta, quest._running(pkg)
if not plan['base']:
if not meta:
raise android.FrameError('Install the APK first for ' + pkg + '; no game data was copied')
plan['checked'] = {'package': pkg, 'label': meta['label'], 'version': meta.get('version'),
'version_code': quest._installed_code(pkg, meta), 'warnings': []}
checked = plan['checked']
if plan['running']:
quest._check_target(pkg, meta, checked)
if not (plan['obb'] or plan['data']):
raise android.FrameError('Close ' + checked['label'] + ' before installing it again')
elif meta and plan['base'] and (plan['splits'] or plan['obb'] or plan['data']):
have, new = quest._installed_code(pkg, meta), checked['version_code']
if have is None or new is None or int(new) < int(have):
raise android.FrameError('Cannot replace ' + checked['label'] + ' with an older or unknown version')
for i, plan in enumerate(plans):
pkg, checked = plan['package'], plan['checked']
report('Importing ' + checked['label'], 20 + i * 60 // len(plans))
if plan['running']:
result = quest.add_game_data(pkg, checked, plan['obb'], plan['data'], report,
checked['warnings'], source='your files')
else:
meta = plan['existing']
if plan['base']:
meta = android.install(str(plan['base']), splits=[str(p) for p in plan['splits']],
source=plan['base'].name,
flatscreen=None if display == 'auto' else display == 'flat')
needs = bool(plan['obb'] or plan['data'])
pending |= needs
message = (f"Installed {meta['label']} in the Steam library." if plan['base'] else
f"{meta['label']}'s game data has not been copied yet.")
if needs:
message += (f" Start {meta['label']} from Steam, leave it open, then drop the same files "
'again to add game data and close it. Relaunch afterwards. '
'If it exits immediately, this data-copy path cannot be used yet.')
warnings = list(checked['warnings']) + list(meta.get('vr_issues', []))
result = {'app': meta, 'message': ' '.join([message, *dict.fromkeys(warnings)])}
results.append(result)
report('Done', 100)
return {'message': ' '.join(r['message'] for r in results),
'apps': [r['app'] for r in results], 'app': results[0]['app'] if len(results) == 1 else None,
'needs_game_data': pending}
+1 -1
View File
@@ -12,7 +12,7 @@ import zlib
# android: attribute resource ids; names can be stripped by shrinkers, ids can't.
ATTR = {0x01010001: 'label', 0x01010002: 'icon', 0x01010003: 'name',
0x01010024: 'value', 0x0101021b: 'versionCode', 0x0101021c: 'versionName', 0x0101020c: 'minSdkVersion',
0x01010202: 'targetActivity'}
0x01010202: 'targetActivity', 0x01010591: 'isSplitRequired'}
T_REF, T_STRING, T_INT_DEC, T_INT_HEX = 0x01, 0x03, 0x10, 0x11
# APKs can come from websites (install links), so nothing read from one may be
# unbounded. zipfile stops at a member's declared size, so checking it is enough.
+4 -5
View File
@@ -256,7 +256,7 @@ def inspect(package_name, base, splits=()):
with zipfile.ZipFile(path) as z:
names |= set(z.namelist())
except (zipfile.BadZipFile, OSError) as e:
raise QuestError(f'{os.path.basename(path)} copied from the Quest is not a readable APK: {e}')
raise QuestError(f'{os.path.basename(path)} is not a readable APK: {e}')
info = android.apk_info(base)
label = info['label']
if info['package'] != package_name:
@@ -457,8 +457,8 @@ def _check_target(pkg, meta, checked):
have, new = _installed_code(pkg, meta), checked['version_code']
if have is None or new is None or int(have) != int(new):
raise QuestError(f"{label} is open on the Frame with a different version (Frame: "
f"{_version(meta.get('version'), have)}; Quest: {_version(checked['version'], new)}). "
"Close it, then import again to install the Quest's version.")
f"{_version(meta.get('version'), have)}; import: {_version(checked['version'], new)}). "
"Close it, then import again to install that version.")
return meta
@@ -474,8 +474,7 @@ def _check_android_package(meta, code):
if not found:
raise QuestError(f'Android in {container} does not have {pkg} installed')
if int(found.group(1)) != int(code):
raise QuestError(f'Android in {container} has {pkg} version code {found.group(1)}, not the '
f"Quest's {code}")
raise QuestError(f'Android in {container} has {pkg} version code {found.group(1)}, expected {code}')
def _close(pkg):
+62 -7
View File
@@ -1126,11 +1126,17 @@
<div class="shelf-head"><h2>Send to Frame</h2></div>
<div class="drop" id="drop" tabindex="0" role="button" aria-label="Choose files to send">
<b><span class="desk-only">Drop files here, or click to choose</span><span class="mobile-only">Tap to choose files</span></b>
Files land in <code>~/Downloads</code>. <code>.apk</code> files install as their own Android app;
a game's <code>.zip</code>, folder or <code>.exe</code> becomes a title in the Steam library.
Android <code>.apk</code>, <code>.xapk</code>, <code>.apks</code> and <code>.apkm</code> files install as apps.
Select a base APK, its splits and OBB files together.
A game's <code>.zip</code>, folder or <code>.exe</code> becomes a title in the Steam library.
<span class="desk-only">You can also drop files anywhere in this window.</span>
<input type="file" id="fileInput" multiple hidden>
</div>
<div class="hint">Android ZIP bundles are detected too. For game data: import once, start the game from Steam,
then drop the same files again to copy data and close the game. Other files go to <code>~/Downloads</code>.
<a href="https://github.com/saphid/frame-control/blob/main/docs/apks.md#games-with-extra-data-or-split-apks" target="_blank" rel="noopener noreferrer">APK and game-data help</a>.
Installing an APK doesn't guarantee the game will run.</div>
<div class="hint" id="androidImportState" role="status"></div>
<label class="sub" for="apkDisplay">Android display</label>
<select id="apkDisplay" aria-label="Android app display mode">
<option value="auto">Automatic (detect VR)</option>
@@ -1300,8 +1306,9 @@
</div>
</main>
<div class="dropzone" id="dropzone" hidden><div><b>Drop to send to the Frame</b>
<span class="sub">Files go to <code>~/Downloads</code>; <code>.apk</code> files install as Android apps;
a game's <code>.zip</code>, folder or <code>.exe</code> becomes a Steam library title.</span></div></div>
<span class="sub">APKs, split sets and Android bundles install as Android apps;
a game's <code>.zip</code>, folder or <code>.exe</code> becomes a Steam library title.
Select companion OBB files with their APK. Other files go to <code>~/Downloads</code>.</span></div></div>
<div class="drawer" id="drawer"><div class="log" id="log"></div></div>
<div class="bottombar" id="bottombar" role="button" aria-label="Show activity">
@@ -2973,9 +2980,43 @@ document.addEventListener("click", async e => {
$("aboutClose").onclick = () => $("aboutDlg").close();
const TITLE_EXT = /\.(zip|exe)$/i;
const ANDROID_EXT = /\.(apk|obb|xapk|apks|apkm)$/i;
async function importAndroidTokens(tokens, dev) {
if (window.connDevice !== dev) throw new Error("Switched headsets; drop the Android files again");
const result = await runJob("Import Android files", "files:" + tokens.join(":"),
() => api("/api/android-import", {action: "import", tokens, display: $("apkDisplay").value}, dev),
j => { if (j.stage) $("androidImportState").textContent = j.stage;
if (j.error && j.apk?.blocker && window.connDevice === dev) checkApkAlternatives(j.apk, dev); });
$("androidImportState").textContent = result?.message || "Import did not finish; see Activity for the reason.";
if (result && !result.needs_game_data) {
for (const app of result.apps || []) await offerTest(app);
}
return result;
}
async function sendAndroidFiles(files, dev) {
const tokens = [];
try {
const empty = files.find(file => !file.size);
if (empty) throw new Error(`${empty.name}: empty files aren't supported; choose the complete nonempty set`);
for (const file of files) {
if (window.connDevice !== dev) throw new Error("Switched headsets; drop the Android files again");
const staged = await upload(file, "android", dev);
tokens.push(staged.token);
}
return await importAndroidTokens(tokens, dev);
} catch (e) {
const message = "Android import failed: " + e.message;
$("androidImportState").textContent = message;
log(message, "e"); toast(message, true);
} finally {
if (tokens.length) api("/api/android-import", {action: "discard", tokens}, dev).catch(() => {});
}
}
async function sendFiles(files, dirs = new Set()) {
const dev = window.connDevice; // the whole batch is for the headset it was dropped on
const sent = new Set();
for (const [i, f] of files.entries()) {
if (sent.has(i)) continue;
if (window.connDevice !== dev) {
toast(`Switched headsets: didn't send ${files.length - i} remaining file${files.length - i === 1 ? "" : "s"}`, true);
break;
@@ -2986,11 +3027,20 @@ async function sendFiles(files, dirs = new Set()) {
else toast(`${f.name}: zip the folder first (folders need the Frame Control app)`, true);
continue;
}
if (ANDROID_EXT.test(f.name)) {
const group = files.filter((file, n) => n >= i && !dirs.has(n) && ANDROID_EXT.test(file.name));
files.forEach((file, n) => { if (n >= i && !dirs.has(n) && ANDROID_EXT.test(file.name)) sent.add(n); });
await sendAndroidFiles(group, dev);
continue;
}
if (!f.size) { toast(`${f.name}: empty files aren't supported`, true); continue; }
if (/\.aab$/i.test(f.name)) {
const message = `${f.name} not installed: an Android App Bundle is a publishing file. Export a device APK/APKS set with the developer's build tools first.`;
log(message, "e"); toast(message, true);
continue;
}
if (TITLE_EXT.test(f.name)) { await sideload(f, path); continue; }
const apk = f.name.toLowerCase().endsWith(".apk");
const res = await act(apk ? `Install ${f.name}` : `Copy ${f.name} to ~/Downloads`, () => upload(f, apk ? "apk" : "push", dev));
if (apk && res && res.app) await offerTest(res.app);
await act(`Copy ${f.name} to ~/Downloads`, () => upload(f, "push", dev));
}
$("fileInput").value = "";
refresh();
@@ -3034,6 +3084,11 @@ async function sideloadOne(file, path, isDir = false, dev = window.connDevice) {
await act(`Copy ${file.name} to ~/Downloads`, () => upload(file, "push", dev));
return;
}
if (r.android) {
try { await importAndroidTokens([r.token], dev); }
finally { api("/api/android-import", {action: "discard", tokens: [r.token]}, dev).catch(() => {}); }
return;
}
// Its own fresh list, so the dialog can say whether this replaces an installed title.
let installed = null;
try { installed = new Set((await api("/api/titles")).titles.map(t => String(t.id).toLowerCase())); } catch {}
+110 -13
View File
@@ -44,6 +44,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent))
import frame_agent # noqa: E402
import frame_assistant # noqa: E402
import frame_android # noqa: E402
import frame_android_import # noqa: E402
from apk_sources import search as apk_search, SourceError # noqa: E402
from apk_sources import fdroid as apk_fdroid # noqa: E402
import frame_apk_versions # noqa: E402
@@ -306,6 +307,8 @@ def _run_job(job, label, work, progress):
fields = {"message": result.get("message") or f"{label}: done", "result": result}
except (Failure, frame_android.FrameError) as e:
fields = {"error": unreachable(str(e)) or str(e)}
if getattr(e, 'apk', None):
fields['apk'] = e.apk
frame_telemetry.diagnostic(f"job {label.split()[0]}", e)
except SourceError as e: # already user-readable, and about a store, not the Frame
fields = {"error": str(e)}
@@ -333,7 +336,7 @@ def start_job(label, work, progress=False):
# normal job (/api/job) that reads "queued" until its turn. A queued install can be
# removed until it starts; each runs on the headset it was queued for, or not at all.
_queue = [] # [job, label, work, progress, headset id], in order; the running one is not here
_queue = [] # [job, label, work, progress, headset id, cleanup], waiting entries only
_queue_state = {"worker": False, "running": None}
@@ -344,12 +347,12 @@ def _restage_locked():
_jobs[job]["stage"] = f"Queued: {n} install{'' if n == 1 else 's'} before it" if n else "Queued: next"
def queue_job(label, work, progress=False):
def queue_job(label, work, progress=False, cleanup=None):
"""Like start_job, but runs after every install queued before it."""
device = LINK.active_device()["id"] if LINK else None
with _jobs_lock:
job = _new_job_locked(label, queued=True, stage=None, percent=None)
_queue.append([job, label, work, progress, device])
_queue.append([job, label, work, progress, device, cleanup])
_restage_locked()
ahead = len(_queue) - 1 + (1 if _queue_state["running"] else 0)
start, _queue_state["worker"] = not _queue_state["worker"], True
@@ -364,7 +367,7 @@ def _drain_queue():
if not _queue:
_queue_state.update(worker=False, running=None)
return
job, label, work, progress, device = _queue.pop(0)
job, label, work, progress, device, cleanup = _queue.pop(0)
_queue_state["running"] = job
_jobs[job].update(queued=False, stage=None)
_restage_locked()
@@ -372,7 +375,11 @@ def _drain_queue():
def guarded(*report, work=work, device=device):
with working(device): # refused if the app switched headsets since it was queued
return work(*report)
_run_job(job, label, guarded, progress)
try:
_run_job(job, label, guarded, progress)
finally:
if cleanup:
cleanup()
def install_queue():
@@ -396,8 +403,14 @@ def queue_action(body):
_jobs[job].update(done=True, queued=False, cancelled=True, stage=None, time=time.time(),
error="Removed from the install queue")
_restage_locked()
return {"message": f"Removed {_jobs[job]['label']} from the install queue"}
raise Failure("that install isn't waiting in the queue (it may have started or finished)", 409)
message = f"Removed {_jobs[job]['label']} from the install queue"
cleanup = q[5]
break
else:
raise Failure("that install isn't waiting in the queue (it may have started or finished)", 409)
if cleanup:
cleanup()
return {"message": message}
def job_status(query):
@@ -1582,8 +1595,83 @@ def apk_installed(info, meta, error, seconds):
frame_android.install_hooks.append(apk_installed)
# ---- Local Android files -------------------------------------------------
# Browsers upload APK/split/OBB selections without building a multi-GB archive
# in memory. One-use tokens stay tied to the headset selected at upload.
_android_uploads = {}
_android_uploads_lock = threading.Lock()
ANDROID_UPLOAD_TTL = 3600
def _purge_android_uploads(now=None):
now = time.time() if now is None else now
with _android_uploads_lock:
expired = [_android_uploads.pop(t) for t, e in list(_android_uploads.items())
if now - e['time'] > ANDROID_UPLOAD_TTL]
for entry in expired:
shutil.rmtree(entry['dir'], ignore_errors=True)
def stage_android_file(path, temp_dir=None):
_purge_android_uploads()
if Path(path).suffix.lower() not in frame_android_import.EXTENSIONS:
raise Failure('Android import needs APK, OBB, XAPK, APKS, APKM or ZIP files', 400)
own = temp_dir
try:
if not own: # local ZIP inspection must never retain/delete the user's original
own = tempfile.mkdtemp(prefix=f'{frame_android_import.TMP_PREFIX}{os.getpid()}-')
copied = Path(own) / Path(path).name
shutil.copyfile(path, copied)
path = copied
size = Path(path).stat().st_size
token = secrets.token_hex(16)
with _android_uploads_lock:
if len(_android_uploads) >= 128 or size + sum(e['size'] for e in _android_uploads.values()) > frame_android_import.MAX_BYTES:
raise Failure('Too many staged Android files; finish the current import or restart the app', 400)
_android_uploads[token] = {'path': str(path), 'dir': own, 'time': time.time(), 'size': size,
'device': LINK.active_device()['id'] if LINK else None}
return {'android': True, 'token': token, 'message': 'Read Android file: ' + Path(path).name}
except BaseException:
if own:
shutil.rmtree(own, ignore_errors=True)
raise
def android_import(body):
_purge_android_uploads()
tokens, action = body.get('tokens'), body.get('action')
if (action not in ('import', 'discard') or not isinstance(tokens, list) or not 1 <= len(tokens) <= 128
or any(not isinstance(t, str) for t in tokens) or len(set(tokens)) != len(tokens)):
raise Failure('Choose staged Android files to import or discard', 400)
display = body.get('display', 'auto')
if display not in ('auto', 'flat', 'vr'):
raise Failure('invalid APK display mode', 400)
device = LINK.active_device()['id'] if LINK and action == 'import' else None
with _android_uploads_lock:
entries = [_android_uploads.get(t) for t in tokens]
if action == 'import' and any(e is None for e in entries):
raise Failure('Android upload expired or was already used; drop the files again', 400)
if action == 'import' and any(e and e['device'] != device for e in entries):
raise Failure('Frame Control switched headsets since these files were read; drop them again', 409)
entries = [_android_uploads.pop(t) for t in tokens if t in _android_uploads]
def cleanup():
for entry in entries:
shutil.rmtree(entry['dir'], ignore_errors=True)
if action == 'discard':
cleanup()
return {'message': 'Discarded Android upload'}
try:
ensure_master()
return queue_job('Import Android files', lambda report: frame_android_import.import_files(
[e['path'] for e in entries], display=display, progress=report), progress=True, cleanup=cleanup)
except BaseException:
cleanup()
raise
# ---- Sideloaded titles (Linux/Windows builds as Steam Devkit Games) --------
#
# Installing is two steps: inspect (a dropped file is uploaded and a zip
# unpacked here, once) returns a token and the detected target and runtime for
# the page to confirm; install then runs in the background with progress the
@@ -1614,6 +1702,8 @@ def _purge_titles(now=None):
def stage_title(path, temp_dir=None, name=None):
"""Inspect a .zip, folder or program and keep it for install; returns the plan and a token."""
_purge_titles()
if Path(path).suffix.lower() == '.zip' and frame_android_import.is_bundle(path):
return stage_android_file(path, temp_dir)
try:
plan = frame_titles.inspect(path, name)
except BaseException as e:
@@ -2197,7 +2287,7 @@ def _pid_alive(pid):
def _tmp_places():
"""Where each kind of staging folder goes, and its name before the server's PID."""
tmp = Path(tempfile.gettempdir())
return [(tmp, WEB_TMP_PREFIX), (tmp, frame_titles.TMP_PREFIX), (tmp, frame_android.TMP_PREFIX),
return [(tmp, WEB_TMP_PREFIX), (tmp, frame_titles.TMP_PREFIX), (tmp, frame_android.TMP_PREFIX), (tmp, frame_android_import.TMP_PREFIX),
(tmp, frame_agent.TMP_PREFIX), (frame_host.cache_dir("apk-sources"), apk_fdroid.TMP_PREFIX)]
@@ -2467,7 +2557,7 @@ POST = {
"/api/agent/approval": agent_approval, "/api/assistant/chat": assistant_chat,
"/api/input": remote_input, "/api/touch": remote_touch,
"/api/settings/artwork": frame_steamgriddb.save_settings,
"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/quest": quest, "/api/meta": meta, "/api/queue": queue_action, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/android-import": android_import, "/api/quest": quest, "/api/meta": meta, "/api/queue": queue_action, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
"/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
"/api/webinstall/cancel": webinstall_cancel,
@@ -2787,7 +2877,8 @@ class Handler(BaseHTTPRequestHandler):
if not isinstance(body, dict):
raise Failure("request body must be a JSON object", 400)
# Not headset work: switching headsets mustn't wait for (or refuse) these.
with (contextlib.nullcontext() if path in NOT_HEADSET_WORK else working(meant)):
local = path in NOT_HEADSET_WORK or (path == '/api/android-import' and action_of(body) == 'discard')
with (contextlib.nullcontext() if local else working(meant)):
result = handler(body)
self.send_json(result)
except ClientGone:
@@ -2890,7 +2981,7 @@ class Handler(BaseHTTPRequestHandler):
def upload(self):
"""Raw file body. X-Filename names it; X-Mode is 'push', 'apk' (install), 'apkinfo' (read only)
or 'title' (a .zip or program to sideload: inspected and kept for /api/titles install)."""
'android' (staged for grouped import), or 'title' (a .zip or program to inspect)."""
name = os.path.basename(unquote(self.headers.get("X-Filename", "")))
mode = self.headers.get("X-Mode", "push")
length = int(self.headers.get("Content-Length") or 0)
@@ -2900,7 +2991,8 @@ class Handler(BaseHTTPRequestHandler):
raise Failure("empty or too-large upload", 400)
if mode in ("apk", "apkinfo") and not name.lower().endswith(".apk"):
raise Failure("APK install needs a .apk file", 400)
tmp = Path(tempfile.mkdtemp(prefix="frame-ui-"))
prefix = f'{frame_android_import.TMP_PREFIX}{os.getpid()}-' if mode in ('android', 'title') else 'frame-ui-'
tmp = Path(tempfile.mkdtemp(prefix=prefix))
keep = False
try:
dest = tmp / name
@@ -2912,6 +3004,10 @@ class Handler(BaseHTTPRequestHandler):
raise Failure("upload interrupted", 400)
f.write(chunk)
remaining -= len(chunk)
if mode == 'android':
result = stage_android_file(dest, str(tmp))
keep = True
return result
if mode == "media":
return push_media(dest)
if mode == "apkinfo":
@@ -3046,6 +3142,7 @@ def main():
if proc.poll() is None:
proc.terminate()
_purge_titles(now=float("inf")) # unconfirmed title uploads
_purge_android_uploads(now=float('inf'))
# Staging folders of work still running (a patched APK mid-copy, an index
# download): leaving below skips the cleanup their TemporaryDirectory would
# get at interpreter exit. sweep_tmp at the next start catches any missed.