feat: macOS preview and local build workflow (#93)

README.md

@@ -223,6 +223,7 @@ screen. In the other order ShadowMount+ finds the file again and puts the icon b
 | `app/` | The application: JSX sources, state, assets, catalog data, and `app.json` (title id `PPSA99058`, version, capabilities) |
 | `ps5-react/` | The framework it runs on, vendored from [half144/ps5-react](https://github.com/half144/ps5-react) |
 | `packaging/make-exfat.sh` | Builds the exFAT image ShadowMount+ installs |
+| `packaging/make-exfat-mac.sh` | The same image, built locally on macOS without `update.zip` |
 | `.github/workflows/build.yml` | The build |
 
 </details>
@@ -249,8 +250,9 @@ What a bundler throws away could not come back: most comments, the JSX syntax (i
 
 Everything runs from the repository root through npm. Once after cloning, `npm run setup`
 (`packaging/setup.sh`) installs every dependency: on Linux the apt packages for the desktop preview,
-the PS5 build and the tests (it asks for `sudo` once), on macOS the Homebrew formulae; then Pillow
-and the framework's npm packages. Read the script first if you'd rather install them yourself.
+the PS5 build and the tests (it asks for `sudo` once); on macOS the Homebrew formulae, a
+`ps5-react/.venv` with Pillow (Homebrew's `python3` refuses a global `pip install`) and the
+framework's npm packages. Read the script first if you'd rather install them yourself.
 
 ```sh
 git clone https://github.com/MyLuxy/overdrive.git
@@ -289,6 +291,42 @@ upstream ps5-react supports macOS only.
 
 </details>
 
+<details>
+<summary><b>Developing on macOS</b></summary>
+<br>
+
+The preview and the PS5 build run on a Mac with Xcode's command line tools, Homebrew, Node 18 or
+newer and Python 3.9 or newer. `npm run setup` installs the Homebrew formulae (CMake, LLVM 18,
+SDL2, ninja, nasm, libarchive, xz, OpenSSL), creates `ps5-react/.venv` with Pillow, and installs the
+framework's npm packages. Homebrew's `python3` is externally managed, so Pillow goes in the venv
+rather than a global `pip install`.
+
+```sh
+xcode-select --install     # once, if you have never built anything on this Mac
+npm run setup              # Homebrew formulae, ps5-react/.venv, framework npm packages
+npm run preview:mac        # the desktop preview
+```
+
+The `:mac` commands leave `preview`, `dev`, `test` and `build` untouched: those remain the ones
+Linux and the other developers use. `preview:mac` and `dev:mac` run through `ps5-react/.venv`, put
+LLVM 18 and Node on `PATH`, and prefer MacPorts at `/opt/local` with an SDL2 built in
+`~/.local/overdrive-sdl2` when they exist. In that case `/usr/local/bin` is left out of `PATH`, so
+CMake does not mix Homebrew's OpenSSL/CURL with the MacPorts copies and break the link.
+
+| Command | What it does |
+| --- | --- |
+| `npm run setup:mac` | Creates `ps5-react/.venv` with Pillow when the native libraries are already installed |
+| `npm run preview:mac` | Opens the desktop preview once |
+| `npm run dev:mac` | Opens it and rebuilds on every save under `app/` |
+| `npm run doctor:mac` | Checks the venv, Pillow, LLVM 18, Node, CMake and clang |
+
+If the preview does not start: `npm run doctor:mac` first; `brew install llvm@18` when LLVM is
+missing; `brew install sdl2` or `CMAKE_PREFIX_PATH=$HOME/.local/overdrive-sdl2` when SDL2 is
+missing; and keep Homebrew's `/usr/local/bin` out of `PATH` when a link error mentions OpenSSL or
+CURL and MacPorts supplies CMake.
+
+</details>
+
 <details>
 <summary><b>Building</b></summary>
 <br>
@@ -311,6 +349,23 @@ downloads its pinned dependencies into `ps5-react/.deps` and writes the app fold
 `.receipt.json` into `ps5-react/dist`. It verifies the result itself (`tools/verify.py`), so a
 green build means the payload structure and signatures check out.
 
+On macOS, `npm run build:mac` does the same app build through `ps5-react/.venv` (Pillow for
+`verify.py`) and LLVM 18, without `sudo` and without packaging. The image is a separate,
+local-only step: `packaging/make-exfat-mac.sh` uses macOS's `newfs_exfat` and `hdiutil`, writes
+`Overdrive.exfat` alone and **no `update.zip`**. The CI keeps using `packaging/make-exfat.sh` on
+Ubuntu and still produces both files.
+
+```sh
+npm run build:mac      # the PS5 app only, in ps5-react/dist/PPSA99058
+npm run package:mac    # Overdrive.exfat from the app already built
+npm run image:mac      # both, in one go
+```
+
+`npm run package:mac` overwrites `Overdrive.exfat` in the repository root, so keep a copy of the
+image you had before. It checks the filesystem with `fsck_exfat` and that every file of the app is
+in the image; `fsck_exfat -n Overdrive.exfat` checks it again afterwards. The image keeps the same
+64 KiB clusters and `OVERDRIVE` label as the Linux one.
+
 The image is a plain exFAT volume whose root is the app folder, with 64 KiB clusters and the
 label `OVERDRIVE`. To release a new version, raise the version in `app/version.js` and
 `app/app.json`, run the workflow, test the draft's image on a console, then publish the draft:

package.json

@@ -16,6 +16,14 @@
     "dev        Same window, rebuilt and restarted on every save under app/. State resets.",
     "test       The framework's checks; run after changing rendering or input in ps5-react/.",
     "",
+    "macOS only: setup:mac creates ps5-react/.venv with Pillow; preview:mac, dev:mac and",
+    "            doctor:mac run through that venv, with LLVM 18, Node and the MacPorts/SDL2",
+    "            libraries when present. The commands above stay unchanged for other hosts.",
+    "build:mac  The PS5 app only, through that venv, into ps5-react/dist/PPSA99058. No sudo.",
+    "package:mac  Local only: builds Overdrive.exfat from the app above with newfs_exfat/hdiutil.",
+    "             It writes no update.zip; the CI keeps producing that on Linux.",
+    "image:mac  build:mac, then package:mac.",
+    "",
     "CC=clang CXX=clang++: the preview is compiled with clang on every platform, since GCC",
     "stops on a bounds warning in archive_preflight.cpp."
   ],
@@ -25,6 +33,13 @@
     "build:app": "bash packaging/build.sh --no-package",
     "preview": "cd ps5-react && CC=clang CXX=clang++ python3 tools/cli.py preview --app-dir ../app",
     "dev": "cd ps5-react && CC=clang CXX=clang++ python3 tools/cli.py preview --app-dir ../app --watch",
-    "test": "cd ps5-react && CC=clang CXX=clang++ npm test"
+    "test": "cd ps5-react && CC=clang CXX=clang++ npm test",
+    "setup:mac": "[ -x ps5-react/.venv/bin/python ] || python3 -m venv ps5-react/.venv; ps5-react/.venv/bin/pip install -r ps5-react/requirements.txt",
+    "doctor:mac": "cd ps5-react && sh tools/mac-run.sh doctor",
+    "preview:mac": "cd ps5-react && CC=clang CXX=clang++ sh tools/mac-run.sh preview --app-dir ../app",
+    "dev:mac": "cd ps5-react && CC=clang CXX=clang++ sh tools/mac-run.sh preview --app-dir ../app --watch",
+    "build:mac": "bash packaging/build-mac.sh",
+    "package:mac": "bash packaging/make-exfat-mac.sh ps5-react/dist/PPSA99058 Overdrive.exfat OVERDRIVE",
+    "image:mac": "npm run build:mac && npm run package:mac"
   }
 }

packaging/build-mac.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# Builds the PS5 app on macOS.
+#
+# Uses ps5-react/.venv for Pillow (verify.py imports it and Homebrew's python3
+# is externally managed), LLVM 18 from Homebrew, and ninja from Homebrew. The
+# desktop preview's MacPorts/SDL2 environment does not apply here.
+#
+# Packaging is a separate, local-only step: packaging/make-exfat-mac.sh.
+set -euo pipefail
+
+root="$(cd "$(dirname "$0")/.." && pwd)"
+venv="$root/ps5-react/.venv"
+if [ ! -x "$venv/bin/python" ]; then
+    echo "error: $venv is missing; run 'npm run setup:mac' from the repository root." >&2
+    exit 1
+fi
+
+llvm="${LLVM:-}"
+if [ -z "$llvm" ]; then
+    for candidate in /usr/local/opt/llvm@18/bin /opt/homebrew/opt/llvm@18/bin; do
+        if [ -x "$candidate/clang" ]; then llvm=$candidate; break; fi
+    done
+fi
+if [ -z "$llvm" ] || [ ! -x "$llvm/clang" ]; then
+    echo "error: LLVM 18 not found. Install it with 'brew install llvm@18' or set LLVM=<its bin directory>." >&2
+    exit 1
+fi
+
+PATH="$venv/bin:$llvm:/opt/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
+export PATH
+
+# The PS5 build does not link the desktop's libraries, but a MacPorts setup keeps
+# CMake away from a mixed Homebrew/MacPorts prefix when one is present.
+prefixes=
+if [ -d "$HOME/.local/overdrive-sdl2" ]; then
+    prefixes=$HOME/.local/overdrive-sdl2
+fi
+if [ -x /opt/local/bin/cmake ]; then
+    prefixes=${prefixes:+$prefixes:}/opt/local
+fi
+if [ -n "$prefixes" ]; then
+    export CMAKE_PREFIX_PATH="$prefixes${CMAKE_PREFIX_PATH:+:$CMAKE_PREFIX_PATH}"
+fi
+
+exec bash "$root/packaging/build.sh" --no-package

packaging/make-exfat-mac.sh

@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+# Packages a built PS5 app folder as the exFAT image ShadowMount+ installs, on
+# macOS. Local only: CI keeps using make-exfat.sh on Linux, which also writes
+# update.zip. This script writes the image alone.
+#
+# usage: make-exfat-mac.sh <app-folder> <out.exfat> [label] [cluster-size]
+set -euo pipefail
+
+app="${1:-}"
+out="${2:-}"
+label="${3:-OVERDRIVE}"
+cluster="${4:-64K}"
+
+if [ -z "$app" ] || [ -z "$out" ]; then
+    echo "usage: $0 <app-folder> <out.exfat> [label] [cluster-size]" >&2
+    exit 2
+fi
+[ -d "$app" ] || { echo "error: no such app folder: $app" >&2; exit 1; }
+[ -f "$app/eboot.bin" ] || { echo "error: $app has no eboot.bin, it is not a built PS5 app" >&2; exit 1; }
+for tool in hdiutil newfs_exfat fsck_exfat; do
+    command -v "$tool" >/dev/null || { echo "error: $tool not found; this script is macOS only" >&2; exit 1; }
+done
+
+case "$cluster" in
+    32K|32k) cluster_bytes=32768 ;;
+    64K|64k) cluster_bytes=65536 ;;
+    128K|128k) cluster_bytes=131072 ;;
+    *) echo "error: cluster size must be 32K, 64K or 128K" >&2; exit 2 ;;
+esac
+
+mount_point="/Volumes/$label"
+if mount | grep -F " on $mount_point " >/dev/null; then
+    echo "error: $mount_point is already mounted; unmount it first" >&2
+    exit 1
+fi
+
+# Size the image from what exFAT actually consumes: every file takes whole
+# clusters, and the app ships thousands of small JSON files, so the payload's
+# apparent size underestimates the volume. This is the same formula as
+# make-exfat.sh, with macOS's stat instead of GNU find -printf.
+slack=$(( ${EXFAT_SLACK_MB:-128} * 1024 * 1024 ))
+clusters=$(find "$app" -type f -exec stat -f%z {} + |
+    awk -v c="$cluster_bytes" '{ n += int(($1 + c - 1) / c) } END { print n + 0 }')
+directories=$(find "$app" -type d | wc -l | tr -d ' ')
+payload=$(( (clusters + directories) * cluster_bytes ))
+size=$(( (payload + slack + 1048575) / 1048576 * 1048576 ))
+printf '→ exFAT: %s MiB, %s clusters, label %s\n' "$((size / 1048576))" "$cluster" "$label"
+
+if [ -e "$out" ]; then
+    echo "→ replacing $out"
+fi
+rm -f "$out"
+truncate -s "$size" "$out"
+
+dev=
+mounted=0
+cleanup() {
+    if [ "$mounted" = 1 ]; then
+        hdiutil detach "$mount_point" >/dev/null 2>&1 ||
+            hdiutil detach -force "$mount_point" >/dev/null 2>&1 || true
+    fi
+    if [ -n "$dev" ]; then
+        hdiutil detach "$dev" >/dev/null 2>&1 || true
+    fi
+}
+trap cleanup EXIT
+
+dev=$(hdiutil attach -imagekey diskimage-class=CRawDiskImage -nomount "$out" |
+    awk '/^\/dev\/disk/{print $1; exit}')
+[ -n "$dev" ] || { echo "error: hdiutil could not attach $out" >&2; exit 1; }
+newfs_exfat -R -v "$label" -b "$cluster_bytes" "$dev" >/dev/null
+hdiutil detach "$dev" >/dev/null
+dev=
+
+attach=$(hdiutil attach -imagekey diskimage-class=CRawDiskImage -nobrowse "$out")
+dev=$(printf '%s\n' "$attach" | awk '$3 ~ /^\/Volumes\// {print $1; exit}')
+[ -n "$dev" ] || dev=$(printf '%s\n' "$attach" | awk '/^\/dev\/disk/{print $1; exit}')
+for _ in 1 2 3 4 5 6 7 8 9 10; do
+    [ -d "$mount_point" ] && break
+    sleep 1
+done
+[ -d "$mount_point" ] || { echo "error: $mount_point did not mount" >&2; exit 1; }
+mounted=1
+
+cp -R "$app/." "$mount_point/"
+
+# Finder and Spotlight may add their own directories; they are not part of the app.
+rm -rf "$mount_point/.Spotlight-V100" "$mount_point/.fseventsd" "$mount_point/.Trashes"
+src=$(find "$app" -type f | wc -l | tr -d ' ')
+dst=$(find "$mount_point" -type f \
+    -not -path "$mount_point/.Spotlight-V100/*" \
+    -not -path "$mount_point/.fseventsd/*" \
+    -not -path "$mount_point/.Trashes/*" | wc -l | tr -d ' ')
+[ "$src" = "$dst" ] || { echo "error: wrote $dst files, the app has $src" >&2; exit 1; }
+
+sync
+hdiutil detach "$mount_point" >/dev/null || hdiutil detach -force "$mount_point" >/dev/null
+mounted=0
+dev=
+
+# Check the finished image itself: its filesystem, and the app's entry point.
+dev=$(hdiutil attach -imagekey diskimage-class=CRawDiskImage -nomount "$out" |
+    awk '/^\/dev\/disk/{print $1; exit}')
+[ -n "$dev" ] || { echo "error: hdiutil could not reattach $out" >&2; exit 1; }
+fsck_exfat -n "$dev" >/dev/null
+hdiutil detach "$dev" >/dev/null
+dev=
+
+echo "wrote $out ($(stat -f%z "$out") bytes, $src files)"

packaging/setup.sh

@@ -8,7 +8,8 @@
 # preview (SDL2, OpenGL, curl, libarchive, OpenSSL, clang), the PS5 build
 # (LLVM 18, ninja, nasm, exfatprogs) and the tests (bsdtar), then Pillow for
 # the asset baking and the framework's npm packages.
-# macOS (Homebrew): the same through brew and pip.
+# macOS (Homebrew): the same through brew, plus a ps5-react/.venv for Pillow
+# (Homebrew's python3 refuses a global pip install).
 set -euo pipefail
 
 root="$(cd "$(dirname "$0")/.." && pwd)"
@@ -28,8 +29,10 @@ Darwin)
     command -v brew >/dev/null || { echo "error: Homebrew is needed: https://brew.sh" >&2; exit 1; }
     echo "→ Homebrew formulae"
     brew install cmake sdl2 llvm@18 nasm ninja curl libarchive xz openssl@3 python node
-    echo "→ Pillow"
-    python3 -m pip install -r "$root/ps5-react/requirements.txt"
+    echo "→ Python environment"
+    [ -x "$root/ps5-react/.venv/bin/python" ] || python3 -m venv "$root/ps5-react/.venv"
+    "$root/ps5-react/.venv/bin/pip" install --disable-pip-version-check \
+        -r "$root/ps5-react/requirements.txt"
     ;;
 *)
     echo "error: unsupported OS $(uname -s)" >&2
@@ -41,10 +44,20 @@ echo "→ framework npm packages"
 (cd "$root/ps5-react" && npm ci)
 
 echo "→ checking"
-python3 -c "import PIL; print('Pillow', PIL.__version__)"
-for tool in cmake clang ninja nasm node mkfs.exfat; do
+case "$(uname -s)" in
+Darwin)
+    "$root/ps5-react/.venv/bin/python" -c "import PIL; print('Pillow', PIL.__version__)"
+    tools="cmake clang ninja nasm node newfs_exfat hdiutil"
+    ;;
+*)
+    python3 -c "import PIL; print('Pillow', PIL.__version__)"
+    tools="cmake clang ninja nasm node mkfs.exfat"
+    ;;
+esac
+for tool in $tools; do
     printf '%-10s %s\n' "$tool" "$(command -v "$tool" || echo MISSING)"
 done
 echo
 echo "Ready. Next: npm run preview (desktop window), npm run build (Overdrive.exfat)."
+echo "On macOS the :mac variants (preview:mac, dev:mac, build:mac) use ps5-react/.venv."
 echo "The first preview and the first build download and compile the pinned dependencies; that takes a while."

ps5-react/package.json

@@ -10,7 +10,10 @@
     "dev": "python3 tools/cli.py preview --watch",
     "build": "python3 tools/cli.py build",
     "test": "python3 tools/cli.py test",
-    "create": "python3 tools/cli.py create"
+    "create": "python3 tools/cli.py create",
+    "doctor:mac": "sh tools/mac-run.sh doctor",
+    "preview:mac": "sh tools/mac-run.sh preview",
+    "dev:mac": "sh tools/mac-run.sh preview --watch"
   },
   "dependencies": {
     "react": "18.3.1"

ps5-react/tools/mac-run.sh

@@ -0,0 +1,54 @@
+#!/bin/sh
+# macOS launcher for the desktop preview.
+#
+# Uses ps5-react/.venv for Pillow (Homebrew's python3 is externally managed) and
+# puts it on PATH so the `python3` calls inside cli.py resolve to the same
+# interpreter. LLVM 18 comes from Homebrew, Node from nvm, and the MacPorts
+# libraries plus a locally built SDL2 are preferred when present.
+#
+# /usr/local/bin is left out when MacPorts supplies cmake: the preview's
+# CMakeLists asks `brew --prefix`, and mixing Homebrew's OpenSSL/CURL with the
+# MacPorts copies breaks the link.
+set -eu
+
+root=$(CDPATH= cd "$(dirname "$0")/.." && pwd)
+venv=$root/.venv
+
+if [ ! -x "$venv/bin/python" ]; then
+    echo "error: $venv is missing. Run 'npm run setup:mac' from the repository root." >&2
+    exit 1
+fi
+
+llvm=
+for dir in /usr/local/opt/llvm@18/bin /opt/homebrew/opt/llvm@18/bin; do
+    if [ -x "$dir/clang" ]; then llvm=$dir; break; fi
+done
+if [ -z "$llvm" ]; then
+    echo "error: LLVM 18 not found. Install it with 'brew install llvm@18'." >&2
+    exit 1
+fi
+
+node=$(command -v node || true)
+if [ -z "$node" ]; then
+    echo "error: node not found; install Node 18 or newer." >&2
+    exit 1
+fi
+node_dir=$(dirname "$node")
+
+prefixes=
+tail=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
+if [ -d "$HOME/.local/overdrive-sdl2" ]; then
+    prefixes=$HOME/.local/overdrive-sdl2
+fi
+if [ -x /opt/local/bin/cmake ]; then
+    prefixes=${prefixes:+$prefixes:}/opt/local
+    tail=/usr/bin:/bin:/usr/sbin:/sbin
+fi
+
+PATH="$venv/bin:$llvm:$node_dir:/opt/local/bin:$tail"
+export PATH
+if [ -n "$prefixes" ]; then
+    export CMAKE_PREFIX_PATH="$prefixes${CMAKE_PREFIX_PATH:+:$CMAKE_PREFIX_PATH}"
+fi
+
+exec "$venv/bin/python" "$root/tools/cli.py" "$@"

ps5-react/tools/rar_worker.py

@@ -101,11 +101,26 @@ def build(output, compiler, flags=(), libraries=(), env=None):
     return output
 
 
+def archive_prefix():
+    """The prefix holding archive.h, as pkg-config, Homebrew or MacPorts reports it."""
+    for command in (["pkg-config", "--variable=prefix", "libarchive"], ["brew", "--prefix", "libarchive"]):
+        try:
+            candidate = subprocess.check_output(command, text=True, stderr=subprocess.DEVNULL).strip()
+        except (OSError, subprocess.CalledProcessError):
+            continue
+        if candidate and (Path(candidate) / "include/archive.h").exists():
+            return candidate
+    for candidate in ("/opt/local", "/usr/local", "/opt/homebrew"):
+        if (Path(candidate) / "include/archive.h").exists():
+            return candidate
+    raise RuntimeError("libarchive not found: install it with Homebrew or MacPorts")
+
+
 def build_host(output):
     """rar-extract for the desktop preview and the archive tests."""
     # macOS hashes with CommonCrypto and keeps Homebrew's libarchive keg-only; elsewhere the driver uses
-    # OpenSSL's EVP and the system's libarchive.
+    # OpenSSL's EVP and the system's libarchive. MacPorts' libarchive is found through pkg-config too.
     if sys.platform == "darwin":
-        prefix = subprocess.check_output(["brew", "--prefix", "libarchive"], text=True).strip()
+        prefix = archive_prefix()
         return build(output, ["clang++"], ["-pthread", "-I", f"{prefix}/include"], ["-L", f"{prefix}/lib", "-larchive"])
     return build(output, ["clang++"], ["-pthread"], ["-larchive", "-lcrypto"])