LowganDev--PlayfinPS4/docs/testing.md

# Testing

    ctest --preset linux-debug      # unit tests + smoke test + UI screenshots
    cmake --preset linux-asan && cmake --build --preset linux-asan && ctest --preset linux-asan

- Unit tests: doctest, `tests/unit/*_test.cpp`, one executable `playfin_tests`.
- Sanitizers: the `linux-asan` preset builds everything with AddressSanitizer and UBSan. `tests/lsan.supp` may only
  suppress leaks inside system graphics libraries. Mesa's driver is unloaded at exit, so its frames print as
  `<unknown module>`: the suppressions name the call that entered the driver (`platform::init`,
  `glnvg__renderFlush`), never a whole library.
- UI tests: `playfin --screen gallery --test-script <file> --test-output <dir>` plays a script and saves screenshots; no window and
  no display are needed (`SDL_VIDEODRIVER=offscreen`, `MESA_LOADER_DRIVER_OVERRIDE=llvmpipe` = CPU rendering,
  identical on every run). Script commands: `wait <frames>`, `press <Action>`, `type <text>`, `screenshot <name>`, `quit`. Images load
  synchronously in test mode. The gallery (`tests/ui/gallery.script`, references in `tests/ui/references/gallery/`)
  covers rows, grid, focus ring, dialog, text editing (CJK) and screen push/pop.
  The clock is fixed at 60 fps in test mode. `tests/ui/run.sh` compares each image of `tests/ui/references/`
  (fails when more than 0.1 % of the pixels differ by more than 8 per channel).
- Shell end to end (`ui_shell`, `tests/ui/run_shell.sh`): `fake_jellyfin --https --port 48920` (self-signed, fixed
  port so the certificate dialog is identical on every run), then `playfin --screen shell --no-discovery` plays
  `tests/ui/shell.script`: add the server, trust its certificate, pick a profile, sign in with a password, browse
  home, a library (sort bar), an episode's details, the player, the menu, then the search tab (empty and with
  results), the settings tab (list, accent chooser), the licences page and a series' details. Network jobs run
  inline in test mode (deterministic) and screen transitions are off. The run fails if the access token appears in
  `playfin.log`. Quick Connect is covered by unit tests only. Runs serially (fixed port); the same references are
  used under ASan. `ui_media` plays `tests/ui/media.script` on `fake_jellyfin --media` (the test server's views:
  Live TV, music) with `--fixed-time 2026-10-16T19:45:00Z --utc-offset 120` (the guide's clock); in script tests
  the music pauses at its first moment (`frozenPlayback`). After an interrupted run the port can stay in TIME-WAIT for up to a minute: the fake server
  then fails with "port in use" and every screenshot after the address shows "cannot reach the server".
- The fake server (`tests/support/fake_jellyfin.*`) also answers what the app screens need: `/Items` honours
  `sortOrder=Descending` (reversed), `filters` (`IsFavorite`: every 10th movie, `IsUnplayed`: odd ones) and
  `includeItemTypes` (`Series` → s-1 "Test Show", `Episode` → e-1 "Pilot", for searches); `/Shows/s-1/Episodes`
  answers per season (se-2 has e-3); `/Users/{id}/Images/Primary` serves a picture. `requests()`, `count(path)` and
  `playbackInfoBody()` let tests check what was asked.
- In the Linux UI editor (no system keyboard) the text starts from the field's value: a test that searches twice
  clears the field first.
- `tests/fixtures/` and the fake server are shared by the unit and end-to-end tests: adding image tags or fields
  to a fixture changes the shell screenshots (regenerate and review them).
- stb_image (inside `nanovg.c`) and stb_image_write are built with `-fno-sanitize=shift` under sanitizers: their
  JPEG code shifts negative values (upstream, harmless).
- Command-line checks (`tests/cli/`, CTest): a test script needs `--test-output`; a startup failure (SDL `dummy`
  driver, no OpenGL) must exit 1 and be written to `playfin.log`.
- Changing the look on purpose: `scripts/update-ui-refs.sh`, then review the new images before committing. The
  references are made by the software renderer (the script's default);
  `ui_gallery_gl` checks that NanoVG's GLES2 backend still draws the same within the imgdiff tolerance.
- Network tests run against `tests/support/fake_jellyfin.*` (cpp-httplib) over HTTP and HTTPS, with test
  certificates in `tests/fixtures/tls/` (a test CA, a CA-signed server, a self-signed server; test-only keys).
- Fuzzing: `cmake --preset linux-fuzz && cmake --build --preset linux-fuzz && ctest --preset linux-fuzz`
  (30 s per target; crashes land in `build/linux-fuzz/fuzz-artifacts/`).
- Under sanitizers the vendored stb is built with `-fno-sanitize=shift` (its JPEG encoder shifts into the sign bit,
  upstream code used only by the fake server's test posters).
- On the PS4, `/data/Playfin/playfin.log` (FTP) is opened before anything else and records each startup step
  (`startup: SDL`, `window`, `OpenGL ES context`, `network`, `canvas`, `fonts and translations`, `first frame`,
  `frame 0 drawn`): the last line before a crash shows where it stopped. Every log line is also sent to the
  kernel log (`sceKernelDebugOutText`), which still works when the file cannot be written.
- PS4 startup trace: `/data/Playfin/boot.txt` is written with raw kernel calls (`sceKernelOpen`/`Write`, no
  stdio, no buffering) from a constructor that runs before main ("loaded"), then "main", "data folder ok",
  the log file open result (errno on failure; append mode falls back to "w") and the first 200 log lines. It is
  rewritten on every launch. On PS4 log timestamps are UTC (no time zone files).
- Rendering: `raster_test` (exact pixels for the fill rule, culling, stencil, paints, scissor, edge alpha, a random
  coverage check against the pixel-centre rule, the constant-colour fast path compared byte for byte with the
  general path, the integer image path within 2 per channel, convex polygons identical to their fans) and
  `nvg_sw_test` (NanoVG scenes: rectangles, circles, holes, crossing strokes, images, text, one thread vs four).
  `ui_gallery` and `ui_shell` render through the software backend against references made by the same renderer;
  `ui_gallery_gl` runs the same script through the GL backend. `fuzz_raster` feeds random paints, states
  and triangles (non-finite and huge values included).
- `playfin_render` is compiled with -O2 in every build type (the per-pixel hot path).
- Player: `engine_test` plays `tests/fixtures/media/clip.mp4` (generated, see its README) through libmpv with
  `ao=null` (frames, seek, pause, end, missing file, external subtitle, non-blocking render);
  `playback_api_test`, `profile_test`, `player_session_test` and the player cases of `shell_screens_test` run
  against the fake server, which streams the clip for every item, answers PlaybackInfo (direct, or transcode for
  `m-transcode`), media segments (`e-1`) and records the playback reports. `ui_shell` screenshots the player bar
  and the track menu over the clip's first picture.

## Test server (music and Live TV)

`scripts/test-server/up.sh` runs a real Jellyfin (Docker image `jellyfin/jellyfin:10.11.8`) on port 8097 of the
PC with generated content, for testing music and Live TV:
- Music (`make-media.sh`, ffmpeg tones, no copyrighted content): artists "Les Ondes Calmes", "Kilohertz",
  "Artistes variés" (with "Duo Sinus" on some tracks); 5 albums, 30 tracks of 30-90 s in MP3, FLAC, AAC (m4a, two
  discs) and Opus, with tags and covers; a playlist "Soirée tests" made by `setup.py`.
- Live TV: 3 channels ("Playfin Info", "Playfin Sport", "Playfin Cinéma"), each a host ffmpeg encoding a test
  pattern with the clock into live HLS (`$DATA/hls/chN`), served by an nginx container on the Docker network as
  `http://streams/chN/index.m3u8` (no port opened on the PC: the PC's ufw stays as it is; Docker publishes 8097
  itself). An M3U tuner and an XMLTV guide (3 days from the day of `up.sh`, remade on each start).
- The console reaches it at `<PC address>:8097` (the PC's own Jellyfin, if any, keeps 8096). On the PS4 keyboard
  ":" is on the symbols page (L2+△ key, "@#:").
- `setup.py` (idempotent) names it "Playfin Test", lists its user on the sign-in screen, completes the wizard, creates the user `playfin` / `playfin-test`, adds the library,
  the tuner, the guide and the recordings folder (`/recordings`), refreshes them, makes the playlist.

What the real server answers (fixtures recorded from it in `tests/fixtures/jellyfin/ts/`, API keys redacted):
- A channel's media source is the tuner's URL (`http://streams/...`, unreachable from the console). With
  `EnableDirectPlay: false` and no `hls` container in the direct play profiles, `PlaybackInfo` returns a
  `TranscodingUrl` (`/videos/{id}/master.m3u8?...&LiveStreamId=...`): the server relays the channel (video copied
  when compatible). The live stream closes with `POST /LiveStreams/Close?liveStreamId=` (not under `/LiveTv`).
- User views: "Live TV" (`UserView`, collection type `livetv`), the music library (`CollectionFolder`, `music`),
  "Playlists" (`UserView`, `playlists`) and "Recordings" (`CollectionFolder` without a collection type).
- Jellyfin answers 503 with an HTML page while it starts; `setup.py` waits.