fstat13--PSCapture/docs/ENGINEERING_LOG.md

PSCapture — Engineering log: problems, diagnoses, work-arounds

This is the long story: every wall we hit while building PSCapture, how it was diagnosed, and what finally fixed (or bypassed) it. Dead ends are kept on purpose — they cost real time and shouldn't cost yours.

(Testbed: retail PS5, firmware 13.60, jailbroken; PC on the same LAN. All work done on personally owned hardware.)

---

Part A — Getting pixels out of the console

A1. The official capture stack is a dead end

Symptoms / approach. First attempts used the console services that look like capture APIs: libSceAvcap2 / libSceVideoRecording etc. What we found. libSceVideoRecording opens a session backed by a 128 MB file and talks to its client over internal IPC; calls from an unprivileged payload fail with the service unavailable. The IPMI (inter-process messaging) client table is limited, and every live instance of our payload holds a session — with ~26 stale instances, a new payload waits forever (connect -> 0x8002000d). Resolution. Abandoned the service path entirely. The work-around that later became permanent: (1) never rely on the recording service; (2) payloads kill their own stale instances at boot ("v17 hygiene"), fingerprinting them by their fixed cacheable mapping in the page tables (PML4 entry #128 — no system process uses it).

A2. The direct path: the display engine's scan-out ring

Diagnosis. The AV-control daemon (SceSysAvControl.elf) exposes the scan-out geometry: ring at 0x200200000, data at +0x4200000, four surfaces 0x600000 apart, rewritten at 60 Hz, stored as P010 (10-bit). First bug — P010 decoding. Extracting the even bytes gave noise: the usable byte of a 10-bit sample is the odd one. Lesson recorded in REVERSE_ENGINEERING.md so nobody re-makes it.

A3. The rotating image (the pan)

Symptom. The picture "rotated" and slid horizontally forever; reading the four surfaces in alternation made it jump every frame. Diagnosis. The scan-out base drifts within the ring: the frame does not start at a fixed offset. The surface content is also quasi-periodic, which made the first metric find a twin minimum 1920 bytes away and oscillate between the two. Dead ends (documented so you skip them):

• a "junction" continuity metric that needed a +960 fudge — produced oscillation between twins;
• chasing the drift with a per-frame tracker — the pursuit itself caused visible horizontal wiggling. Final metric ("exact cut"): strip [row r, 0..maxj] must match [row r-1, c..], minimizing Σ |Y[r][j*2+1] − Y[r-1][(c+j)*2+1]| over sampled rows; conversion pan = ((1920 − c) % 1920) * 2. Resolution. Field-verified fixed pan = 2230 (the drift is ~2 B/s, below visibility), and later a user-tunable fine shift on the PC side (even pixels only, see D2).

A4. The color saga (or: "v10 colors were right")

Symptom. Backgrounds looked red/amber instead of blue; users kept saying "too red", "now too gray", "the colors of v10 were right". Diagnosis path. The chroma plane does not start after the luma: it begins at byte 0x3EA300 (= 4 104 960), 11 rows before the presumed boundary, with dead zones before and after (the "green bar" of legend was the tail read into the gap → replicate the last good row). But position alone wasn't the whole story. Comparing a live frame byte for byte against an old reference frame of the same screen showed the output matched only with U/V pairs swapped: in this plane the 16-bit words are stored V-first. Mean absolute error dropped 15.5 → 1.7 → 0.06 once the pair order was corrected in the P010→NV12 conversion (a second shuffle mask for the chroma region only). Honest note. Before that discovery, the wrong knob was shipped: a chroma gain to "lower the red" — it made the cast less visible but never right. The gain knob survives as an optional tuning, default 1.0 (= raw). How the truth was found. Three independent anchors: neutral white stays (128,128) either way, the bluish background flips the right way only with the swap, and the byte-level diff against the reference settles it numerically. Rendering the same bytes with two different YUV formulas long ago had also caused a wild goose chase — the rule became: compare numerically, never eye-render across periods (see Lessons).

A5. Kernel reads without crashing the console (three crashes, one rule)

Crash #1/#2. Full console lock-ups (all TCP ports mute, ICMP half-dead) after storms of live payload instances + IPMI maintenance traffic. Recovered only by physical power cycle. Root cause never proven; the permanent hygiene (single instance, kill-istanze at boot) removed the whole class. Crash #3 (diagnosed exactly). A kernel-dump experiment read kernel VA through kernel_copyout — a fault there is a kernel-mode fault: instant panic. Rule, now absolute: kernel memory is only ever read via a page table walk with present-bit validation + 8-byte reads through the direct map; kernel_copyout on kernel addresses is banned. Related guard bug (fun one). The cacheable-mapping builder refused physical addresses above 16 GB as "suspicious" — but the display surfaces live at 0x459200000 (≈18.7 GB). Result: silent fallback to the slow path (45 MB/s, 6 fps). Fix: raise the guard to 32 GB; the real safety net is the page-table validation, not the numeric limit.

A6. Speed: from 45 MB/s to 4 GB/s

Diagnosis. Reading the ring through syscall copies serialized inside the process (multi-threading didn't help — the copies bottleneck in the kernel path). Fix. Build a cacheable mapping of the DCE physical range inside the payload process: allocate our own PDPT/PD pages, replicate the daemon's PDE flags (write-back, no PCD/PWT), write one PML4 entry, done. Bulk frame reads went to ~4 GB/s. Tearing. The DCE rewrites a surface at 60 Hz; a read during a rewrite yields a mixed frame (ghosts, colored tails). Fix: read, wait 2 ms, compare; if the surface changed it was alive — move to the next one.

---

Part B — The PC side: from frames to a "webcam"

B1. Pipeline shape

Feeder (TCP receiver) → shared memory ring → DirectShow source filter (ps5cam.c → pscapture.dll) that every consumer sees as a camera called PS5 Capture.

B2. Discord error 2014 (spinner → no camera), root cause found in their binary

Symptoms. Camera selectable, infinite spinner, then Discord error 2014 — with any virtual camera (ours and OBS Virtual Camera alike). Diagnosis. Discord's own media logs contained the smoking gun: Failed to create capturer (video_capture_module_source.cpp:105). Reading their capture engine (WebRTC's DShow path inside the Discord binary): the engine's device picker requests 1280×720@30 and only accepts a device that lists a matching capability. Ours offered only 1920×1080 → no match → capturer never created. OBS Virtual Camera exposed zero caps → same fate. Fix — "real webcam" capability profile. The driver now exposes 11 UVC-like capabilities (YUY2/NV12/I420/RGB24/RGB32 at 1080p/720p/480p, negative-height variants first), plus a DevicePath string in the registry (the modern identifier consumers use instead of the friendly name). Verified in their logs: Capturer running at 1280x720@30, first frame in 10–13 ms. Tooling discoveries that made this tractable:

• Discord can be driven with --remote-debugging-port; from it the media engine is scriptable (device selection, camera on/off) without touching the UI — see docs/ history and our notes.
• The binary is searchable for log strings ((file:line) hints) and RIP-relative xrefs; that's how the picker logic was pinned down.

B3. Flips and orientations

Symptom. 720p output upside-down for one consumer but not another. Diagnosis. DIB convention sign vs YUV conventions differ between consumers: ffmpeg/WebRTC treat YUV as top-down regardless of biHeight sign, but XRGB-style DIBs as bottom-up. Fix. Flip only for RGB subtypes when biHeight > 0; all YUV paths stay top-down. Verified with ffmpeg grabs.

B4. Alignment ("more space on the left than on the right")

Symptom. The image sat a few dozen pixels off; users described margins, not offsets. Diagnosis. Measured inside the frame: the centered title text of the screen had its glyph center at x≈993 (960 = screen center) → the capture was ~33 px to the right; button margins agreed (~29 px, arrow ~18 px — the title is the cleanest anchor). Fix. A cyclic horizontal shift in the feeder (whole luma plane and UV plane, same byte count; even pixels only — an odd shift re-labels U/V pairs and hue-inverts), exposed as the live shift px knob. Long-term. Bake the value into the payload's pan and drop the PC-side shift (planned).

B5. The 1080p question (and what Discord really does)

Users compared with capture cards: "Elgatos stream at 1080p!". Findings. (1) The capture path in Discord calls is capped by Discord's engine (it asks every camera for 720p — capture cards included); (2) as a generic DirectShow source, PSCapture already delivers 1920×1080 to OBS and ffmpeg (verified with an ffmpeg 1080p grab while Discord was live) and that's the road for 1080p streaming; (3) 1080p inside a Discord call via screen share is possible up to 1080p60 with Nitro.

---

B6. The installer crash-course: a cryptic module error and a lying log

Symptom. Pressing Install driver showed "the specified module could not be found" - the classic rundll32 message, with no other clue anywhere. Diagnosis. Two faults stacked: when the app was not elevated, writing pscapture.dll into %ProgramFiles% failed silently - the old code logged the extraction as successful anyway - and then rundll32 was asked to load a file that did not exist (hence the generic, useless message). On top of that, rundll32 never reports what went wrong back to the caller. Fix. Every filesystem step is now verified (write -> check size -> report), the install aborts honestly when the dll cannot be placed, and registration happens in-process (LoadLibrary + DllRegisterServer) so failures come back as real HRESULTs. The driver check also verifies that the registered dll still exists on disk before declaring victory.

B7. The modern control center UI

The first GUI iteration was a plain Win32 dialog. It was rebuilt as a dark custom-chrome window: painted title bar with integrated minimize/close, rounded window region, owner-drawn buttons, a live status pill with a state dot, and a dark monospace console pane. Everything is still pure C with zero frameworks - the UI is drawn by hand.

Part C — The road to 60 fps

The math. Raw NV12 1080p is 3.11 MB/frame → 30 fps ≈ 93 MB/s (a gigabit LAN is already at the wall at ~110 MB/s); 60 fps ≈ 186 MB/s → needs 2.5 GbE or compression.

Raw path (2.5 GbE adapter on the console). The user's adapter is a TP-Link UE302C (Realtek RTL8156B). Findings: the console kernel is FreeBSD 9.0 based; the matching driver (ure) appears in FreeBSD 13+, and Sony ships only the drivers it uses — so nothing will recognize the adapter out of the box. Two enablers were secured anyway: the PLD payload format and the kernel payload loader on port 9019 (how the scene's kernel payloads ship), plus the kernel offsets table for 13.60 — enough to deliver a custom kernel module when the port is ready. (Full port = a separate workstream; the homework is in REVERSE_ENGINEERING.md §4.)

Compressed path (stays on gigabit). The console has a hardware JPEG encoder (libSceJpegEnc). At ~250-350 KB/frame, 60 fps ≈ 15-21 MB/s — five times inside a gigabit. The API surface was recovered from the library itself (exports, NIDs, argument structs, validation rules — see the notes in REVERSE_ENGINEERING.md), and a measuring payload is ready to run on the console (each call in a forked child with timeout — Sony APIs abort on wrong arguments, so a bad call must die alone).

---

Lessons (the rules we now follow)

1. Never hand a kernel address to kernel_copyout — page-walk + present check + direct-map reads, always.

2. One live payload instance. Kill stale ones at boot; never accumulate.

3. Measure with bytes, not eyes: same formula, same reference frame, signed deltas. Eye-comparison across renderers lies.

4. Even-pixel shifts only (U/V pairing), for the same reason.

5. Never call sceAvControlChangeScanInMode from a payload (it can wedge the display engine).

6. When an API can abort the host on bad arguments, isolate the call in a child process with a timeout.

7. When a suspicious "fix" targets a symptom (the chroma gain), keep looking for the identity of the problem (the U/V swap).

8. Never log success without checking the I/O result. A lying log (the installer claiming "extracted ok" on a failed write) costs far more time than a missing one - it sends you hunting in the wrong place.