0 Fork 0

mirror of https://github.com/Swordpdf/ps5-framegen.git synced 2026-10-17 18:03:09 +02:00

Files

T

swordpdfandClaude Opus 5.5 99edde64ca Review fixes: barriers, resets, the latency hold's limits (AI-assisted)

interpolate_next() makes the first frame's still history and quad motions
visible to its compute reads; the quads' motions are cleared on a reset and a
scene change; the refinement scores an off-screen sample as the candidates
do; the GPU timer's optical flow sum is reset with the rest. Docs and the
example: no frame limit counts as tight, the user's own second limit is
taken in, the multiplier is a most (as many frames as the game's interval
has refreshes, less one).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaNprJCmpBhrkpzRHVPLXM

2026-10-07 08:36:33 +03:00

4.5 KiB

Raw Blame History

Testing

There is no debugger on the console, so the frame generation is developed on a PC. tools/fg-test runs the very same interpolator and shaders on any Vulkan device. On SwiftShader (a CPU Vulkan driver) that means no GPU at all: a 1080p frame takes a few seconds.

CXX=clang++ VULKAN_HEADERS=/path/to/Vulkan-Headers/include tools/build-fg-test.sh build/fg-test
export VK_ICD_FILENAMES=/path/to/vk_swiftshader_icd.json

Synthetic scenes (a known middle frame)

build/fg-test --scene NAME out.png [--bg BG.png] [--pan N] [--obj N] [--head-move N] [--debug 1]

A 960x540 background (noise, or a 1280x720 or larger picture such as a frame of a game) pans N pixels a frame (12). On top of it, in place on screen like a game's HUD or the character the camera follows:

Scene	What it adds
`pan`	the background alone
`head`	a head and a body in the middle; `--head-move N` moves them N px a frame
`hud`	HUD text, a minimap and a button prompt (cut from BG.png when given)
`all`	the head and the HUD, and a checkerboard square moving `--obj` px a frame (20)

The true middle frame is rendered too. The result is scored against it by region (head edge, HUD, square edge; mean absolute error, 0-255), next to the plain blend and the two frames. out.png.cmp.png shows A, B, the truth, the blend, the result and its error around each region.

--gen N makes N frames between the last two (multi-frame generation): out.png, out.png.1.png, ... For a scene, each one is measured against its own true frame. On the "all" scene:

Multiplier	Moment	Error	Head edge	HUD	Square edge
x2	1/2	0.76	4.79	3.45	10.25
x3	1/3	0.86	4.49	3.05	13.61
x3	2/3	0.87	4.66	2.91	15.72
x4	1/4	0.61	4.13	2.48	8.22
x4	1/2	0.74	4.88	3.16	10.40
x4	3/4	0.64	4.25	2.43	11.14

Real frames

build/fg-test --seq out.png f1.png f2.png ... fN.png     # the frame between the last two
python3 tools/truth.py build/fg-test,other/fg-test frames/*.png

truth.py takes consecutive frames of a game. For each of the last frames i, it feeds every other frame up to i and compares the frame generated between i-2 and i with the real i-1. That is twice the motion of real use, so it is a hard test. The real middle frame is rarely where an even motion would put it (cameras speed up and slow down), so each 32x32 block is compared after the best shift within 6 pixels: the score is artifacts, not timing. Several fg-test builds (variants made with OUT=... tools/build-shaders.sh -DFG_... and tools/build-fg-test.sh OUT INC_DIR) are compared in one run.

Results of m74 (RPCS3-PS5, 2026-10-07)

Real frames: GTA IV, 1920x1080, Niko walking while the camera turns, 4 tests.

Variant	Error	Moving	Bad (>40)	Specks	PSNR
plain blend	3.913	11.92	92,189	516	34.44
m73d: FSR3's field, its cells, the block's flow	1.200	2.95	13,585	1,749	40.66
m73g: + the 4 neighbour blocks, own match, consistency penalty	1.259	2.86	13,279	1,730	40.68
+ sub-pixel refinement, off-screen samples	1.230	2.85	12,806	1,310	40.72
+ a local search where nothing agrees (dropped)	1.230	2.99	13,738	1,769	40.17
+ skipping unchanged quads (< 0.02) (dropped)	1.346	2.85	12,952	1,448	40.51
m74: refinement where the error is > 0.03, off-screen, last frame's motion	1.237	2.84	12,887	1,449	40.72

Synthetic scenes (a GTA IV background; mean absolute error of the region):

Variant	head-move 0	head-move 2	head-move 4	hud: HUD	all: head	all: HUD	all: square
m73g	4.67	9.73	8.92	5.58	4.57	3.04	11.39
m74	5.35	8.14	9.01	6.75	4.79	3.45	10.25
m74 with unchanged quads skipped (dropped)	5.34	8.12	8.98	6.75	4.76	3.44	32.47

The HUD regions' extra error in m74 is in the moving background between the letters, not in the text: 0.20 on the text pixels against 0.18 for m73g.

On the console

RPCS3-PS5 logs a status line every 10 s:

the game's frame rate and the rate shown;
the GPU time per generated frame, split into optical flow and interpolation;
how often a present had to wait for an image, and the generated frames left out to shorten the queue.

It can also dump bursts of frames (real and generated, flagged) for fg-test --seq and truth.py.

4.5 KiB Raw Blame History