steam-frame/Swordpdf--ps5-framegen
1
0 Fork 0
mirror of https://github.com/Swordpdf/ps5-framegen.git synced 2026-10-17 12:01:27 +02:00
Code Issues Packages Projects Releases Wiki Activity

CI on lavapipe, the scene regression check, docs for both PS5 drivers (AI-assisted)

Browse Source 
- .github/workflows/ci.yml: on every push and pull request, fg-test's
  scenes on Mesa's lavapipe against tools/check-scenes.py's limits (about
  1.5x today's scores, plus at most a fifth of a plain blend's error, plus
  BGRA = RGBA), the pacer's test with GCC and Clang -Werror, the example
  compiled, and the shaders rebuilt with glslang 16.2.0 (pinned, cached)
  and compared with src/framegen/fg_shaders.inc.
- README: what the template gives, both emulators it runs in, the quick
  start on lavapipe, the integration with the pacer, the PS5 notes.
- docs/integration.md: the pacer, the frames' format, your own loader,
  RADV and ps5vk, the 120 Hz param.json (attribute3 524352) and
  PS5VK_PARAM_JSON, ps5vk's binding numbers.
- docs/testing.md: what CI runs and how to run it locally, the baseline
  scores on SwiftShader and lavapipe.
- LICENSE.md: copyright swordpdf; .gitignore: *.pyc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaNprJCmpBhrkpzRHVPLXM
This commit is contained in:
swordpdfandClaude Opus 5.5 committed 2026-10-08 22:31:15 +03:00
1 parent 11937f654f
commit cb951d5ddd
8 files changed
+379 -74
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/ci.yml
+99
View File
@@ -0,0 +1,99 @@
# CI: the interpolator and its shaders checked on Mesa's lavapipe (a CPU Vulkan driver), the pacing's unit test, and the shaders
# rebuilt from shaders/ to match src/framegen/fg_shaders.inc. docs/testing.md has what each check means.
name: CI
on:
push:
pull_request:
workflow_dispatch:
permissions:
contents: read
env:
# glslang 16.2.0 (the commit fg_shaders.inc was built with; 15.x can't parse FSR3's HLSL). Change both together.
GLSLANG_SHA: f0bd0257c308b9a26562c1a30c4748a0219cc951
GLSLANG_TAG: 16.2.0
jobs:
test:
name: Scenes on lavapipe, pacing
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Vulkan loader, headers and lavapipe
run: |
sudo apt-get update -q
sudo apt-get install -y -q --no-install-recommends mesa-vulkan-drivers libvulkan-dev clang
ls /usr/share/vulkan/icd.d/
- name: Pacing unit test (GCC and Clang)
run: |
mkdir -p build
g++ -std=c++17 -O2 -Wall -Wextra -Werror -Isrc -o build/test_pacing tests/test_pacing.cpp
build/test_pacing
clang++ -std=c++17 -O2 -Wall -Wextra -Werror -Isrc -o build/test_pacing_clang tests/test_pacing.cpp
build/test_pacing_clang
- name: The example compiles
run: |
for cxx in g++ clang++; do
$cxx -std=c++17 -Wall -Wextra -Wno-missing-field-initializers -fsyntax-only -Isrc examples/present_loop.cpp
done
- name: Build fg-test
run: CXX=g++ VULKAN_HEADERS=/usr/include tools/build-fg-test.sh build/fg-test
- name: Synthetic scenes against their limits
env:
VK_ICD_FILENAMES: /usr/share/vulkan/icd.d/lvp_icd.json
run: python3 tools/check-scenes.py build/fg-test build/scenes
- name: Keep the pictures
if: always()
uses: actions/upload-artifact@v4
with:
name: scenes
path: build/scenes/
if-no-files-found: ignore
shaders:
name: Shaders rebuilt match fg_shaders.inc
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Cached glslang
id: cache
uses: actions/cache@v4
with:
path: ~/glslang-bin
key: glslang-${{ env.GLSLANG_SHA }}-hlsl-noopt
- name: Build glslang
if: steps.cache.outputs.cache-hit != 'true'
run: |
git init -q ~/glslang-src
cd ~/glslang-src
git fetch -q --depth 1 https://github.com/KhronosGroup/glslang.git "$GLSLANG_SHA" \
|| git fetch -q --depth 1 https://github.com/KhronosGroup/glslang.git "refs/tags/$GLSLANG_TAG"
git checkout -q FETCH_HEAD
echo "glslang at $(git rev-parse HEAD)"
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_HLSL=ON -DENABLE_OPT=OFF -DALLOW_EXTERNAL_SPIRV_TOOLS=OFF \
-DGLSLANG_TESTS=OFF -DENABLE_GLSLANG_BINARIES=ON
cmake --build build --target glslang-standalone -j"$(nproc)"
mkdir -p ~/glslang-bin
cp build/StandAlone/glslang ~/glslang-bin/glslangValidator
- name: Rebuild the shaders and compare
run: |
~/glslang-bin/glslangValidator --version | head -1
mkdir -p build
GLSLANG=~/glslang-bin/glslangValidator OUT=build/fg_shaders.inc tools/build-shaders.sh
if ! cmp build/fg_shaders.inc src/framegen/fg_shaders.inc; then
echo "::error::src/framegen/fg_shaders.inc is not what shaders/ compiles to: run tools/build-shaders.sh (glslang $GLSLANG_TAG) and commit it"
diff <(grep -n 'name\|binding' build/fg_shaders.inc) <(grep -n 'name\|binding' src/framegen/fg_shaders.inc) | head -40 || true
exit 1
fi
echo "fg_shaders.inc matches shaders/"
.gitignore
+1
View File
@@ -1,3 +1,4 @@
build/
__pycache__/
*.pyc
*.spv
LICENSE.md
+2 -2
View File
@@ -3,8 +3,8 @@
| Path | Licence |
|---|---|
| `shaders/` | MIT: AMD FidelityFX SDK files and the passes and changes made to them here ([shaders/LICENSE.txt](shaders/LICENSE.txt)) |
| `src/`, `tools/` (but `tools/stb/`), `examples/`, `docs/`, `README.md` | GPL-2.0-only ([LICENSES/GPL-2.0-only.txt](LICENSES/GPL-2.0-only.txt)), as in RPCS3-PS5, which they come from |
| `tools/stb/` | stb_image and stb_image_write by Sean Barrett: public domain or MIT (the end of each file) |
| `src/framegen/fg_shaders.inc` | generated from `shaders/`: MIT |
| `src/` (but `fg_shaders.inc`), `tests/`, `tools/` (but `tools/stb/`), `examples/`, `docs/`, `.github/`, `README.md` | GPL-2.0-only ([LICENSES/GPL-2.0-only.txt](LICENSES/GPL-2.0-only.txt)), copyright 2026 swordpdf |
| `tools/stb/` | stb_image and stb_image_write by Sean Barrett: public domain or MIT (the end of each file) |
The GPL-2.0-only part is the work of its author, who can relicense it (for example to MIT) for use in other projects.
README.md
+76 -44
View File
@@ -1,24 +1,39 @@
# ps5-framegen
Frame generation for PS5 homebrew on Vulkan: from two frames of a game, the frame halfway between them, shown in between. A game held
at 30 fps is shown at 60 on a 120 Hz display, a game at 60 at 120. It needs nothing from the game but its frames: no motion vectors and
no depth. That makes it fit emulators and ports, where the game never hands those over.
[![CI](https://github.com/Swordpdf/ps5-framegen/actions/workflows/ci.yml/badge.svg)](https://github.com/Swordpdf/ps5-framegen/actions/workflows/ci.yml)
It can also make 2 or 3 frames in between (multi-frame generation: 40 fps or 30 fps shown at 120), at the cost of more input lag.
Frame generation for PS5 homebrew on Vulkan. From two frames of a game it makes the frame halfway between them and shows it in between:
a game at 60 fps is shown at 120 on a 120 Hz display, and a game at 30 fps at 60. It needs nothing from the game but its frames: no
motion vectors and no depth. That makes it fit emulators and ports, where the game never hands those over.
It comes from [RPCS3-PS5](https://github.com/Swordpdf)'s frame generation (the PS3 emulator's PS5 port), where it runs on the
console in GTA IV, inFamous and others. This repository is that code taken out on its own, as a template to start from.
It runs on the console today in two emulators:
- **Optical flow:** AMD FidelityFX FSR3's (SDK 2.3.0, MIT), compiled from HLSL to SPIR-V with glslang (no DXC, no DirectX).
- **Interpolation:** written for this, in place of FSR3's (see below), running FSR3's optical-flow-only path.
- **Host side:** plain Vulkan in one class, `ps5::framegen::interpolator` (`src/ps5_framegen.{h,cpp}`). It has no dependency on
the emulator, the console's SDK or AMD's SDK backend.
- **[RPCS3-PS5](https://github.com/Swordpdf)** (PS3) on the RADV driver: GTA IV, inFamous, Ratchet & Clank: Quest for Booty;
- **[PS5SX2](https://github.com/Swordpdf/PS5SX2)** (PS2) on the ps5vk driver, since 2.00.
This repository is that code on its own, as a template to start from. Press **Use this template** and copy `src/` into your
renderer.
## What you get
- **The interpolator:** `ps5::framegen::interpolator` (`src/ps5_framegen.{h,cpp}`), plain Vulkan in one class. It has no dependency on
an emulator, the console's SDK or AMD's SDK backend, and works on RADV, ps5vk, lavapipe and SwiftShader.
- **The pacing:** `ps5::framegen::pacer` (`src/ps5_framegen_pacing.h`, header-only). It decides when generated frames help and for
how many refreshes each is shown: only at an even frame rate with room on the display, never self-feeding, and with a safety net
that backs off when they cost an emulator its speed.
- **The shaders:** AMD FidelityFX FSR3's optical flow (SDK 2.3.0, MIT), compiled from HLSL to SPIR-V with glslang (no DXC or
DirectX), and an interpolation pass written for this in place of FSR3's (below).
- **Tests you can run without a console:** `tools/fg-test` runs the same class and shaders on a CPU Vulkan driver against synthetic
scenes with a known middle frame. `tests/test_pacing.cpp` checks the pacing. CI runs both on every push (`.github/workflows/ci.yml`).
- **A worked integration:** `examples/present_loop.cpp`, with the pacing, the latency hold and the presents.
- **The PS5 specifics, written down:** the 120 Hz mode's param.json, ps5vk's binding numbers, swapchain depth
([docs/integration.md](docs/integration.md#the-ps5)).
## Why not just FSR3's interpolation
FSR3's frame interpolation expects the game's motion vectors and depth. Without them it falls back to its optical flow, whose 8x8
blocks drag whatever stays still on screen along with whatever moves behind it. That includes the HUD, text and the character the
camera follows. The interpolation here (`computeFrameinterpolationQuad` in
blocks drag whatever stays still on screen along with whatever moves behind it: the HUD, text, and the character the camera
follows. The interpolation here (`computeFrameinterpolationQuad` in
`shaders/framegeneration/fsr3/include/gpu/frameinterpolation/ffx_frameinterpolation.h`) tries several motions for every 2x2 quad
of pixels:
@@ -29,54 +44,73 @@ of pixels:
- the best of these refined to a fraction of a pixel.
Each pixel keeps the motion under which the previous and the current frame agree. A motion that is not the optical flow's own where
it takes its samples is penalised, because two dark samples agree by chance. Where nothing agrees, something in front covers or
uncovers what is behind it, and the sample that another motion explains is dropped. A pixel that has not changed for 8 frames (HUD,
text) stays still. [docs/algorithm.md](docs/algorithm.md) has the details and the reasons, case by case.
it takes its samples is penalised, because two dark samples can agree by chance. Where nothing agrees, something in front covers
or uncovers what is behind it, and the sample that another motion explains is dropped. A pixel that has not changed for 8 frames
(HUD, text) stays still. [docs/algorithm.md](docs/algorithm.md) has the details and the reasons, case by case.
## Layout
| Path | What |
|---|---|
| `src/ps5_framegen.h`, `src/ps5_framegen.cpp` | the interpolator: images, pipelines, the passes in order, a GPU timer |
| `src/ps5_framegen_pacing.h` | the pacer: when to generate, how many presents, the safety net |
| `src/framegen/fg_shaders.inc` | the compiled shaders (SPIR-V arrays with their bindings), made by `tools/build-shaders.sh` |
| `shaders/` | the HLSL: FSR3's optical flow (unchanged but for portability), the cut-down interpolation passes ([shaders/README.md](shaders/README.md)) |
| `tools/fg-test.cpp` | a host test on any Vulkan device, SwiftShader included: synthetic scenes with a known middle frame, real frames |
| `shaders/` | the HLSL: FSR3's optical flow (unchanged but for portability) and the cut-down interpolation passes ([shaders/README.md](shaders/README.md)) |
| `examples/present_loop.cpp` | how a renderer drives it |
| `tests/test_pacing.cpp` | the pacer's unit test (911 checks) |
| `tools/fg-test.cpp` | the host test on any Vulkan device: synthetic scenes with a known middle frame, real frames |
| `tools/check-scenes.py` | the scene regression check CI runs, with its limits |
| `tools/truth.py` | real-frame tests: the frame generated between frame i and i+2 against the real frame i+1 |
| `examples/present_loop.cpp` | how a renderer drives it: pacing, the latency hold, the presents |
| `docs/` | the algorithm, the integration (pacing, latency, colour, PS5 specifics), testing and results |
## Quick start
```sh
# shaders -> src/framegen/fg_shaders.inc (glslangValidator with HLSL: the Vulkan SDK's)
GLSLANG=glslangValidator tools/build-shaders.sh
On a PC with no GPU (Ubuntu 24.04 shown; any Vulkan device works):
# the host test, on a software Vulkan device (SwiftShader's ICD comes with Chromium builds)
CXX=clang++ VULKAN_HEADERS=/path/to/Vulkan-Headers/include tools/build-fg-test.sh build/fg-test
export VK_ICD_FILENAMES=/path/to/vk_swiftshader_icd.json
build/fg-test --scene all out.png # a head, a HUD and a square over a panning background, scored by region
build/fg-test --seq out.png f1.png f2.png f3.png # real frames in order: writes the frame between the last two
python3 tools/truth.py build/fg-test frames/*.png # real-frame scores
```sh
sudo apt install mesa-vulkan-drivers libvulkan-dev # lavapipe, a CPU Vulkan driver
CXX=g++ VULKAN_HEADERS=/usr/include tools/build-fg-test.sh build/fg-test
export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json
build/fg-test --scene all out.png # a head, a HUD and a square over a panning background, scored by region
python3 tools/check-scenes.py build/fg-test # every scene against its limits, as CI does
c++ -std=c++17 -Isrc -o build/test_pacing tests/test_pacing.cpp && build/test_pacing
# after changing shaders/ (glslang 16.2.0 with HLSL; Ubuntu's 15.x can't parse FSR3's HLSL)
GLSLANG=glslangValidator tools/build-shaders.sh # -> src/framegen/fg_shaders.inc
```
In a renderer (see [docs/integration.md](docs/integration.md) and `examples/present_loop.cpp`):
```cpp
ps5::framegen::interpolator fg;
fg.create(physical_device, device, width, height, error); // the size of the picture shown
ps5::framegen::pacer pacing;
fg.create(physical_device, device, width, height, error, swapchain_format); // the size of the picture shown
// every game frame, in the frame's command buffer:
// every new frame of the game (ticks: the source clock's vblanks since the last one; already: presents made since then):
const auto d = pacing.frame(ticks, already, source_hz, display_hz, now_seconds, speed_percent, limiter_nominal);
if (!d.engaged) { present_the_game_frame(); return; }
fg.prepare(cmd);
copy_or_draw_this_frame_into(fg.frame_image()); // exactly as it will be shown (colour, scaling)
if (fg.record(cmd, reset)) // optical flow always, interpolation unless reset
present(fg.output_image(), repeats); // the in-between frame first, then the game's frame as usual
draw_this_frame_into(fg.frame_image()); // exactly as it will be shown (colour, scaling)
if (fg.record(cmd, d.reset)) // optical flow always, the in-between frame unless reset
present(fg.output_image(), d.presents); // the in-between frame first...
present_the_game_frame(); // ...then the game's frame as usual
fg.advance();
```
## On the PS5
- Vulkan 1.1 with `shaderStorageImageExtendedFormats` and subgroup quad operations in compute. RADV and ps5vk both have them.
- **120 Hz:** the drivers offer the 119.88 Hz mode only to an app whose `sce_sys/param.json` has `"attribute3": 524352`. Without
it you get 60 Hz, where only 30 fps games gain anything.
- **ps5vk:** `create()` renumbers the shaders' bindings 0..n-1, which ps5vk's pipeline compiler needs. RADV doesn't mind.
[docs/integration.md](docs/integration.md#the-ps5) has the details.
## Cost and latency
- **GPU time:** on the PS5 at 3840x2160, about 2.1-2.5 ms per generated frame (m73). The renderer's status line splits it into
the optical flow and the interpolation (m74).
- **GPU time:** on the PS5 at 3840x2160, about 2.1-2.5 ms per generated frame (RADV). The status line splits it into the optical flow
and the interpolation.
- **Input lag:** interpolation shows each game frame half a frame later than it would be shown otherwise (8 ms at 60 fps, 17 ms at
30), plus the GPU time. Nothing can remove that half frame.
- **What the integration avoids on top of that:** a FIFO swapchain that fills up (three refreshes more, 25 ms at 120 Hz, when a
@@ -94,18 +128,16 @@ On real GTA IV frames (1920x1080, the frame between frames i and i+2 against the
| m73g (+ neighbours, + consistency penalty) | 1.259 | 13,279 | 1,730 | 40.68 |
| **m74 (this)** | **1.237** | **12,887** | **1,449** | **40.72** |
[docs/testing.md](docs/testing.md) has the synthetic scenes, the variants that were tried and dropped, and how to run all of it.
On the synthetic `all` scene CI runs, the generated frame's error is 0.22 against 4.93 for a plain blend, and 3.2-3.5 against 66 at
the moving square's edge. [docs/testing.md](docs/testing.md) has every scene, the variants that were tried and dropped, and how to
run all of it.
## Licences
- **`shaders/`:** AMD FidelityFX SDK files under the MIT licence ([shaders/LICENSE.txt](shaders/LICENSE.txt)). The changes and
the new passes there are under the same licence.
- **`src/`, `tools/`, `examples/`, `docs/`:** written for RPCS3-PS5 and GPL-2.0-only like it
([LICENSES/GPL-2.0-only.txt](LICENSES/GPL-2.0-only.txt)). The author can relicense this part (for example to MIT) if it should
go into non-GPL projects.
- **`shaders/`:** AMD FidelityFX SDK files under the MIT licence ([shaders/LICENSE.txt](shaders/LICENSE.txt)). The changes and the
new passes there are under the same licence.
- **`src/`, `tests/`, `tools/`, `examples/`, `docs/`:** GPL-2.0-only ([LICENSES/GPL-2.0-only.txt](LICENSES/GPL-2.0-only.txt)),
copyright swordpdf. Ask if you need another licence for a non-GPL project.
- **`tools/stb/`:** stb_image and stb_image_write (public domain or MIT, as stated in the files).
Nothing here comes from Sony's SDK. The code uses only Khronos Vulkan: a Vulkan 1.1 device with `shaderStorageImageExtendedFormats`
and subgroup quad operations in compute shaders ([docs/integration.md](docs/integration.md#the-device)). On the PS5 it was run on
RADV for the PS5 (presenting through VideoOut), and on PCs on
SwiftShader.
Nothing here comes from Sony's SDK. The code uses only Khronos Vulkan.
docs/integration.md
+66 -24
View File
@@ -1,7 +1,14 @@
# Putting it in a renderer
`examples/present_loop.cpp` is the same as code. RPCS3-PS5's own integration is in its `rpcs3/Emu/RSX/VK/VKPresent.cpp` (the
`RPCS3_PLATFORM_PS5` block in `VKGSRender::flip`).
`examples/present_loop.cpp` is the same as code. Two real integrations:
- **RPCS3-PS5:** `rpcs3/Emu/RSX/VK/VKPresent.cpp` (the `RPCS3_PLATFORM_PS5` block in `VKGSRender::flip`), on RADV.
- **PS5SX2:** `pcsx2/GS/Renderers/Vulkan/GSDeviceVK.cpp` and `GSRenderer.cpp`, on ps5vk, with the pacer in
`src/ps5_framegen_pacing.h`.
Copy `src/` into your project: `ps5_framegen.{h,cpp}`, `framegen/fg_shaders.inc` and `ps5_framegen_pacing.h`. Nothing else is
needed at run time. To call Vulkan through your own loader (volk, a function table), build with
`-DPS5_FRAMEGEN_VULKAN_HEADER='"your_header.h"'`. That header must declare the `vk*` commands under their usual names.
## The device
@@ -10,12 +17,14 @@ Vulkan 1.1, with two things the shaders use that `create()` does not check:
- the `shaderStorageImageExtendedFormats` feature, enabled on the device (the optical flow's R8 and R16G16 storage images);
- subgroup quad operations in compute shaders (`VK_SUBGROUP_FEATURE_QUAD_BIT`; FSR3's luminance pyramid).
RADV (the PS5's driver) and SwiftShader have both. `tools/fg-test.cpp` shows a device set up for it.
RADV, ps5vk, SwiftShader and lavapipe have both. `tools/fg-test.cpp` shows a device set up for it.
## The frame's picture
The interpolator works on `R8G8B8A8_UNORM` images of the size given to `create()`, normally the size of the picture shown (the
swapchain's).
The interpolator works on images of the size given to `create()`, normally the size of the picture shown (the swapchain's). The
game's frames are `R8G8B8A8_UNORM` by default. Pass `VK_FORMAT_B8G8R8A8_UNORM` as `create()`'s last argument when your own
pipelines draw into `frame_image()` in a BGRA swapchain's format (PS5SX2 does). The passes only sample the frames, so the result
is the same (CI checks it). `output_image()` is always `R8G8B8A8_UNORM`: blit it, which converts.
1. `prepare(cmd)` once a frame. It puts its images in `VK_IMAGE_LAYOUT_GENERAL` the first time.
2. Copy, blit or draw this frame's picture into `frame_image()`, in `GENERAL` layout.
@@ -42,23 +51,40 @@ generated frames skipped that, every other frame was brighter: GTA IV sets a gam
V-Sync must be on, with FIFO presenting: each present is shown for at least one refresh, in order.
The generated frame is presented `repeats` times, then the game's frame once (FIFO holds it until the next present):
`ps5::framegen::pacer` (`src/ps5_framegen_pacing.h`, header-only) decides each frame. You give it the game's new frame counted in
**ticks** of the clock that paces the game, not wall-clock time:
```text
half = (game frame interval in refreshes) / 2
repeats = 1 if half < 1.9, 2 if half < 2.9, 3 above
(back down only below 1.8 and 2.8, so that it does not flicker between two)
- in an emulator, the emulated machine's vblanks since the last new frame (a PS2: 1 at 60 fps, 2 at 30, at 59.94 or 50 Hz);
- in a port, your game loop's own ticks.
```cpp
const ps5::framegen::pacer_decision d = pacing.frame(ticks, already, source_hz, display_hz, now_s, speed_pct, limiter_nominal);
// d.engaged: generate for this frame (record() it, with d.reset)
// d.presents: show the generated frame that many times (0: none this time), then the game's frame once
```
| Game rate | Display | repeats | What is shown |
|---|---|---|---|
| 30 fps | 120 Hz | 2 | generated 2 refreshes, the game's 2 |
| 45 fps | 120 Hz | 1 | generated 1, the game's 1 or 2 |
| 60 fps | 120 Hz | 1 | generated 1, the game's 1 |
| any | 60 Hz | 1 | |
What it does (the header has the reasons):
Measure the interval as a running mean of the time between the game's flips (0.85/0.15 here), ignoring gaps under 4 ms or over
250 ms.
| Game | Display | Decision |
|---|---|---|
| 60 fps | 120 Hz | generated 1 refresh, the game's 1 |
| 30 fps | 120 Hz | generated 2, the game's 2 |
| 30 fps | 60 Hz | generated 1, the game's 1 |
| 60 fps | 60 Hz | none: no room |
| 45-58 fps, uneven | any | none: uneven frame lengths look worse with generated frames than without |
| any, after a load or at the start | | none for 8 frames |
**Why ticks and not the wall clock:** the first pacing (RPCS3-PS5's, and this example's before) timed the game's frames between
presents. At 60 Hz it fed itself. A generated frame took a refresh, the next game frame came a refresh later, and that looked like a
30 fps game that should get a generated frame. A PS2 game ran at half speed. Ticks don't depend on the presents.
`already` is the presents made since the last new frame: the game's last frame (1), plus any repeat your renderer presented. While
the pacer is generating, a tick that only repeats the game's last frame should present nothing, because the generated frame used its
refresh.
**The safety net** (for emulators): when the game falls below 95% speed for 3 s while frames are generated, the pacer pauses them
(10 s, doubling up to 160 s if they were the cause). A native port passes `speed_pct = 100`. Log `take_event()` and `state()`: they
say why nothing is generated.
## More than one frame in between (the multiplier)
@@ -77,7 +103,8 @@ Present each generated frame for one refresh, then the game's frame:
| x4 | 30 fps | 120 Hz | 3 refreshes (25 ms), against 2 at x2 |
So more frames mean smoother motion for more input lag. RPCS3-PS5 keeps x2 as the default (ps5.yml "Frame Generation Multiplier":
2, 3 or 4). The multiplier is a most: RPCS3-PS5 generates as many frames as the game's frame interval has refreshes, less one. A
2, 3 or 4). The multiplier is a most: generate as many frames as the game's frame interval has refreshes, less one
(`examples/present_loop.cpp` takes it from the ticks). A
game slower than the multiplier wants would otherwise get its generated frames bunched together and its own frame held for the rest
(x3 at 30 fps: 1/3, 2/3, 1, 1).
@@ -123,9 +150,24 @@ They are read back `k_slots` frames later, never waiting. On the PS5 (RADV throu
## The PS5
- **Driver:** Vulkan on the console is RADV for the PS5, presenting through VideoOut. The display is a
`VK_KHR_display` surface, 3840x2160 at 119.88 Hz (or 59.94), FIFO, 4 swapchain images.
- **GPU time:** about 2.1-2.5 ms per generated frame at 3840x2160 (m73). The PS3 games RPCS3 runs leave most of the GPU free.
- **Testing:** there is no debugger on the console. Develop against `tools/fg-test` on SwiftShader (it runs the very same
`interpolator` and shaders), and dump frames on the console to compare (RPCS3-PS5 dumps the real and the generated frames,
- **Drivers:** it runs on both Vulkan drivers PS5 homebrew uses:
- **RADV** (Mesa's AMD driver, ported to the PS5): RPCS3-PS5;
- **ps5vk** (a native PS5 Vulkan driver): PS5SX2.
Both present through VideoOut: a `VK_KHR_display` surface, 3840x2160 at 119.88 Hz (or 59.94), FIFO. RADV gives 4 swapchain
images (see the multiplier above), ps5vk 2 to 3.
- **ps5vk and binding numbers:** FSR3's shaders number their bindings sparsely (0, 100, 200, 300, 405...). ps5vk sizes a set's table
to its highest binding number and keeps a binding number in a byte. The first PS5SX2 build failed in `vkCreateComputePipelines`
("internal error"). `create()` renumbers every pass's bindings 0..n-1 in the SPIR-V, the set layout and the writes, so this is
handled. A pass added through `tools/build-shaders.sh` gets the same treatment, because its bindings are read back into
`fg_shaders.inc`.
- **120 Hz needs the app's param.json:** both drivers offer the 119.88 Hz modes only when the title's `sce_sys/param.json`
declares high frame rates: `"attribute3": 524352` (`0x80040`), as PS5SX2 ships it. Without it, the display stays at 60 Hz and only
30 fps games get generated frames. ps5vk reads `/app0/sce_sys/param.json`, which after the jailbreak may not lead to the app's
folder.
PS5SX2 sets `PS5VK_PARAM_JSON` to the first that opens of: the running app's folder as the sandbox mounts it, `/app0`, the
install folder. After changing param.json, re-register the app, or the mode change can be refused (`0x80290016`).
- **GPU time:** about 2.1-2.5 ms per generated frame at 3840x2160 on RADV (RPCS3-PS5, m73). PS3 and PS2 games leave most of the
GPU free.
- **Testing:** there is no debugger on the console. Develop against `tools/fg-test` on lavapipe or SwiftShader (it runs the very
same `interpolator` and shaders). Then dump frames on the console to compare (RPCS3-PS5 dumps the real and the generated frames,
flagged, to a zstd file).
docs/testing.md
+36 -2
View File
@@ -8,6 +8,40 @@ CXX=clang++ VULKAN_HEADERS=/path/to/Vulkan-Headers/include tools/build-fg-test.s
export VK_ICD_FILENAMES=/path/to/vk_swiftshader_icd.json
```
## What CI runs
`.github/workflows/ci.yml`, on every push and pull request:
| Check | Command | What fails it |
|---|---|---|
| Synthetic scenes on lavapipe | `python3 tools/check-scenes.py build/fg-test` | a region's error above its limit, a generated frame above a fifth of the plain blend's error, or B8G8R8A8 frames giving another picture than R8G8B8A8 |
| Pacing unit test | `tests/test_pacing.cpp` (911 checks), GCC and Clang with `-Werror` | any check |
| The example compiles | `examples/present_loop.cpp`, `-fsyntax-only` | a compile error |
| Shaders rebuilt | `tools/build-shaders.sh` with glslang 16.2.0, compared with `src/framegen/fg_shaders.inc` | a shader changed in `shaders/` without the `.inc` rebuilt (or the other way round) |
The same on a PC (Ubuntu 24.04: `apt install mesa-vulkan-drivers libvulkan-dev`):
```sh
CXX=g++ VULKAN_HEADERS=/usr/include tools/build-fg-test.sh build/fg-test
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json python3 tools/check-scenes.py build/fg-test
c++ -std=c++17 -Wall -Wextra -Isrc -o build/test_pacing tests/test_pacing.cpp && build/test_pacing
```
glslang 15 (Ubuntu 24.04's `glslang-tools`) can't parse FSR3's HLSL. Use 16.2.0: the Vulkan SDK's, or a source build with
`-DENABLE_HLSL=ON -DENABLE_OPT=OFF` (the workflow shows one).
The limits in `check-scenes.py` are about 1.5x today's scores on the default noise background (960x540, 12 px pan, the square at 20
px a frame). SwiftShader and lavapipe give nearly the same:
| Scene | Whole frame | Head edge | HUD | Square edge | Plain blend |
|---|---|---|---|---|---|
| `all` on SwiftShader | 0.22 | 1.27 | 0.72 | 3.16 | 4.93 |
| `all` on lavapipe | 0.22 | 1.28 | 0.73 | 3.48 | 4.93 |
| `head --head-move 6` on lavapipe | 0.29 | 1.75 | | | 4.37 |
| `hud` on lavapipe | 0.08 | | 1.04 | | 3.52 |
When a change makes a scene better, lower its limit in the same commit. When one makes it worse on purpose, say why in the commit.
## Synthetic scenes (a known middle frame)
```sh
@@ -55,7 +89,7 @@ best shift within 6 pixels: the score is artifacts, not timing. Several fg-test
## Results of m74 (RPCS3-PS5, 2026-10-07)
Real frames: GTA IV, 1920x1080, Niko walking while the camera turns, 4 tests.
These came from frames of a game, which can't be shipped here, so CI can't repeat them. Real frames: GTA IV, 1920x1080, Niko walking while the camera turns, 4 tests.
| Variant | Error | Moving | Bad (>40) | Specks | PSNR |
|---|---|---|---|---|---|
@@ -80,7 +114,7 @@ against 0.18 for m73g.
## On the console
RPCS3-PS5 logs a status line every 10 s:
RPCS3-PS5 logs a status line every 10 s (PS5SX2's `[fg]` lines: one a while, plus one whenever the pacing's state changes):
- the game's frame rate and the rate shown;
- the GPU time per generated frame, split into optical flow and interpolation;
shaders/README.md
+2 -2
View File
@@ -1,11 +1,11 @@
# shaders: AMD FSR3 frame interpolation, optical flow only (from RPCS3-PS5)
# shaders: AMD FSR3 frame interpolation, optical flow only
The shaders of the frame generation (`src/ps5_framegen.{h,cpp}`): AMD FidelityFX SDK 2.3.0's FSR3 optical flow
and frame interpolation, MIT (see LICENSE.txt), compiled from HLSL to SPIR-V by glslang's HLSL front end into
`src/framegen/fg_shaders.inc` with `tools/build-shaders.sh` (no DXC needed). The SDK itself is DX12 only;
the host side here is plain Vulkan code of its own, not the SDK's backend.
A game seen only through its frames (a PS3 game under RPCS3) gives no motion vectors or depth, so only FSR3's optical flow path runs (`FFX_FG_OF_ONLY`). FSR3's game vector path,
A game seen only through its frames (a PS3 game under RPCS3, a PS2 game under PCSX2) gives no motion vectors or depth, so only FSR3's optical flow path runs (`FFX_FG_OF_ONLY`). FSR3's game vector path,
disocclusion mask, inpainting and UI composition are not built.
## What is here
tools/check-scenes.py
Executable
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
# check-scenes.py FG_TEST [OUT_DIR]: the regression check CI runs. It runs tools/fg-test's synthetic scenes (a known true frame) and
# fails when the generated frames get worse than the limits below, which are about 1.5x what SwiftShader and lavapipe give today
# (docs/testing.md). It also checks that frames in B8G8R8A8_UNORM (create()'s frame_format) give the very same picture.
#
# Needs a Vulkan device: VK_ICD_FILENAMES=.../lvp_icd.json (Mesa's lavapipe) or SwiftShader's ICD, or a GPU.
#
# Copyright (C) 2026 swordpdf
# SPDX-License-Identifier: GPL-2.0-only
import os
import re
import subprocess
import sys
# name, fg-test arguments after OUT.png, limits: 'all' is the whole frame, the others fg-test's regions (mean abs error, 0-255).
# Every generated frame is also held to at most a fifth of the plain blend's error.
CASES = [
('pan', ['--scene', 'pan'], {'all': 0.05, 'rest': 0.05}),
('hud', ['--scene', 'hud'], {'all': 0.2, 'HUD': 1.4}),
('head', ['--scene', 'head', '--head-move', '6'], {'all': 0.5, 'head edge': 3.0}),
('all', ['--scene', 'all'], {'all': 0.4, 'head edge': 2.0, 'HUD': 1.1, 'square edge': 5.5}),
('all-x3', ['--scene', 'all', '--gen', '2'], {'all': 0.6, 'head edge': 2.0, 'HUD': 1.1, 'square edge': 13.0}),
('all-x4', ['--scene', 'all', '--gen', '3'], {'all': 0.4, 'head edge': 2.0, 'HUD': 1.1, 'square edge': 8.0}),
]
BLEND_SHARE = 0.2
def run(fg_test, out, args, env=None):
cmd = [fg_test, args[0], args[1], out] + args[2:]
r = subprocess.run(cmd, capture_output=True, text=True, env=env)
if r.returncode != 0:
print(r.stdout[-3000:], r.stderr[-3000:])
raise SystemExit('FAIL: %s exited with %d' % (' '.join(cmd), r.returncode))
return r.stdout + r.stderr
def parse(text):
# The first frame's table, then each further generated frame's block (--gen).
frames = []
m = re.search(r'interpolated ([\d.]+), plain blend of A and B ([\d.]+)', text)
if not m:
raise SystemExit('FAIL: no score in the output:\n' + text[-2000:])
first = {'all': float(m.group(1)), 'blend': float(m.group(2))}
table = text[m.end():].split('\n')
for line in table[2:]:
r = re.match(r'\s{2}(\S.*?)\s{2,}([\d.]+)\s+([\d.]+)\s+[\d.]+\s+[\d.]+\s+\d+$', line)
if not r:
break
first[r.group(1).strip()] = float(r.group(2))
frames.append(first)
for block in re.finditer(r'mean abs error against its true frame: ([\d.]+) \(plain blend ([\d.]+)\)((?:\n {2}\S[^\n]*?\s{2,}[\d.]+)*)', text):
f = {'all': float(block.group(1)), 'blend': float(block.group(2))}
for line in block.group(3).strip('\n').split('\n'):
r = re.match(r'\s{2}(\S.*?)\s{2,}([\d.]+)$', line)
if r:
f[r.group(1).strip()] = float(r.group(2))
frames.append(f)
return frames
def main():
if len(sys.argv) < 2:
raise SystemExit(__doc__ if __doc__ else 'usage: check-scenes.py FG_TEST [OUT_DIR]')
fg_test = sys.argv[1]
out_dir = sys.argv[2] if len(sys.argv) > 2 else 'build/scenes'
os.makedirs(out_dir, exist_ok=True)
failures = 0
for name, args, limits in CASES:
frames = parse(run(fg_test, os.path.join(out_dir, name + '.png'), args))
for i, f in enumerate(frames):
label = name if len(frames) == 1 else '%s frame %d/%d' % (name, i + 1, len(frames))
bad = []
for key, limit in limits.items():
if key in f and f[key] > limit:
bad.append('%s %.2f > %.2f' % (key, f[key], limit))
if f['all'] > f['blend'] * BLEND_SHARE:
bad.append('%.2f is more than %d%% of the plain blend\'s %.2f' % (f['all'], BLEND_SHARE * 100, f['blend']))
scores = ', '.join('%s %.2f' % (k, v) for k, v in f.items() if k != 'blend')
print('%-4s %-18s %s (blend %.2f)%s' % ('FAIL' if bad else 'ok', label, scores, f['blend'],
': ' + '; '.join(bad) if bad else ''))
failures += 1 if bad else 0
# The same scene with B8G8R8A8 frames: the same picture, byte for byte.
rgba = os.path.join(out_dir, 'all.png')
bgra = os.path.join(out_dir, 'all-bgra.png')
run(fg_test, bgra, ['--scene', 'all'], dict(os.environ, FG_TEST_BGRA='1'))
same = open(rgba, 'rb').read() == open(bgra, 'rb').read()
print('%-4s %-18s B8G8R8A8 frames give %s picture' % ('ok' if same else 'FAIL', 'all-bgra', 'the same' if same else 'another'))
failures += 0 if same else 1
if failures:
raise SystemExit('FAIL: %d check(s)' % failures)
print('PASS: every scene within its limits')
if __name__ == '__main__':
main()