Overlay in HDR, DualSense marker bug, launcher mouse takeover, item search outside menus

Overlay in HDR
- ImGui draws sRGB colours, and the overlay wrote them straight into the
  game's HDR buffer, where they came out far too bright and washed out.
  The DX12 backend's pixel shader now converts them: to scRGB for an FP16
  buffer, and to HDR10 (BT.2020, PQ) for a 10-bit buffer while the display
  is in HDR, with white at 200 nits. An SDR buffer is drawn as before.
- The log says which was chosen ("Overlay output: ..."), and now also says
  when the game presents in a format the overlay does not draw to, which
  left its menus hidden with no reason given.
- Not seen on an HDR display. The changed shader compiles with the D3D
  compiler.
DualSense
- While a menu holds the game's input, the reports the game reads from a
  PlayStation pad are rewritten as untouched. USB and Bluetooth lay a
  report out differently, and the two were told apart by the size of the
  buffer the game read into. A USB DualSense read into a larger buffer had
  its d-pad byte written as "up", which placed a session marker whenever T
  or Insert opened something. The device's own report length decides now.
- The log line for the first rewritten report gives the pad, USB or
  Bluetooth, the report and the two lengths.
- Not tested with a DualSense.
Launcher
- The controller pointer learned where the right stick rests while no pad
  was being read, which includes the launcher being in the background, and
  took the middle for it. A stick that rests off the middle, or a wheel or
  throttle on that axis, then steered the pointer for good once the
  launcher had the focus again, and the mouse stopped responding until a
  restart. Nothing is learned while no pad is read, and a pad that goes
  away gives the pointer back. Two tests for it.
Item search
- The cosmetic search bar showed in the world: a grid keeps its focused
  item after its screen has gone. It needs the game's menu to be up, and
  to have been for a second, as well.

Extension/Customization/item_browser.cpp

@@ -654,6 +654,13 @@ void update(std::uintptr_t base) noexcept {
         if (sh.status.exchange(false)) { log_status(s); log_roots(s, context); }
         // The search belongs to one visit: it is cleared when the grids are gone, or have been out of
         // focus for longer than one of the game's own cards (a dialog over the grid) usually stays up.
+        // A grid keeps its focus after its screen has gone, so the game's menu has to be up as
+        // well, and to have been for a moment: the game's menu flag is also up for the second
+        // or so it rebuilds the skater after a bail, when the search must not flash up.
+        static ULONGLONG menu_since{};
+        if (!in_menu) menu_since = 0;
+        else if (!menu_since) menu_since = now;
+        if (!menu_since || now - menu_since < 1000) active = nullptr, focused = nullptr;
         if (active) s.last_open = now;
         s.open = active != nullptr;
         if (!s.open && (s.grids.empty() || now > s.last_open + 8000) && (!s.search.empty() || s.filter != Filter::all)) {

Extension/UI/Overlay/input_capture.cpp

@@ -41,9 +41,13 @@ HeldInput& held_input() { static auto* value = new HeldInput; return *value; }
 
 struct InputCaptureHooks {
     Hook hid_attributes, read_file, overlapped, overlapped_ex, completion, completion_ex, close_handle;
-    struct HidRead { HANDLE file; void* buffer; DWORD capacity; PlayStationPad kind; };
+    // input_length: the device's own input report length (64 over USB, longer over Bluetooth),
+    // which says how a report is laid out; 0 when Windows would not say, and then the size of
+    // the buffer the game reads into stands in for it.
+    struct HidRead { HANDLE file; void* buffer; DWORD capacity; PlayStationPad kind; std::size_t input_length{}; };
     std::mutex hid_mutex;
     std::map<HANDLE, PlayStationPad> hid_devices;
+    std::map<HANDLE, std::size_t> hid_lengths; // of the PlayStation pads among them, looked up once
     std::map<OVERLAPPED*, HidRead> hid_reads;
     Hook register_raw;
     Hook async_key, key, keyboard, raw_data, raw_buffer, capture, release_capture, cursor, show_cursor, physical_cursor, cursor_position;
@@ -77,12 +81,30 @@ BOOLEAN __stdcall captured_hid_attributes(HANDLE file, PHIDD_ATTRIBUTES attribut
     SetLastError(error);
     return result;
 }
+// The pad's input report length as Windows gives it, or 0.
+std::size_t hid_input_length(HANDLE file) noexcept {
+    const auto error = GetLastError();
+    std::size_t length{};
+    PHIDP_PREPARSED_DATA preparsed{};
+    if (HidD_GetPreparsedData(file, &preparsed) && preparsed) {
+        HIDP_CAPS caps{};
+        if (HidP_GetCaps(preparsed, &caps) == HIDP_STATUS_SUCCESS) length = caps.InputReportByteLength;
+        HidD_FreePreparsedData(preparsed);
+    }
+    SetLastError(error);
+    return length;
+}
 void neutral_hid(const InputCaptureHooks::HidRead& read, DWORD size) {
     if (!block_polled_input() || !read.buffer || size > read.capacity) return;
     auto* bytes = static_cast<std::uint8_t*>(read.buffer);
-    if (!parse_playstation_report(read.kind, playstation_bluetooth(read.capacity), bytes, size)) return;
+    // USB or Bluetooth decides where the buttons are in a report with id 1. It is the device's
+    // report length that tells them apart, not how large a buffer the game happens to read
+    // into: a game reading a USB DualSense into a larger buffer had its d-pad byte written as
+    // "up" here, and placed a session marker whenever a menu took the input.
+    const bool bluetooth = playstation_bluetooth(read.input_length ? read.input_length : read.capacity);
+    if (!parse_playstation_report(read.kind, bluetooth, bytes, size)) return;
     const bool sense_full = read.kind == PlayStationPad::dualsense &&
-        (bytes[0] == 0x31 || (bytes[0] == 1 && !playstation_bluetooth(read.capacity)));
+        (bytes[0] == 0x31 || (bytes[0] == 1 && !bluetooth));
     const unsigned offset = bytes[0] == 0x31 ? 2 : bytes[0] == 0x11 ? 3 : 1;
     auto* data = bytes + offset;
     std::fill_n(data, 4, std::uint8_t{128});
@@ -107,8 +129,10 @@ void neutral_hid(const InputCaptureHooks::HidRead& read, DWORD size) {
         std::memcpy(bytes + size - 4, &crc, sizeof(crc));
     }
     static std::atomic<bool> reported{};
-    if (!reported.exchange(true)) logging::write(logging::Level::info, logging::Channel::input,
-        "Native PlayStation HID report neutralized; overlay retains its independent input stream.");
+    if (!reported.exchange(true)) logging::printf(logging::Level::info, logging::Channel::input,
+        "Native PlayStation HID report neutralized; overlay retains its independent input stream. (%s over %s, report 0x%02x of %lu bytes, "
+        "device report length %zu, the game's buffer %lu.)", read.kind == PlayStationPad::dualsense ? "DualSense" : "DualShock 4",
+        bluetooth ? "Bluetooth" : "USB", bytes[0], size, read.input_length, read.capacity);
 }
 void complete_hid(OVERLAPPED* request, DWORD bytes, bool success, bool finished) {
     auto& h = input_hooks();
@@ -148,6 +172,9 @@ BOOL WINAPI captured_read_file(HANDLE file, LPVOID buffer, DWORD size, LPDWORD b
         const auto found = h.hid_devices.find(file);
         if (found != h.hid_devices.end() && found->second != PlayStationPad::none) {
             read.kind = found->second;
+            const auto length = h.hid_lengths.find(file);
+            if (length != h.hid_lengths.end()) read.input_length = length->second;
+            else read.input_length = h.hid_lengths[file] = hid_input_length(file);
             if (request) h.hid_reads[request] = read;
         }
     }
@@ -191,6 +218,7 @@ BOOL WINAPI captured_close_handle(HANDLE file) {
     auto& h = input_hooks();
     {
         std::lock_guard lock(h.hid_mutex);
+        h.hid_lengths.erase(file);
         if (h.hid_devices.erase(file))
             std::erase_if(h.hid_reads, [&](const auto& entry) { return entry.second.file == file; });
     }

Extension/UI/Overlay/overlay_render.cpp

@@ -8,6 +8,7 @@
 #include "chat_emotes.h"
 #include "input_capture.h"
 #include "cursor.h"
+#include <dxgi1_6.h>
 #include <imgui_internal.h>
 
 namespace dingosdk::overlay::detail {
@@ -221,8 +222,15 @@ bool setup_graphics() {
     // queue-type validation are the portable checks here.
     const auto format = description.BufferDesc.Format;
     if (format != DXGI_FORMAT_R8G8B8A8_UNORM && format != DXGI_FORMAT_B8G8R8A8_UNORM
-        && format != DXGI_FORMAT_R10G10B10A2_UNORM && format != DXGI_FORMAT_R16G16B16A16_FLOAT)
+        && format != DXGI_FORMAT_R10G10B10A2_UNORM && format != DXGI_FORMAT_R16G16B16A16_FLOAT) {
+        // Said once for each: with this the overlay never draws, and nothing else says why.
+        static std::atomic<int> said{-1};
+        if (said.exchange(static_cast<int>(format)) != static_cast<int>(format))
+            dingosdk::logging::printf(dingosdk::logging::Level::warning, dingosdk::logging::Channel::graphics,
+                "Overlay: the game presents in a format the overlay does not draw to (DXGI format %d); its menus stay hidden.",
+                static_cast<int>(format));
         return false;
+    }
 
     D3D12_DESCRIPTOR_HEAP_DESC heap{};
     heap.Type = D3D12_DESCRIPTOR_HEAP_TYPE_RTV;
@@ -334,6 +342,25 @@ bool setup_graphics() {
     info.LegacySingleSrvCpuDescriptor = s.srvs->GetCPUDescriptorHandleForHeapStart();
     info.LegacySingleSrvGpuDescriptor = s.srvs->GetGPUDescriptorHandleForHeapStart();
     if (s.win32_ready) s.dx12_ready = ImGui_ImplDX12_Init(&info);
+    // ImGui draws sRGB colours. A game in HDR presents another kind of buffer, where those same
+    // numbers are far too bright and washed out: scRGB (FP16, linear) or HDR10 (10 bits, BT.2020
+    // with the PQ curve, while the display is in HDR). The backend's shader converts for those.
+    {
+        bool hdr_display{};
+        ComPtr<IDXGIOutput> output;
+        ComPtr<IDXGIOutput6> output6;
+        DXGI_OUTPUT_DESC1 display{};
+        if (SUCCEEDED(s.swapchain->GetContainingOutput(&output)) && output && SUCCEEDED(output.As(&output6)) &&
+            SUCCEEDED(output6->GetDesc1(&display)))
+            hdr_display = display.ColorSpace == DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020;
+        constexpr float hdr_white_nits = 200.0f; // how bright the overlay's white is on an HDR display
+        const int transfer = format == DXGI_FORMAT_R16G16B16A16_FLOAT ? 1
+                           : format == DXGI_FORMAT_R10G10B10A2_UNORM && hdr_display ? 2 : 0;
+        ImGui_ImplDX12_SetOutputTransfer(transfer, hdr_display ? hdr_white_nits : 80.0f);
+        dingosdk::logging::printf(dingosdk::logging::Level::info, dingosdk::logging::Channel::graphics,
+            "Overlay output: %s (buffer format %d, display %s).",
+            transfer == 1 ? "scRGB" : transfer == 2 ? "HDR10" : "SDR", static_cast<int>(format), hdr_display ? "in HDR" : "in SDR");
+    }
     // Font upload uses the exact swapchain queue, and this pinned backend waits
     // for its own upload fence before returning from device-object creation.
     const bool objects = s.dx12_ready && ImGui_ImplDX12_CreateDeviceObjects();

External/imgui/backends/imgui_impl_dx12.cpp

@@ -114,6 +114,14 @@ struct VERTEX_CONSTANT_BUFFER_DX12
     float   mvp[4][4];
 };
 
+// [ReSkate] mode, white in nits, unused, unused: see ImGui_ImplDX12_SetOutputTransfer.
+static float g_ReSkateOutputTransfer[4] = { 0.0f, 200.0f, 0.0f, 0.0f };
+void ImGui_ImplDX12_SetOutputTransfer(int mode, float white_nits)
+{
+    g_ReSkateOutputTransfer[0] = (float)mode;
+    g_ReSkateOutputTransfer[1] = white_nits;
+}
+
 // Functions
 static void ImGui_ImplDX12_SetupRenderState(ImDrawData* draw_data, ID3D12GraphicsCommandList* command_list, ImGui_ImplDX12_RenderBuffers* fr)
 {
@@ -163,6 +171,7 @@ static void ImGui_ImplDX12_SetupRenderState(ImDrawData* draw_data, ID3D12Graphic
     command_list->SetPipelineState(bd->pPipelineState);
     command_list->SetGraphicsRootSignature(bd->pRootSignature);
     command_list->SetGraphicsRoot32BitConstants(0, 16, &vertex_constant_buffer, 0);
+    command_list->SetGraphicsRoot32BitConstants(0, 4, g_ReSkateOutputTransfer, 16); // [ReSkate] see ImGui_ImplDX12_SetOutputTransfer
 
     // Setup blend factor
     const float blend_factor[4] = { 0.f, 0.f, 0.f, 0.f };
@@ -440,8 +449,8 @@ bool    ImGui_ImplDX12_CreateDeviceObjects()
         param[0].ParameterType = D3D12_ROOT_PARAMETER_TYPE_32BIT_CONSTANTS;
         param[0].Constants.ShaderRegister = 0;
         param[0].Constants.RegisterSpace = 0;
-        param[0].Constants.Num32BitValues = 16;
-        param[0].ShaderVisibility = D3D12_SHADER_VISIBILITY_VERTEX;
+        param[0].Constants.Num32BitValues = 20;                        // [ReSkate] the matrix, then the output transfer
+        param[0].ShaderVisibility = D3D12_SHADER_VISIBILITY_ALL;       // [ReSkate] the pixel shader reads the transfer
 
         param[1].ParameterType = D3D12_ROOT_PARAMETER_TYPE_DESCRIPTOR_TABLE;
         param[1].DescriptorTable.NumDescriptorRanges = 1;
@@ -582,12 +591,35 @@ bool    ImGui_ImplDX12_CreateDeviceObjects()
               float4 col : COLOR0;\
               float2 uv  : TEXCOORD0;\
             };\
+            cbuffer vertexBuffer : register(b0) \
+            {\
+              float4x4 ProjectionMatrix; \
+              float4 OutputTransfer; \
+            };\
             SamplerState sampler0 : register(s0);\
             Texture2D texture0 : register(t0);\
             \
+            float3 srgb_to_linear(float3 c)\
+            {\
+              return lerp(c / 12.92, pow(abs((c + 0.055) / 1.055), 2.4), step(0.04045, c)); \
+            }\
             float4 main(PS_INPUT input) : SV_Target\
             {\
               float4 out_col = input.col * texture0.Sample(sampler0, input.uv); \
+              if (OutputTransfer.x > 0.5) \
+              {\
+                float3 lin = srgb_to_linear(saturate(out_col.rgb)); \
+                if (OutputTransfer.x < 1.5) \
+                {\
+                  out_col.rgb = lin * (OutputTransfer.y / 80.0); \
+                }\
+                else \
+                {\
+                  float3 wide = float3(dot(lin, float3(0.6274, 0.3293, 0.0433)), dot(lin, float3(0.0691, 0.9195, 0.0114)), dot(lin, float3(0.0164, 0.0880, 0.8956))); \
+                  float3 y = pow(saturate(wide * (OutputTransfer.y / 10000.0)), 0.1593017578125); \
+                  out_col.rgb = pow((0.8359375 + 18.8515625 * y) / (1.0 + 18.6875 * y), 78.84375); \
+                }\
+              }\
               return out_col; \
             }";
 

External/imgui/backends/imgui_impl_dx12.h

@@ -51,6 +51,11 @@ IMGUI_IMPL_API bool     ImGui_ImplDX12_Init(ImGui_ImplDX12_InitInfo* info);
 IMGUI_IMPL_API void     ImGui_ImplDX12_Shutdown();
 IMGUI_IMPL_API void     ImGui_ImplDX12_NewFrame();
 IMGUI_IMPL_API void     ImGui_ImplDX12_RenderDrawData(ImDrawData* draw_data, ID3D12GraphicsCommandList* graphics_command_list);
+// [ReSkate] How the colours ImGui draws (sRGB) are written to the render target. 0: as they are
+// (an SDR target). 1: scRGB, linear with 1.0 at 80 nits (an FP16 HDR target). 2: HDR10, BT.2020
+// primaries with the ST.2084 (PQ) curve (a 10-bit HDR target). `white_nits`: how bright white
+// is drawn in 1 and 2. Read at each ImGui_ImplDX12_RenderDrawData.
+IMGUI_IMPL_API void     ImGui_ImplDX12_SetOutputTransfer(int mode, float white_nits);
 
 #ifndef IMGUI_DISABLE_OBSOLETE_FUNCTIONS
 // Legacy initialization API Obsoleted in 1.91.5

Launcher/Test/gamepad_input_tests.cpp

@@ -455,6 +455,34 @@ int main() {
         check(scroll > 100, "The left stick scrolls the panel under the mouse");
     }
 
+    {
+        // A stick that rests well off the middle (a worn pad, or a wheel's pedal on that axis)
+        // never steers the pointer: not at first, and not after the launcher was in the
+        // background, where the pad reads as not there.
+        fresh();
+        PadFeed feed;
+        ImGui::GetIO().AddMousePosEvent(400, 300);
+        const auto off = pad(0, 12000, 0);   // about 0.37: past the rest limit and the dead zone
+        for (int i = 0; i < 60; ++i) frame(feed, off);
+        check(!feed.pointing(), "A stick resting off the middle does not take the pointer");
+        for (int i = 0; i < 60; ++i) frame(feed, PadState{});   // in the background for a second
+        for (int i = 0; i < 60; ++i) frame(feed, off);
+        check(!feed.pointing() && !ImGui::GetIO().MouseDrawCursor,
+              "A stick resting off the middle does not take the pointer after the launcher was in the background");
+        check(ImGui::GetIO().MousePos.x == 400 && ImGui::GetIO().MousePos.y == 300, "The mouse stays where it was");
+    }
+    {
+        // A pad that goes away while it has the pointer gives it back to the mouse.
+        fresh();
+        PadFeed feed;
+        ImGui::GetIO().AddMousePosEvent(400, 300);
+        for (int i = 0; i < 30; ++i) frame(feed, pad());
+        for (int i = 0; i < 5; ++i) frame(feed, pad(0, 30000, 0));
+        check(feed.pointing(), "The right stick takes the pointer");
+        frame(feed, PadState{});
+        check(!feed.pointing(), "A pad that goes away gives the pointer back");
+    }
+
     // ------------------------------------------------ the Steam Deck's trackpad
     {
         fresh();

Launcher/gamepad_input.cpp

@@ -112,7 +112,15 @@ void PadFeed::update(ImGuiIO& io, const PadState& input, float scale) {
 
     const PointerFeel& feel = trackpad_ ? trackpad_feel : stick_feel;
     const float raw_x = axis(pad.right_x), raw_y = -axis(pad.right_y);   // screen Y grows downward
-    if (std::abs(raw_x - last_x_) < rest_jitter && std::abs(raw_y - last_y_) < rest_jitter &&
+    if (!pad.connected) {
+        // No pad is being read (none there, or the launcher is in the background, which reads
+        // as none): that is not the stick at rest. Taking the middle for its rest here made a
+        // stick that rests off it, or a wheel or throttle on that axis, steer the pointer for
+        // good once the launcher had the focus again, and the mouse with it.
+        steady_ = 0;
+        rest_known_ = false;
+        pointing_ = false;
+    } else if (std::abs(raw_x - last_x_) < rest_jitter && std::abs(raw_y - last_y_) < rest_jitter &&
         std::sqrt(raw_x * raw_x + raw_y * raw_y) < rest_limit) {
         steady_ += io.DeltaTime;
         if (steady_ >= rest_seconds) {