Roadmap: multiview to look into later; refresh rate steps in main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

port/android/VR_ROADMAP.md

@@ -17,7 +17,7 @@ named where code is taken over.
 | Area | What |
 | --- | --- |
 | Headsets | Quest 3; Steam Frame (Lepton): its controllers' layout, a Steam library tile that installs, starts, updates and quits the game, data kept in `~/Documents/RingfallMjolnirVR` |
-| Rendering | Both eyes in one pass (multiview) on the Quest; one by one on SteamVR, where multiview's eyes did not fuse; lens flares at their depth in stereo; HUD, menus, scope, crosshair and names as layers of their own |
+| Rendering | The eyes drawn one by one (both at once, multiview, only with `debug.multiview`: its eyes did not always fuse, see the open points); lens flares at their depth in stereo; HUD, menus, scope, crosshair and names as layers of their own |
 | Aiming | The weapon in the hand, its shots from the muzzle, laser and crosshair options |
 | Body | Physical crouch (`vr.crouch_limit`) beside the crouch button, leaning (`vr.lean_limit`), melee by swinging, grenades thrown with the left hand, two-handed weapons |
 | Vehicles | The Warthog's wheel turned with the hands, right trigger throttle, left brake; first-person seats; turrets with handles |
@@ -42,6 +42,7 @@ named where code is taken over.
 | 3c | Weapons held physically, holsters at shoulder and hip (the buttons stay) | |
 | 3d | Other VR players shown with their heads, hands and bodies (today their bodies move as flat players' do: only their aim and throws come from their hands) | |
 | 4 | Later: disc image import in the launcher; moving by swinging the arms (in place of the stick, never faster than it: online too), and a sprint by swinging them harder (faster than the game allows: offline only) | import **done** (in `main` 2026-10-06): the app's import (tried on the Quest 3: all 24 maps; on the Frame kept in `~/Documents/RingfallMjolnirVR`), a Steam Frame installer from Desktop Mode (`STEAM_FRAME.md`; tried on the Frame: the old folder renamed, the shortcut added by Steam), a release signing key (CI and local builds the same). Arm swinging and its offline sprint still open |
+| 5 | Later: multiview again (both eyes in one pass, half the drawing's CPU cost): find why its eyes do not fuse in big online games on the Quest 3 (and now and then on the Steam Frame), then turn it back on | open: off by default since 2026-10-16 (`debug.multiview`); findings in the open points |
 | R | Release as **Ringfall: Mjolnir VR** (no "Halo" in the product's name): app name, id `io.github.cesarus85.ringfall`, icon and library pictures (`tools/ringfall_art.py`), Frame folder and installer, release assets, README; the repository renamed | **done** (in `main` 2026-10-06; Quest 3 and Steam Frame good) |
 
 ## Step 2, first part (done)
@@ -518,6 +519,34 @@ see that nothing is missing or slower.
 
 ## Open points outside the order
 
+- Multiview's doubled pictures (2026-10-16): on the developer's Quest 3,
+  doubled and shifted pictures from the start of big online games (26-35
+  players) as the head turned, nauseating; none in a local game with 48
+  bots; none online either with the eyes drawn one by one. Not the frame
+  rate: one by one, online at 72 Hz, the compositor repeated as many
+  frames (about 4 a second) and nothing was seen. Not the facing jumps
+  (respawns only) nor the camera (it moves every frame). Before: the
+  Steam Frame's (2026-10-05), each eye alone sharp. So the eyes are drawn
+  one by one by default now (`debug.multiview` for both at once). To do:
+  find what the shared pass draws wrongly online, e.g. by saving both
+  eyes' pictures in a big online game and comparing what is doubled
+  (sprites and effects projected by the game itself, `mv_depth`; programs
+  made on the shader worker; the sky rows), then turn it back on: it
+  halves the drawing's CPU cost.
+- Big online games (2026-10-16): doubled pictures on a 32-player server,
+  none in a local game with 48 bots. Not the GPU (60–80 %): the ticks cost
+  3.3–4.6 ms as a client there (1.3–1.9 ms with bots or 16 players), so
+  47–70 of 150 frames with a tick missed 90 Hz's 11.1 ms (the doubling
+  itself was multiview's, above). In `main` 2026-10-16 (Quest 3 good):
+  `vr.refresh_rate_auto`
+  (VR options MORE, "Lower it when frames are late", on by default) steps
+  the rate down a step (90, 80, 72 Hz) while more than one frame in 32
+  comes late, and back up once they would keep up; "VR: refresh rate" in
+  the log. First try: 90 to 80 Hz a few seconds into a 30-player game, no
+  doubled picture seen; 2-5 stale frames a second left at 80, hence one
+  in 32 (was 16). Later:
+  why a client's ticks cost that much per remote player.
+
 - Doubled pictures when turning on the developer's Quest 3 (2026-10-12):
   with every build tried, back to 2026-10-06's, at 90 and at 72 Hz, even
   in seconds the compositor reported no stale frame. That Quest runs
@@ -526,6 +555,8 @@ see that nothing is missing or slower.
   default (smooth there). To do: try this and later builds with "high" on
   a Quest 3 outside PTC (the developer's wife's), and set the default
   back to "high" or "auto" if it holds.
+  Resolved 2026-10-16: multiview (above), not the Horizon OS PTC.
+
 - Multiview on the Steam Frame: why the eyes do not fuse there (suspected
   Mesa zink/Turnip); tools in branch `frame/doubled-images` (86335828),
   not merged.

port/linux/include/halo_vr.h

@@ -50,7 +50,7 @@ void halo_vr_eye_camera(int eye, const float game_position[3],
 /* copies the back buffer, just drawn for an eye, to that eye's image */
 void halo_vr_capture_eye(int eye);
 
-/* Both eyes in one draw (vr.multiview, where GL_OVR_multiview2 is): whether
+/* Both eyes in one draw (debug.multiview, where GL_OVR_multiview2 is): whether
 this frame draws so; the camera between the eyes whose view takes in both
 (as halo_vr_eye_camera), which also starts drawing into both eyes' layers;
 and after the draw, the layers taken for the eyes' images. */

port/linux/src/port_config.c

@@ -200,6 +200,12 @@ static const struct config_setting config_settings[] =
 	{ "vr.refresh_rate", _config_real, "90.0", "HALO_VR_REFRESH_RATE", _environment_value, _platform_android,
 		"The display refresh rate to ask the headset for, in Hz (72, 90, 120);\n"
 		"it picks the nearest it has at or below. 0 for the highest." },
+	{ "vr.refresh_rate_auto", _config_boolean, "true", "HALO_VR_REFRESH_RATE_AUTO", _environment_value,
+		_platform_android,
+		"While frames come late at that rate (big online games), step down to the\n"
+		"next the headset has (90 to 72 Hz), and back up once they would keep up\n"
+		"again; the headset shows a late frame twice, a doubled picture when\n"
+		"turning. Only where the headset lets the game choose (not SteamVR)." },
 	{ "vr.graphics", _config_string, "\"medium\"", "HALO_VR_GRAPHICS", _environment_value, _platform_android,
 		"Which of the costlier effects are drawn (\"medium\" by default): \"auto\" all of them while the\n"
 		"GPU keeps up, leaving out one after another (the shield's glow, object\n"
@@ -227,9 +233,6 @@ static const struct config_setting config_settings[] =
 		"away from the player's body (walls are not checked)." },
 	{ "vr.crouch_limit", _config_real, "0.9", "HALO_VR_CROUCH_LIMIT", _environment_value, _platform_android,
 		"How far, in metres, the view follows your head down." },
-	{ "vr.multiview", _config_boolean, "true", "HALO_VR_MULTIVIEW", _environment_value, _platform_android,
-		"Draw both eyes in one pass (GL_OVR_multiview2), from between them: about\n"
-		"half the drawing's cost on the processor. false: one eye after the other." },
 	{ "vr.log_performance", _config_boolean, "false", "HALO_VR_LOG_PERFORMANCE", _environment_value, _platform_android,
 		"Log the frame rate and where the frame time goes every five seconds\n"
 		"(adb logcat -s halo)." },
@@ -780,10 +783,12 @@ static const struct config_setting config_settings[] =
 		"machines of twice it), to test the netcode as over the internet; 0 none." },
 	{ "debug.network_loss", _config_real, "0.0", "HALO_NETWORK_LOSS", _environment_value, _platform_all,
 		"Percent of datagrams received that are dropped, for the same; 0 none." },
-	{ "debug.multiview_steamvr", _config_boolean, "false", "HALO_MULTIVIEW_STEAMVR", _environment_value,
-		_platform_android,
-		"On SteamVR (the Steam Frame) the eyes are drawn one by one, whatever\n"
-		"vr.multiview says: true draws them at once there too, for testing." },
+	{ "debug.multiview", _config_boolean, "false", "HALO_MULTIVIEW", _environment_value, _platform_android,
+		"Draw both eyes in one pass (GL_OVR_multiview2), from between them: about\n"
+		"half the drawing's cost on the processor, but the eyes' pictures did not\n"
+		"always fuse (doubled as the head turned: on the Steam Frame now and then,\n"
+		"on the Quest 3 in big online games). Off, one eye after the other; on,\n"
+		"for finding why." },
 	{ "debug.gl_thread", _config_integer, "-1", "HALO_GL_THREAD", _environment_value, _platform_android,
 		"Mesa's GL thread (its drivers: the Steam Frame's): 1 on, 0 off, -1 as the\n"
 		"driver chooses. Read when the app starts." },

port/linux/src/vr.c

@@ -138,7 +138,7 @@ static struct
 	float seat_frame_yaw;
 	int seat_frame_valid;
 	float seat_turn;
-	int multiview;              /* vr.multiview: both eyes in one draw */
+	int multiview;              /* debug.multiview: both eyes in one draw */
 	int multiview_logged;
 	int multiview_logged_views;
 	/* vr.foveation's level last logged (+1; 0: none yet) */
@@ -298,6 +298,15 @@ static struct
 	vr.resolution_scale as the app started with it */
 	float refresh_rate_wanted;
 	float resolution_scale;
+	/* vr.refresh_rate_auto (refresh_governor): the rate asked for now (the
+	one wanted, or lower while frames come late), the rates below and above
+	it, and the window's frames: all, late at this rate, and those that
+	would be late at the one above */
+	int refresh_auto;
+	float refresh_rate_now, refresh_lower, refresh_higher, refresh_period_ms;
+	unsigned long long refresh_window_us, refresh_changed_us;
+	unsigned long refresh_frames, refresh_late, refresh_tight;
+	int refresh_bad_windows, refresh_good_windows;
 } vr;
 
 /* d3d8_gl.c */
@@ -585,12 +594,23 @@ static void vr_read_live_settings(void)
 		/* (the refresh rate changes at once where the runtime lets it) */
 		float refresh_rate = (float)config_real("vr.refresh_rate");
 
-		if (vr.active && refresh_rate != vr.refresh_rate_wanted)
+		int automatic = config_boolean("vr.refresh_rate_auto");
+
+		if (vr.active && (refresh_rate != vr.refresh_rate_wanted ||
+			(!automatic && vr.refresh_rate_now != refresh_rate)))
 		{
 			platform_log("VR: refresh rate %.0f Hz asked for", refresh_rate);
 			host_xr_set_refresh_rate(refresh_rate);
 		}
+		if (refresh_rate != vr.refresh_rate_wanted || automatic != vr.refresh_auto)
+		{
+			/* (the governor starts over from the rate chosen) */
+			vr.refresh_rate_now = refresh_rate;
+			vr.refresh_window_us = 0;
+			vr.refresh_bad_windows = vr.refresh_good_windows = 0;
+		}
 		vr.refresh_rate_wanted = refresh_rate;
+		vr.refresh_auto = automatic;
 	}
 }
 
@@ -625,7 +645,7 @@ void halo_vr_initialize(void)
 	vr.two_hand_reach = (float)config_real("vr.two_hand_reach");
 	vr.melee_swing_speed = (float)config_real("vr.melee_swing_speed");
 	vr.grenade_throw_speed = (float)config_real("vr.grenade_throw_speed");
-	vr.multiview = config_boolean("vr.multiview");
+	vr.multiview = config_boolean("debug.multiview");
 	vr.vehicle_first_person = config_boolean("vr.vehicle_first_person");
 	vr.vehicle_spawn = config_boolean("debug.vehicle_spawn");
 	vr.turret_handles = config_boolean("vr.turret_handles");
@@ -658,16 +678,13 @@ void halo_vr_initialize(void)
 	vr.active = 1;
 	platform_log("VR: %s, eyes %dx%d at %.0f Hz", vr.info.system_name, vr.info.eye_width, vr.info.eye_height,
 		vr.info.refresh_rate);
-	/* SteamVR (the Steam Frame): each eye drawn apart. With both drawn at
-	once the eyes' pictures did not fuse, now and then: each alone was
-	sharp, the two together doubled as the head turned (2026-10-05), and
-	one by one costs it no more (its time goes to the driver either way).
-	debug.multiview_steamvr draws them at once there again, for finding why */
-	if (vr.multiview && strstr(vr.info.system_name, "SteamVR") && !config_boolean("debug.multiview_steamvr"))
-	{
-		vr.multiview = 0;
-		platform_log("VR: SteamVR: the eyes drawn one by one (debug.multiview_steamvr)");
-	}
+	/* Each eye drawn apart (debug.multiview draws both at once). At once,
+	the eyes' pictures did not always fuse: on the Steam Frame now and then
+	(each alone sharp, the two together doubled as the head turned,
+	2026-10-05; one by one costs it no more there), and on the Quest 3 in
+	big online games (doubled from the start as the head turned, none with
+	48 bots; one by one, none online either, 2026-10-16) */
+	platform_log("VR: the eyes drawn %s (debug.multiview)", vr.multiview ? "at once" : "one by one");
 }
 
 int halo_vr_active(void)
@@ -889,6 +906,93 @@ static void performance_phases(unsigned long long now)
 	vr.frame_ticks = 0;
 }
 
+/* vr.refresh_rate_auto: frames late at the headset's rate (a game's big
+online games: the ticks cost as a client in step with the players) are
+shown twice by the headset, a doubled picture when turning. While more
+than one in 32 of them come late, two windows of REFRESH_WINDOW_US
+running, the rate steps down to the next the headset offers (the Quest
+3's: 90, 80, 72 Hz); while hardly any would be late at the next one up (one in a
+hundred), ten windows running, it steps back up, never above
+vr.refresh_rate. Frames of a load (REFRESH_LOAD_MS and more) count for
+neither; a step waits REFRESH_SETTLE_US (down) or REFRESH_RAISE_US (up)
+after the last. */
+#define REFRESH_WINDOW_US 2000000ULL
+#define REFRESH_SETTLE_US 10000000ULL
+#define REFRESH_RAISE_US 30000000ULL
+#define REFRESH_LOAD_MS 50.0f
+
+static void refresh_rates_around(void)
+{
+	float rates[16], current = 0.0f;
+	int count = host_xr_refresh_rates(rates, 16, &current);
+	float ceiling = vr.refresh_rate_wanted > 1.0f ? vr.refresh_rate_wanted + 0.5f : 1e6f;
+	int index;
+
+	vr.refresh_lower = vr.refresh_higher = 0.0f;
+	if (current <= 1.0f)
+		current = vr.refresh_rate_now;
+	vr.refresh_period_ms = current > 1.0f ? 1000.0f / current : 0.0f;
+	for (index = 0; index < count; index++)
+	{
+		if (rates[index] < current - 0.5f && rates[index] > vr.refresh_lower)
+			vr.refresh_lower = rates[index];
+		if (rates[index] > current + 0.5f && rates[index] <= ceiling &&
+			(!vr.refresh_higher || rates[index] < vr.refresh_higher))
+			vr.refresh_higher = rates[index];
+	}
+}
+
+static void refresh_step(float rate, unsigned long long now, const char *why)
+{
+	platform_log("VR: refresh rate %.0f Hz (from %.0f): %s (%lu of %lu frames late, %lu too slow for the rate above)",
+		rate, 1000.0f / vr.refresh_period_ms, why, vr.refresh_late, vr.refresh_frames, vr.refresh_tight);
+	host_xr_set_refresh_rate(rate);
+	vr.refresh_rate_now = rate;
+	vr.refresh_changed_us = now;
+	vr.refresh_bad_windows = vr.refresh_good_windows = 0;
+}
+
+static void refresh_governor(void)
+{
+	unsigned long long now = now_us();
+	float work_ms = (float)(now - vr.frame_start_us) * 1e-3f;
+
+	if (!vr.refresh_auto)
+		return;
+	if (!vr.refresh_window_us)
+	{
+		refresh_rates_around();
+		vr.refresh_window_us = now;
+		vr.refresh_frames = vr.refresh_late = vr.refresh_tight = 0;
+		return;
+	}
+	if (work_ms < REFRESH_LOAD_MS && vr.refresh_period_ms > 0.0f)
+	{
+		vr.refresh_frames++;
+		if (work_ms + 1.0f > vr.refresh_period_ms)
+			vr.refresh_late++;
+		if (vr.refresh_higher > 1.0f && work_ms + 1.0f > 1000.0f / vr.refresh_higher)
+			vr.refresh_tight++;
+	}
+	if (now - vr.refresh_window_us < REFRESH_WINDOW_US)
+		return;
+	if (vr.refresh_frames)
+	{
+		vr.refresh_bad_windows = vr.refresh_late * 32 > vr.refresh_frames ? vr.refresh_bad_windows + 1 : 0;
+		vr.refresh_good_windows = vr.refresh_tight * 100 <= vr.refresh_frames ? vr.refresh_good_windows + 1 : 0;
+		if (vr.refresh_lower > 1.0f && vr.refresh_bad_windows >= 2 && now - vr.refresh_changed_us >= REFRESH_SETTLE_US)
+			refresh_step(vr.refresh_lower, now, "frames come late");
+		else if (vr.refresh_higher > 1.0f && vr.refresh_good_windows >= 10 &&
+			now - vr.refresh_changed_us >= REFRESH_RAISE_US)
+		{
+			refresh_step(vr.refresh_higher, now, "frames would keep up again");
+		}
+	}
+	refresh_rates_around();
+	vr.refresh_window_us = now;
+	vr.refresh_frames = vr.refresh_late = vr.refresh_tight = 0;
+}
+
 static void performance_report(int stereo)
 {
 	unsigned long long now = now_us();
@@ -1886,6 +1990,7 @@ void halo_vr_present(unsigned int back_buffer_texture)
 	vr.last_overlay_mode = vr.overlay_mode;
 	vr.last_frame_stereo = submit.eyes;
 	vr.frame_begun = 0;
+	refresh_governor();
 	if (vr.log_performance)
 		performance_report(submit.eyes);
 }
@@ -2053,7 +2158,7 @@ float halo_vr_refresh_rate(void)
 	return vr.active ? vr.info.refresh_rate : 0.0f;
 }
 
-/* ---------- both eyes at once (vr.multiview)
+/* ---------- both eyes at once (debug.multiview)
 
 The scene is drawn once, from a camera between the eyes whose view takes in
 both eyes' (with a margin, for what is near: the eyes stand apart), into two

source/interface/ui_widget.c

@@ -7468,6 +7468,7 @@ enum
 	_vr_option_graphics,
 	_vr_option_foveation,
 	_vr_option_refresh_rate,
+	_vr_option_refresh_auto,
 	_vr_option_resolution,
 	_vr_option_controls,
 	_vr_option_quit,
@@ -7521,8 +7522,8 @@ static short const vr_option_pages[NUMBER_OF_VR_OPTION_PAGES][VR_OPTIONS_PER_PAG
 		_vr_option_button_first + HALO_VR_ACTION_CROUCH, _vr_option_button_first + HALO_VR_ACTION_ZOOM,
 		_vr_option_button_first + HALO_VR_ACTION_MELEE, _vr_option_button_first + HALO_VR_ACTION_TALK,
 		_vr_option_buttons_reset, _vr_option_controls, NONE, NONE },
-	{ _vr_option_graphics, _vr_option_foveation, _vr_option_refresh_rate, _vr_option_resolution, _vr_option_quit,
-		NONE, NONE, NONE, NONE, NONE, NONE, NONE },
+	{ _vr_option_graphics, _vr_option_foveation, _vr_option_refresh_rate, _vr_option_refresh_auto,
+		_vr_option_resolution, _vr_option_quit, NONE, NONE, NONE, NONE, NONE, NONE },
 };
 
 /* the page a row is on, and its place there */
@@ -7942,6 +7943,9 @@ static void ui_vr_options_change(
 		}
 		break;
 	}
+	case _vr_option_refresh_auto:
+		config_write_boolean("vr.refresh_rate_auto", !config_boolean("vr.refresh_rate_auto"));
+		break;
 	case _vr_option_resolution:
 		ui_vr_options_step("vr.resolution_scale", vr_resolution_scales, NUMBEROF(vr_resolution_scales), change);
 		break;
@@ -8651,7 +8655,10 @@ static void ui_vr_options_render(
 			short count = ui_vr_options_refresh_rates(rates, &chosen, &current);
 
 			label = "Refresh rate";
-			if (count > 0)
+			/* (lowered while frames come late: vr.refresh_rate_auto) */
+			if (count > 0 && current > 1.0f && current < rates[chosen] - 0.5f)
+				snprintf(value, sizeof(value), "%.0f Hz (now %.0f)", rates[chosen], current);
+			else if (count > 0)
 				snprintf(value, sizeof(value), "%.0f Hz", rates[chosen]);
 			else if (current > 1.0f)
 				snprintf(value, sizeof(value), "%.0f Hz (system)", current);
@@ -8660,6 +8667,17 @@ static void ui_vr_options_render(
 			unused = count <= 1;
 			break;
 		}
+		case _vr_option_refresh_auto:
+		{
+			/* (dim where the headset offers one rate: SteamVR's) */
+			real rates[16], current;
+			short chosen;
+
+			label = "Lower it when frames are late";
+			snprintf(value, sizeof(value), "%s", config_boolean("vr.refresh_rate_auto") ? "On" : "Off");
+			unused = ui_vr_options_refresh_rates(rates, &chosen, &current) <= 1;
+			break;
+		}
 		case _vr_option_resolution:
 		{
 			real scale = vr_resolution_scales[ui_vr_options_nearest((real)config_real("vr.resolution_scale"),
@@ -8717,12 +8735,11 @@ static void ui_vr_options_render(
 	}
 	if (page == _vr_options_page_buttons)
 	{
-		/* (what the buttons' rows do) */
-		ui_join_code_draw_text(128, 512, 302, "An action moved to a button swaps with the one there;", FALSE, 0,
+		/* (what the buttons' rows do: below the page's ten rows) */
+		ui_join_code_draw_text(128, 512, 320, "An action moved to a button swaps with the one there;", FALSE, 0,
 			&dim_color);
-		ui_join_code_draw_text(128, 512, 320, "left-handed and mirrored, the buttons are mirrored too.", FALSE, 0,
+		ui_join_code_draw_text(128, 512, 338, "mirrored, the buttons are too. The menus keep theirs.", FALSE, 0,
 			&dim_color);
-		ui_join_code_draw_text(128, 512, 338, "The menus keep their buttons.", FALSE, 0, &dim_color);
 	}
 	if (page == _vr_options_page_more)
 	{
@@ -8739,6 +8756,8 @@ static void ui_vr_options_render(
 			&dim_color);
 		ui_join_code_draw_text(128, 512, 320, "Foveation draws the view's edges coarser (Quest only).", FALSE, 0,
 			&dim_color);
+		ui_join_code_draw_text(128, 512, 338, "Late frames: the rate steps down (90, 80, 72) while they last.", FALSE, 0,
+			&dim_color);
 	}
 	ui_join_code_draw_text(126, 514, VR_OPTIONS_FOOTER + 8, "%a-button Change   %b-button Done", TRUE, 0, &text_color);
 }