screens/roomview.h tells whether Room View is on: forced by HANDS_ROOMVIEW, else
off when the passthrough shared state says disabled, else the last
RoomViewShown/Hidden event in a game, else off in a game and the construct's
setting outside one. `cutouts state` says which; `cutouts roomview auto|on|off`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Live test 2026-10-09: taps took several tries and the laser jumped between
hands. The owner's ray dropping for a moment (closing the fingers turns the
hand) handed the laser to the other raised hand at once, for good; a pinch
by the other hand pressed where the old owner's ray pointed; a pinch whose
ray had just dropped became a press at a stale spot, or only woke the
pointer.
- The owner keeps the laser through POINTER_RAY_GRACE (0.5 s) of its ray
being down; the other hand takes it by pinching, by raising its ray while
the owner's is down, or after the grace (pointer/helper/rayowner.h).
- A pinch on a ray that dropped within the grace, or is on its way up,
brings it back (fh_ray.h ShowForPinch) and is a ray pinch.
- Aim lock: the press goes along the ray from just before the fingers
began to close, after a pose along it has gone out; a tap that ends
before its press clicks there too.
- Each ray follows one tracked hand through a brief flip of ft-hands'
left/right call.
- Log: owner changes with the reason, the owner's ray down and back, and
one line per pinch saying what it clicked or why not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hands/handsmode.py starts and stops ft-camd, ft-hands and the hand view
(frametop-handview.service, when installed) by HANDS_MODE: frametop =
while the desktop runs (session hooks), always = wanted by
steamvr.service (and kept running through game pauses), off = the
default, only ft-handsctl on as before. ft-handsctl mode sets it;
missing ft-camd capabilities start nothing and say what to run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Turn a bare hand palm up toward your face and look at it: after 0.3 s a
small ring shows above the palm and follows it, like the Vision Pro's
home indicator. A thumb-index pinch tap by that hand runs
`ft-steam menu` (opens the Steam menu, or closes the dashboard); the
ring is blue while the pinch is held. ft-screens owns it
(screens/palmmenu.cpp) and reports that hand near while the ring shows,
so the pointer helper doesn't also gaze-click the pinch. The helper is
unchanged. HANDS_PALM_MENU=0 turns it off; `palmmenu on|off|state`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fork's CI published frametop-2.0.0-1 (Vibepollo 2.0.0 with Frametop's
changes, now including the HDR-off guard for SDR display streams). $BuildUrl
points at its sunshine.exe, so a PC needs only host/windows; the first CI
build (a84b6cfc) is upgraded like the hand-built one. Tested on the test PC:
over the first CI build, the setup downloaded the release, matched its
SHA-256, swapped it in and restarted Vibepollo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's note said Frametop didn't work on the 0.4 beta. Gaze reads
0.4's eye tracker layout now, and the missing taskbar in #15 wasn't the
beta's doing (0.4.5 has the same KWin and Plasma as 0.3.0), so the note
now says what the update changes and to run doctor.sh after it. The gaze
README covers Track Dominant Eye Only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The desktop is a KDE session, so it ran /etc/xdg/autostart/steam.desktop.
Steam is already running (the desktop starts from it), so `steam -silent`
only reached that client as a command line it ran (ExecCommandLine in its
console log). SteamOS 0.4 adds -vrdisable -deckard to that entry, for
Desktop Mode, where Plasma starts Steam itself. The session now hides it
with the other two. The marker goes to 2, so desktops that hid the first
two get Steam's copy once, and an entry someone brought back stays.
The Steam client's XCURSOR_THEME=steam reaches the desktop through its
environment, and KWin and the apps prefer it to the desktop's setting.
There was no such theme, so they fell back to Breeze; SteamOS 0.4's
holo-cursors adds one. The session sets the desktop's own theme (Breeze
unless changed in its System Settings) and leaves the size as it was.
ft-screens doesn't draw KWin's cursor, so this shows over remote access
and with the gamescope backend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Joins with hands-screens: ft-hands keeps 30 Hz for a middle pinch as well
as for a want-hz reader; GesturePublisher keeps both use() (replay dumps)
and write()'s middle-pinch slots; both test groups run in test-c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The driver's default role is the treadmill (4), hinted from Activate;
the helper's POINTER_ROLE defaults to treadmill. Needs the driver
reinstalled and a SteamVR restart to take effect.
- The "no hand role" release skips a device whose hint is the treadmill.
- Hand gestures: none begins on a side whose controller is in use, or by a
hand at a controller (by position); a held one ends, as lost, when its
side turns controller.
- The moving-controller release no longer applies while a bare hand's ray
drives the pointer (that ray goes only when its own side turns controller).
- update-check.py: the virtual controller's path, and the touch components.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The driver takes "role treadmill" and hints its role from Activate, so a
treadmill gets /user/treadmill (from gaze-first 87a6393); the compositor
binding has the /user/treadmill entries. The default role stays the right
hand until the helper side is ready. docs/design.md says why (Treadmill role).
Probes for frame-hands/notes/hand-interaction.md section 7:
- pointer/probe/lasertest (from gaze-first): who has the laser, claim time
- pointer/probe/ctlnoise: controller pose noise, activity level, charging
- vrprobe prints each device's activity level and charging
- input/vrws.py --touch prints touch and value components
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-layout reset reopens the profile in use (the custom arrangement with an
active profile) with use NAME, so its screens, hidden screens, remote
displays and apps all go back as the profile has them. Without one it runs
apply, as before. Meta+Shift+R and the Reset Screen Layout entry
(ft-layout-reset), the input relay's layout_reset action (a mapped mouse or
controller button), and the reset button on a screen's bar now run it.
Display Settings' Arrange now still only arranges.
Before, every quick reset ran apply, which put the desktop screens back but
left the profile's hidden screens, remote displays and apps alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving a profile records the remote displays connected then, with their
places and hidden state, the way it records the apps open then. Opening the
profile (use, open, desktop start), or arranging while it's the profile in
use, connects each of them whose host answers on Vibepollo's Web UI port (at
its address or its dongle's, as its route allows) and puts it back where it
was saved. A host that doesn't answer within 1.5 s is skipped, and its
displays stay disconnected. Like apps, displays the profile doesn't have are
left as they are: a profile connects, but never disconnects.
Before, a profile kept the places of every display that had one, connected
or not, and opening it never reconnected a display.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
install.sh's desktop step builds ft-stream and installs Remote Displays' menu entry.
uninstall.sh removes the entry and offers to delete the remote displays' pairings and
sign-ins with the other settings. docs/reference.md describes the app and the PC setup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A remote display is a screen of ft-screens whose picture comes from another computer,
streamed by Vibepollo with Moonlight's protocol. ft-screens (screens/remote.c) runs one
ft-stream per remote screen (101 and up) over a socket pair: ft-stream (stream/, GPLv3:
moonlight-common-c and moonlight-embedded's libgamestream, pinned) decodes on the iris
decoder into a ring of three RGBA buffers that ft-screens shows as a panel with a
screen's controls, layout, profiles, curve, pins and hand cutouts. Mouse, keys and scroll
go back to the host; a button's release goes through the stream it was pressed on, as
Vibepollo wants, and a press carried onto another remote panel moves the pointer there.
A panel out of sight gets 10 fps; the host's sound plays from one stream per host.
ft-stream signs in with an API token and the host's pinned public key
(stream/host.cpp), and chooses its path: a Steam Link dongle on the Frame's hotspot when
it answers, or the network (the layout's "route": auto, network or dongle). ft-layout
gets "remote" commands (list, add, connect, disconnect, host route and direct), starts
the connected displays with the desktop and keeps disconnected ones parked. ft-gaze and
ft-pointer know the remote panels. ft-screens now shuts its VR side down before its
streams and waits for SteamVR before exiting (a vrcompositor crash at quit, in standby).
docs/remote-displays.md has the design, the spikes and the test results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteamVR's laser and ComputeOverlayIntersection size an overlay's hit area
from its mouse scale, not its texture. At the default 1x1 every control
was hit as a square as tall as it is wide, so the grab bar (a 256x24
texture) caught clicks meant for the bottom tenth or so of the screen
above it.
MakeChrome now sets each control's mouse scale to its texture size. The
controls only read button and scroll events, never the mouse position,
so nothing else changes. Measured on the Frame (SteamOS 0.3.0 build
20260922) with a 0.2 m wide 256x24 overlay: a hit area 199 mm tall at
the default scale, 18 mm at 256x24 (the bar is drawn 18.8 mm tall); an
intersection mask doesn't change it. The pointer helper's measure command
gave the live grab bars square hit areas (0.197 x 0.197 m, 0.212 x
0.212 m) before the change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019u6dFg7wXBokn1TWD7Cg6X
reference.md's ft-screens keys paragraph, the relay's top docstring and
hazards.md's key releases section now say that a mouse button passed
through as a key in pointer mode goes to the screen the pointer is on,
not where typing goes, only while one shows and Frametop isn't paused,
that ft-screens lets its release through and releases it itself when
the pointer leaves the screens, its screen hides, or Frametop pauses,
and that a relay that starts releases mouse buttons too. hazards.md
adds that the button still reaches the virtual mouse, which gamescope
reads, and that a keyboard's media keys come from its Consumer Control
node, which stays ungrabbed, so they reach gamescope as well as the
desktop even while typing goes to the desktop. design.md says why the
buttons follow the pointer: as keys, a release was lost whenever typing
moved, and wlroots counts presses per button. reference.md names
scripts/test-relay-buttons.sh, and keys-test's wider scope.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With PR #26 ft-gaze reads the SteamOS 0.4.x beta's layout too, but
update-check.py still knew only stable's: on the beta it reported FAIL (warn
without gaze) and said to "update the offsets in gaze/ft-gaze.cpp", as in
issue #15's report, though the gaze worked.
It now tries both layouts with ft-gaze's own test (EyeFile::Detect: the
timestamp within 2 s of the clock and moving on, both set-1 directions unit
vectors), says which one it found, and fails only when neither fits, for
1.5 s so a blink or a moment with an eye lost doesn't fail it. The failure
says to add the new layout to EyeFile::Detect, and to check again after the
tracker's warm-up when the headset just went on. EYE_NEED is ft-gaze's
kNeed now (it was 0x1d3 against ft-gaze's 0x1f3, now 0x1f8). design.md's
update-check sentence and the offset table in gaze/tracker/findings.md say
there are two layouts.
Checked with made-up files: stable and beta ok, the fields moved by 3
bytes, and a direction always zero, FAIL, an eye lost for two samples ok,
an idle counter skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
design.md's notes on KWin's script engine get the quirk the watchdog from #35
is for: callDBus never calls back when a call fails (an error reply, the name
gone from the bus, or the 25-second timeout), and KWin only logs it. The long
poll's description says how the watchdog calls again (after 30 seconds, then
longer each time while no reply comes) and why the 30 seconds have to stay
above KWin's timeout. floating-windows.md's line on the poll says the script
also calls again after 30 seconds without an answer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frametop.conf.example said HANDS_SWAP_SIDES=0, and desktops.sh copies it on a
fresh install, so every new install forced ft-camd's side camera names and
turned the hand tracker's own side check off. Some SteamVR restarts swap those
names, and then hands land beside their cutouts and recordings are mislabelled.
The example now says auto. scripts/conf-migrate.sh, run by install.sh and
hands/rec/install.sh, replaces the old line only where it's still exactly as
the example wrote it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pause section of reference.md now says the relay releases a click
still held as it lets go of the 3D mouse, and lists
input/test/pause-buttons-test.py with the other pause test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two installers give ft-camd its capabilities: hands/run.sh install and the
Hand Recorder's (through hands/run.sh caps). #30 made hands/run.sh uninstall
take them back unconditionally, before it removed the units. With the
recorder installed too, its next session then stopped at "ft-camd needs its
capabilities", and so did ft-cutouts. The recorder's own uninstall, the one
hand-recorder.md points to, still left them on the binary.
One rule now: hands/run.sh uncaps takes them back only while neither the
Hand Recorder's menu entry nor the frametop-camd unit is installed, and says
which one keeps them otherwise. It checks through on_frame, in one call, so
it works from a PC too. hands/run.sh uninstall runs it after removing the
units, and hands/rec/install.sh uninstall after removing the entry, so each
stops counting itself. uninstall.sh removes both and always takes them back.
The docs say which commands ask for the password, and AGENTS.md says
uninstall.sh also has to undo file capabilities.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since b61b4d6, <file>.ft-bak is written only before the first repair. That
keeps the first config, but no repair after it had a backup, and the log
still said "(backup: <file>.ft-bak)" for each one. The repair writes 0 over
a moved panel's old screen number, so a backup is the only record of where
the panel was. Restoring the file the log named after a later repair also
threw away every Plasma change since the first one: panels, widgets, pinned
apps, wallpapers. Repeated repairs are the case b61b4d6 was written for.
Tried on a copy: after a first repair, a new top panel on screen 2 with
pinned apps, and a drop to one screen, the second repair moved that panel
and named a backup that had neither.
Before each repair the config now also goes to <file>.ft-bak.last (whole or
not at all, like <file>.ft-bak). <file>.ft-bak stays write-once. The log
names both, and the docstring and docs/design.md describe both. Both live
in ~/.config/frametop, which uninstall.sh already deletes with the
settings.
The new test repairs twice, with a widget changed between the runs, and
checks that .last holds the changed config, .ft-bak the first one, and the
log names both. test_repair checks that the first repair writes both and a
run with nothing to repair writes neither.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After 9c66475 and the previous commit, ft-layout's log is
$XDG_RUNTIME_DIR/frametop-layout.log, but report.sh still collected
/tmp/frametop-layout.log, so bug reports lost it, and reference.md still
named the /tmp path.
report.sh reads the new path (it already sets XDG_RUNTIME_DIR to
/run/user/<uid>). reference.md and the comment on RunLayout name the host's
runtime directory, which ft-screens and the desktop start share. The nested
desktop has its own, /run/user/<uid>/frametop, removed at every desktop start
and stop, and the log isn't there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The desktop has its own XDG_CONFIG_HOME (~/.config/frametop), and SteamVR's tools and OpenVR
read their path registry from there. So from a terminal in the desktop:
- pointer/driver/install.sh (install.sh step 4, every reinstall or update from Konsole) wrote
a new registry, ~/.config/frametop/openvr/openvrpaths.vrpath, with only our driver and
"runtime": null, and never registered the driver with SteamVR. That stray file then hid
SteamVR from every OpenVR program started in the desktop.
- scripts/update-check.py (and so doctor.sh and report.sh) reported OpenVR "can't connect to
SteamVR as a background app" (VRInitError_Init_PathRegistryNotFound, or
InstallationNotFound with the stray file) and "vrcmd --overlays lists no overlays". A user's
report of 2026-10-05 showed both, with SteamVR and Frametop's services fine.
- ft-layout's vrcmd calls (the gamescope backend) found no overlays.
They now run with XDG_CONFIG_HOME=~/.config: the driver installer (install, uninstall, probe,
aimhere), update-check.py (for all its checks: nothing there reads the desktop's own config),
report.sh's vrpathreg show, and ft-layout's vrcmd. report.sh doesn't set it for the whole
report, since session/fix-panels.py --check reads the desktop's Plasma config through it.
The driver installer also removes the stray registry (only vrpathreg's, with no runtime), and
update-check.py warns about one. And vrpathreg runs `xdg-open vrmonitor://driverinstalled` to
tell SteamVR's desktop monitor, which the Frame doesn't have: in the desktop that showed "could
not read file vrmonitor://driverinstalled" on every install. A stand-in xdg-open on its PATH
takes it, so install.sh no longer filters the step's output.
Tested in a fake HOME with a copy of SteamVR's registry, a stray one, and the desktop's
XDG_CONFIG_HOME: the new installer removes the stray file, registers the driver in the copy,
and never reaches xdg-open; the old one wrote the stray file, left the copy alone, and ran
xdg-open. update-check.py from that environment: OpenVR and vrcmd ok (the old one fails both);
its stray-registry warning fires on a seeded file. ft-layout's vrcmd: 119 overlays (was 0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each check starts two gdbus processes (about 5.5 ms of CPU each on the
Frame), so checking once a second cost about 1% of a core for as long as
the desktop ran. On SteamOS 0.3.0 native activation fails, so the watcher
always runs. A registry left behind now goes within 5 seconds instead of 1.
design.md also says where the watcher matters: started from the VR
launcher, the desktop runs in steam.service, which doesn't stop with it,
so keep-apps.sh and the unit's stop don't clean up there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts with experimental's pause_toggle, steam_menu and command: actions and its
AnnounceOverlay: both kept. The spin bindings join the Meta tap in the defaults, spinning
doesn't need pointer mode, and like other actions it does nothing while Frametop is paused.
AnnounceOverlay now sends through the PR's SendPointer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plasma 6.2.5 keeps a panel on a screen number and never moves one whose
number is past the screen count, so a taskbar saved on a spare output
(#18, lastScreen=8 with three screens) or on a screen a smaller layout
dropped stayed hidden. Before Plasma starts, session/fix-panels.py moves
such a panel and its tray's containment to screen 0 (the primary),
keeping its widgets, unless screen 0 already has a panel on that edge.
doctor.sh checks the panels' screens, and report.sh lists them with the
live outputs and panels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GetComponentStateForDevicePath with no input source handle fails for
every render model component while a VR game runs (checked 2026-10-03
with a game up: all 21 components of frame_controller_right). TipOffset
then fell back to the controller's pose, which aims 40 degrees above the
Frame controller's laser. In games, pointing at a screen's middle missed
it and pointing below it hit, so the new aim-to-laser only worked from
the bottom; the controls' reveal and pin/roll aim were off the same way.
GetComponentState still answers then, with the same tip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteamVR's own floating windows take the laser while a controller points
at them in a game and give it back when it points away. Frametop's
panels didn't: with the controllers left to the game (outside_games, the
default, or dashboard), they couldn't be clicked without the dashboard.
ft-screens now sets MakeOverlaysInteractiveIfVisible on a screen or
floating window while a hand controller's laser pose meets it, its
controls, or its popups (UpdateAim; curved screens hit on their
cylinder), and clears it 0.3 s after the aim leaves a wider margin. A
drag or a held button keeps it on. The keyboard, one overlay, uses
ComputeOverlayIntersection and now follows the mode when a game starts
or ends while it's open. This replaces the reset button's own aim zone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8 is about 0.2 degrees on a 3.4 m wide 3440-pixel screen 2 m away, so a
trigger press turned into a drag unless the hand was very still. 32
(about 0.9 degrees) felt much better in the headset (2026-10-03).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each desktop screen gets a reset button left of its bar (a reticle). It
puts every screen back in its layout around where you are now, like
Meta+Shift+R (ft-layout apply).
In a VR game the screens leave the controllers to the game (the
outside_games and dashboard modes), so a controller couldn't click any
of their controls. Aiming a hand controller at the reset button now sets
MakeOverlaysInteractiveIfVisible on that button's overlay alone, so the
trigger clicks it; the flag clears half a second after the aim leaves a
zone twice as wide, and the game gets the controllers back. The aim
comes from the laser poses ft-screens already reads to show the controls.
The ft-layout spawn is now RunLayout(cmd), shared with the arrange.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ft-screens "spin next|prev|<degrees>": every unpinned screen and
floating window turns together about a vertical axis through your
head (0.3 s, eased), so the next panel on the right or left comes to
straight ahead; the arrangement stays as it is. Taps during a spin
add to it, from where the panels are headed; grabbing a panel or
placing it (ft-layout, ft-floatd) takes it out of the spin
- when a spin settles, the panel in front gets the pointer (recenter),
typing (as after a click), and KWin's active window: its floating
window, or the top window on a screen (ft-floatd "front N", the KWin
script's activate-output). KWin's outputs follow the screens'
new places (ft-layout scale), as after a move
- the input relay: spin_next and spin_prev actions, Meta+Alt+Tab and
Meta+Alt+Shift+Tab by default; Frametop Input Settings lists them.
Not Meta+Tab: that's Cmd+Tab on a Mac reached through a remote
desktop like RustDesk, and the relay would take the Mac's app
switcher. Meta+Alt+Tab (Cmd+Option+Tab) is unused on macOS,
Windows, and KDE
Used on the Frame (SteamOS 0.3.0 build 20260922) with one screen and
three or four floating windows, through RustDesk to a Mac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
