The ray places it; the wrist's turn no longer tilts or turns it. Controllers
and the mouse still carry it rigidly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The handle (the laser's hit area, all outside the corner) is 1.8 buttons
wide, and the laser's corner glow reaches as far off the edges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The laser's aim at a corner gets the same white L a hand's pinch gets
(a third guide overlay, frametop.guide.laser), brightest while resizing.
The handle stays just outside the corner as the laser's hit area.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The line from the eyes through the palm lit panels off to the side of a palm
that wasn't facing them. The line is now the palm's own, along its normal,
with the margins seen from the palm. HANDS_GRAB_THAT_AIM=eyes brings the old
line back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spotwatch's "spin to N" still turned the whole ring after the merge, the panel in the
front slot with it, while next and prev bring one panel to the slot. With
SCREEN_SPIN_BRING on (the default) it now brings panel N there too; the ring turn stays
for SCREEN_SPIN_BRING=0 and for degrees. Docs: reference.md's spin, spotwatch.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Far stretch: both palms held out at the same far panel, then two fists
within 0.6 s, scales it round its centre (a floating window by pixels)
until a hand opens. Aims at the panel both hands' lines meet, or the
line through the palms' midpoint. Grab that holds its fire meanwhile, and
both hands are claimed from the pointer helper so a fist passing through
a pinch doesn't click or drag under a raised hand ray. Near misses log why.
- Reset button: a hand's tap shows "Reset the layout? Tap" under it for
5 s; a tap on it (or the button again) resets. A laser click resets at once.
- Lazy susan: with the other hand still or unseen, a looser shape, 18 cm
and 0.5 m/s count; any stroke rides out 2 frames off its shape; misses
log open/tilt/lead against their limits.
- Pointer helper: the two-hand gaze drag no longer logs "with the right
button" (it is the left alone).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Throws: a carry's let-go takes the fastest outward stretch in the last
150 ms before the fingers opened (a hand slows as it lets go), at 0.45 m/s
within 70 deg (was 0.55 within 60, the last 100 ms only). Every let-go
moving 0.3 m/s or more is logged.
Sent away (thrown, flicked, pushed), a panel goes along that line out as
far and as big as it was before a hand or "grab that" took it, not back
to its old spot (layouts are for that). Put down past 1.2 m, that's its
distance from then on; never taken, it flies to 3 m as before. The front
slot's screen leaves the slot the same way.
The bar's buttons sit a button's width apart, and the reset button
resets the layout on a tap (HANDS_RESET_HOLD_MS=0; 500 before).
Grab that's glow: ft_pointer's laser sits on the panel with the cursor
all the time (the 3D mouse, the gaze), so "no glow under a laser" kept
the glow off that panel. The pointer helper now tells ft-screens which
hand rays are up ("ray L R", on change and every second); our laser
counts as one only then. Controllers' lasers count as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The "no glow under a laser" rule counted any overlay event as a laser, and
SteamVR sends panels events from the headset and from no device at all, so
the glow stayed off nearly everywhere and only the last one drawn (the
32:9 remote) was ever seen. Only a controller's events (the pointer
helper's 3D mouse and hand ray are ft_pointer controllers) count now. The
log says when a laser holds the glow back, and the glow's size per panel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 5120x1440 remote in the front slot was held to 100 deg wide (51 px/deg,
hard to read). Panels 3:1 or wider may now go to SCREEN_SPIN_MAX_WIDE_DEG
(150) instead, and their curve may wrap that far round you.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User 2026-10-11: "the lazy susan can be considered to be in use and remain active until I
move it out of its spot manually through either the eye tracking or grabbing it with
hand-tracking". Looking away no longer sends it home (SCREEN_FRONT_AWAY=1 brings that
back, with the keep-by-use); a gaze-pinch drag, a hand grab, a push or grab that
releases it, and the next stroke sends it home.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user's 2026-10-11 test of gaze + hands:
- Two-hand drag as they meant it: hold one hand's pinch (nudge it into place), tap
with the other: index presses the left button where the pointer is, middle the
right; the first hand drags (POINTER_GAZE_TWO_HAND). The tap-first right-drag is
off by default (it could catch a tap followed by a nudge).
- 9 held pinches ended as "hand tracking stopped (1.5 s)" while ft-hands still held
them: 4 s with nothing pressed, 2.5 s with a press, and the line now says the file's
seq and ages to find the cause.
- A gaze pinch held back ends without a click when the hand closes into a fist
(ft-hands' grip may not take it past 0.1 s now); a pinch begun while the palm
moves faster than 0.6 m/s (a stroke) doesn't click (POINTER_PINCH_MAX_SPEED).
- Three-finger scroll: 3.5 cm and 0.4 s to join.
- Gaze mode hides the dot and collapses its laser for 1.5 s after each key
(POINTER_TYPING_HIDE): it sat big and bright on the text being typed.
- The front slot's keep counts clicks, scrolls, drags and keys, not the gaze pointer
passing over it (every brought screen was kept 3 s in).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User 2026-10-11, gaze + hands becoming the main input:
- Right-drag: an index tap on one hand, then an index pinch on the other within 1 s,
holds the right button at the gaze lock; that hand drags until it ends
(POINTER_GAZE_RIGHT_DRAG). It replaces the chord (POINTER_GAZE_CHORD=0).
- Three-finger scroll: the thumb on both the index and middle tips (3 cm) scrolls where
you looked as the hand moves, like the ray's middle-pinch stick (POINTER_GAZE_SCROLL).
- The fist guard (a hand closing into a fist: no click) now covers gaze and chord
pinches too, from each hand's own frame (not the ray's).
- A hand's laser shows only its dot: the beam starts just short of it, as the eye's
does, held still through a carry (POINTER_RAY_BEAM=0; SteamVR's beam width is one
global setting it reads only at start).
- The front slot gives a screen up to manual controls: grab that, a push, flick or
throw, or a carry by hand or laser releases it (it moves like any other and stays
where you put it); the next stroke puts it home and brings the next.
- The lazy susan counts only palm-led strokes (the return stroke does nothing; the
other hand palms the other way; HANDS_SWEEP_BACK=1 for both).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user's proposals (2026-10-11): the lazy susan and grab that were two systems that
fought over a screen (grab that took one the susan brought: 2.3 m wide at 1.5 m).
Now one front slot, one screen at a time (the one there goes home first):
- reading distance (the lazy susan, 1 m) and arm's reach (grab that), both readable
(ReadableSize, the same view at either); grab that at the one there pulls it closer;
- a push, flick or throw steps it back: arm's reach -> reading -> home;
- one home per screen: every dismissal returns it there; moved by hand or laser it
lives where you put it; resized in the slot it keeps that view next time;
- turned away from it goes home, unless used 3 s (kept; SCREEN_FRONT_KEEP_MS);
- a strip of the screens' names, left to right as they live, for 2.5 s after each
stroke, the one in the slot lit and the next each way marked (keyboard.cpp's font,
now DrawLabel for other panels too);
- SCREEN_UI_SCALE for a remote display's own UI scale (a Mac's Retina: 2);
- the reset button needs a 0.5 s hold, laser or finger (a stray poke reset the layout;
HANDS_RESET_HOLD_MS).
grab that's remembered widths, send-away widths and view fit go (the slot sizes it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lazy susan (the third live test): a brought screen comes 1 m ahead, as big as its
resolution needs for text to read: its logical pixels (KWin's scale undone, now passed
to vr.cpp) at 26 px/deg, at most 100x55 deg keeping its shape, at least 40 deg wide,
curved round you past 60 deg (grabthat::ReadableSize; SCREEN_SPIN_DISTANCE, _PPD,
_MAX_DEG, _MAX_TALL_DEG). Its curve comes back on the way home. The strokes cycled
104, 1, 104 with three screens up: a screen still flying home read as straight ahead;
each one is now taken where it lives (home, or where it's going).
Accidental clicks: most were pinches with no laser up pressing where the pointer was
last left ("no ray up: a press where the pointer is", a dozen in two minutes). With
gaze off they now only wake the pointer (POINTER_PINCH_NO_RAY=1: the old air mouse).
The hand's laser shows only with the thumb and index within 6 cm (gone past 10;
POINTER_RAY_READY), and a fist drops it: fh_ray's "ready" and "open" tests, the fist
read as the index and the other three both curled under 1.1 (FistCurl; measured on 47k
flagged hands: the other three alone read as a fist in a fifth of the pinches). A ray
pinch made while the hand closes into a fist doesn't click. Input Settings switches for
all three (POINTER_ ones reload the helper).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lazy susan (the sweep, Meta+Alt+Tab) no longer turns the room: the next screen
round, by where each one lives, comes in front of you at a distance it spans at most
60 deg of the view at, and takes the typing. The next stroke sends it home and brings
the one after; pushed, flicked or thrown it goes home; turned away from for 0.6 s it
goes home; moved by hand or laser it stays. SCREEN_SPIN_BRING=0 turns the room again
(Input Settings switch too).
Sweeps were hard to make (counted strokes came at 30-35 cm, just over the 30 asked):
25 cm in 0.5 s, the hand open 1.4, tilted up to ~45 deg off its edge, and hovering near
a screen counts (the brought one sits right there). The flags replayed: one stroke in
49 flags of other use.
Grab that brought a remote display back at the 1.62 m it was resized to, 0.45 m ahead:
it filled the view and every poke went through it (both flags of the second live test).
A wide one now comes as far out as it spans at most 75 deg (HANDS_GRAB_THAT_VIEW_DEG).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-11: "make every interaction able to be enabled or disabled".
The Hands page gets a "Hand interactions" section: 24 switches in four
groups (tapping, moving screens, gestures, feedback), each writing its
frametop.conf key and telling ft-screens at once ("handsetting KEY 0|1",
SetHandSetting; number-valued ones go back to their defaults when switched
on). They follow the direct touch switch above.
The command is checked before the helper's "hands L R allowed" word: that
pattern matched "handsetting X 0" too and scrambled the hands word.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-11: "paw with a mostly horizontal hand back and forth like
pushing branches out of the way, left or right, and lazy susan through
windows and displays"; decided: one panel per stroke; the palm or the back
of the hand leading, never a flat hand; every stroke counts, but a large one.
An open hand on its edge (fingers open, the palm's normal within 0.55 of
level and along the motion), away from every panel, 18 cm or more in front
of the eyes, from 50 cm under them to 15 over, sweeping 30 cm sideways
within 0.6 s, drifting at most 15 cm: Event::Spin -> the spin's next (swept
left: the panel on the right comes) or prev. Not while typing, pressing,
holding, pinching or arming grab that. HANDS_SWEEP.
Tests: 40 cm in 0.4 s either way spins once; flat, slow (1.5 s), short
(15 cm) don't; back and forth spins each stroke. Flag replays: no strokes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 11:00 live test:
- The finger's line overshot on the native screens: the screen sits low,
the hand comes up steeply, and the line from a tip a few cm out met the
screen 2-4 cm ahead of the drawn fingertip (the flags' clips show the dot
there). The aim is now the tip's path: where it's heading as it comes in,
carried on to the surface, at most 1.2 times its distance off the tip's
own point; still or slow, the tip's own point (HANDS_TAP_AIM=path|finger|
straight).
- Slow deliberate pushes on the remote displays were refused (too slow,
reaching through): once the aim has frozen on the way in, a pointing
finger pushed 1 cm through the panel (on its way in: within 5 cm, 10
frames) taps at the aim, whatever its speed or angle (HANDS_LOCK_PUSH).
- A hand at a corner's zone shows the corner's glow before the tap dot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 10:49 live test:
- Taps were offset from where the user aimed, most at the edges: the tip's
own point (straight in from it) sits short of where the finger points when
it comes in at 30-50 degrees, by the tip's distance times that angle's
tangent (about 1.3 cm with the aim locked 1.5 cm out). The aim, the dot,
the cursor and the press now use the line from the index's middle joint
through its tip, met with the panel (at most 4 cm off; HANDS_FINGER_RAY).
- With a hand near, the grown buttons overlapped the screen: the bar's row
sits below the screen by the taller of the bar and a button.
- The "grab that" glow doesn't show on a panel a laser is on (the gesture
still works there).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-11: after arranging a window it's almost impossible to keep it
exactly where it was: the hand moves a little as the fingers (or the whole
hand) let go. Only the let-go is treated differently, so carrying has no
added smoothing or latency:
- The panel stops following once the fingers open 8 mm past their tightest
this hold (a fist: its curl 0.3), or three times the reading's own wobble,
on two frames running, and goes back to where it was 70 ms before the
opening first showed; closed again, it follows on from there (the carry
or the two hands' stretch starts afresh). HANDS_RELEASE_FREEZE.
- A slow let-go the freeze didn't catch: back to where it was 120 ms before
(Event::Settle; HANDS_RELEASE_REWIND_MS). Throws are left alone.
- The same hand doesn't take the panel it let go of again within 250 ms.
Test: a let-go with the hand sagging 2 cm leaves the panel 1.3 mm from where
it was held (14.7 without). Flag replays: 2 pinch freezes, 2 fist ones.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tapping, after the 2026-10-11 10:10 live test:
- A soft white dot on the panel where a tap would land (Guide::Touch,
HANDS_TOUCH_DOT): smaller and brighter as the fingertip comes in, lit with
a ring while pressed; it shows with the cutouts off.
- The aim locks 1.5 cm out (HANDS_TAP_LOCK): the press lands where the tip
was as it came within lockOut, not where its jitter or slide put it on
crossing; the cursor holds there too.
- A press by motion (HANDS_MOTION_PRESS): a tip that came in at 0.25 m/s or
more, 3 cm or more, and stopped or turned back within 1.5 cm of the surface
presses there (the depth reads short of the touch).
- A hand at a panel makes its round buttons at least 2.4 cm across
(HANDS_BUTTON_SIZE), and they take a fingertip or pinch 2.5 cm round
(HANDS_BUTTON_REACH, was 1.5).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 10:10 live test: pokes went down at one spot and up
10-30 cm of pixels away (the tip slides down the panel as the finger curls
back off it), so apps saw a drag. A press doesn't drag in its first 250 ms
(HANDS_TAP_MS). Two-finger scrolls started three times and ended at once:
a swipe in the air drifts off the surface; it now ends only 5 cm off, the
fingers 10 cm apart, or not two fingers for 6 frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 09:43 live test:
- Two fists anywhere near one panel (palms 30 cm in front to 15 behind, 30 cm
round it) resize it by their shape alone (HANDS_FIST_PAIR): ft-hands' grip
only begins on a hand seen open just before, at the panel.
- Two fingers (index and middle straight, together, level; ring and little
curled) touching the content: a swipe scrolls (a notch per 1.2 cm, the
content following), a lift unmoved clicks, held still they act as a poke
(HANDS_TWO_FINGER_SCROLL).
- Stuck presses: a press isn't held through a tracking loss (only grabs are);
a poke lets go as the index curls; a pinch-click as the tips are seen 4.5 cm
apart; either held still 1.5 s (HANDS_PRESS_STILL_MS).
- The hand ray hides within 22 cm in front of a panel and for 0.8 s after
(HANDS_RAY_HIDE_REACH); the pointer helper ignores a ray-less pinch there
without the gaze (it woke the laser).
- The push/flick hint glow is off (HANDS_PUSH_GLOW): it covered near panels
the palm faced while clicking.
- Grab that gives panels that look small up to 10 deg more aim margin
(HANDS_GRAB_THAT_SMALL).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 09:25 live test:
- A fist only resizes (HANDS_FIST_RESIZE_ONLY, default on): one fist holds the
panel still as an anchor; two hands with a fist among them scale it about
its centre, no move or turn. Pinches still carry and turn.
- Throws need 0.55 m/s (was 1.0): the user's releases were 0.62-0.96 m/s.
- Grab that brings a panel back at the width it had when it was last sent
away from arm's reach (g_nearWidth), not grab that's maxWidth.
- A natural reach arms grab that: palmArm 0.2 (was 0.45), aimFacing 80 deg
(was 55), aim margin 10 deg (was 6). Flag replays: 11 fires (was 8).
- Curve: a curved screen's radius follows its distance from the head after
every move, and it bends at most SCREEN_CURVE_MAX_DEG (100) round: one
curved up close and thrown 3 m out at full width had wrapped into a
cylinder.
- The pointer helper releases its laser while a hand touches a panel
(ft-screens' near claim): the laser from the head to the fingertip's
cursor got in the way and held the button, refusing pokes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fifth live test (2026-10-10 22:55): grab that brought the desktop in, it was
carried and thrown 3 m out, then five gestures at it or the remote displays
logged "no panel there". The open palm is held out below the line of sight,
so the line from the eyes through it runs 12-15 deg under where the head
points (the user's flag 150115). Since the fourth live test the line had to
meet a panel exactly, and a thrown 1.35 m tall desktop 3 m out is only
12.7 deg each way, so the line passed just under it. One of the flag's two
fires misses such a panel that way.
Pick now takes the panel the line meets (unchanged, nearest wins), else the
one whose outline it passes nearest within HANDS_GRAB_THAT_MARGIN_DEG
(6 deg seen from the eyes; 0 = the old rule). The gesture itself is
unchanged, so it stays as hard to trigger, and held, pinned, pressed and
near panels are still never taken. "no panel there" now logs where the line
went relative to the head and the nearest panel it may take, with its angle.
Test: TestGrabThatAfterThrow runs the user's sequence with synthetic hands:
grab that brings the desktop in, a bar carry is thrown out and up at
1.3 m/s, then grab that at it again (missed by 0.8 deg with the old rule,
taken now), a remote display after that, precision cases, and the flag's
two fires (2 of 2 now, 1 with the old rule).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example config gets HANDS_FIST_REACH, HANDS_JOIN_REACH, HANDS_BUTTON_REACH,
HANDS_PINCH_BUTTONS, HANDS_EDGE_GLOW, HANDS_GRAB_THAT_FROM, HANDS_GRAB_THAT_MARGIN and
HANDS_GRAB_THAT_FACING; hands/README.md and docs/reference.md describe them, the white
halo, the corner, edge and button glows, and that near panels go away by push, flick or
throw now, not by grab that. `handinput state` counts joins, pinches that waited to be
fists, and buttons pinched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fourth live test: "The glow is too powerful. It should be a white glow and far more
subtle. It should only be around the outside of the window." And for resizing, "almost
like a cutout of the soft glow ... just around the edges and corners".
- The target glow (grab that, and dimmer a push or flick) is a white halo outside the
panel's edges, about 1.3 degrees wide, at most half opaque, clear over the content,
in the panel's plane after it (frametop.guide.target).
- Each hand's guide (handinput's Guide) as the same halo: an L round the corner a pinch
would resize from, a segment along the edge it would move by, a ring round a button;
brighter closer, lit while held (frametop.guide.hand.left/right). Shown with the
cutouts off. With HANDS_EDGE_GLOW (default) the resize handle no longer follows hands.
- softglow.h: the textures as pure code, tested.
- Grab that takes only panels shown, unpinned, not held by a laser or a hand, not pressed,
and (grabthat's nearest) farther than 0.8 m; a fist or pinch waits on it only while it
aims at one of those; a send-away quiets it for 1 s; its claim likewise.
- Settings: HANDS_JOIN_REACH, HANDS_FIST_REACH, HANDS_BUTTON_REACH, HANDS_PINCH_BUTTONS,
HANDS_EDGE_GLOW, HANDS_GRAB_THAT_FROM, HANDS_GRAB_THAT_MARGIN, HANDS_GRAB_THAT_FACING.
The log names a pressed control.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A glow on the panel a gesture under way would take: "grab that" (blue, fading in as the open
hand held out to it arms, by its own Pick) or a push or flick (warm, dimmer: an open palm close
in front of it, handinput's throwAims). One overlay, frametop.guide.target, sized and curved to
the panel, 4 mm in front of it: an edge glow and a faint tint, drawn with cutouts off.
HANDS_TARGET_GLOW.
- Sent away (thrown, flicked, pushed, or grab that's send-back) a panel keeps its width; one that
grab that shrank to come to arm's reach and that's still that wide gets its width back
(HANDS_SEND_AWAY_WIDTH=restore, the default; keep: never). In the third live test the 2.5 m
desktop came at 0.6 m and was thrown 3 m out at 0.6.
- A hand holding a panel or KWin's button holds on while the pointer helper's word says its
side is "off" (not a controller's) for up to holdLostMs: the word flickered off 24 times in the
session's 7 minutes and dropped three holds.
- The palm menu gets "grab that" as busy and whether the hand is at a panel (near input's near,
or HandAtPanel within 17.5 cm and 10 cm round a panel).
- Settings: HANDS_PUSH, HANDS_PUSH_SPEED, HANDS_STRETCH_TURN, HANDS_TRACK_GATE, HANDS_TARGET_GLOW,
HANDS_SEND_AWAY_WIDTH; `handinput state` and `palmmenu state` count the new things.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A second hand taking a panel the first already holds (a stretch) by a
corner moved the resize handle there; it stays where it is now. And the
controls' comment is back over UpdateControls, UpdateCorner's over it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A hand pinching the resize handle after it moved to another corner
(a laser or the gaze took it there) resized from the bottom right:
handinput read the tab as that corner always. It takes the corner
from where the handle is now.
- A floating window's last resize can be answered after the release:
its opposite corner stays kept for half a second after a resize ends,
not reset to the top left at once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session/frametop.conf.example: HANDS_POKE_TRAVEL=0.017, the card
pinch's fingers, and the new HANDS_HOLD_LOST_MS, HANDS_FLICK,
HANDS_FLICK_SPEED, HANDS_FLICK_OPENING, HANDS_THROW, HANDS_THROW_SPEED,
HANDS_THROW_DISTANCE and POINTER_GAZE_CONTROL_HOLD. scripts/conf-migrate.sh
moves the example's exact HANDS_POKE_TRAVEL=0.025 line (hands-touch,
never released) to 0.017. hands/README.md's near input section,
docs/reference.md's .resize, settings and handinput state, and the
gesture tables at the top of handinput.h and vr.cpp say what each
gesture does now and what wins. The README notes that the hand glows
are drawn into the cutout buffer, so they don't show with the cutouts
off (the default since the drawn hands).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-10. Both send the panel 3 m from the eyes
(HANDS_THROW_DISTANCE), facing them, over 400 ms easing out; "grab that"
brings it back from there.
- Flick (HANDS_FLICK): a hand at a panel (its palm within 12 cm in
front of it or 4 cm behind, over it, and nearly still there: under
0.6 m/s) flicks out: away from the eyes (within 40 deg) at 1.2 m/s or
more (HANDS_FLICK_SPEED, a fit over 100 ms), 12 cm or more from there
within 200 ms, the fingers open (mean extension 1.5 or more) and
opening on the way by 0.15 (HANDS_FLICK_OPENING; 0 lets an open hand
pushed out flick too). The panel goes out along the line from the
eyes through its centre. Not a poke (a pointing hand's fingers read
about 1.3; an open hand coming through a panel faster than 0.8 m/s
doesn't poke it), not a wave (the direction), not "grab that" (its
pull back is toward the eyes). Without the opening rule the
recordings' probe moves, with a panel 0.3-0.45 m ahead, flicked 6
times in 11 minutes; with it once. None in the 13 "grab that" flags.
- Card throw (HANDS_THROW): a carry by one hand (a card, a fist, a pinch
on the border, bar or title; not a resize or two hands) let go moving
at 1.0 m/s or more (HANDS_THROW_SPEED; a fit over the 100 ms before
the fingers opened), within 60 deg of away from the eyes: the panel
goes on along that line to 3 m from the eyes. A slower release leaves
it where it is. The velocity's track starts afresh when the hand comes
back from a gap, and a flick doesn't reach back over one.
- vr.cpp animates a throw like a "grab that" move (ease-out), and
forgets that the panel came by "grab that": the throw is its place
now, so the gesture brings it back and sends it back there.
- "grab that" finds a panel along the palm's line within 3 deg seen
from the eyes (or 10 cm), so a 0.6 m panel 3 m out isn't missed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-10: Vision Pro-style resize from all four corners. Only the
bottom right worked by laser (the .resize handle), and a resize kept the
screen centred.
- One resize handle per panel still (SteamVR's 128 overlays are tight:
a third remote display once didn't fit), but it moves to the corner
being aimed at (vr.cpp UpdateCorner): a laser's ray (a controller's,
or the pointer helper's device, which the gaze and the hand ray
drive) landing within a fifth of the screen's height (5-20 cm) of a
corner, or a hand's pinch at one (handinput's corners()). Its texture
turns for that corner, it brightens the closer the aim (from the
controls' idle 0.55 to 1), and it's lit blue while resizing. With no
aim it lingers 0.4 s, then goes back to the bottom right.
- Any corner resizes and the opposite one stays put: screens and
remote displays keep their aspect (the corner follows the diagonal,
ResizeFromCorner, on the cylinder for curved ones: SurfacePose),
floating windows take the pixels from that corner (ft-floatd's new
size is placed keeping the opposite corner, keepX/keepY; every corner
of a window resizes now, the title bar's ends included).
- Near hands: a pinch at any corner resizes the same way (the carry
now gives the pose as well as the width).
- Gaze pinches (gazehand.h): with the pointer on a screen's resize
handle or grab bar, the pinch is a press after 0.12 s still
(POINTER_GAZE_CONTROL_HOLD), not 0.5: nothing to click there, and the
hand drags the resize. The chrome overlays are announced to the
helper as they're made, so its ray and the gaze find a new remote
display's handle at once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Losing the hand for 250 ms (or ft-hands ending its pinch as lost) dropped
whatever it held. Now a pinch, card or fist grab, and a poke's or a
pinch's press in KWin, last until a clear release: the hand seen with
its gesture up and its fingers open (thumb to index tip over 3.5 cm; a
fist's fingers over 1.35 extended) on 2 hands frames in a row, or the
gesture up for 6 frames whatever the fingers say. A pinch that
flickers up for a frame or two in fast motion, or that ft-hands ended
as lost while the hand comes back still pinched, holds on. Unseen for
1.75 s (HANDS_HOLD_LOST_MS), it ends where it is.
While unseen (or on a frame whose point jumped more than 12 cm, up to
2 in a row: a glitch), the carried point coasts on the hand's last
velocity (a fit over 110 ms), slowing to a stop over 150 ms. When the
hand is back, the smoothing restarts there moving at the gap's average
velocity (fh_smooth Point::Seed, new) and the panel blends from the
coast to it (smoothstep over 100 ms, longer for more than 0.6 m/s on
average). The raw track starts afresh, so a throw's release velocity
never sees the gap.
Synthetic carries at 0.6 m/s losing the hand for 3, 6 and 10 frames:
held, the panel's largest step 12-17 mm a tick (6.7 moving; 13-24 with
3 mm of noise), back under the hand after. Two glitch frames 30 cm off:
11 mm. Release now comes 2 frames (67 ms) after the fingers open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The border band reached 4 cm in front of a panel, the touch zone 7.5: a
pinch 5-7.5 cm out by an edge was claimed (the helper left it) but took
nothing. The band's depth is now the zone's (at least 4 cm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session/frametop.conf.example: HANDS_NEAR_REACH=0.075, the new
HANDS_POKE_TRAVEL, HANDS_GRAB_BAND, HANDS_PINCH_CLICK, HANDS_CARD_PINCH and
HANDS_FIST_GRAB. scripts/conf-migrate.sh moves the example's exact 0.30
line (hands-reach, never released) to 0.075. hands/README.md's near input
section, docs/reference.md's settings and handinput state, and the
gesture table at the top of handinput.h and vr.cpp say what each gesture
does near a panel and what wins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The near report to the pointer helper ("near <ns> <L> <R>") was one flag
for two things: the hand's ray hides, and its pinches and grips are
ft-screens'. With the 0.075 m zone, its 2.5 cm leave margin and 200 ms
linger (there so the ray doesn't flicker at the edge) would still have
swallowed pinches made just outside the zone, the live test's complaint.
Now ft-screens says 1 when the hand is within the zone itself (or
pressing, grabbing, or waiting on "grab that"): its gestures are ours; 2
in the margin: the ray stays hidden, the gestures are the helper's (ray or
gaze); 0 otherwise. The helper takes 1 as the claim and 1 or 2 as near for
the ray; an older helper takes 2 as 1, as before. ft-screens' own pinch
and fist begins need the zone itself too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the live test of 2026-10-10 "grab that" fired three times, each
bringing another remote display to arm's reach (they stacked up in one
spot), and never sent one back: a panel had to be within 15 cm of where
it came to go back.
Now (grabthat::Decide): at a panel that came by the gesture and is within
arm's reach (0.45 + 0.35 m of the eyes), wherever it was moved about
there, it goes back where it was; at a panel within arm's reach that
didn't come by it, it centres it at arm's reach (and the gesture again
sends it back); a panel farther away comes as before. One at a time: a
panel that came before and is still where it came goes back as another
one comes.
The touch zone and the fist grab don't take the gesture's hand any more
(the previous commit: a fist or pinch on a hand the gesture has waits on
it). The user's flag replayed through hand input and the detector, with a
panel 0.35-0.45 m ahead where the gesture brings one, still fires twice,
with no press or grab.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the live test of 2026-10-10 (handinput.h has the rules and numbers):
- The touch zone (HANDS_NEAR_REACH) is 0.075 m (was 0.30: "1 foot is way
too far, it should be closer to 3 inches"). A hand stays near until
2.5 cm farther out, then 200 ms; a hand resting low by the body is near
only within 4 cm. Beyond the zone a pinch is the ray's or the gaze's.
ft-hands is asked for 30 Hz while a hand is within 0.30 m of a panel, so
a tap from outside the zone isn't seen at a still hand's 15 Hz.
- Pokes: a decisive tap (2.5 cm in on average over the last 300 ms, on its
way in on the frame before too, 0.10 m/s or faster) presses on its first
frame past the surface; others still need two. The finger's angle is
taken knuckle to tip (the 2 cm last bone turned 15-20 degrees with a few
mm of noise), the index counts as straight from 1.5 (was 1.6), a new
press needs the tip back 1 cm out (a noisy pull-back pressed twice), and
a finger past the surface more than 5 frames or 4 cm deep is reaching
through. On noisy synthetic taps (3-6 mm of hand noise) 50-56 of 60
shallow taps (0.5 cm past) press, against 34-36 with the old rules; a
still finger 1.5-2.5 cm out presses no more often than before.
- Logs: each press and release (panel, place, depth, speed, frames), each
poke attempt that crossed the surface without a press with the rules
that held it back, each grab and its end, printed by ft-screens as
"hands: ...". handinput state counts them.
- Pinch grabs test where the thumb and index meet (the gestures file's
point is the knuckles, 5-8 cm behind them, which the 3 cm deep zones
rarely had): a band 6 cm outside to 2.5 cm inside the border, 4 cm deep
either side (HANDS_GRAB_BAND); corners (6 cm) resize, the rest moves.
The glow shows it before the pinch: amber at a corner, blue on the
border, green for a card pinch.
- A near pinch on the content clicks where the fingertip's cursor was (or
where the fingers meet), held while the pinch is (HANDS_PINCH_CLICK).
- Card pinch (HANDS_CARD_PINCH): the thumb and index either side of the
panel's plane at its border, the hand along it; the panel goes rigidly
with the hand, position and full turn, past 1 cm and 3 degrees.
- Fist grab (HANDS_FIST_GRAB): a grip begun with the palm or knuckles at a
panel carries it and turns it with the wrist; two fists stretch it. A
fist or pinch on a hand "grab that" has waits until it fires or lets the
hand go (at most 0.6 s): on the user's flag, with a panel where the
gesture brings one, a pinch-click on the closing hand stole the second
fire before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the live test of 2026-10-10 the cursor followed the fingertip but taps
on remote displays didn't click. ft-screens' log shows the pokes did reach
the streams (remote 104 got 10 button downs and ups, 100-200 ms each), so
the press itself wasn't the only problem: SteamVR's laser moves (the
pointer helper's head or gaze pointer, a controller's laser) on the same
panel still went to KWin or to the remote host between the hand's press
and release, so the host saw a drag from the fingertip to the other
pointer and back, not a click, and while the finger hovered the host's
pointer jumped between the two.
Now, while a hand holds the button, SteamVR's mouse moves go nowhere, and
on a panel whose pointer a fingertip moved within the last 250 ms
(kHandPointerMs), neither do its moves nor its focus leaving.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflict: docs/reference.md's hands settings line (typing's POINTER_TYPING_*
keys next to reach's HANDS_NEAR_REACH, HANDS_GRAB_ROTATE* and
HANDS_GRAB_THAT*): both kept.
Typing and grab that: a hand typing holds back (the helper's "typing" word)
doesn't arm or fire "grab that" either, so it never claims a fist as near;
vr.cpp's UpdateGrabThat leaves it out, as Poke and Grab already do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hold an open hand out to a screen, remote display or floating window,
palm to it, then pull the arm back making a fist: the panel comes over
250 ms to 45 cm ahead of the eyes, at the fist's height, facing you
(user, 2026-10-10). Screens wider than HANDS_GRAB_THAT_WIDTH (0.6 m)
come at that width. The same gesture at a panel that came, still where
it came to, sends it back; the reset button sends floating windows back
(the layout reset places screens and remote displays itself).
screens/grabthat.{h,cpp}: the detector (pure logic): open fingers, palm
ahead and the wrist 28 cm out for 0.2 s arm it; the fingers closing with
the palm still to the panel, then on one frame a fist, the wrist 8 cm
back within 0.5 s and the palm turned over fire it. The panel is the one
the eye-through-palm line meets, else the one the head points at. From
the closing (thumb clear of the index: not a pinch) to 0.4 s after the
fire the hand counts as near, so the helper doesn't take the fist as a
gaze grip. Hands frames carry the head pose at capture now
(Hands::captureHead).
Tests replay the user's flag (fires twice, right hand, at -11.35 and
-5.89 s; expected -11.4 and -5.9), and the other 12 flags and 15 minutes
of recordings fire nothing (1 of 143 pinches, 3 of 12 grips begin on a
claimed hand); synthetic waves, pushes, fists without the pull, slow
pulls, a hand turned over first, held-out pinches, a drop to the lap and
typing don't fire. handinput-test --grabthat FLAG_DIR... scans flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While one hand carries a panel, the hand's turn since the grab turns it
too, 1:1 about yaw, pitch and roll (user, 2026-10-10: line displays up).
The turn comes from the wrist and the index and little knuckles, which
hold still while the fingers pinch, smoothed with fh_smooth's Direction
filter, relative to facing the head across the floor from the hand, and
is applied on top of the panel facing the head: carrying a panel round
you still only keeps it facing you. Nothing turns until the hand has
turned HANDS_GRAB_ROTATE_DEADZONE (5 deg), then it turns from there
without a jump; within 2.5 deg of level it snaps level. Let go, it keeps
the turn, and the next grab starts from it. Corner and tab grabs only
resize; a stretch is unchanged and the hand left holding starts afresh.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens takes the pointer helper's "typing <L> <R>" (lapses after 3 s):
such a hand presses nothing, grabs nothing, and the cursor doesn't follow its
fingertip; a press or grab already held carries on. handinput's state counts
them as held back (typing).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The near claim reaches 0.30 m (was 0.15): within it the pointer helper
hides that hand's ray and leaves its pinches alone, and the index pokes
(user, 2026-10-10: "near = poke within a foot"). A hand near a panel
stays near until 5 cm farther out, then 250 ms on, so the ray doesn't
flicker at the zone's edge. A hand resting low by the body (the poke's
resting rule) is near only within 0.15 m, so a low panel within reach
doesn't take gaze pinches from the lap. The poke's rules are unchanged:
on the PR #3 desk and touch replays the presses are the same as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Spotlight-like search pill (docs/spotwatch.md). Meta+Space (the input
relay's new spotwatch action, bindable to mouse and controller buttons
too) opens a faux-glass pill straight ahead of the head. Typing finds
apps and remote displays ({host} - {display}). Enter brings a running
app's most recent window to the front with the lazy susan, launches an
app that isn't running as a floating window, or connects and spins to a
remote display. Shift+Enter or a right click opens the item's menu: the
app's desktop actions, Open new window, Open docked on this screen, and
its windows; a display's Go to and Disconnect.
- spotwatch/ft-spotwatch (Python, in the Plasma session's autostart):
the index, ranking (spotsearch.py), keys via libxkbcommon, usage.
- spotwatch/ft-spotpanel.cpp (SteamVR overlay, built by
screens/build.sh): the pill, drawn on the CPU; --render for a look
without a headset.
- ft-screens: "spin to N", "front N", and "spotwatch on|off", which
routes every key to the search while it's open (a lease, renewed
every second) and dismisses it on a click elsewhere or a pause.
- ft-floatd: "spotlist", "activate ID", and launch action=/screen=;
the KWin script tracks when each window was last active.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
