The line from the eyes through the palm lit panels off to the side of a palm
that wasn't facing them. The line is now the palm's own, along its normal,
with the margins seen from the palm. HANDS_GRAB_THAT_AIM=eyes brings the old
line back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 5120x1440 remote in the front slot was held to 100 deg wide (51 px/deg,
hard to read). Panels 3:1 or wider may now go to SCREEN_SPIN_MAX_WIDE_DEG
(150) instead, and their curve may wrap that far round you.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lazy susan (the third live test): a brought screen comes 1 m ahead, as big as its
resolution needs for text to read: its logical pixels (KWin's scale undone, now passed
to vr.cpp) at 26 px/deg, at most 100x55 deg keeping its shape, at least 40 deg wide,
curved round you past 60 deg (grabthat::ReadableSize; SCREEN_SPIN_DISTANCE, _PPD,
_MAX_DEG, _MAX_TALL_DEG). Its curve comes back on the way home. The strokes cycled
104, 1, 104 with three screens up: a screen still flying home read as straight ahead;
each one is now taken where it lives (home, or where it's going).
Accidental clicks: most were pinches with no laser up pressing where the pointer was
last left ("no ray up: a press where the pointer is", a dozen in two minutes). With
gaze off they now only wake the pointer (POINTER_PINCH_NO_RAY=1: the old air mouse).
The hand's laser shows only with the thumb and index within 6 cm (gone past 10;
POINTER_RAY_READY), and a fist drops it: fh_ray's "ready" and "open" tests, the fist
read as the index and the other three both curled under 1.1 (FistCurl; measured on 47k
flagged hands: the other three alone read as a fist in a fifth of the pinches). A ray
pinch made while the hand closes into a fist doesn't click. Input Settings switches for
all three (POINTER_ ones reload the helper).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lazy susan (the sweep, Meta+Alt+Tab) no longer turns the room: the next screen
round, by where each one lives, comes in front of you at a distance it spans at most
60 deg of the view at, and takes the typing. The next stroke sends it home and brings
the one after; pushed, flicked or thrown it goes home; turned away from for 0.6 s it
goes home; moved by hand or laser it stays. SCREEN_SPIN_BRING=0 turns the room again
(Input Settings switch too).
Sweeps were hard to make (counted strokes came at 30-35 cm, just over the 30 asked):
25 cm in 0.5 s, the hand open 1.4, tilted up to ~45 deg off its edge, and hovering near
a screen counts (the brought one sits right there). The flags replayed: one stroke in
49 flags of other use.
Grab that brought a remote display back at the 1.62 m it was resized to, 0.45 m ahead:
it filled the view and every poke went through it (both flags of the second live test).
A wide one now comes as far out as it spans at most 75 deg (HANDS_GRAB_THAT_VIEW_DEG).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-11 09:43 live test:
- Two fists anywhere near one panel (palms 30 cm in front to 15 behind, 30 cm
round it) resize it by their shape alone (HANDS_FIST_PAIR): ft-hands' grip
only begins on a hand seen open just before, at the panel.
- Two fingers (index and middle straight, together, level; ring and little
curled) touching the content: a swipe scrolls (a notch per 1.2 cm, the
content following), a lift unmoved clicks, held still they act as a poke
(HANDS_TWO_FINGER_SCROLL).
- Stuck presses: a press isn't held through a tracking loss (only grabs are);
a poke lets go as the index curls; a pinch-click as the tips are seen 4.5 cm
apart; either held still 1.5 s (HANDS_PRESS_STILL_MS).
- The hand ray hides within 22 cm in front of a panel and for 0.8 s after
(HANDS_RAY_HIDE_REACH); the pointer helper ignores a ray-less pinch there
without the gaze (it woke the laser).
- The push/flick hint glow is off (HANDS_PUSH_GLOW): it covered near panels
the palm faced while clicking.
- Grab that gives panels that look small up to 10 deg more aim margin
(HANDS_GRAB_THAT_SMALL).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fifth live test (2026-10-10 22:55): grab that brought the desktop in, it was
carried and thrown 3 m out, then five gestures at it or the remote displays
logged "no panel there". The open palm is held out below the line of sight,
so the line from the eyes through it runs 12-15 deg under where the head
points (the user's flag 150115). Since the fourth live test the line had to
meet a panel exactly, and a thrown 1.35 m tall desktop 3 m out is only
12.7 deg each way, so the line passed just under it. One of the flag's two
fires misses such a panel that way.
Pick now takes the panel the line meets (unchanged, nearest wins), else the
one whose outline it passes nearest within HANDS_GRAB_THAT_MARGIN_DEG
(6 deg seen from the eyes; 0 = the old rule). The gesture itself is
unchanged, so it stays as hard to trigger, and held, pinned, pressed and
near panels are still never taken. "no panel there" now logs where the line
went relative to the head and the nearest panel it may take, with its angle.
Test: TestGrabThatAfterThrow runs the user's sequence with synthetic hands:
grab that brings the desktop in, a bar carry is thrown out and up at
1.3 m/s, then grab that at it again (missed by 0.8 deg with the old rule,
taken now), a remote display after that, precision cases, and the flag's
two fires (2 of 2 now, 1 with the old rule).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two fists (flags 215945 and 215955, the live log): the right hand's fist began as a
pinch with the thumb on the index while the other fingers were still closing, waited
0.6 s on "grab that" (armed by the open hand held out to the near panel), then clicked
the content and held the click 9 s, so the left fist's hold never got a second anchor.
- A second hand at a panel the other holds (15 cm in front, 12 behind, 10 round) joins
it with a pinch or fist anywhere on it: the stretch (move, scale, turn). No click,
poke, push or flick on a held panel (HANDS_JOIN_REACH).
- A pinch on the content by a closing hand waits up to 150 ms to become a fist; a wait
on grab that ends as a join, a fist or a pinch, as the hand is then.
- Fists reach 11 cm in front and 10 behind; a curled hand is claimed that far
(HANDS_FIST_REACH).
- Controls: the nearest wins (the bar took the curve button's pokes), round buttons get
a 1.5 cm hit area, a pinch presses them (HANDS_BUTTON_REACH, HANDS_PINCH_BUTTONS); the
log names the control.
- Guides: per hand, what its soft glow shows (corner, edge, button; lit while held).
- A carried panel turns at most 6 deg a tick about the hand's point.
- Grab that: the line through the palm has to meet the panel (no margin, no head cone),
panels within 0.8 m aren't taken, the arming hold faces the aim within 55 deg and is
still under 0.5 m/s, and a send-away quiets it for 1 s.
Tests: buttons, guides, the fist settle and reach, joins, two-fist flag replays (before:
a click and no two-hand hold; now: 9.4 s of stretch with the panel 4 cm nearer or 8
cm farther), grab that at near and far panels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the 2026-10-10 19:28 session (ten flags, ft-screens' and the helper's logs):
- Tracking gates (Params::gate, HANDS_TRACK_GATE): a held hand's frame that no hand makes
(faster than 4 m/s, or more than 0.5*100 m/s^2*dt^2 + 2 cm off where its velocity had it) is
dropped unless three in a row agree; its frame turning faster than 540 deg/s likewise. One
camera's frames and frames near the edge of view (50-80 deg off the head's forward) are smoothed
harder and don't turn the panel; past 80 deg the hand counts as unseen (the panel holds). On the
side-grab flag (203635) the left hand read alternately at two places 10-20 cm apart with its frame
flipping 100-180 deg.
- CarryPose faces the head from a point between the hand and the panel's centre when the panel is
held near the eyes by a far point (kCarryShare): facing from the centre had no steady answer
there and spun 0.6 m panels held by an edge at arm's reach. A centre near the eyes stops turning.
- A pinch by a closed hand (index <= 1.15, the others <= 1.05) is a fist grab, not a pinch click:
ft-hands called the user's two fists pinches (flag 203215) and the content was clicked.
- Two hands turn a panel on every axis with the line between them and roll it with their wrists
(StretchTurn, Params::stretchTurn, HANDS_STRETCH_TURN); fists find a panel by its edge too.
- A pinch the touch zone claimed takes the border 3 cm past the band's depth (Params::pinchSlack).
- The push (Params::push, HANDS_PUSH): an open hand, palm to the panel, pushed straight into it
at 0.5 m/s (5 cm in) or 15 cm in slowly, throws it 3 m out like a flick. A flat or pushing open
hand doesn't poke. throwAims() says which panel a push or flick would throw.
- The palm menu: facing 45 deg (60 to leave), the palm still, not at a panel (Input::atPanel,
HandAtPanel) nor for a second after it or after a hold; a pinch with the middle finger in it
does nothing on the ring.
- "Grab that": Aim() for the target's glow.
Tests: gates (alternating misreads, real moves, a real jump, the edge of view, flipping frames,
one camera), fist pinches and two-fist stretch and turn, StretchTurn, push vs poke, rest, wave and
the back of the hand, pinch slack, the stricter palm menu; replays: --carries (steadiness of
carried panels on the flags, gates on and off), --grabthat with pushes and the palm menu.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
User, 2026-10-10. Both send the panel 3 m from the eyes
(HANDS_THROW_DISTANCE), facing them, over 400 ms easing out; "grab that"
brings it back from there.
- Flick (HANDS_FLICK): a hand at a panel (its palm within 12 cm in
front of it or 4 cm behind, over it, and nearly still there: under
0.6 m/s) flicks out: away from the eyes (within 40 deg) at 1.2 m/s or
more (HANDS_FLICK_SPEED, a fit over 100 ms), 12 cm or more from there
within 200 ms, the fingers open (mean extension 1.5 or more) and
opening on the way by 0.15 (HANDS_FLICK_OPENING; 0 lets an open hand
pushed out flick too). The panel goes out along the line from the
eyes through its centre. Not a poke (a pointing hand's fingers read
about 1.3; an open hand coming through a panel faster than 0.8 m/s
doesn't poke it), not a wave (the direction), not "grab that" (its
pull back is toward the eyes). Without the opening rule the
recordings' probe moves, with a panel 0.3-0.45 m ahead, flicked 6
times in 11 minutes; with it once. None in the 13 "grab that" flags.
- Card throw (HANDS_THROW): a carry by one hand (a card, a fist, a pinch
on the border, bar or title; not a resize or two hands) let go moving
at 1.0 m/s or more (HANDS_THROW_SPEED; a fit over the 100 ms before
the fingers opened), within 60 deg of away from the eyes: the panel
goes on along that line to 3 m from the eyes. A slower release leaves
it where it is. The velocity's track starts afresh when the hand comes
back from a gap, and a flick doesn't reach back over one.
- vr.cpp animates a throw like a "grab that" move (ease-out), and
forgets that the panel came by "grab that": the throw is its place
now, so the gesture brings it back and sends it back there.
- "grab that" finds a panel along the palm's line within 3 deg seen
from the eyes (or 10 cm), so a 0.6 m panel 3 m out isn't missed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the live test of 2026-10-10 "grab that" fired three times, each
bringing another remote display to arm's reach (they stacked up in one
spot), and never sent one back: a panel had to be within 15 cm of where
it came to go back.
Now (grabthat::Decide): at a panel that came by the gesture and is within
arm's reach (0.45 + 0.35 m of the eyes), wherever it was moved about
there, it goes back where it was; at a panel within arm's reach that
didn't come by it, it centres it at arm's reach (and the gesture again
sends it back); a panel farther away comes as before. One at a time: a
panel that came before and is still where it came goes back as another
one comes.
The touch zone and the fist grab don't take the gesture's hand any more
(the previous commit: a fist or pinch on a hand the gesture has waits on
it). The user's flag replayed through hand input and the detector, with a
panel 0.35-0.45 m ahead where the gesture brings one, still fires twice,
with no press or grab.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hold an open hand out to a screen, remote display or floating window,
palm to it, then pull the arm back making a fist: the panel comes over
250 ms to 45 cm ahead of the eyes, at the fist's height, facing you
(user, 2026-10-10). Screens wider than HANDS_GRAB_THAT_WIDTH (0.6 m)
come at that width. The same gesture at a panel that came, still where
it came to, sends it back; the reset button sends floating windows back
(the layout reset places screens and remote displays itself).
screens/grabthat.{h,cpp}: the detector (pure logic): open fingers, palm
ahead and the wrist 28 cm out for 0.2 s arm it; the fingers closing with
the palm still to the panel, then on one frame a fist, the wrist 8 cm
back within 0.5 s and the palm turned over fire it. The panel is the one
the eye-through-palm line meets, else the one the head points at. From
the closing (thumb clear of the index: not a pinch) to 0.4 s after the
fire the hand counts as near, so the helper doesn't take the fist as a
gaze grip. Hands frames carry the head pose at capture now
(Hands::captureHead).
Tests replay the user's flag (fires twice, right hand, at -11.35 and
-5.89 s; expected -11.4 and -5.9), and the other 12 flags and 15 minutes
of recordings fire nothing (1 of 143 pinches, 3 of 12 grips begin on a
claimed hand); synthetic waves, pushes, fists without the pull, slow
pulls, a hand turned over first, held-out pinches, a drop to the lap and
typing don't fire. handinput-test --grabthat FLAG_DIR... scans flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
