A left hand's skeleton could flip to a right hand's mirror image and stay
that way: with the knuckles allowed to bend 35 degrees backward, the mirror
follows half-curled fingers well enough, so curling them showed the fingers
bending back. Live on 2026-10-19 the left hand was fitted as a right one in
41% of frames, knuckles at -18 degrees.
A fit as the other side than the model's call now goes back to the model's
side as soon as its four knuckles bend backward on average (below -10
degrees), and a mirror that needs that never wins the side.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands fits a skeleton to each hand's 21 landmarks after the 3D step
(track/skeleton.{h,cpp}):
- Bone lengths and palm shape are learned per side from clean stereo
frames (medians, then a slow running median), with the model's average
hand times Hand::scale, or the other side, as the prior.
- Each frame: the palm as a rigid body, then each finger and the thumb as
a chain with joint limits (Gauss-Newton), the chain bases allowed a few
mm off the rigid palm. Landmarks are weighted by their triangulation
residual and down-weighted along the line of sight.
- The pose (palm centre, palm rotation, joint angles) is filtered, not the
points; each finger's cutoff follows its own speed.
- An elbow from the arm model in frame-hands' notes/forearm.md.
The hands file goes to version 2: version 1's struct is unchanged and an
fh_hand_fit_t per hand follows it (joints as position + quaternion,
bone lengths, angles, elbow, pinch point, fit quality, head-frame
velocities). Version 1's points and capsules come from the skeleton
(HANDS_SKELETON=0 or --skeleton 0 for the old ones). The hand
recorder's reader accepts version 2.
ft-handreplay reports the fit (track/skeval.h): time, residual,
calibration, bone-length spread, jitter, tip lag, elbow, and with
--labels the distance to frame-hands' auto-labels. SetReader reads a
hand-recorder take's later parts (sets-2.bin, ...).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
