--publish-dump writes what ft-hands would have published after each processed set (the hands
and gestures files' bytes), for ft-screens' offline hand input tests. want-hz: a reader
(ft-screens with a hand near a panel) asks for up to 30 Hz by keeping the file fresh.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each chain runs Gauss-Newton from the measured bone directions and from
last frame's angles, and keeps the better fit, so it hops less between
two poses that fit about as well: tip jitter tails 2-6% lower in the lit
and dim recordings, for 25 -> 42 us median per hand on a big core.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands fits a skeleton to each hand's 21 landmarks after the 3D step
(track/skeleton.{h,cpp}):
- Bone lengths and palm shape are learned per side from clean stereo
frames (medians, then a slow running median), with the model's average
hand times Hand::scale, or the other side, as the prior.
- Each frame: the palm as a rigid body, then each finger and the thumb as
a chain with joint limits (Gauss-Newton), the chain bases allowed a few
mm off the rigid palm. Landmarks are weighted by their triangulation
residual and down-weighted along the line of sight.
- The pose (palm centre, palm rotation, joint angles) is filtered, not the
points; each finger's cutoff follows its own speed.
- An elbow from the arm model in frame-hands' notes/forearm.md.
The hands file goes to version 2: version 1's struct is unchanged and an
fh_hand_fit_t per hand follows it (joints as position + quaternion,
bone lengths, angles, elbow, pinch point, fit quality, head-frame
velocities). Version 1's points and capsules come from the skeleton
(HANDS_SKELETON=0 or --skeleton 0 for the old ones). The hand
recorder's reader accepts version 2.
ft-handreplay reports the fit (track/skeval.h): time, residual,
calibration, bone-length spread, jitter, tip lag, elbow, and with
--labels the distance to frame-hands' auto-labels. SetReader reads a
hand-recorder take's later parts (sets-2.bin, ...).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The runtime files move to /run/user/UID/frametop-hands/: the desktop
session deletes /run/user/UID/frametop at every start.
- The cutout copy shader runs at highp: mediump (16-bit on Adreno)
stepped 1.7 texels across a 3440-pixel screen.
- One hand no longer pinches both sides after its left/right call flips
mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm
facing down (typing on a lap keyboard).
- ft-camd judges a colour frame fresh by its luma rows only, and logs
per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1.
- hands/run.sh caps skips the setcap when ft-camd already has them.
- The replay tool dumps poses (--poses), and its pinch events carry the
hand id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Programs: ft-camd (the camera broker), ft-hands (the tracker), and
ft-handreplay and ft-ringplay for recordings, built by hands/build.sh
into hands/build/ with one Makefile. The first build fetches ncnn at
frame-hands' pinned tag and builds it with the same options.
- ft-camd gets its privileges from file capabilities (CAP_SYS_PTRACE,
CAP_PERFMON, CAP_DAC_READ_SEARCH) that hands/run.sh install sets with
sudo, and drops them once set up. It still works under sudo. It runs
on the host, linked statically, as frametop-camd.service. ft-hands
runs in the dev container as frametop-hands.service. Both start and
stop with SteamVR.
- Files move to /run/user/UID/frametop/ (cam-ring, hands, gestures),
not $XDG_RUNTIME_DIR, which a terminal in the Frametop desktop has
its own of. SIGUSR1 recordings go to ~/.local/share/frametop/hands.
- The calibration is read through /run/host in the container.
- Settings: HANDS_SWAP_SIDES and HANDS_CPUS in frametop.conf.
- install.sh offers hand tracking as an optional last step.
- The container gets jsoncpp-devel, glibc-static, and NumPy and OpenCV
for the Python tools.
- tools/ring.py reads the ring, and models/NOTICE credits the
Apache-2.0 models.
Checked: ft-handreplay gives identical summaries and byte-identical
depth dumps to frame-hands' fh-replay on both 2026-09-29 recordings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trackd/ becomes track/, and the calibration helper the Python tools
import moves from the prototype's folder into tools/. The model
development tools (nettest and the scripts that compare it with the
Python models or cut int8 calibration crops) stay in frame-hands with
the prototype they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
