mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-27 10:01:01 +01:00
7dfdfd24e1603ed65671b60274afac72ed00a211
54
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DeeJanuzandClaude Opus 5.5
|
3fe498ffc3 |
Merge branch 'hands-r6' into experimental: hand tracking as the main input
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
80bce4cf66 |
Gaze + hands: the two-hand drag works like Meta+J/K; held pinches no longer drop on a clock race; easier lazy susan
User 2026-10-11, the second gaze test: - "hand tracking stopped" ends were an unsigned age: ft-hands publishing just after the loop's tnow made it negative, wrapped to 584 years, and ended held pinches, nudges, drags and scrolls. Signed now. - The two-hand drag mirrors the keys: hold one pinch (aim, nudge), pinch with the other: the left button goes down where the dot is (as Meta+K with Meta+J held); the drag lasts while either pinch holds, and the hand still pinching drags. A second pinch during a drag joins it. - The fast-hand filter refused dozens of still pinches (frame-to-frame palm speed is noisy): smoothed, 1.0 m/s. - A three-finger scroll goes on when ft-hands swaps which finger it calls pinched. - Lazy susan: counted strokes came at 25-30 cm in 230-470 ms, at both limits: 22 cm, quick somewhere along it (0.7 m/s peak) within 0.9 s; looser hand shape. The flags replayed: 2 strokes in 45 flags of other use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0ce6843338 |
Front slot: a brought screen stays until you move it
User 2026-10-11: "the lazy susan can be considered to be in use and remain active until I move it out of its spot manually through either the eye tracking or grabbing it with hand-tracking". Looking away no longer sends it home (SCREEN_FRONT_AWAY=1 brings that back, with the keep-by-use); a gaze-pinch drag, a hand grab, a push or grab that releases it, and the next stroke sends it home. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0b72710474 |
Gaze + hands: two-hand drag (hold, tap the other), pinches held through tracking pauses, no clicks from fists or strokes, the pointer hides while typing; the slot keeps only on real use
The user's 2026-10-11 test of gaze + hands: - Two-hand drag as they meant it: hold one hand's pinch (nudge it into place), tap with the other: index presses the left button where the pointer is, middle the right; the first hand drags (POINTER_GAZE_TWO_HAND). The tap-first right-drag is off by default (it could catch a tap followed by a nudge). - 9 held pinches ended as "hand tracking stopped (1.5 s)" while ft-hands still held them: 4 s with nothing pressed, 2.5 s with a press, and the line now says the file's seq and ages to find the cause. - A gaze pinch held back ends without a click when the hand closes into a fist (ft-hands' grip may not take it past 0.1 s now); a pinch begun while the palm moves faster than 0.6 m/s (a stroke) doesn't click (POINTER_PINCH_MAX_SPEED). - Three-finger scroll: 3.5 cm and 0.4 s to join. - Gaze mode hides the dot and collapses its laser for 1.5 s after each key (POINTER_TYPING_HIDE): it sat big and bright on the text being typed. - The front slot's keep counts clicks, scrolls, drags and keys, not the gaze pointer passing over it (every brought screen was kept 3 s in). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
02f1575c52 |
Gaze + hands first: right-drag by two taps, three-finger scroll, fist guard on every pinch; the slot lets go to manual controls; palm-only susan; dot-only hand laser
User 2026-10-11, gaze + hands becoming the main input: - Right-drag: an index tap on one hand, then an index pinch on the other within 1 s, holds the right button at the gaze lock; that hand drags until it ends (POINTER_GAZE_RIGHT_DRAG). It replaces the chord (POINTER_GAZE_CHORD=0). - Three-finger scroll: the thumb on both the index and middle tips (3 cm) scrolls where you looked as the hand moves, like the ray's middle-pinch stick (POINTER_GAZE_SCROLL). - The fist guard (a hand closing into a fist: no click) now covers gaze and chord pinches too, from each hand's own frame (not the ray's). - A hand's laser shows only its dot: the beam starts just short of it, as the eye's does, held still through a carry (POINTER_RAY_BEAM=0; SteamVR's beam width is one global setting it reads only at start). - The front slot gives a screen up to manual controls: grab that, a push, flick or throw, or a carry by hand or laser releases it (it moves like any other and stays where you put it); the next stroke puts it home and brings the next. - The lazy susan counts only palm-led strokes (the return stroke does nothing; the other hand palms the other way; HANDS_SWEEP_BACK=1 for both). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
90114ed043 |
The front slot: one place in front of you for the lazy susan and grab that
The user's proposals (2026-10-11): the lazy susan and grab that were two systems that fought over a screen (grab that took one the susan brought: 2.3 m wide at 1.5 m). Now one front slot, one screen at a time (the one there goes home first): - reading distance (the lazy susan, 1 m) and arm's reach (grab that), both readable (ReadableSize, the same view at either); grab that at the one there pulls it closer; - a push, flick or throw steps it back: arm's reach -> reading -> home; - one home per screen: every dismissal returns it there; moved by hand or laser it lives where you put it; resized in the slot it keeps that view next time; - turned away from it goes home, unless used 3 s (kept; SCREEN_FRONT_KEEP_MS); - a strip of the screens' names, left to right as they live, for 2.5 s after each stroke, the one in the slot lit and the next each way marked (keyboard.cpp's font, now DrawLabel for other panels too); - SCREEN_UI_SCALE for a remote display's own UI scale (a Mac's Retina: 2); - the reset button needs a 0.5 s hold, laser or finger (a stray poke reset the layout; HANDS_RESET_HOLD_MS). grab that's remembered widths, send-away widths and view fit go (the slot sizes it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
12b8159d4e |
Lazy susan: readable at 1 m, cycles every screen; the laser only for pinching, none in a fist
Lazy susan (the third live test): a brought screen comes 1 m ahead, as big as its
resolution needs for text to read: its logical pixels (KWin's scale undone, now passed
to vr.cpp) at 26 px/deg, at most 100x55 deg keeping its shape, at least 40 deg wide,
curved round you past 60 deg (grabthat::ReadableSize; SCREEN_SPIN_DISTANCE, _PPD,
_MAX_DEG, _MAX_TALL_DEG). Its curve comes back on the way home. The strokes cycled
104, 1, 104 with three screens up: a screen still flying home read as straight ahead;
each one is now taken where it lives (home, or where it's going).
Accidental clicks: most were pinches with no laser up pressing where the pointer was
last left ("no ray up: a press where the pointer is", a dozen in two minutes). With
gaze off they now only wake the pointer (POINTER_PINCH_NO_RAY=1: the old air mouse).
The hand's laser shows only with the thumb and index within 6 cm (gone past 10;
POINTER_RAY_READY), and a fist drops it: fh_ray's "ready" and "open" tests, the fist
read as the index and the other three both curled under 1.1 (FistCurl; measured on 47k
flagged hands: the other three alone read as a fist in a fifth of the pinches). A ray
pinch made while the hand closes into a fist doesn't click. Input Settings switches for
all three (POINTER_ ones reload the helper).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
a546469db6 |
Lazy susan brings the next screen to you; easier sweeps; grab that fits wide screens to the view
The lazy susan (the sweep, Meta+Alt+Tab) no longer turns the room: the next screen round, by where each one lives, comes in front of you at a distance it spans at most 60 deg of the view at, and takes the typing. The next stroke sends it home and brings the one after; pushed, flicked or thrown it goes home; turned away from for 0.6 s it goes home; moved by hand or laser it stays. SCREEN_SPIN_BRING=0 turns the room again (Input Settings switch too). Sweeps were hard to make (counted strokes came at 30-35 cm, just over the 30 asked): 25 cm in 0.5 s, the hand open 1.4, tilted up to ~45 deg off its edge, and hovering near a screen counts (the brought one sits right there). The flags replayed: one stroke in 49 flags of other use. Grab that brought a remote display back at the 1.62 m it was resized to, 0.45 m ahead: it filled the view and every poke went through it (both flags of the second live test). A wide one now comes as far out as it spans at most 75 deg (HANDS_GRAB_THAT_VIEW_DEG). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
6164766efa |
Input Settings: switch every hand interaction on or off, live
User, 2026-10-11: "make every interaction able to be enabled or disabled".
The Hands page gets a "Hand interactions" section: 24 switches in four
groups (tapping, moving screens, gestures, feedback), each writing its
frametop.conf key and telling ft-screens at once ("handsetting KEY 0|1",
SetHandSetting; number-valued ones go back to their defaults when switched
on). They follow the direct touch switch above.
The command is checked before the helper's "hands L R allowed" word: that
pattern matched "handsetting X 0" too and scrambled the hands word.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
8135bd351d |
Input Settings: hand input switches on the Hands page
Three switches, each on its own: pinch to click where you look (gaze on), a raised hand's ray (gaze off), and direct touch on Frametop's screens (gaze on or off). They write through handsmode.method_writes, reload the helper and relay, and tell ft-screens direct touch live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
2c472879ed |
Spotwatch: Meta+Space search for apps and remote displays
A Spotlight-like search pill (docs/spotwatch.md). Meta+Space (the input
relay's new spotwatch action, bindable to mouse and controller buttons
too) opens a faux-glass pill straight ahead of the head. Typing finds
apps and remote displays ({host} - {display}). Enter brings a running
app's most recent window to the front with the lazy susan, launches an
app that isn't running as a floating window, or connects and spins to a
remote display. Shift+Enter or a right click opens the item's menu: the
app's desktop actions, Open new window, Open docked on this screen, and
its windows; a display's Go to and Disconnect.
- spotwatch/ft-spotwatch (Python, in the Plasma session's autostart):
the index, ranking (spotsearch.py), keys via libxkbcommon, usage.
- spotwatch/ft-spotpanel.cpp (SteamVR overlay, built by
screens/build.sh): the pill, drawn on the CPU; --render for a look
without a headset.
- ft-screens: "spin to N", "front N", and "spotwatch on|off", which
routes every key to the search while it's open (a lease, renewed
every second) and dismisses it on a click elsewhere or a pause.
- ft-floatd: "spotlist", "activate ID", and launch action=/screen=;
the KWin script tracks when each window was last active.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuz
|
ca1e5c38ab |
Merge branch 'experimental' into hands-full: a Mac's real monitors through Sunshine, Frametop Voice
# Conflicts: # input-settings/ft_input_settings.py # input-settings/main.qml # input/game_pause.py # session/frametop.conf.example |
||
|
DeeJanuzandClaude Opus 5.5
|
018677bcb1 |
Input Settings: a Voice page, dictation actions, right-hand taps
- Voice page: Frametop Voice on or off (enable/disable --now on the
host), its keys (shown, set on the Keyboard page), controller buttons
(capture or pick; read by ft-voiced, so games keep them) and hold or
toggle, typing as you talk, sounds, the wake word and its phrases,
and the voice commands in the file. ft-voiced's "status" every 2 s.
- Dictation: start or stop / while held, for key combinations and
mouse buttons; not on the Controllers page, which would take the
button from games.
- A captured tap of a right-hand modifier binds that key alone ("97",
shown as Right Ctrl tap), as the relay now reads it. The defaults
include the Right Ctrl tap once Frametop Voice is installed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
bd7b15613f |
Input Settings: the gaze pinch's nudge dead zone and speed on the Gaze page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
85d34c73a4 |
Input Settings: a Hands page for when hand tracking runs and the cutout margin
Hand tracking: With Frametop / Always / Off, applied at once through hands/handsmode.py on the host, with a status line that says what runs and why it can't (not installed, no capabilities, cameras). Hand cutout margin: 0-3 degrees in 0.25 steps, written to HANDS_MATTE_MARGIN and sent live to ft-screens (the still margin only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
185cb766b1 |
SteamOS updates: keep the Bluetooth fixes and the eye grabber through them
A SteamOS update deletes every /etc file its keep list (/usr/lib/rauc/atomic-update-keep.conf) doesn't name. The list keeps units but not what they run, so after the 0.4.5 update steamframe-bt-fixups.service and frametop-eyegrab.service failed with 203/EXEC: Bluetooth LE devices stopped reconnecting, and gaze mode fell back to SteamVR's tracker, which had no calibration. - Both installers add a drop-in to /etc/atomic-update.conf.d naming their file; uninstall removes it. - update-check.py (doctor.sh) fails when a unit's program is gone and warns when it isn't kept. - ft-gazed's log and Input Settings' Bluetooth page say what to reinstall. - setup/README no longer says /etc survives updates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
6dd9fb1ad8 |
One launch switch: scripts/in-box runs a program in the container it was built for
Every place an installed Frametop started a program with `distrobox enter dev --` now goes through scripts/in-box: the pointer and power units, ft-gazed (ft-gaze, ft-eyes), the gaze check's panel, the desktop's ft-screens, the remote desktop's krdp and VNC, and the three settings apps. in-box starts the container in a scope of its own (container-up.sh, which now takes the container's name) and becomes `distrobox enter BOX --`, so callers behave as before. BOX is "dev" for a source install. A release names its own container in .frametop-release, so the same tree runs from an image without editing nine call sites. FRAMETOP_BOX overrides both. Development tools (frame.sh, the headless test, the gaze probe and lab) and hand tracking, which install.sh doesn't install, keep using the dev container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9d7c8a0b91 |
Experimental (#29)
* Input relay: typing with the pointer helper down no longer ends the relay Typing on a pass-through keyboard tells the helper "typing". With the helper not running (SteamVR off), that send raised ConnectionRefusedError and the relay exited, dropping every grab until systemd restarted it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-steam: open Steam's menu through Steam's own UI steam/ft-steam menu opens the SteamVR dashboard on Steam's menu, or closes the dashboard if it's up, without pointer mode: it asks Steam's UI over its debugging port to show its dashboard overlay (ShowVROverlay, what Steam calls itself) and focus the Steam frame's left menu (MenuStore.OpenMainMenu). ft-steam check says whether those calls still exist, and update-check.py runs it, since a Steam client update can rename them. The CDP client moves from display-settings/steam_settings.py to steam/steamui.py, so both use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Shortcuts: Steam menu, commands, and modifier taps Key combinations (and mouse and controller buttons) get two new actions: - steam_menu: Open Steam menu / close dashboard (steam/ft-steam menu). - command:CMD: run CMD with sh -c, as the relay's service, with layout/, float/ and steam/ on its PATH. Input Settings offers it for key combinations as Run a command... Both work without pointer mode. A modifier on its own is now a key combination too: a tap, pressed and released with no other key, mouse button, or scroll in between. A bound tap sends the desktop F24 before the release, so Plasma's launcher stays shut. The defaults gain a Meta tap for the Steam menu; this replaces META_DASHBOARD, which only worked in pointer mode. input/test/keys-test.py runs the relay against fake devices with every outgoing socket renamed, so it's safe next to the live relay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Relay: share a key combination's Meta release with frame-voice A Meta+key combination (Meta+J for gaze_left, say) hides Meta's release from the desktop, and the relay skipped share_key for it too. frame-voice saw Meta go down on @frametop_keys and never come up, so it held all dictated text back, waiting for that release. Keys of grabbed keyboards are now shared as pressed, before key_binding() decides what the desktop gets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-cutouts, the hand cutouts without pinches and grips hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient user units with ft-hands' new --no-gestures: hands are published for ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks or drags and a closing hand doesn't raise the tracking rate. It needs a build and ft-camd's capabilities, not hands/run.sh install. Its units conflict with ft-handsctl's, so each stops the other, and they stop with SteamVR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-cutouts status: only the current run's tracker lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: say why gaze mode can't work yet, and open the calibration whenever it's missing Turning gaze mode on without a calibration opened Calibrate only on the off-to-on change, and only if it could open right then. With the headset off, the eye tracker silent, or the panel not built, or with gaze mode already on when the gaze service started, nothing opened and nothing said why: the pointer just stayed a mouse. - The gaze service now checks every second: gaze mode on, no calibration for the tracker in use, eyes seen -> the full calibration opens. One that closes unfinished opens again only after the headset comes off and on, gaze mode off and on, or Calibrate, so it doesn't loop. A start that fails retries every 10 s. - Its status says why gaze mode can't work yet (checks.problem): not calibrated and opening, open, closed unfinished, or can't open and why. - Input Settings shows that under the Gaze pointer switch, along with the gaze service not installed or not running and our tracker missing its frame grabber. - ft-gazectl on notes a missing calibration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: install our eye tracker, prefer it, and say why a calibration dot wasn't taken A user on a fresh install got "Calibration failed: only 0 of 21 dots" with no reason. The installer never installed our tracker, so gaze used SteamVR's, and the only way SteamVR's tracker rejects a dot is losing an eye for most of the look. The panel just showed a red ring. - install.sh: step 9/10 installs our tracker (gaze/tracker/install.sh) after gaze mode, yes by default; it needs sudo, so --yes runs it only when sudo won't prompt. If it fails, gaze keeps SteamVR's tracker. Configs that still say GAZE_TRACKER=steam (the old template) are asked whether to switch. - GAZE_TRACKER=auto, the new default: ours when it's installed (the frame grabber, its unit, and ft-eyes' Python), else SteamVR's. ft-gazed rechecks every second, so installing it switches over. Input Settings lists Own tracker first as recommended, and says how to install it when it's missing (checking the host's /etc through /run/host from the dev container). - The calibration panel has a note line, orange over the instructions: why a dot wasn't taken (an eye lost, a blink, the eyes disagreeing for SteamVR's tracker, from steady_samples' new drop counts; ft-eyes' reply for ours), what a click is still waiting for after 1.5 s, and a failed calibration's most common reason, which the Gaze page shows too. steady_samples keeps the same samples as before (checked on 2037 windows of recordings); a lost eye is named before a blink, since its openness reads 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: link the Frametop Discord Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Wait for a new container to finish setting up before entering it container-up.sh starts the dev container in a scope of its own, so distrobox enter finds it running and skips its wait for distrobox-init. On a fresh install, init was still setting up passwordless sudo when dev-container.sh ran sudo dnf install, and sudo asked for a password with no terminal to read it from. container-up.sh now waits for container_setup_done itself, and the container's sudo calls use -n, so a password prompt fails at once with a clear message. Fixes #9 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Prevent small desktop overlay pointer movements from starting a drag * Clear reported drag state on controller release * Click stability: only a hand controller's press starts it The 3D mouse drives SteamVR's laser through the ft_pointer virtual controller, so its events reach the screens the same way a controller's do. The filter held every press, which turned the mouse's short drags (selecting a character or two, nudging a slider) into clicks. Mark button events from hand controllers and start the filter only on those. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Input relay: retry a new device until udev gives it to the input group A new /dev/input node is root:root 0600 until udev applies GROUP=input. The scan probed each new node once and marked it seen even when the open failed, so a node caught in that gap was never opened. Behind a KVM, a switch brings back a hub of devices at once: on the Frame, four nodes failed with EACCES in one switch, the keyboard was never grabbed, and its keys went to gamescope instead of the desktop screens. A node that isn't readable yet now waits for the next scan. * Screens: take a screen's overlays from one copy in the catcher While a button pressed on a screen is held, UpdateCatcher checks every tick whether the laser is still on one of the screen's overlays. It built that list from s.All().begin() and s.All().end(), but All() returns a std::array by value: iterators into two different temporaries, which is undefined behaviour. A clang build of ft-screens got a garbage length, threw std::length_error, and aborted on the first click, taking KWin and the desktop with it. * Gaze: leave SteamVR's gaze action alone during VR games From curiousjtuber's PR #13: with the gaze service running, SteamVR restarted its eye tracker every 10 to 13 s of Beat Saber, as if the headset came off, and each restart took input focus from the game. The PR stopped every read in a game. Only the action path reaches SteamVR (UpdateActionState on the gaze set at overlay-global priority, then GetEyeTrackingDataRelativeToNow); the mmap and our tracker are read-only files. So only the action is skipped while a scene app runs, and gaze keeps moving the pointer over the dashboard in a game. The action source is only used with --source action. Co-Authored-By: CuriousJ <curious.j.tuber@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: --record-hz, and the hand recorder's design (hands/rec/DESIGN.md) ft-hands --record-hz N records at most N frame sets a second, for the hand recorder (10). DESIGN.md lays out the recorder: the headset panel, the session runner and its script, the files, review and export, consent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-handpanel, the hand recorder's headset panel A head-locked SteamVR overlay for the hand recorder (hands/rec/DESIGN.md): 1.2 m ahead, 12 degrees up, 36 degrees wide, drawn with stb_truetype into three shared DMA-BUFs as ft-gazepanel does. It shows the title, step, wrapped instruction, note, countdown, hand chips, near/far bar and a "Paused" cover, driven over @ft_handpanel. It also places the touch target, a 2 cm dot in its own overlay fixed in the room where the head was at the first command for that point, and logs head and controller poses to poses.jsonl at 250 Hz from a thread of its own. Both threads take one lock around OpenVR calls. --no-vr prints each picture's state to stdout (and --dump writes the pictures), for testing without a headset. hands/rec/build.sh builds it in the dev container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the recorder's worn check goes by the panel's backlight, as frame-job does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's session runner and guided script hands/rec/session.py runs a recording session from script.json: it starts ft-camd and a tracking ft-hands as transient units only if they aren't running, records each section as one take (ft-hands --record-only at 10 sets/s, a new sets-N.bin after each pause), drives ft-handpanel, and writes session.json, calibration.json (identifying fields removed), prompts.jsonl and take.json. Feedback comes from the live hands file and, in the controller sections, from the panel's device poll. It also runs from the command line (--dry-run, --speed, --ring, --no-start). hands/rec/script.json: 11 sections, about 9 minutes without the object and controller sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's window, review and export hands/rec/ft_handrec.py + main.qml (Kirigami, dev container; host launcher hands/rec/ft-handrec): consent (CONSENT.md, asked again when its version changes; profile.json with a random contributor id), the before-you-start checklist with the lighting and free-space checks, the session controls (Space pauses, Esc stops), review with a frame-set viewer that deletes ranges, takes and sessions, export with progress and cancel (warns while the headset is worn), and the upload page (UPLOAD.md, the huggingface-cli command; HF_DATASET is a placeholder). --dry-run runs sessions without processes, for testing. hands/rec/takes.py (standard library): indexes sets.bin and sets-N.bin without reading pixels, reads one set's cameras, keeps deleted ranges in take.json, and exports: deleted sets left out, zstd -10 -T2 at nice 19, manifest.json and SHA256SUMS, nothing left behind on cancel. CONSENT.md and UPLOAD.md are drafts pending a legal review; the window says contributions aren't open yet. The dev container gains zstd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: upload from the hand recorder's window, export checks, a rehearsal hands/rec/validate.py (standard library; Linux and Windows, Python 3.12+) checks an export before upload and when it's received: SHA256SUMS, an allow-list of files, the manifest's schema and keys, the consent version, a uuid4 contributor, no identifying fields in calibration.json or device.json, every sets.bin.zst decompressed to its end as a stream with each FHSET01 header checked against the manifest, jsonl lines, the total size. It decompresses with compression.zstd, zstandard or the zstd program. validate.py DIR [--json]. hands/rec/hub.py uploads an export with huggingface_hub, as a pull request to contributions/<contributor>/<session>: validate first, refuse a repeat of the same export, check the login (whoami) and access (auth_check), upload_folder(create_pr=True) with the manifest summary as the description, then record the PR under "uploads" in session.json. Errors are explained (terms not accepted, not found, 401/403, network). --dry-run makes no network calls. FT_HANDREC_DATASET overrides HF_DATASET (DeeJanuz/frametop-hands); while the texts are drafts a real upload needs FT_HANDREC_ALLOW_UPLOAD=1. The Upload page shows the login with "Check again" and how to run hf auth login in a terminal (the token never enters the window), then Upload with a phase, progress and Cancel (hub.py as a child process), the PR link, and a warning for an export uploaded before. The manual command stays as the fallback. ft-handrec --hub-dry-run. session.py also saves device.json: cv.cad_from_cal and head from /persist/device_config.json, the labeller's shape, nothing identifying; export copies it. Session ids with a -N suffix are accepted everywhere. hands/rec/rehearse.sh runs it all without the headset: ft-ringplay plays 30 s of a capture into a ring, session.py records a short test script with ft-handpanel --no-vr and a tracker, then export, validate and a dry-run upload (--repo ID uploads for real). It runs in one frame-job scope, deletes its data and stops its processes, also on Ctrl+C. hands/rec/tests/test_validate.py covers good and broken exports and hub.py without the network. The dev container gains python3-huggingface-hub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: Frametop doesn't work on the SteamOS beta yet Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit |
||
|
DeeJanuzandClaude Opus 5.5
|
c257000a23 |
A one-line installer, and gaze mode in install.sh
- get.sh: curl ... | bash asks for stable (main) or experimental, clones or updates ~/frametop, and runs install.sh; run it again to update or switch - install.sh offers gaze mode (step 8, yes by default) and notes what to do if an SSH connection drops - Input Settings' Gaze page says when the gaze service isn't installed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0ac2b10cd7 |
Remove leftovers before a release
- gaze/tracker/lab/eyes_track.py: nothing used it - Input Settings: the pointer role backend, whose page was removed - ft-screens: no log line for every floating-window resize - hands: ft-hands --help gives the palm-down default (1: off), the uninstall removes ft-handsctl's link, .frame-job is ignored - Display Settings: "Save as profile…", not "Save current arrangement" - two stale comments (ft-pointer's grabprobe, ft-gaze) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
20e8d6254a |
Merge gaze-calibration into experimental
The gaze checks and calibration in a headset panel (quick, five, full, headset fit; shared-buffer drawing, click-to-capture), keyboard and mouse gaze clicks (Meta+J/K, the mouse's buttons alike, mouse moves only while a button is held, double right/Meta+K tilts a drag), one 55 degree learning limit with a quick check past it, eye presence for "the headset went on", the gaze probe as a development tool, and the relay releasing keys the desktop has down that no keyboard holds. Conflicts with the profiles: the relay keeps known_action/needs_pointer and the gaze defaults (Meta+J/K and the float key); Input Settings keeps the profile actions everywhere and the gaze actions in key combinations only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
45b1b06cdb |
Calibration all in the headset panel; the gaze probe is a development tool
Check headset fit now runs in the headset panel too ("fitcheck"): a card per
eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
seen) and the hints, from the probe's fitcheck.py, updated at most twice a
second; left click or Meta+J runs its guided check, right click or Meta+K closes
it. So Quick check, Calibrate, and Check headset fit all happen in one place.
The gaze probe moves to the Gaze page's overflow menu as "Gaze probe
(development)", and its app menu entry says it's a development tool. Texts that
sent users to it ("use Calibrate… with Own tracker") point at Calibrate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
1f46616c75 |
Gaze mode: the mouse's buttons work like Meta+J and Meta+K
The right button's press is held back like the left's: hold it, the pointer stops where you look, move onto the target, and the right click comes on the release (held still, it's a real right press). Right during the left's held-back press now starts a drag where the pointer is, as Meta+J then Meta+K does, in place of a right click there; letting go of either drops it. So once you've moved, the left alone only clicks, and the right starts a drag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
949f629338 |
Input Settings: the mouse movement explanation is a tooltip
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
a2a5a4c98e |
Gaze mode: the mouse only corrects, and left-then-right right-clicks
POINTER_GAZE_MOUSE_MOVE=held, the default (the Gaze page's Mouse movement switch, with why it's on): while the gaze has the pointer, moving the mouse does nothing; it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so lessons aren't polluted by mouse moves to somewhere else. With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual. "free" is the old behaviour. Pressing the right button while the left one's press is held back right-clicks where the pointer is instead (correct with the left, then right-click); both releases are then nothing. Meta+J then Meta+K stays a drag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
8998fc1ddd |
Gaze learning limit: 55 degrees, past it a quick check; a solid panel
Drops the quick check's second step (
|
||
|
DeeJanuzandClaude Opus 5.5
|
362968198b |
Quick check: put the pointer on the dot, and one limit for learning
After the quick check's capture, in gaze mode, the dot stays where it is in the
room (the panel's new "lock") and the gaze has the pointer again. You put the
pointer on the dot as you'd correct a click, with the mouse or Meta+J and the
head. That click clicks nothing: the helper sends it as "calverify", learned
whatever its size. Over POINTER_GAZE_NUDGE_MAX, the capture runs again, up to
3 times. A right click or Meta+K skips it.
POINTER_GAZE_NUDGE_MAX is now the one limit: 15 degrees by default (was 8),
and ft-gazed's own limits (8 for SteamVR's tracker, 25 for ours) follow it. A
keyboard correction past it isn't learned but opens a quick check ("recheck"),
in place of the 30 degree keyboard limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
858dbe1562 |
Profiles: named layouts that open apps
A profile is a named layout plus the screens it hides and its apps' windows (docs/profiles.md). ft-layout save captures them (ft-floatd's "windows", after the KWin script reports every window as it is now); use opens them: the screens move, open windows of each app go to their places (on a screen, maximized or not, or floating), and missing apps start, once and then again for each window still missing 3 s after the first. Nothing closes. The desktop starts in FT_PROFILE or default_profile (ft-layout start, from the session script). Each profile gets a launcher entry (Frametop: NAME, in SteamVR's Launch a program list) that switches to it or starts the desktop in it. The relay's profile:NAME action and Input Settings' "Open profile" entries put one on a key, mouse button, or controller button. Display Settings' Layout page becomes Layout & profiles: Save as profile, Open profile, the profile's apps, and Start in profile. Plasma's own session restore is off in the Frametop session. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
c4c10e6d92 |
Gaze checks and calibration in a panel fixed to the headset
ft-gazepanel (gaze/panel) is a SteamVR overlay that stays put in your view and shows dots at known head-relative directions; the gaze service runs it and drives it (gaze/gazecheck.py): - a one-dot quick check 3 s after the headset goes on, when our tracker asks for a click (reseat), at most every 2 minutes, and from Quick check on the Gaze page; five dots follow if the next 3 lessons are still over 2 degrees off - the full calibration (the probe's three rounds, dark to bright) when gaze mode comes on without one, or from Calibrate; quitting it with still no calibration turns gaze mode off Each dot takes the gaze once it has held still for 0.6 s; a left click or Meta+J takes it at once, a right click or Meta+K closes the panel (the pointer helper hides its dot and passes those presses on while "calpanel" lasts). Our tracker gets clicks and its own calibration; SteamVR's gets lessons per eye, or a new calibration.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuz
|
6ea70f32bc |
Merge branch 'experimental' into gaze-calibration
# Conflicts: # input-settings/ft_input_settings.py # input-settings/main.qml # input/input-relay.py |
||
|
DeeJanuzandClaude Opus 5.5
|
dc3a1b8208 |
Keyboard clicks at the gaze: Meta+J and Meta+K
gaze_left and gaze_right (Meta+J and Meta+K by default) click where you look.
A tap clicks where the dot was at the press and tells the gaze tracker it was
right there. Held, the pointer stays put in your view, so turning your head
carries it onto the target; the release clicks there and the correction is a
lesson (judged by the net correction, not the head's path). Held still for
POINTER_GAZE_HOLD it's a real press that the head drags, and Meta+K while
Meta+J aims presses where the dot is now, to correct and then drag.
A Meta combination now also sends the desktop an F24 press and Meta's release
at once: Meta's release no longer opens Plasma's launcher, and the click isn't
Meta+click (KWin's window move and resize, which swallowed right clicks). The
desktop gets Meta back for the next key if it's still held.
Also fixes
|
||
|
DeeJanuzandClaude Opus 5.5
|
1e83f96b0f |
Float key in the input relay, docking everything, and the plan for profiles
The input relay owns the float key now: float_toggle (Meta+Shift+F unless the rules have their own key combinations) floats the window under the pointer, or the active one over the wallpaper, and docks it if it floats. dock_all puts every floating window back. Both are mappable to mouse and controller buttons, and work without pointer mode. The KWin script no longer registers a shortcut, and ft-floatd drops the old one. Key combinations move to Input Settings' Keyboard page. docs/floating-windows.md records the decisions from 2026-09-30 (the title bar button, Launch as Standalone, profiles), and docs/profiles.md plans profiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
e16b7588e2 |
Gaze mode is a mouse feature: drop its controller parts
Steam reads the Frame controllers itself, outside SteamVR's bindings, and every press and release it sees takes SteamVR out of laser mode, so controller clicks at the gaze can't be done cleanly (docs/gaze-controllers.md, from the gaze-first branch's tests). - Gaze precision and Gaze drag take a mouse button or a key combination only; the controller source, its aim steering, and POINTER_PRECISION_GAIN, POINTER_PRECISION_DEADZONE and POINTER_GAZE_DRAG_GAIN are gone. - The gaze actions (including Gaze pointer on/off) can't be mapped to controller buttons: the relay ignores them there and doesn't ask the helper for those buttons, and Input Settings no longer offers them. - Last used wins in gaze mode too: picking up a controller hands it the laser, as docs/design.md already said. - The gaze dot shows all the time (POINTER_GAZE_DOT=always, the default; moving brings back the old behaviour), with a switch on the Gaze page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
97f853130d |
Merge eye-tracking into experimental
Our own eye tracker joins the gaze folder: ft-eyegrab (the root frame grabber, frametop-eyegrab.service) and ft-eyes, run by the gaze service when GAZE_TRACKER=own, with its lab tools. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
55d92b411b |
Input Settings: drop the pointer role choice
The hand role the pointer takes isn't something to choose by hand. The helper still reads POINTER_ROLE (right by default) for experiments. Tried and reverted (not committed): in gaze mode, taking the stylus role by reconnecting. SteamVR gave the device no role at all (hint 5, role none) and kept the dashboard laser on the held controllers, so the mouse couldn't click either. The dashboard laser needs a hand role, and a held Frame controller takes its hand's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
d666fa031f |
Gaze first: precision and drag buttons, key combinations, pointer role
Two new actions for mouse buttons, controller buttons, and key
combinations: gaze_precision (hold: the pointer stops where you look and
the button's device steers it, a controller by where it points at
POINTER_PRECISION_GAIN, the mouse by its moves; release: click there) and
gaze_drag (the same with a real press at once, dragging until the
release). The relay sends "precision|gazedrag <source> 1|0" to the
helper, which runs them through the same holds as pinches and grips.
- Key combinations on any keyboard ("key_bindings" in the rules, e.g.
Ctrl+Alt+G for gaze on/off): the last key isn't typed.
- POINTER_GAZE_MOUSE: in gaze mode the left button is a precision button
(the default, as before) or clicks right away (direct).
- In gaze mode a moving controller no longer takes the pointer away.
- POINTER_ROLE (right, left, stylus): the driver takes a "role" command,
so the pointer can stay off the hand holding the precision controller,
which would otherwise take the role back. Needs the rebuilt driver.
- Input Settings: the actions on the Controllers and Buttons pages; on the
Gaze page the mouse choice, the role, the precision sliders, and key
combinations (captured from any keyboard).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
5714978e65 |
Add a keyboard for text fields on the desktop
Fixes #7: SteamVR's keyboard never came up for the desktop's apps, and opening it for our panels doesn't work well on the Frame (it's Steam's own panel, mounted in the dashboard's scene, it follows the laser between panels, and it takes the controllers over to SteamVR's laser). - KWin starts input/ft-textinput as the desktop's input method. It tells the input relay when a text field gains or loses focus, and the relay asks ft-screens to open or close the keyboard. The session drops the QT_IM_MODULE=xim and GTK_IM_MODULE=xim that the gamescope session sets, or Qt and GTK apps never report text fields. - The keyboard is ft-screens' own panel (screens/keyboard.cpp): a US laptop layout, typed with a controller's laser or the 3D mouse. It opens 0.7 m in front of you, below your eyes and facing you. It has a grab bar to move it, a Close key, latching Shift, Ctrl and Alt, and repeat on a held key. It's drawn into shared DMA-BUFs, so it doesn't flicker. Its keys reach the focused screen as key presses, so every app takes them. - It steps aside while the Steam menu or Steam's own keyboard is up and comes back after. A layout reset closes it, and it doesn't open without a head pose. - Frametop Input Settings has a Keyboard page: open it for every text field, only while no keyboard is connected (the default), only from a mapped button (the new Open/close keyboard action, for mice and controllers), or never. A switch keeps it open until you close it. - The pointer helper treats every frametop.* overlay as a real panel. The keyboard's shared texture reports 0x0 like SteamVR's scene-graph controls, and the helper had given it their wide catch radius. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
ed75614f62 |
Bring our own eye tracker into the gaze folder, run by the gaze service
frame-eyes, a separate project until now, becomes gaze/tracker:
- ft-eyegrab (was fe-bufprobe) copies the eye-camera frames, read-only, out of
SteamVR's eyetracking process. It runs as the system service
frametop-eyegrab.service, which gaze/tracker/install.sh installs to
/etc/frametop with sudo. It keeps only CAP_SYS_PTRACE, CAP_DAC_READ_SEARCH, and
CAP_CHOWN, and copies frames only while /dev/shm/frametop-eyes-want is fresh,
holding none of the tracker's buffers otherwise.
- ft-eyes (was fe-trackd) runs under ft-gazed in the dev container, with
build/venv's pinned numpy and OpenCV: while Eye tracker is Own tracker, or on
the probe's lease ("eyes SECONDS"). No sudo password or fe-live script at
run time any more.
- lab/ holds the research tools (ft-eyes-score, -e2e, -record, -replay,
-session) and findings.md. Recordings live outside the repo, in
~/.local/share/frametop/eyes/captures; .gitignore catches stray frame dumps.
Its socket is now @ft_eyes, its output /dev/shm/frametop-eyes-gaze, and its state
~/.local/state/frametop/gaze/eyes. On practice1 -> practice2 the whole live path
(ft-eyes-e2e) gives 1.30 deg median and 3.18 for the worst tenth, as before the
move (1.30, 3.21).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
5565a25444 |
Let the gaze pointer use our own eye tracker, and weight the eyes
Frametop Input Settings' Gaze page gets two settings, saved in frametop.conf and read again by ft-gazed when the file changes: Eye tracker (GAZE_TRACKER: SteamVR's or our own, frame-eyes' fe-trackd) and Eye bias (GAZE_EYE: auto, left, right). ft-gazed now combines the eyes, each calibrated on its own: SteamVR's set 2 eyes with the probe's Left eye and Right eye calibrations, or our tracker's eyes as they come. Without per-eye calibrations, or with --source, it keeps the older one-source path. A pointer nudge finds its look from the raw gaze the helper echoes back; with our tracker it goes to fe-trackd as a click. The bias leans instead of choosing (gazecal.EyeWeights): on 306 live clicks the eyes' errors partly cancelled, both together 0.65 deg off against 0.96 and 1.11 for either alone. Left or Right counts that eye twice; auto weights each eye by its RMS miss at its last 20 nudges, since the calibration's fit picked the wrong eye on SteamVR's test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
9c04207bb9 |
Let the pointer pass through panels you pick, like a performance overlay
A head-locked performance overlay kept catching the 3D mouse. It has no
input method, so SteamVR's laser passes through it, but the helper hit
tests every visible overlay with ComputeOverlayIntersection, and the dot
stuck to it whenever it crossed that corner of the view.
POINTER_IGNORE in frametop.conf now lists overlay keys the helper leaves
out of the collision, comma-separated shell patterns, so "vendor.app*"
covers a whole app, including panels it opens later. The laser starts
just before the cursor point, so an ignored panel nearer to you doesn't
catch it either.
Frametop Input Settings has a new Ignored panels page. It asks the
helper for SteamVR's overlays ("overlays", answered from the list's
thread once vrcmd has run again, even while the pointer is off), groups
them by app, and has a checkbox per panel and one for the whole app.
Frametop's own screens aren't offered, since ignoring one would leave
nothing to click the app on with the mouse. Entries for apps that
aren't open are listed so they can be removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
51e7e0fa85 |
Warn in Input Settings when SteamVR hasn't loaded the pointer driver
When SteamVR crashes, safe mode can block the newest add-on, and then it skips the ft_pointer driver at every start. The cursor still moves (the pointer helper draws it), but clicks, scrolling and mapped actions go through the driver's virtual controller, so none of them do anything, and nothing said why (#4). Input Settings now shows a warning at the top of every page with the fix: Manage Add-Ons in SteamVR's settings, unblock ft_pointer, restart SteamVR. It reads steamvr.vrsettings (~/.config/openvr/config on the Frame) for blocked_by_safe_mode, a disabled driver, or SteamVR's own safe mode. When none of those is set but SteamVR is running (the helper answers) and @ft_pointer doesn't, it says SteamVR runs without the driver: unblocked but not restarted yet, or not installed. The driver ignores the "ping" it sends. It checks at startup and every 30 minutes, since the driver only changes when SteamVR restarts. Fixes #4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
f01c84f9a3 |
Use the other eye when the tracker loses one, and add a headset fit check
SteamVR's combined gaze keeps going on one eye, but it holds the lost eye's yaw, so the gaze moves half as far sideways as the eyes do. ft-gaze now reads each eye's tracking uncertainty and raw measurement from eye-server.mmap. When the tracker loses an eye, ft-gazed takes the gaze from the other one, plus the offset that eye usually shows against both, learned while both are seen. On a recording, one eye alone came out a median 0.8 degrees from both eyes' gaze. Glances down at the keyboard, past every screen, aren't sent. The pointer stays put, and eyes lost there don't count as lost. The gaze probe gets a Headset fit mode. It shows per-eye tracking, openness and confidence, maps where each eye gets lost, gives hints, and has a guided check. The settings app opens it from the Gaze page and shows how often each eye is lost. The probe can also test each eye alone, and its side panel now collapses to a title bar so the dot isn't hidden behind it. Snapping to UI elements is deferred; the mouse drag is the correction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0857a54fab |
Don't let resting controllers take the laser from the mouse
A controller released the 3D mouse's pointer on a single pose sample faster than 0.35 m/s or 2 rad/s, once the mouse had been still for 500 ms. The helper polls about every 8 ms, so one noisy sample was enough: a knock on the desk, or a tracking jump when the headset's cameras pick a resting controller up again (#1). A controller now has to stay over the limit for 100 ms in a row, and only samples with a normal tracking result (Running_OK) count. The new POINTER_CONTROLLER_PICKUP setting (1 by default, 0.5 to 5) scales both limits; it's a slider on the Pointer page of Frametop Input Settings and applies live. A controller picked up for real still gets the laser back through SteamVR's hand role, which follows its touch sensors. The release log now records the speed and spin that triggered it, to tune the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
ed3c360f0d |
Add a Global input switch to the Controllers page
SteamVR's "Enable global input from overlays (Experimental)" could only be turned on from the settings app: its notice, with the button, hid once the setting was on. It's now a switch, and the notice shows only when buttons are mapped and the setting is off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0d50efec45 |
Map the Frame controllers' buttons; make gaze clicks correctable
The Frame controllers aren't input devices on the host, so the pointer helper reads them with SteamVR input (vrbuttons.h, actions/), one action set per button, and sends presses to the relay, which does the mapped action like for a mouse button. Only mapped buttons are taken, at an overlay-global priority (SteamVR's experimental "Enable global input from overlays"), and only outside games unless In games is on. Frametop Input Settings gets a Controllers page to map them. Gaze mode: a left press while the gaze has the pointer isn't sent at once. The pointer stops, you drag it onto what you meant with the button held, and the release clicks there; a press held still for POINTER_GAZE_HOLD (0.5 s) becomes a real press, for drags. The dot shows only while the mouse moves it (POINTER_GAZE_SHOW), while a press is held, and as a pulse per click. Outside games the pointer stays on while gaze mode is on. ft-gazed takes a third of each lesson's offset instead of all of it: in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. Frametop Input Settings gets a Gaze page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
10f21273a6 |
Add gaze mode to the pointer: it goes where you look
MAGIC pointing (Zhai et al. 1999), off by default: with POINTER_GAZE=1, "gaze on", or a button mapped to the new gaze_toggle action, the cursor ray is the corrected gaze from ft-gazed while the gaze has the pointer. Moving the mouse takes the pointer from where the gaze left it; looking more than POINTER_GAZE_RETAKE (5 deg) away with the mouse still gives it back. The pointer is aimed at the gaze, never steered toward it, and only fresh gaze moves it, so a stopped service or a blink leaves it where it is. A mouse nudge of up to POINTER_GAZE_NUDGE_MAX (8 deg) before a click is sent to ft-gazed as a lesson in the eye tracker's error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
cfe5990b12 |
Mark head follow as experimental
It works but is only lightly tested, and what's left is mostly tuning its settings. Frametop Input Settings now says so under the Head follow switch, and the README, design notes, and button action name say it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
9addd8c669 |
Move the head-follow cursor only after the head passes the leash
Easing the reference toward the head all the time moved the cursor on every small head movement, so a cursor parked in a corner of the view drifted. Now nothing moves while the head stays within the leash. Once the head has been past it for POINTER_LEASH_DELAY (0.2 s, so a glance out and back doesn't count), the reference eases all the way to where you face (POINTER_LEASH_RETURN) and the cursor lands back in its place in the view, then the leash waits again. Leaning inside the leash no longer moves the cursor either. Leash 0 stays head-locked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
9f4b61729d |
Settle head follow back on where you look; let the mouse reach 70 degrees
The leash only moved its reference when the head reached the leash's end, so after turning back it sat up to the leash off, and recentring meant overshooting with the head. The reference now eases toward the head's facing (POINTER_LEASH_RETURN, 0.2 s) and never lags more than the leash, so the cursor settles back to its place in the view once the head stops. The mouse can now move the cursor up to POINTER_FOLLOW_REACH (70 degrees) from the middle of the view, up from a fixed 40. Both are sliders on the Pointer page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
137705cbfd |
Add head follow: the pointer comes along when you turn your head
Off by default. With POINTER_FOLLOW=1 (a switch on the Pointer page of Frametop Input Settings, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction leashed POINTER_LEASH_DEG (10) from where you face. Within the leash it stays put in the room; past it, it turns with your head and keeps its offset. A leash of 0 locks it to your view. Head roll is ignored, the cursor stays within 40 degrees of the reference, and it holds still in the room while the left button is down so the head can't nudge a click or a drag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |