From 07ce0d9d59122f9f49afed66ae90e4b567622017 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Fri, 9 Oct 2026 11:12:46 -0600 Subject: [PATCH] Hand ray: the laser stays with one hand, and a pinch clicks where it pointed Live test 2026-10-09: taps took several tries and the laser jumped between hands. The owner's ray dropping for a moment (closing the fingers turns the hand) handed the laser to the other raised hand at once, for good; a pinch by the other hand pressed where the old owner's ray pointed; a pinch whose ray had just dropped became a press at a stale spot, or only woke the pointer. - The owner keeps the laser through POINTER_RAY_GRACE (0.5 s) of its ray being down; the other hand takes it by pinching, by raising its ray while the owner's is down, or after the grace (pointer/helper/rayowner.h). - A pinch on a ray that dropped within the grace, or is on its way up, brings it back (fh_ray.h ShowForPinch) and is a ray pinch. - Aim lock: the press goes along the ray from just before the fingers began to close, after a pose along it has gone out; a tap that ends before its press clicks there too. - Each ray follows one tracked hand through a brief flip of ft-hands' left/right call. - Log: owner changes with the reason, the owner's ray down and back, and one line per pinch saying what it clicked or why not. Co-Authored-By: Claude Opus 5.5 --- docs/reference.md | 2 +- hands/README.md | 2 +- hands/include/fh_ray.h | 33 ++- pointer/helper/ft-pointer.cpp | 324 +++++++++++++++++++-------- pointer/helper/rayowner.h | 263 ++++++++++++++++++++++ pointer/tests/rayowner-test.cpp | 373 ++++++++++++++++++++++++++++++++ session/frametop.conf.example | 1 + 7 files changed, 892 insertions(+), 106 deletions(-) create mode 100644 pointer/helper/rayowner.h create mode 100644 pointer/tests/rayowner-test.cpp diff --git a/docs/reference.md b/docs/reference.md index 468286c..0c25b9e 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -281,7 +281,7 @@ Your hands show over the screens: where a tracked hand is between an eye and a s - `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`. The Hand Recorder's installer (`hands/rec/install.sh`) sets them the same way. `hands/run.sh uncaps` takes them back while neither the services nor the Hand Recorder is installed. `hands/run.sh uninstall` and `hands/rec/install.sh uninstall` run it after removing their own part, and `uninstall.sh` always takes them back. - `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring. - The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live. -- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, `POINTER_PINCH_TYPING`, and the hand ray's `POINTER_HAND_RAY`, `POINTER_RAY_DEADZONE`, `POINTER_RAY_PUSH`, `POINTER_RAY_SCROLL`, and `POINTER_RAY_RELEASE`. The example config explains each. +- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, `POINTER_PINCH_TYPING`, and the hand ray's `POINTER_HAND_RAY`, `POINTER_RAY_DEADZONE`, `POINTER_RAY_PUSH`, `POINTER_RAY_SCROLL`, `POINTER_RAY_RELEASE`, and `POINTER_RAY_GRACE`. The example config explains each. Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md). diff --git a/hands/README.md b/hands/README.md index 73d01dd..c29f996 100644 --- a/hands/README.md +++ b/hands/README.md @@ -227,7 +227,7 @@ Settings in `~/.config/frametop.conf`: - `POINTER_GRIP_GAIN` (1): a grip moves the pointer this many times the hand's angle. - `POINTER_GRIP_BELOW` (0.35): grips that begin more than this many metres below the eyes are ignored, because hands resting on a desk curl like a loose fist. Pinches have no such limit: deliberate ones sat 0.35-0.45 m below the eyes with the elbow resting. - `POINTER_PINCH_TYPING` (1): no pinch begins within this many seconds of a key press, because typing touches thumb to index. -- `POINTER_HAND_RAY` (1), `POINTER_RAY_DEADZONE` (2.5 degrees the ray turns during a pinch before the press drags), `POINTER_RAY_PUSH` (1), `POINTER_RAY_SCROLL` (1; -1 turns scrolling round), `POINTER_RAY_RELEASE` (2 s after the ray goes before a pointer it woke lets go): the hand ray. +- `POINTER_HAND_RAY` (1), `POINTER_RAY_DEADZONE` (2.5 degrees the ray turns during a pinch before the press drags), `POINTER_RAY_PUSH` (1), `POINTER_RAY_SCROLL` (1; -1 turns scrolling round), `POINTER_RAY_RELEASE` (2 s after the ray goes before a pointer it woke lets go), `POINTER_RAY_GRACE` (0.5 s a hand keeps the laser after its ray drops; the other hand's ray takes it after that, or at once by pinching or raising): the hand ray. ## Recordings diff --git a/hands/include/fh_ray.h b/hands/include/fh_ray.h index 994bcfc..2860e2f 100644 --- a/hands/include/fh_ray.h +++ b/hands/include/fh_ray.h @@ -39,7 +39,10 @@ // claim, or the fingertip at an overlay) // not a fist no grip down a grip down // While a ray pinch is held (Hold), the ray stays up whatever the tests say, until hideFor after -// the release. +// the release. A pinch may also bring a ray up (ShowForPinch): one that hid within a moment (the +// pointer helper's grace; not one hidden for "off", "near" or a reset), or one whose show tests +// all pass but not yet for showFor. Closing the fingers turns the hand a little, so a ray near a +// hide limit can drop just as the pinch begins. // // Filter (RayFilter): One Euro filters (Casiez, Roussel and Vogel, CHI 2012) on the ray's yaw and // pitch (minCutoff 0.8 Hz, beta 0.02 per deg/s, dCutoff 1 Hz) and per axis on the origin @@ -288,14 +291,14 @@ public: } } else { if (!sideIsHand) { - Hide("off"); + Hide("off", f.t); } else if (f.near && !held_) { - Hide("near"); + Hide("near", f.t); } else if (keepOk || held_ || f.t < holdUntil_) { failSince_ = -1; } else { if (failSince_ < 0) failSince_ = f.t; - if (f.t - failSince_ >= p_.hideFor - 1e-9) Hide(tests_.fails.empty() ? "pose" : tests_.fails); + if (f.t - failSince_ >= p_.hideFor - 1e-9) Hide(tests_.fails.empty() ? "pose" : tests_.fails, f.t); } } if (shown_) out_ = filter_(raw_, f.t); @@ -303,15 +306,24 @@ public: // Each tick, with the time now on the capture clock: the hand lost (no frame for lostFor) hides it. void Tick(double now, bool sideIsHand) { if (!shown_) return; - if (!sideIsHand) Hide("off"); - else if (!held_ && now - lastFrame_ > p_.lostFor && now >= holdUntil_) Hide("lost"); + if (!sideIsHand) Hide("off", now); + else if (!held_ && now - lastFrame_ > p_.lostFor && now >= holdUntil_) Hide("lost", now); } // A ray pinch is held: the ray stays up until hideFor after it ends. void Hold(bool on, double t) { if (held_ && !on) holdUntil_ = t + p_.hideFor; held_ = on; } - void Reset() { Hide("reset"); } + void Reset() { Hide("reset", -1e9); } + // A pinch began on this hand at t: may its ray show for it (see the top)? within: how long ago + // it may have hidden. + bool CanShowForPinch(double t, double within) const { + return shown_ || showSince_ >= 0 || (revivable_ && t - hidAt_ <= within); + } + // Show it (filter started afresh from the hand now), for a pinch that may (CanShowForPinch). + void ShowForPinch() { + if (!shown_) Show(lastFrame_); + } bool Shown() const { return shown_; } const Ray &Current() const { return out_; } // filtered @@ -329,9 +341,10 @@ private: out_ = filter_(raw_, t); event = "show"; } - void Hide(const std::string &why) { + void Hide(const std::string &why, double t) { if (!shown_) return; shown_ = false, failSince_ = -1, showSince_ = -1, held_ = false, holdUntil_ = 0; + hidAt_ = t, revivable_ = why != "off" && why != "near" && why != "reset"; event = "hide " + why; } int side_; @@ -342,8 +355,8 @@ private: std::string why_; V3 lastK_; double lastT_ = 0, lastFrame_ = -1e9, lastStereo_ = -1e9, reach_ = 0; - double showSince_ = -1, failSince_ = -1, holdUntil_ = 0; - bool haveLast_ = false, shown_ = false, held_ = false; + double showSince_ = -1, failSince_ = -1, holdUntil_ = 0, hidAt_ = -1e9; + bool haveLast_ = false, shown_ = false, held_ = false, revivable_ = false; }; } // namespace fhray diff --git a/pointer/helper/ft-pointer.cpp b/pointer/helper/ft-pointer.cpp index 1044636..d7ca6b9 100644 --- a/pointer/helper/ft-pointer.cpp +++ b/pointer/helper/ft-pointer.cpp @@ -286,18 +286,29 @@ // pages, overlays and Frametop's panels like a controller's laser. The model (body, shoulders, the // pointing tests, filters) is hands/include/fh_ray.h, shared with ft-screens. Per side, from the // hands file at each new frame; a side whose controller is in use (Handoff), hands not allowed (a -// game), or the mouse used within a second, has no ray. The first ray to show drives the pointer; -// a pinch by the other hand's ray takes it over. +// game), or the mouse used within a second, has no ray. Each side's ray follows one tracked hand +// through a brief flip of ft-hands' left/right call. Which ray drives the pointer is rayowner.h's: +// the first ray to show; its owner keeps it through POINTER_RAY_GRACE (0.5 s) of its ray being down +// (the pointer holds that ray's last pose; closing the fingers can drop a ray for a moment), and +// the other hand takes it by pinching, by raising its ray while the owner's is down, or once the +// grace runs out. Changes go to the log ("hand ray: right -> left (left pinched)"), and so do the +// owner's ray going down and coming back. // A ray appearing wakes the pointer (role, show) and claims the laser, unless a controller that // pressed a button within 2 s has it. A controller's trigger then takes the laser natively: our dot // hides (rayInactive) until a ray pinch claims it back. The pose is the ray's own origin and // direction (the beam shows, at POINTER_LASER_WIDTH), not the eye-to-cursor trick; our dot shows // only where the ray meets a panel, and nothing catches the laser in free space. -// Gestures with the ray up are ray gestures, decided once at the gesture's begin: a pinch is the -// left button, pressed 30 ms after a claim it had to make (the claim lands in 11-15 ms), held as -// long as the pinch, so a tap clicks and a hold drags. The ray's direction holds still at the press -// until it has turned POINTER_RAY_DEADZONE (2.5 deg; the evaluation saw ray shifts of 1.3-1.6 deg -// median, 2.4-3.5 at the 90th percentile, 150 ms into a pinch), then follows at gain 1. A middle +// Gestures by a hand whose ray is up, or may come up for it (it dropped within the grace, or is on +// its way up: fh_ray.h's ShowForPinch; it then shows), are ray gestures, decided once at the +// gesture's begin. A pinch by the other hand takes the ray and presses along its own ray. A pinch +// is the left button, pressed along the ray as it was just before the fingers began to close (the +// aim lock, rayowner.h), once a pose along it has gone out and 30 ms after a claim it had to make +// (the claim lands in 11-15 ms), held as long as the pinch, so a tap clicks and a hold drags; a +// tap that ends before its press could go out clicks then (released 40 ms on). The direction holds +// still at the lock until the ray has turned POINTER_RAY_DEADZONE (2.5 deg) from where it was at +// the begin (the evaluation saw ray shifts of 1.3-1.6 deg median, 2.4-3.5 at the 90th percentile, +// 150 ms into a pinch), then follows at gain 1. Each pinch logs one line: what it did ("pinch +// right: clicked on frametop.screen.0 (right ray, aim 100 ms before, held 0.15 s)") or why not. A middle // pinch (thumb to middle finger, fh_gestures v3) is a right click on a tap; held and moved, it // scrolls: the virtual stick deflects with the hand's displacement from the begin (0.5 cm dead zone, // full at 4 cm; up and down, and across the ray), times POINTER_RAY_SCROLL (1; -1 turns it round). @@ -373,7 +384,7 @@ // POINTER_HANDS (0), POINTER_PINCH_GAIN (0.5), POINTER_PINCH_DEADZONE (1.5 deg), // POINTER_GRIP_GAIN (1), POINTER_GRIP_BELOW (0.35 m), POINTER_PINCH_TYPING (1 s): hands, above. // POINTER_HAND_RAY (1), POINTER_RAY_DEADZONE (2.5 deg), POINTER_RAY_PUSH (1), POINTER_RAY_SCROLL (1), -// POINTER_RAY_RELEASE (2 s): the hand ray, above. +// POINTER_RAY_RELEASE (2 s), POINTER_RAY_GRACE (0.5 s): the hand ray, above. // POINTER_GAZE_MOUSE (precision), POINTER_HEAD_DEADZONE (0.5 deg), POINTER_KEY_TAP (0.25 s), // POINTER_ROLE (right): gaze precision and keyboard clicks, above. #include @@ -381,6 +392,7 @@ #include "../../hands/include/fh_ray.h" #include "handoff.h" #include "handsin.h" +#include "rayowner.h" #include "vrbuttons.h" #include "vrmath.h" @@ -793,9 +805,9 @@ int main() { bool handsOn = false; double pinchGain = 0.5, pinchDeadzone = 1.5, gripGain = 1.0, handBelow = 0.35, typingHold = 1.0; // The hand ray (see "Hand ray" at the top): POINTER_HAND_RAY, POINTER_RAY_DEADZONE, POINTER_RAY_PUSH, - // POINTER_RAY_SCROLL, POINTER_RAY_RELEASE. + // POINTER_RAY_SCROLL, POINTER_RAY_RELEASE, POINTER_RAY_GRACE. bool rayOn = true, rayPush = true; - double rayDeadzone = 2.5, rayScroll = 1.0, rayRelease = 2.0; + double rayDeadzone = 2.5, rayScroll = 1.0, rayRelease = 2.0, rayGrace = 0.5; // Gaze precision (see the top): POINTER_GAZE_MOUSE (precision: the mouse's left button // holds back its press in gaze mode; direct: it clicks at once), POINTER_ROLE. bool gazeMousePrecision = true; @@ -840,6 +852,7 @@ int main() { rayPush = ConfDouble(conf, "POINTER_RAY_PUSH", 1) != 0; rayScroll = std::clamp(ConfDouble(conf, "POINTER_RAY_SCROLL", 1.0), -5.0, 5.0); rayRelease = std::clamp(ConfDouble(conf, "POINTER_RAY_RELEASE", 2.0), 0.0, 60.0); + rayGrace = std::clamp(ConfDouble(conf, "POINTER_RAY_GRACE", 0.5), 0.0, 3.0); const auto gm = conf.find("POINTER_GAZE_MOUSE"); gazeMousePrecision = gm == conf.end() || gm->second != "direct"; const auto mm = conf.find("POINTER_GAZE_MOUSE_MOVE"); @@ -1031,11 +1044,17 @@ int main() { bool keyDrag = false; // gaze_right pressed while gaze_left aimed: a left press, either key ends it double pressYaw = 0, pressPitch = 0; // the pointer at the press (a quick tap clicks there) // A ray hold (see "Hand ray" at the top): kind 0 a pinch (the left button), 2 a middle pinch - // (right click or scroll); its press waits for the laser claim (pressAt); scrolling, and the - // pinch point in the room at the begin; push and pull (pushOn, from reach h0 and distance d0). - int rayKind = 0; - bool pendingPress = false, scrolling = false, pushOn = false; - Clock::time_point pressAt{}; + // (right click or scroll); its press waits for the laser claim (pressAt) and for a pose along + // the locked aim to have gone out (aimSent); scrolling, and the pinch point in the room at the + // begin; push and pull (pushOn, from reach h0 and distance d0). raySlot: the ray's side (the + // ray that follows the pinching hand; side is the gesture's). released: the pinch ended before + // its press went out, which then clicks (released at releaseAt). For the log: took (it took + // the ray from the other hand), lockAge (the aim's age at the begin, s; < 0 none), the hit. + int rayKind = 0, raySlot = -1; + bool pendingPress = false, scrolling = false, pushOn = false, aimSent = false, released = false, took = false; + Clock::time_point pressAt{}, releaseAt{}, beganAt{}; + double lockAge = -1; + std::string pressHit; Vec3 beginPoint; double h0 = 0, d0 = 0, scrollX = 0, scrollY = 0; Vec3 origin; @@ -1087,6 +1106,12 @@ int main() { // pointer, and what happened to the laser. fhray::Body body; fhray::HandRay rays[2] = {fhray::HandRay(0), fhray::HandRay(1)}; + // Which ray drives the pointer, which hand each ray follows, and each ray's recent aims for a + // pinch's aim lock (rayowner.h); rayHid: why each ray last hid, for the log. + rayown::Owner rayOwner; + rayown::Slots raySlots; + rayown::AimHistory rayAims[2]; + std::string rayHid[2]; bool rayInactive = false; // a controller took the laser: our dot hides until a pinch claims bool rayWoke = false; // the ray woke the pointer (it goes again when the ray has gone a while) Clock::time_point rayGoneAt{}; // when the last ray went @@ -1102,7 +1127,6 @@ int main() { }; std::vector pendingBegins; bool gripDown[2] = {false, false}; // a grip is down on that side (the gestures file, last read) - Clock::time_point rayShownAt[2] = {}; // when each side's ray last showed (the first one keeps the pointer) bool rayHoldCancel = false; // the driving ray went mid-hold: the hold ends, as lost // Let go of the virtual controller (its role and laser), as the other releases in the main loop do. auto releasePointer = [&](const char *why) { @@ -1139,13 +1163,9 @@ int main() { } else if (!laserOurs() && !controllerBusy(t) && !claimPending && !claimHeld) { claimPending = true, claimAt = t; } - if (debug) std::printf("ray %s drives the pointer\n", s ? "right" : "left"); - if (debug) std::fflush(stdout); }; auto rayLeave = [&](bool controller) { if (raySide < 0) return; - if (debug) std::printf("ray %s let go of the pointer%s\n", raySide ? "right" : "left", controller ? " (controller)" : ""); - if (debug) std::fflush(stdout); raySide = -1; rayGoneAt = Clock::now(); rayHoldCancel = true; @@ -1154,6 +1174,19 @@ int main() { // as when a controller moves (last used wins), unless the mouse is in use if (controller && active && Clock::now() - lastMouse > std::chrono::seconds(1)) releasePointer("hand ray off (a controller, or a game)"); }; + // The ray owner's changes (rayowner.h), to the log, one line each; apply: the pointer follows + // (rayEnter, rayLeave). + auto ownerChanges = [&](bool apply) { + for (const std::string &l : rayOwner.log) std::printf("%s\n", l.c_str()); + for (const rayown::Change &c : rayOwner.changes) { + std::printf("hand ray: %s -> %s (%s)\n", rayown::SideName(c.from), rayown::SideName(c.to), c.why.c_str()); + if (!apply) continue; + if (c.to >= 0) rayEnter(c.to); + else rayLeave(c.off); + } + if (!rayOwner.log.empty() || !rayOwner.changes.empty()) std::fflush(stdout); + rayOwner.log.clear(), rayOwner.changes.clear(); + }; // Gaze mode outside games, and its dot (see the top): lastMove/lastHeld/pulseAt. bool inGame = false, gazeAwake = false, gameSent = false; Clock::time_point inGameAt{}, gazeAwakeAt{}, gameSentAt{}; @@ -1482,6 +1515,8 @@ int main() { if (raySide >= 0 || rays[0].Shown() || rays[1].Shown()) { rays[0].Reset(), rays[1].Reset(); rays[0].event.clear(), rays[1].event.clear(); + rayOwner.Drop("the mouse"); + ownerChanges(false); if (raySide >= 0) raySide = -1, rayGoneAt = Clock::now(), rayHoldCancel = true; } rayWoke = false; @@ -2007,11 +2042,14 @@ int main() { const double tc = double(handsRoom.capture_ns) / 1e9; const Vec3 hp = Position(capHead), hf = Rotate(capHead, Vec3{0, 0, -1}); body.Update(tc, {hp.x, hp.y, hp.z}, {hf.x, hf.y, hf.z}); + // the hand each side's ray follows: ft-hands' side call, held through a brief flip (rayowner.h) + std::vector seenHands; + for (const hin::RoomHand &c : handsRoom.hands) seenHands.push_back({c.id, c.side}); + int pick[2]; + raySlots.Update(tc, seenHands, pick); for (int s = 0; s < 2; ++s) { - const hin::RoomHand *h = nullptr; - for (const hin::RoomHand &c : handsRoom.hands) - if (c.side == s) h = &c; - if (!h) continue; + if (pick[s] < 0) continue; + const hin::RoomHand *h = &handsRoom.hands[size_t(pick[s])]; auto fv = [](Vec3 v) { return fhray::V3{v.x, v.y, v.z}; }; fhray::HandFrame f; f.t = tc, f.stereo = h->stereo; @@ -2038,33 +2076,36 @@ int main() { } } rays[s].Frame(body, f, mayPoint(s)); + rayAims[s].Add(tc, rays[s].Current(), rays[s].Shown(), h->pinchD[0]); } } for (int s = 0; s < 2; ++s) { rays[s].Tick(secsOf(tnow), mayPoint(s)); if (rays[s].event.empty()) continue; - if (rays[s].event == "show") rayShownAt[s] = tnow; + if (rays[s].event.rfind("hide ", 0) == 0) rayHid[s] = rays[s].event.substr(5); if (debug) { const fhray::Tests &ts = rays[s].LastTests(); std::printf("ray %s %s (below %.2f m, ahead %.2f m, palm %.2f, pitch %.0f, yaw off %.0f)\n", s ? "right" : "left", rays[s].event.c_str(), ts.below, ts.ahead, ts.palm, ts.pitch, ts.yawOff); std::fflush(stdout); } - // "hide off": its side has a controller in use, or hands aren't allowed now (a VR game) - if (raySide == s && !rays[s].Shown()) rayLeave(rays[s].event == "hide off"); rays[s].event.clear(); } - if (raySide < 0) { - int pick = -1; - for (int s = 0; s < 2; ++s) - if (rays[s].Shown() && (pick < 0 || rayShownAt[s] < rayShownAt[pick])) pick = s; - if (pick >= 0) rayEnter(pick); - } + // Which ray drives the pointer: the owner keeps it through a grace after its ray goes down + // (rayowner.h); one whose side may not point (a controller in use, a VR game) leaves at once. + rayown::Params op; + op.grace = rayGrace; + rayOwner.SetParams(op); + const bool shown[2] = {rays[0].Shown(), rays[1].Shown()}, may[2] = {mayPoint(0), mayPoint(1)}; + rayOwner.Update(secsOf(tnow), shown, may, rayHid); + ownerChanges(true); if (raySide >= 0 && !active && !headsetOff) { // a wake that was held off (just released): again wake(tnow); rayWoke = active; } - } else if (raySide >= 0) { + } else if (raySide >= 0 || rayOwner.Side() >= 0) { + rayOwner.Drop("hand input off"); + ownerChanges(false); rayLeave(false); } // The ray woke the pointer and has been gone a while (no mouse, no gaze meanwhile): release it. @@ -2142,18 +2183,31 @@ int main() { hold.lastYaw = yaw, hold.lastPitch = pitch; lastMove = tnow; // the dot shows while it moves (gaze mode) }; + // One line per pinch (index or middle) in the log: what it did (clicked where), or why not. + auto pinchLog = [&](int side, int kind, const std::string &what) { + std::printf("%s %s: %s\n", kind == 2 ? "middle pinch" : "pinch", side ? "right" : "left", what.c_str()); + std::fflush(stdout); + }; + // A pinch held with no ray up (see "Hands" at the top); its log line says so. auto beginHold = [&](int side, bool grip, const fh_pinch_t &g) { handUsed = tnow; if (hold.src != Src::None) { // A grip takes over a pinch on the way to it (closing the hand passes through // one); otherwise one hold at a time. - if (hold.src != Src::Hand || !grip || hold.grip) return; + if (hold.src != Src::Hand || !grip || hold.grip) { + if (!grip) pinchLog(side, 0, "ignored (no ray up; a press is already held)"); + return; + } aimHeld = aimHand = false; hold = {}; } - if (leftHeld || aimHeld || clickPress || clickRelease) return; // the mouse's button is busy + if (leftHeld || aimHeld || clickPress || clickRelease) { // the mouse's button is busy + if (!grip) pinchLog(side, 0, "ignored (no ray up; the button is held)"); + return; + } if (!active) { wake(tnow); // the first gesture only wakes the pointer, like the first mouse move + if (!grip) pinchLog(side, 0, "no ray up: it woke the pointer, no click"); return; } vr::HmdMatrix34_t head; @@ -2163,7 +2217,7 @@ int main() { // puts them straight ahead in the head's frame, where ft-hands can't tell). const Vec3 at = Position(head) + Rotate(head, Vec3{g.begin_point[0], g.begin_point[1], g.begin_point[2]}); if (!grip && tnow - lastTyping < std::chrono::duration(typingHold)) { - if (debug) std::printf("hand pinch ignored: typing\n"); + pinchLog(side, 0, "ignored (no ray up; typing)"); return; } if (grip && Position(head).y - at.y > handBelow) { @@ -2177,6 +2231,9 @@ int main() { // A grip, or a pinch without gaze mode: a real press where the pointer is (where // you look, in gaze mode), dragging with the hand until it opens. startHold(grip || !gazeOn); + if (!grip) + pinchLog(side, 0, gazeOn ? "no ray up: held back at the gaze, clicks where the pointer is at the release" + : "no ray up: a press where the pointer is, on " + (lastHit.empty() ? std::string("nothing") : lastHit)); if (debug) std::printf("hand %s %s began\n", side ? "right" : "left", grip ? "grip" : "pinch"); if (debug) std::fflush(stdout); }; @@ -2260,7 +2317,10 @@ int main() { keyPresses.clear(); if (calOpened) { // the calibration panel came up: a press in progress ends, no click calOpened = false; - if (hold.src == Src::Ray) rays[hold.side].Hold(false, secsOf(tnow)), SendTo(out, "ft_pointer", "scroll 0 0"); + if (hold.src == Src::Ray) { + pinchLog(hold.side, hold.rayKind, "no click (the gaze calibration panel came up)"); + rays[hold.raySlot].Hold(false, secsOf(tnow)), SendTo(out, "ft_pointer", "scroll 0 0"); + } if (hold.src != Src::None) endHold(true); if (leftHeld) releaseLeft(); aimHeld = aimHand = clickPress = false; @@ -2278,6 +2338,7 @@ int main() { } fh_gestures_t hg; const bool handOk = handsOn && handFile.Read(hg); + const char *const kLostPinch = "ft-hands lost the hand, or a grip took the pinch"; // A ray hold (see "Hand ray" at the top): the pinch's press goes out once the laser claim // has landed; a middle pinch is a right click (a tap) or scrolling (held and moved). auto rayRoom = [&](const float p[3], uint64_t t_ns, Vec3 &out) { @@ -2286,14 +2347,34 @@ int main() { out = hin::ToRoom(head, p); return true; }; - auto beginRayHold = [&](int side, int kind, const fh_pinch_t &g) { + // A ray pinch (see "Hand ray" at the top). side: the gesture's; rs: the ray's (the one that follows + // the pinching hand). The aim is locked where the ray pointed just before the fingers began to + // close (rayowner.h), and the dead zone counts from the ray now. + auto beginRayHold = [&](int side, int rs, int kind, const fh_pinch_t &g) { handUsed = tnow; - if (hold.src != Src::None || leftHeld || aimHeld || clickPress || clickRelease) return; - if (raySide != side) rayEnter(side); // a pinch by the other hand claims the pointer + if (hold.src != Src::None || leftHeld || aimHeld || clickPress || clickRelease) { + pinchLog(side, kind, "ignored (a press is already held)"); + return; + } + const bool took = rayOwner.Pinched(rs, true) == rayown::Owner::Pinch::Take; + rays[rs].ShowForPinch(); // a ray that dropped as the fingers closed comes back + rays[rs].event.clear(); + ownerChanges(true); // a pinch by the other hand takes the ray (rayEnter) Hold h; - h.src = Src::Ray, h.side = side, h.rayKind = kind, h.pressed = false; + h.src = Src::Ray, h.side = side, h.raySlot = rs, h.rayKind = kind, h.pressed = false; + h.took = took, h.beganAt = tnow; + auto angles = [](const fhray::V3 &d, double &y, double &p) { + const Vec3 n = Normalize({d.x, d.y, d.z}); + y = std::atan2(-n.x, -n.z) * 180 / M_PI; + p = std::clamp(std::asin(std::clamp(n.y, -1.0, 1.0)) * 180 / M_PI, -85.0, 85.0); + }; + angles(rays[rs].Current().dir, h.refYaw, h.refPitch); + fhray::Ray lock; + if (rayAims[rs].Lock(double(g.begin_ns) / 1e9, handOk && hg.end_m > 0 ? hg.end_m : 0.035, lock, h.lockAge)) + angles(lock.dir, yaw, pitch); + else + yaw = h.refYaw, pitch = h.refPitch, h.lockAge = -1; h.startYaw = h.lastYaw = yaw, h.startPitch = h.lastPitch = pitch; - h.refYaw = yaw, h.refPitch = pitch; if (!rayRoom(g.begin_point, g.begin_ns, h.beginPoint)) h.beginPoint = rayOrigin; // the laser: a pinch always claims it (if it isn't ours), and the press waits 30 ms for it rayInactive = false; @@ -2302,35 +2383,62 @@ int main() { h.pendingPress = kind == 0; h.pressAt = tnow + std::chrono::milliseconds(ours_ ? 0 : 30 + (claimHeld ? 0 : 8)); hold = h; - rays[side].Hold(true, secsOf(tnow)); - if (debug) std::printf("ray %s %s began\n", side ? "right" : "left", kind ? "middle pinch" : "pinch"); + rays[rs].Hold(true, secsOf(tnow)); + if (debug) std::printf("ray %s %s began\n", rs ? "right" : "left", kind ? "middle pinch" : "pinch"); if (debug) std::fflush(stdout); }; - auto endRayHold = [&](bool lost) { - const int side = hold.side; - if (hold.rayKind == 0) { - if (hold.pendingPress && !lost) clickPress = true; // a tap before the press went out: a click - else if (hold.pressed && leftHeld) releaseLeft(); - } else { - if (hold.scrolling) SendTo(out, "ft_pointer", "scroll 0 0"); - else if (!lost) pressRight = clickPress = true; // a middle tap: a right click, where the ray is - } - if (debug) - std::printf("ray %s %s %s\n", side ? "right" : "left", hold.rayKind ? "middle pinch" : "pinch", - lost ? "lost" : hold.scrolling ? "scrolled" : "released"); - if (debug) std::fflush(stdout); + // What a ray pinch did, for its log line. + auto rayPinchSays = [&](const char *what) { + char b[256]; + const double held = std::chrono::duration(tnow - hold.beganAt).count(); + char aim[48] = "aim now"; + if (hold.lockAge >= 0) std::snprintf(aim, sizeof aim, "aim %.0f ms before", hold.lockAge * 1000); + std::snprintf(b, sizeof b, "%s (%s ray%s, %s, held %.2f s)", what, hold.raySlot ? "right" : "left", + hold.took ? ", took it" : "", aim, held); + return std::string(b); + }; + auto hitName = [&](const std::string &k) { return k.empty() ? std::string("nothing") : k; }; + auto finishRayHold = [&] { + const int rs = hold.raySlot; hold = {}; - if (side >= 0) rays[side].Hold(false, secsOf(tnow)); + if (rs >= 0) rays[rs].Hold(false, secsOf(tnow)); + }; + // A ray pinch ended (lost: by losing the hand, a grip, or the ray; why says which). + auto endRayHold = [&](bool lost, const char *why) { + const int side = hold.side, kind = hold.rayKind; + if (!lost && !hold.pressed && (kind == 0 || !hold.scrolling)) { + // a tap before its press went out (or a middle tap): it clicks once the press can go + // (the press block below), along the locked aim + hold.released = hold.pendingPress = true; + return; + } + if (kind == 0 && hold.pressed && leftHeld) releaseLeft(); + if (kind == 2 && hold.scrolling) SendTo(out, "ft_pointer", "scroll 0 0"); + std::string what; + if (kind == 2 && hold.scrolling) what = "scrolled"; + else if (!hold.pressed) what = std::string("no click: ") + why; + else if (lost) what = "pressed on " + hitName(hold.pressHit) + ", let go: " + why; + else what = (hold.engaged ? "dragged from " : "clicked on ") + hitName(hold.pressHit); + pinchLog(side, kind, rayPinchSays(what.c_str())); + finishRayHold(); }; // A gesture that began: by the ray, held back like the gaze (beginHold), or nothing. + // rayFor: the ray that follows the gesture's hand (rayowner.h's Slots), else its side's. + auto rayFor = [&](int s, const fh_pinch_t &g) { + const int r = raySlots.SideOf(g.hand_id); + return r >= 0 ? r : s; + }; + auto canRay = [&](int rs) { return rays[rs].CanShowForPinch(secsOf(tnow), rayGrace); }; auto startGesture = [&](int s, int kind, const fh_pinch_t &g, bool ray) { - ray = ray && rays[s].Shown(); // (a ray that went while ft-screens was asked: not a ray gesture) + const int rs = rayFor(s, g); + // (and still: a ray that went for good while ft-screens was asked isn't; a grip needs it up) + ray = ray && (kind == 1 ? rays[rs].Shown() : canRay(rs)); if (ray && kind != 1) { - beginRayHold(s, kind, g); + beginRayHold(s, rs, kind, g); } else if (ray || kind == 2) { // a fist with the ray up, or a middle pinch without it: nothing - if (debug) std::printf("hand %s %s ignored (%s)\n", s ? "right" : "left", kind == 1 ? "grip" : "middle pinch", - ray ? "the ray is up" : "no ray"); + if (kind == 2) pinchLog(s, kind, "ignored (no ray up)"); + else if (debug) std::printf("hand %s grip ignored (the ray is up)\n", s ? "right" : "left"); } else { beginHold(s, kind == 1, g); } @@ -2340,13 +2448,13 @@ int main() { if (hold.src == Src::Ray) return hold.rayKind == kind; return hold.src == Src::Hand && hold.grip == (kind == 1); }; - auto endGesture = [&](bool lost) { - if (hold.src == Src::Ray) endRayHold(lost); + auto endGesture = [&](bool lost, const char *why) { + if (hold.src == Src::Ray) endRayHold(lost, why); else endHold(lost); }; - if (rayHoldCancel) { // the driving ray went (a controller, the hand lost): its hold ends, as lost + if (rayHoldCancel) { // the driving ray went (a controller, the mouse, hand input off): its hold ends, as lost rayHoldCancel = false; - if (hold.src == Src::Ray) endRayHold(true); + if (hold.src == Src::Ray) endRayHold(true, "the ray went (a controller, the mouse, or hand input off)"); } if (handOk && (hg.seq != handSeq || handFile.opens != handOpens)) { handSeq = hg.seq; @@ -2360,7 +2468,7 @@ int main() { for (int s = 0; s < 2; ++s) rebase = rebase || slots[k][s].begins < seenBegins[k][s] || slots[k][s].ends < seenEnds[k][s]; if (rebase) { - if (hold.src == Src::Hand || hold.src == Src::Ray) endGesture(true); + if (hold.src == Src::Hand || hold.src == Src::Ray) endGesture(true, "ft-hands started over"); pendingBegins.clear(); for (int k = 0; k < 3; ++k) for (int s = 0; s < 2; ++s) seenBegins[k][s] = slots[k][s].begins, seenEnds[k][s] = slots[k][s].ends; @@ -2377,10 +2485,14 @@ int main() { const bool began = g.begins != seenBegins[k][s], ended = g.ends != seenEnds[k][s]; const bool lost = g.flags & FH_PINCH_LOST; seenBegins[k][s] = g.begins, seenEnds[k][s] = g.ends; - if (holdingKind(s, k) && ended) endGesture(lost); // the one held ended (another may have begun) + if (holdingKind(s, k) && ended) endGesture(lost, kLostPinch); // the one held ended (another may have begun) for (PendingBegin &p : pendingBegins) if (p.side == s && p.kind == k && ended) p.ended = true, p.lost = lost; - if (!began || !allowed) continue; + if (!began) continue; + if (!allowed) { + if (k != 1) pinchLog(s, k, "ignored (hands not allowed: a VR game, or the headset off)"); + continue; + } // Not on a side that has a controller in use, and not by a hand at a controller // (reaching for it or holding it: by position, whatever ft-hands calls its side). bool atController = arbiter.IsController(s); @@ -2388,25 +2500,30 @@ int main() { if (!atController && rayRoom(g.begin_point, g.begin_ns, at)) atController = arbiter.NearController({at.x, at.y, at.z}); if (atController) { - if (debug) std::printf("hand %s %s ignored: at a controller\n", s ? "right" : "left", - k == 1 ? "grip" : k ? "middle pinch" : "pinch"); + if (k != 1) pinchLog(s, k, "ignored (at a controller)"); + else if (debug) std::printf("hand %s grip ignored: at a controller\n", s ? "right" : "left"); continue; } - // Ray or gaze is decided once, here, from the state at the begin. - const bool ray = raySide >= 0 ? rays[s].Shown() : rays[s].Shown() && active; + // Ray or gaze is decided once, here, from the state at the begin: a ray pinch when the + // pinching hand's ray is up, or may come up for it (fh_ray.h's ShowForPinch). + const bool ray = canRay(rayFor(s, g)); if (screensNear && nearNs < g.begin_ns) { PendingBegin p{s, k, g, ray, tnow + std::chrono::milliseconds(60)}; p.ended = !(g.flags & FH_PINCH_DOWN), p.lost = lost; pendingBegins.push_back(p); continue; } - if (screensNear && nearSide[s]) continue; // ft-screens' (a grab or a poke) + if (screensNear && nearSide[s]) { // ft-screens' (a grab or a poke) + if (k != 1) pinchLog(s, k, "left to ft-screens (the hand is near its panel)"); + continue; + } startGesture(s, k, g, ray); // begun and ended since the last read (a quick tap): its release too - if (!(g.flags & FH_PINCH_DOWN) && holdingKind(s, k)) endGesture(lost); + if (!(g.flags & FH_PINCH_DOWN) && holdingKind(s, k)) endGesture(lost, kLostPinch); } // A side that turns controller mid-gesture: the gesture ends, as lost. - if ((hold.src == Src::Hand || hold.src == Src::Ray) && arbiter.IsController(hold.side)) endGesture(true); + if ((hold.src == Src::Hand || hold.src == Src::Ray) && arbiter.IsController(hold.side)) + endGesture(true, "its side has a controller in use"); // The held gesture's hand moves the pointer: past the dead zone (a tap's jitter, // the pinch point shifting as the fingers close), then at the gain, from there. if (hold.src == Src::Hand) { @@ -2452,43 +2569,61 @@ int main() { const PendingBegin q = p; pendingBegins.erase(pendingBegins.begin() + long(i)); if (reported && nearSide[q.side]) { - if (debug) std::printf("hand %s gesture left to ft-screens (near)\n", q.side ? "right" : "left"); + if (q.kind != 1) pinchLog(q.side, q.kind, "left to ft-screens (the hand is near its panel)"); + else if (debug) std::printf("hand %s grip left to ft-screens (near)\n", q.side ? "right" : "left"); continue; } startGesture(q.side, q.kind, q.g, q.ray); - if (q.ended && holdingKind(q.side, q.kind)) endGesture(q.lost); + if (q.ended && holdingKind(q.side, q.kind)) endGesture(q.lost, kLostPinch); } - // A ray pinch's press, once the claim has had its 30 ms. + // A ray pinch's press, once the claim has had its 30 ms and a pose along the locked aim has gone + // out (so the press lands there, and lastHit is what it hits). // (and once a pointer the ray just woke has been bound: the wake's claim comes 300 ms on) - if (hold.src == Src::Ray && hold.pendingPress && tnow >= hold.pressAt && active && + // A pinch that already ended (a tap, or a middle tap: the right button) clicks: released 40 ms on. + if (hold.src == Src::Ray && hold.pendingPress && hold.aimSent && tnow >= hold.pressAt && active && tnow - wokeAt > std::chrono::milliseconds(350)) { hold.pendingPress = false; hold.pressed = true; gazeBack = false; + pressRight = hold.rayKind == 2; pressLeft(); - // push and pull: on a carry (a press on no panel's content: a grab bar, a scene-graph - // control, an edge, one of ft-screens' controls); a panel that starts to move turns it on below - hold.h0 = rays[hold.side].Reach(), hold.d0 = lastDistance; - hold.pushOn = rayPush && (lastHit.empty() || pass.onEdge || pass.scene || (lastHit.rfind("frametop.", 0) == 0 && !FramePanel(lastHit))); + hold.pressHit = lastHit; + if (hold.released) { + hold.releaseAt = tnow + std::chrono::milliseconds(40); + } else { + // push and pull: on a carry (a press on no panel's content: a grab bar, a scene-graph + // control, an edge, one of ft-screens' controls); a panel that starts to move turns it on below + hold.h0 = rays[hold.raySlot].Reach(), hold.d0 = lastDistance; + hold.pushOn = rayPush && (lastHit.empty() || pass.onEdge || pass.scene || (lastHit.rfind("frametop.", 0) == 0 && !FramePanel(lastHit))); + } + } + if (hold.src == Src::Ray && hold.released && hold.pressed && tnow >= hold.releaseAt) { + if (leftHeld) releaseLeft(); + pinchLog(hold.side, hold.rayKind, + rayPinchSays(((hold.rayKind == 2 ? "right click on " : "clicked on ") + hitName(hold.pressHit)).c_str())); + finishRayHold(); } // ft-hands stopped (or hung) with a gesture held: it's over, as lost. const uint64_t nowNs = uint64_t(std::chrono::duration_cast(tnow.time_since_epoch()).count()); if ((hold.src == Src::Hand || hold.src == Src::Ray) && (!handOk || !active || nowNs - handPublished > 1'500'000'000ull)) - endGesture(true); + endGesture(true, "hand tracking stopped, or the pointer went"); if (hold.src != Src::None && hold.src != Src::Hand && hold.src != Src::Ray && !active) endHold(true); // The driving ray (see "Hand ray" at the top): the pointer's origin and direction. During a ray // pinch the direction holds still until the ray has turned rayDeadzone, then follows (steer); // a middle pinch holds it. Push and pull move the origin along the ray while a carry is held. + // rayMode: a hand's ray has the pointer, driving it or (its owner's grace, rayowner.h) holding + // its last pose. const bool rayDrives = active && raySide >= 0 && rays[raySide].Shown(); + const bool rayMode = active && raySide >= 0; if (rayDrives) { const fhray::Ray &r = rays[raySide].Current(); rayOrigin = {r.origin.x, r.origin.y, r.origin.z}; rayDir = Normalize({r.dir.x, r.dir.y, r.dir.z}); const double ry = std::atan2(-rayDir.x, -rayDir.z) * 180 / M_PI; const double rp = std::asin(std::clamp(rayDir.y, -1.0, 1.0)) * 180 / M_PI; - if (hold.src == Src::Ray && hold.side == raySide) { + if (hold.src == Src::Ray && hold.raySlot == raySide) { if (hold.rayKind == 0) steer(ry, rp, rayDeadzone, 1.0); } else { yaw = ry, pitch = std::clamp(rp, -85.0, 85.0); @@ -2522,7 +2657,7 @@ int main() { // Head follow (see the top): past the leash (for the delay), ease the reference to the // head's facing, and turn the cursor with it. Not in gaze mode: the gaze places it. - if (follow && !gazeOn && active && anchored && hmd.bPoseIsValid && !rayDrives) { + if (follow && !gazeOn && active && anchored && hmd.bPoseIsValid && !rayMode) { const Vec3 head = LimitPitch(Vec3{-hm[0][2], -hm[1][2], -hm[2][2]}, 85); if (followReset) { followRef = head, followLag = 0, leashOutSince = {}; @@ -2747,7 +2882,7 @@ int main() { // depths). Anything in front of the point on the eye's line of sight wins. // Not for a hand's ray: like a controller's, it lands where it points. const double toPoint = std::sqrt(Dot(point - eye, point - eye)); - const Hit front = rayDrives ? Hit{} : nearest(eye, Normalize(point - eye)); + const Hit front = rayMode ? Hit{} : nearest(eye, Normalize(point - eye)); if (front.along < toPoint - 0.02) { occluded = true; bestKey = front.key, bestScene = front.scene; @@ -2765,8 +2900,8 @@ int main() { const bool onScene = !dragging && ((bestScene && best < 1e8) || onEdge); const bool onPanel = dragging || (best < 1e8 && !onScene); const Vec3 sight = Normalize(point - eye); - if (rayDrives) onVrSettings = false; // the beam starts at the hand: SteamVR finds the page itself - if (!dragging && !reuse && !rayDrives) { + if (rayMode) onVrSettings = false; // the beam starts at the hand: SteamVR finds the page itself + if (!dragging && !reuse && !rayMode) { // SteamVR Settings (see the top): the dashboard's main panel is hidden and its // scene-graph panel shows the page. onVrSettings = false; @@ -2791,7 +2926,7 @@ int main() { catcherHidesHit = onVrSettings; } - if (rayDrives) { + if (rayMode) { // A hand's ray (see "Hand ray" at the top): the beam shows, as a controller's does, and // our dot only where it meets a panel. No laser-catching dot in free space: the beam just // ends. None while a controller has taken the laser (rayInactive). @@ -2851,14 +2986,14 @@ int main() { // Controller ray: from the eye, aimed at the cursor point, converted from the // standing universe to raw tracking space via the HMD's pose in both. // A hand's ray: from its origin along its direction (push and pull already in the anchor). - const Vec3 aimStanding = rayDrives ? dir : Normalize(point - eye); + const Vec3 aimStanding = rayMode ? dir : Normalize(point - eye); const auto &S = hmd.mDeviceToAbsoluteTracking, &R = hmdRaw.mDeviceToAbsoluteTracking; const Vec3 aim = Rotate(R, RotateInverse(S, aimStanding)); // raw <- head <- standing // Origin partway along the line of sight to the cursor (smaller hit dot). const double toPoint = std::sqrt(Dot(point - eye, point - eye)); double originDist = std::max(0.0, std::min(toPoint * originFraction, toPoint - originMargin)); if (onVrSettings) originDist = std::min(originDist, SETTINGS_ORIGIN); // SteamVR finds the page - const Vec3 originStanding = rayDrives ? anchor : eye + Normalize(point - eye) * originDist; + const Vec3 originStanding = rayMode ? anchor : eye + Normalize(point - eye) * originDist; const Vec3 eyeRaw = Position(R) + Rotate(R, RotateInverse(S, originStanding - eye)); lastPoint = point, lastOrigin = originStanding, lastAim = aimStanding; // tilt starts from here havePoint = true; @@ -2901,6 +3036,7 @@ int main() { SendTo(out, "ft_pointer", msg); posed = {true, tnow, eyeRaw, ayaw, apitch}; } + if (hold.src == Src::Ray) hold.aimSent = true; // its press can go out next frame (see above) } // A held-back press (see the top): held still long enough, it's a real press (a drag); diff --git a/pointer/helper/rayowner.h b/pointer/helper/rayowner.h new file mode 100644 index 0000000..37417f9 --- /dev/null +++ b/pointer/helper/rayowner.h @@ -0,0 +1,263 @@ +// Which bare hand's ray drives the pointer, which tracked hand each ray follows, and where a ray +// pinch clicks. The pointer helper's side of the hand ray ("Hand ray" in ft-pointer.cpp; the ray +// itself is hands/include/fh_ray.h). Header-only and free of OpenVR, so pointer/tests/rayowner-test.cpp +// can drive it with made-up timelines. +// +// Owner: one ray drives the pointer (one SteamVR laser). With none, the first ray to show takes it. +// The owner keeps it while its ray is up, and through a grace (0.5 s, POINTER_RAY_GRACE) after its +// ray goes down: the pointer holds the ray's last pose meanwhile, and the ray coming back carries on. +// Closing the fingers turns the hand, and a ray near a limit of fh_ray's gate drops for a moment; +// without the grace the other hand's ray took the laser at once and kept it (the laser "jumping +// between hands", live test 2026-10-09). The other hand takes over: +// - when it pinches with its ray up (or able to come up, fh_ray's ShowForPinch): last used wins; +// - when its ray shows while the owner's is down (a deliberate raise; a ray that was up all along +// doesn't count); +// - when the owner's ray has been down for the grace and its ray is up. +// The owner leaves at once when its side may not point (its controller in use, a game, the mouse) or +// its hand went near a panel (fh_ray's "near": ft-screens' poke and grab, no frozen beam meanwhile), +// and after the grace when no other ray is up. +// +// Pinch: a pinch on the owner's hand is a ray pinch (its ray brought back if it dropped within the +// grace). A pinch on the other hand is a ray pinch when that hand's ray is up or can come up: it +// takes the ray, and the click is where its own ray pointed (not where the old owner's did). +// Otherwise it isn't one (the helper's gaze-style hold). +// +// Slots: which of ft-hands' hands each side's ray follows. ft-hands calls each hand left or right +// per frame, and a call can flip for a few frames (or a pair swap): following the call, a ray's +// knuckle point jumped to the other hand and back. A ray keeps the hand it follows (by id) until +// ft-hands has called it the other side for flipHold (0.3 s), or it's gone; then it takes the hand +// called its side that the other ray doesn't follow. +// +// Aim lock: where a ray pinch clicks. ft-hands sees a pinch once the thumb and index tips are +// within begin_m, and closing them turns the hand (1.3-1.6 deg median in the first 150 ms after the +// begin, 2.4-3.5 at the 90th percentile, on recordings; more before it). So the press goes out +// along the ray as it was just before the fingers began to close: the newest frame, within lockBack +// (0.35 s) before the begin, with the tips still lockOpen (1 cm) further apart than the pinch's +// release distance (end_m, 3.5 cm); without a distance (no skeleton) or with fingers resting nearly +// closed, the frame lockLead (0.1 s) before the begin. Only frames with the ray up count. The rays +// here are fh_ray's filtered ones, what the beam showed. +// +// Times are seconds on one steady clock (CLOCK_MONOTONIC: ft-hands' capture times as seconds). +#pragma once + +#include "../../hands/include/fh_ray.h" + +#include +#include +#include +#include +#include + +namespace rayown { + +inline const char *SideName(int s) { return s < 0 ? "none" : s ? "right" : "left"; } + +struct Params { + double grace = 0.5; // the owner's ray down this long: it's gone + double flipHold = 0.3; // a hand called the other side this long moves to that side's ray + double lockBack = 0.35; // aim lock: how far back before a pinch's begin to look + double lockLead = 0.10; // aim lock: the frame this long before the begin, without a distance + double lockOpen = 0.01; // aim lock: the tips this much further apart than end_m: still open +}; + +// A change of owner: from, to (-1 none), why, and off (the old owner's side may not point: the +// pointer goes back at once, see rayLeave in ft-pointer.cpp). +struct Change { + int from, to; + std::string why; + bool off = false; +}; + +class Owner { +public: + explicit Owner(Params p = {}) : p_(p) {} + void SetParams(const Params &p) { p_ = p; } + int Side() const { return side_; } + // The owner's ray is down, within the grace: the pointer holds its last pose. + bool Waiting() const { return side_ >= 0 && downSince_ >= 0; } + + // Each tick. shown: each side's ray is up; may: each side may point at all (fh_ray's sideIsHand); + // why: why each ray is down, for the log (fh_ray's hide reason). + void Update(double t, const bool shown[2], const bool may[2], const std::string why[2]) { + for (int s = 0; s < 2; ++s) { + if (shown[s] && !was_[s]) shownAt_[s] = t; + was_[s] = shown[s]; + } + if (side_ < 0) { + int pick = -1; + for (int s = 0; s < 2; ++s) + if (shown[s] && (pick < 0 || shownAt_[s] < shownAt_[pick])) pick = s; + if (pick >= 0) Set(pick, "raised", false); + return; + } + const int o = side_, other = 1 - o; + if (!may[o]) { + Set(-1, "off: its controller, a game, or the mouse", true); + return; + } + if (!shown[o] && why[o] == "near") { + Set(-1, "near a panel", false); + return; + } + if (shown[o]) { + if (downSince_ >= 0) { + char b[96]; + std::snprintf(b, sizeof b, "hand ray %s: back after %.2f s", SideName(o), t - downSince_); + log.push_back(b); + } + downSince_ = -1; + return; + } + if (downSince_ < 0) { + downSince_ = t, downWhy_ = why[o].empty() ? "pose" : why[o]; + log.push_back(std::string("hand ray ") + SideName(o) + ": down (" + downWhy_ + ")"); + } + if (shown[other] && shownAt_[other] > downSince_) { + Set(other, std::string(SideName(other)) + " raised while " + SideName(o) + "'s was down", false); + } else if (t - downSince_ >= p_.grace - 1e-9) { + char b[128]; + std::snprintf(b, sizeof b, "%s's down %.1f s (%s)%s%s", SideName(o), p_.grace, downWhy_.c_str(), + shown[other] ? ", up: " : "", shown[other] ? SideName(other) : ""); + Set(shown[other] ? other : -1, b, false); + } + } + + enum class Pinch { Ray, Take, None }; + // A pinch began on side s; canRay: its ray is up or may come up for it (fh_ray's CanShowForPinch). + // Ray: the owner's, a ray pinch; Take: the other hand's, it takes the ray and it's a ray pinch; + // None: not a ray pinch. + Pinch Pinched(int s, bool canRay) { + if (!canRay || s < 0 || s > 1) return Pinch::None; + if (s == side_) { + downSince_ = -1; // brought back for the pinch + return Pinch::Ray; + } + Set(s, std::string(SideName(s)) + " pinched", false); + return Pinch::Take; + } + // The pointer went elsewhere (the mouse took it, hands turned off): no owner, at once. + void Drop(const std::string &why) { + if (side_ >= 0) Set(-1, why, false); + } + + std::vector changes; // since the caller last cleared it + std::vector log; // the owner's ray going down and coming back, likewise + +private: + void Set(int to, const std::string &why, bool off) { + if (to == side_) return; + changes.push_back({side_, to, why, off}); + side_ = to, downSince_ = -1; + } + Params p_; + int side_ = -1; + double downSince_ = -1, shownAt_[2] = {-1e9, -1e9}; + bool was_[2] = {false, false}; + std::string downWhy_; +}; + +// One tracked hand this frame, as ft-hands calls it. +struct SeenHand { + uint32_t id = 0; + int side = 0; // 0 left, 1 right +}; + +class Slots { +public: + explicit Slots(Params p = {}) : p_(p) {} + // This frame's hands: which one each side's ray follows (pick[s], an index into hands; -1 none). + void Update(double t, const std::vector &hands, int pick[2]) { + pick[0] = pick[1] = -1; + for (int s = 0; s < 2; ++s) { + if (!bound_[s]) continue; + int at = -1; + for (size_t i = 0; i < hands.size(); ++i) + if (hands[i].id == id_[s]) at = int(i); + if (at < 0) { + bound_[s] = false; // gone + continue; + } + if (hands[at].side == s) { + flipSince_[s] = -1; + } else { + if (flipSince_[s] < 0) flipSince_[s] = t; + if (t - flipSince_[s] >= p_.flipHold - 1e-9) { + bound_[s] = false; // ft-hands has settled on the other side + continue; + } + } + pick[s] = at; + } + for (int s = 0; s < 2; ++s) { + if (pick[s] >= 0) continue; + for (size_t i = 0; i < hands.size(); ++i) + if (hands[i].side == s && int(i) != pick[1 - s]) { + pick[s] = int(i); + break; + } + bound_[s] = pick[s] >= 0; + if (bound_[s]) id_[s] = hands[pick[s]].id, flipSince_[s] = -1; + } + } + // The side whose ray follows hand id, or -1. + int SideOf(uint32_t id) const { + for (int s = 0; s < 2; ++s) + if (bound_[s] && id_[s] == id) return s; + return -1; + } + +private: + Params p_; + uint32_t id_[2] = {0, 0}; + bool bound_[2] = {false, false}; + double flipSince_[2] = {-1, -1}; +}; + +// One side's recent rays, for the aim lock. +class AimHistory { +public: + explicit AimHistory(Params p = {}) : p_(p) {} + // A hands frame at capture time t: the ray (filtered), whether it's up, and the thumb to index + // distance (m; < 0 unknown). + void Add(double t, const fhray::Ray &ray, bool shown, double pinchD) { + h_.push_back({t, ray, shown, pinchD}); + while (!h_.empty() && h_.front().t < t - 1.0) h_.pop_front(); + } + // The ray to press along for a pinch that began at tBegin (see the top); endD: the pinch's release + // distance (end_m). False: no frame with the ray up within lockBack. age: how long before the begin. + bool Lock(double tBegin, double endD, fhray::Ray &out, double &age) const { + const double openD = endD + p_.lockOpen; + const S *open = nullptr, *lead = nullptr; + for (auto it = h_.rbegin(); it != h_.rend(); ++it) { + if (it->t > tBegin + 1e-6) continue; + if (it->t < tBegin - p_.lockBack - 1e-6) break; + if (!it->shown) continue; + if (!open && it->d >= openD) open = &*it; + if (!lead && it->t <= tBegin - p_.lockLead + 1e-6) lead = &*it; + } + const S *pick = open ? open : lead; + if (!pick) { // nothing that far back: the oldest up within the window + for (const S &e : h_) + if (e.shown && e.t <= tBegin + 1e-6 && e.t >= tBegin - p_.lockBack - 1e-6) { + pick = &e; + break; + } + } + if (!pick) return false; + out = pick->ray, age = tBegin - pick->t; + return true; + } + void Clear() { h_.clear(); } + +private: + struct S { + double t; + fhray::Ray ray; + bool shown; + double d; + }; + Params p_; + std::deque h_; +}; + +} // namespace rayown diff --git a/pointer/tests/rayowner-test.cpp b/pointer/tests/rayowner-test.cpp new file mode 100644 index 0000000..3c4d116 --- /dev/null +++ b/pointer/tests/rayowner-test.cpp @@ -0,0 +1,373 @@ +// Unit tests for the hand ray's owner, slots and aim lock (pointer/helper/rayowner.h), and a pinch +// bringing a ray back (hands/include/fh_ray.h's ShowForPinch): made-up timelines at the helper's +// 8 ms frame and ft-hands' 30 Hz. +// Build and run: just test-c (or g++ -std=c++17 -I. pointer/tests/rayowner-test.cpp). +#include "pointer/helper/rayowner.h" + +#include +#include +#include +#include +#include + +using namespace rayown; +using fhray::kDeg; + +static int failures = 0; +#define CHECK(cond, ...) \ + do { \ + if (!(cond)) { \ + std::printf("FAIL %s:%d: ", __FILE__, __LINE__); \ + std::printf(__VA_ARGS__); \ + std::printf("\n"); \ + ++failures; \ + } \ + } while (0) + +namespace { + +constexpr double kDt = 0.008; +constexpr double kFrame = 1 / 30.0; + +// The owner, stepped with each side's ray up or down (and allowed to point). +struct World { + Owner o; + double t = 0; + bool shown[2] = {false, false}, may[2] = {true, true}; + std::string why[2] = {"palm", "palm"}; + std::vector changes; + std::vector log; + explicit World(Params p = {}) : o(p) {} + void Step() { + o.Update(t, shown, may, why); + for (const Change &c : o.changes) changes.push_back(c); + for (const std::string &l : o.log) log.push_back(l); + o.changes.clear(), o.log.clear(); + t += kDt; + } + void Run(double secs, const std::function &each = {}) { + const double end = t + secs - 1e-9; + while (t < end) { + if (each) each(); + Step(); + } + } +}; + +void TestFirstRaised() { + World w; + w.Run(0.2); + CHECK(w.o.Side() == -1 && w.changes.empty(), "no ray: no owner"); + w.shown[1] = true; + w.Step(); + CHECK(w.o.Side() == 1, "the first ray up takes it: %d", w.o.Side()); + CHECK(w.changes.size() == 1 && w.changes[0].from == -1 && w.changes[0].to == 1 && w.changes[0].why == "raised", + "one change, none -> right (raised)"); + // the other one showing later doesn't take it + w.shown[0] = true; + w.Run(2); + CHECK(w.o.Side() == 1 && w.changes.size() == 1, "a second ray up leaves the owner: %d, %zu changes", w.o.Side(), + w.changes.size()); +} + +// The live complaint: both hands up, the owner's ray dropping for a moment as the fingers close. +void TestBriefDropKeepsTheOwner() { + World w; + w.shown[1] = true; + w.Step(); + w.shown[0] = true; // the left hand up too, all along + w.Run(0.5); + w.changes.clear(); + // ten drops of 0.3 s, one a second + for (int i = 0; i < 10; ++i) { + w.Run(0.7); + w.shown[1] = false; + w.Step(); + w.Run(0.29, [&] { CHECK(w.o.Side() == 1 && w.o.Waiting(), "right keeps it while its ray is down"); }); + w.shown[1] = true; + } + w.Run(0.1); + CHECK(w.changes.empty(), "brief drops: no owner change (%zu)", w.changes.size()); + CHECK(!w.o.Waiting(), "back up: not waiting"); + int downs = 0, backs = 0; + for (const std::string &l : w.log) { + downs += l.find(": down (palm)") != std::string::npos; + backs += l.find(": back after") != std::string::npos; + } + CHECK(downs == 10 && backs == 10, "each drop logs once down, once back: %d %d", downs, backs); + + // as it was (no grace): every drop handed the laser to the other hand, and it stayed there + Params none; + none.grace = 0; + World old(none); + old.shown[1] = true; + old.Step(); + old.shown[0] = true; + old.Run(0.5); + old.shown[1] = false; + old.Run(0.3); + old.shown[1] = true; + old.Run(0.5); + CHECK(old.o.Side() == 0, "without the grace a drop gives the other hand the ray: %d", old.o.Side()); +} + +void TestGraceRunsOut() { + World w; + w.shown[0] = w.shown[1] = true; + w.Step(); + CHECK(w.o.Side() == 0 || w.o.Side() == 1, "an owner"); + const int o = w.o.Side(), other = 1 - o; + w.Run(0.5); + w.changes.clear(); + w.shown[o] = false; + const double from = w.t; + double took = -1; + w.Run(1, [&] { + if (took < 0 && w.o.Side() == other) took = w.t; + }); + CHECK(took >= from + 0.5 - 1e-9 && took < from + 0.5 + 2 * kDt, "the other hand's ray takes it after the grace: %.3f", + took - from); + CHECK(w.changes.size() == 1 && w.changes[0].why.find("down 0.5 s (palm)") != std::string::npos && + w.changes[0].why.find("up: ") != std::string::npos, + "why: %s", w.changes.empty() ? "-" : w.changes[0].why.c_str()); + // with no other ray up, the owner goes after the grace + World v; + v.shown[1] = true; + v.Run(0.3); + v.shown[1] = false; + v.Run(0.45); + CHECK(v.o.Side() == 1, "within the grace: still right"); + v.Run(0.1); + CHECK(v.o.Side() == -1 && v.changes.back().to == -1 && !v.changes.back().off, "after it: none, not off"); +} + +void TestFreshRaiseTakesAtOnce() { + World w; + w.shown[1] = true; + w.Run(0.3); + w.shown[1] = false; // the right hand comes down + w.Run(0.1); + w.shown[0] = true; // and the left goes up (its ray shows, after the right's went) + w.Step(); + CHECK(w.o.Side() == 0, "a ray raised while the owner's is down takes it at once: %d", w.o.Side()); + CHECK(w.changes.back().why.find("left raised") != std::string::npos, "why: %s", w.changes.back().why.c_str()); +} + +void TestOffLeavesAtOnce() { + World w; + w.shown[0] = w.shown[1] = true; + w.Run(0.2); + const int o = w.o.Side(); + w.may[o] = false, w.shown[o] = false; // its controller in use: fh_ray hides it ("off") + w.Step(); + CHECK(w.changes.back().from == o && w.changes.back().to == -1 && w.changes.back().off, "off: leaves at once, off"); + w.Step(); + CHECK(w.o.Side() == 1 - o, "then the other ray up takes it: %d", w.o.Side()); + // near a panel (ft-screens' poke): at once too, not off + World v; + v.shown[1] = true; + v.Run(0.2); + v.shown[1] = false, v.why[1] = "near"; + v.Step(); + CHECK(v.o.Side() == -1 && !v.changes.back().off && v.changes.back().why == "near a panel", "near: leaves at once"); +} + +void TestPinches() { + World w; + w.shown[0] = w.shown[1] = true; + w.Step(); + const int o = w.o.Side(), other = 1 - o; + w.changes.clear(); + CHECK(w.o.Pinched(o, true) == Owner::Pinch::Ray, "the owner's pinch: a ray pinch"); + CHECK(w.o.Pinched(other, false) == Owner::Pinch::None, "the other hand without a ray: not a ray pinch"); + CHECK(w.o.Side() == o, "and it leaves the owner"); + CHECK(w.o.Pinched(other, true) == Owner::Pinch::Take, "the other hand with its ray up: takes it"); + CHECK(w.o.Side() == other && w.o.changes.back().why.find("pinched") != std::string::npos, "taken by the pinch"); + w.o.changes.clear(); + // the owner's ray down, a pinch on it within the grace: back, a ray pinch, no change + w.shown[other] = false; + w.Run(0.2); + CHECK(w.o.Waiting(), "waiting"); + CHECK(w.o.Pinched(other, true) == Owner::Pinch::Ray && !w.o.Waiting(), "the owner's pinch brings it back"); + w.shown[other] = true; + w.Run(1); + CHECK(w.o.Side() == other && w.changes.empty(), "still the owner, no change"); + // no owner at all: a pinch with a ray that can come up takes it + World v; + CHECK(v.o.Pinched(1, true) == Owner::Pinch::Take && v.o.Side() == 1, "no owner: the pinch takes it"); + v.o.Drop("the mouse"); + CHECK(v.o.Side() == -1 && v.o.changes.back().why == "the mouse", "dropped"); +} + +void TestSlots() { + Params p; + Slots sl(p); + int pick[2]; + double t = 0; + std::vector hands = {{3, 1}, {4, 0}}; + sl.Update(t, hands, pick); + CHECK(pick[0] == 1 && pick[1] == 0, "by ft-hands' call: %d %d", pick[0], pick[1]); + CHECK(sl.SideOf(3) == 1 && sl.SideOf(4) == 0 && sl.SideOf(9) == -1, "SideOf"); + // the calls swap for 0.2 s: each ray keeps its hand + for (int i = 0; i < 6; ++i) { + t += kFrame; + sl.Update(t, {{3, 0}, {4, 1}}, pick); + CHECK(pick[1] == 0 && pick[0] == 1, "a brief swap: the rays keep their hands (%d %d)", pick[0], pick[1]); + } + t += kFrame; + sl.Update(t, hands, pick); + CHECK(pick[0] == 1 && pick[1] == 0, "back: unchanged"); + // ft-hands settles on the other call: after 0.3 s the rays move + double moved = -1; + const double from = t + kFrame; + for (int i = 0; i < 20 && moved < 0; ++i) { + t += kFrame; + sl.Update(t, {{3, 0}, {4, 1}}, pick); + if (pick[0] == 0) moved = t; + } + CHECK(moved >= from + 0.3 - 1e-9 && moved < from + 0.3 + kFrame + 1e-9, "a held call moves the rays after 0.3 s: %.3f", + moved - from); + CHECK(pick[1] == 1 && sl.SideOf(3) == 0 && sl.SideOf(4) == 1, "and both follow the call (%d %d)", pick[0], pick[1]); + // both called right: the right ray keeps its hand, the left keeps its own for the hold + Slots d(p); + t = 0; + d.Update(t, {{7, 0}, {8, 1}}, pick); + t += kFrame; + d.Update(t, {{7, 1}, {8, 1}}, pick); + CHECK(pick[0] == 0 && pick[1] == 1, "both called right: no hand changes ray (%d %d)", pick[0], pick[1]); + // one hand gone, a new one called that side + t += kFrame; + d.Update(t, {{8, 1}, {9, 0}}, pick); + CHECK(pick[0] == 1 && pick[1] == 0 && d.SideOf(9) == 0, "a new hand takes the free ray (%d %d)", pick[0], pick[1]); + // a lone hand's call flips for a frame: no ray changes hand + Slots one(p); + t = 0; + one.Update(t, {{5, 1}}, pick); + t += kFrame; + one.Update(t, {{5, 0}}, pick); + CHECK(pick[1] == 0 && pick[0] == -1, "a lone hand's flip: still the right ray's (%d %d)", pick[0], pick[1]); +} + +fhray::Ray RayAt(double yawDeg) { + fhray::Ray r; + r.dir = fhray::Dir(yawDeg * kDeg, 0); + r.origin = {0.2, 1.3, -0.4}; + return r; +} +double YawDeg(const fhray::Ray &r) { return fhray::YawOf(r.dir) / kDeg; } + +void TestAimLock() { + // pointing at yaw 0 with the fingers open (6 cm), then closing over 150 ms, turning the ray 3 deg + // down to the begin (2 cm, when ft-hands sees the pinch) + AimHistory h; + const double begin = 2.0; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) { + const double closing = std::clamp((t - (begin - 0.15)) / 0.15, 0.0, 1.0); + h.Add(t, RayAt(-3 * closing), true, 0.06 - 0.04 * closing); + } + fhray::Ray r; + double age = -1; + CHECK(h.Lock(begin, 0.035, r, age), "a lock"); + CHECK(std::fabs(YawDeg(r)) < 1.3, "the aim before the fingers closed far: %.2f deg", YawDeg(r)); + CHECK(age > 0.05 && age < 0.15, "from the frame the tips were last apart: %.3f s", age); + // without the distance (no skeleton): the frame 0.1 s before + AimHistory n; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) n.Add(t, RayAt(-20 * (t - 1)), true, -1); + CHECK(n.Lock(begin, 0.035, r, age), "a lock without distances"); + CHECK(age >= 0.1 - 1e-9 && age < 0.1 + kFrame, "0.1 s back: %.3f", age); + // fingers resting nearly closed (never past end_m in the window): 0.1 s back too + AimHistory c; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) c.Add(t, RayAt(0), true, 0.025); + CHECK(c.Lock(begin, 0.035, r, age) && age >= 0.1 - 1e-9 && age < 0.1 + kFrame, "resting closed: %.3f", age); + // frames with the ray down don't count; none up in the window: no lock + AimHistory d; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) d.Add(t, RayAt(t > 1.8 ? 10 : 0), t > 1.8, 0.06); + CHECK(d.Lock(begin, 0.035, r, age) && std::fabs(YawDeg(r) - 10) < 1e-6, "only frames with the ray up"); + AimHistory e; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) e.Add(t, RayAt(0), t < 1.5, 0.06); + CHECK(!e.Lock(begin, 0.035, r, age), "nothing up within 0.35 s: no lock"); + // the ray just showed (less than 0.1 s of it, fingers closed): its first frame + AimHistory f; + for (double t = 1.0; t <= begin + 1e-9; t += kFrame) f.Add(t, RayAt(t), t > begin - 0.07, 0.01); + CHECK(f.Lock(begin, 0.035, r, age) && age <= 0.07 + 1e-9 && age > 0.03, "a ray just up: its oldest frame (%.3f)", age); +} + +// fh_ray's ShowForPinch: a pinch brings back a ray that just dropped, not one sent away. +fhray::HandFrame Pointing(double t) { + fhray::HandFrame f; + f.t = t, f.stereo = true; + f.knuckle = {0.18, 1.35, -0.45}; + f.wrist = {0.17, 1.33, -0.36}; + f.palmNormal = fhray::Norm({0, -0.5, -1}); + f.elbow = {0.2, 1.15, -0.1}; + return f; +} + +void TestShowForPinch() { + const fhray::V3 eye{0, 1.6, 0}, fwd{0, 0, -1}; + fhray::Body b; + fhray::HandRay r(1); + double t = 0; + auto run = [&](double secs, const std::function &make, bool may = true) { + const double end = t + secs - 1e-9; + while (t < end) { + b.Update(t, eye, fwd); + r.Frame(b, make(t), may); + t += kFrame; + } + }; + auto palmIn = [](double tt) { + fhray::HandFrame f = Pointing(tt); + f.palmNormal = {0, 0, 1}; + return f; + }; + run(0.5, Pointing); + CHECK(r.Shown(), "pointing: shown"); + run(0.4, palmIn); + CHECK(!r.Shown() && r.event == "hide palm", "palm toward you: hidden (%s)", r.event.c_str()); + CHECK(r.CanShowForPinch(t, 0.5), "just hid: a pinch may bring it back"); + r.ShowForPinch(); + r.Hold(true, t); + CHECK(r.Shown(), "brought back"); + run(1, palmIn); + CHECK(r.Shown(), "held by the pinch whatever the pose"); + r.Hold(false, t); + run(0.8, palmIn); + CHECK(!r.Shown(), "after the pinch: hides again"); + run(0.6, palmIn); + CHECK(!r.CanShowForPinch(t, 0.5), "hid more than the grace ago: no"); + // hidden for "off" (its controller in use): never + fhray::HandRay q(1); + t = 0; + while (t < 0.5) b.Update(t, eye, fwd), q.Frame(b, Pointing(t), true), t += kFrame; + b.Update(t, eye, fwd), q.Frame(b, Pointing(t), false); + CHECK(!q.Shown() && !q.CanShowForPinch(t, 0.5), "hidden off: a pinch can't bring it back"); + // not shown yet but every show test passing (raised and pinched within 120 ms): yes + fhray::HandRay s(1); + t = 0; + b.Update(t, eye, fwd), s.Frame(b, Pointing(t), true); + CHECK(!s.Shown() && s.CanShowForPinch(t, 0.5), "on its way up: a pinch may show it"); + fhray::HandRay lap(1); + lap.Frame(b, palmIn(0), true); + CHECK(!lap.CanShowForPinch(0, 0.5), "never up, failing a test: no"); +} + +} // namespace + +int main() { + TestFirstRaised(); + TestBriefDropKeepsTheOwner(); + TestGraceRunsOut(); + TestFreshRaiseTakesAtOnce(); + TestOffLeavesAtOnce(); + TestPinches(); + TestSlots(); + TestAimLock(); + TestShowForPinch(); + if (failures) { + std::printf("%d failures\n", failures); + return 1; + } + std::printf("rayowner-test: ok\n"); + return 0; +} diff --git a/session/frametop.conf.example b/session/frametop.conf.example index 3d099df..112eec7 100644 --- a/session/frametop.conf.example +++ b/session/frametop.conf.example @@ -70,6 +70,7 @@ POINTER_RAY_DEADZONE=2.5 # hand ray: degrees the ray turns during a pinch befo POINTER_RAY_PUSH=1 # hand ray: while carrying a panel, reaching out pushes it away and pulling in brings it near POINTER_RAY_SCROLL=1 # hand ray: scroll speed for a held middle pinch moved up or down (-1 turns the direction round) POINTER_RAY_RELEASE=2 # hand ray: seconds after the ray goes before the pointer it woke lets go of the laser +POINTER_RAY_GRACE=0.5 # hand ray: seconds a hand keeps the laser after its ray drops (the other hand's ray takes it after this, or at once by pinching or raising) SHARE_KEYS=0 # 1 = keys of keyboards grabbed for the desktop also go to @frametop_keys, for hotkey tools (any local process can listen) DISPLAY_OFF_MIN=0 # minutes without use before frametop-power turns the displays off, even if the headset seems worn (0 = never) DISPLAY_MOVE_MM=5 # displays off: the headset or a controller moving this far (mm) within 10 s counts as use