FrameYap

On-device voice typing for Steam Frame. Hold a button, speak, check the text, and type it into whatever app has focus. Speech recognition runs on the headset itself.

In-headset UI, captured from the video tutorial on YouTube.

Install

1. On the Frame, switch to desktop mode and open Konsole (or connect over SSH).

2. Paste this and press Enter:

   curl -fsSL https://github.com/baketnk/frame-yap/releases/latest/download/install.sh | sh
   

3. Answer y to the three questions:

   • install FrameYap,
   • download the speech model (about 180 MB),
   • install the speech runtime (about 200 MB download, roughly 1–1.5 GB on disk).

   The downloads can take a few minutes with little output; that is normal.

Everything installs into your home folder. No sudo, no compiler, no Steam store page. Prefer to read the script first? Download install.sh and open it; it is a single file.

Launch

Open FrameYap from the desktop application menu, or in the headset from your Steam library's Non-Steam section. The panel appears in front of you; drag the bar under it to move it, or the corner handle to resize.

Use it

Button	What it does
X (right), hold	Record while held; let go to transcribe.
A (right)	Type the text into the focused app. With nothing to type, A presses Enter (so A, A types and submits).
B (right)	Cancel / discard.
Y (right)	Quick phrases; press again to pick the next one.
Left grip, double-tap	Type + Enter.

You can also click the panel's buttons with the laser pointer. The Review tab shows Open Plan, Open Keyboard, and Open Draw for companion apps installed on the Frame. Tap Open Keyboard to show it; hold for 0.8 seconds to recenter it in front of you. Nothing is ever typed or submitted without you pressing a button. Buttons can be remapped with Bindings on the panel.

The header shows battery levels for your controllers and headset. The Buttons paused badge means the Steam menu is open: controller buttons go to Steam, so use the pointer instead (or close the menu).

Troubleshooting

• Panel says Unavailable, or shows an error while loading: the speech model or runtime is missing (for example, you answered n during install). Run these, then restart FrameYap:

  sh ~/.local/share/frameyap/current/bin/install.sh --install-model --backend redux --yes
  sh ~/.local/share/frameyap/current/bin/install.sh --install-runtime --yes
  

• Text went to the wrong place: FrameYap types into the currently focused window. Click the target text field first.

• Controller buttons do nothing: check for the Buttons paused badge (see above).

Privacy

Recognition runs locally with the Parakeet Redux model. FrameYap itself does not collect or send analytics. Its separately installed Moondream/Kestrel runtime includes upstream telemetry; see Kestrel's telemetry disclosure.

Settings → Moondream analytics is OFF by default, including for existing settings files without this option. With it OFF, FrameYap suppresses the Kestrel reporter before starting recognition. Turning it ON requires confirming the in-app disclosure and successfully saving the preference. ON leaves the upstream reporter enabled: Kestrel 0.8.0 sends hostname, model/service name, device details, timestamps, random instance/report IDs, and aggregate request/error/token counts to the default endpoint https://api.moondream.ai/v1/photon/telemetry. Reports are attempted at model startup, about every 60 seconds (even while idle), and on graceful runtime shutdown. On FrameYap's CPU path, device information identifies CPU and zero GPU memory. The audited reporter does not send audio or transcripts.

Changing the setting restarts recognition and discards current recording, review text and queued dictation. Turning it OFF stops the old worker without a final graceful-shutdown report; it cannot recall requests already sent. The choice is saved as moondream_analytics in ~/.config/frameyap/config.json. If saving OFF fails, it still applies for this session; the panel warns that the old saved choice may return next launch and offers Save analytics OFF to retry.

Kestrel 0.8.0 has no supported opt-out, so OFF uses a narrow in-process monkey-patch for moondream 2.4.0 / Kestrel 0.8.0. It does not edit installed dependency files or affect other programs. With OFF selected, an unknown version or changed reporter interface stops recognition with Telemetry protection unavailable rather than silently allowing analytics. Quit FrameYap, then restore the pinned runtime using the --install-runtime --yes command in Troubleshooting; upgrades need a fresh compatibility review. ON does not apply that patch or its version guard. This is not a network sandbox for every third-party library. Earlier releases without this setting did not suppress Kestrel telemetry. See worker implementation and verification.

The installer downloads FrameYap from GitHub, the model from Hugging Face and the Python packages (moondream, Kestrel, CPU Torch) from PyPI and PyTorch. Check for updates contacts GitHub when pressed. Once installed, recognition can run offline. By default the microphone stays open while FrameYap is running and idle audio is thrown away; turn on Settings → Close mic when idle to close it between recordings (it may clip the first syllable). Recordings and transcripts are not kept, unless you turn on Advanced debug, whose logs may contain speech.

Update and uninstall

• Update: run the install command again and answer y to FrameYap. Answer n to the model and runtime questions unless the release notes say they changed.
• Check for updates: Settings → scroll to Check for updates. FrameYap contacts GitHub only when you press it. If a newer published release is available, Install update... opens a desktop terminal; quit FrameYap and press Enter there to install. Nothing is downloaded by the check itself.
• Settings: Animated background is off by default; Settings → Animated background turns it on and saves your choice. ~/.config/frameyap/config.json holds Quick phrases, theme and placement; restart FrameYap after manual edits.
• Uninstall: sh ~/.local/share/frameyap/current/bin/install.sh --uninstall --unregistered. Your settings and downloaded model are kept. Delete ~/.local/share/frameyap and ~/.config/frameyap to remove everything.

More

• Development notes: build from source, validation status, full control details.
• Third-party components and licenses. FrameYap itself is MIT licensed.
• The release archive contains FrameYap, SDL3 and the OpenVR client library only; the speech runtime and model are downloaded on your machine, not redistributed.