FauxxYT--StreamXR/docs/Future-Plans.md

# Future plans: StreamXR inside a BrowseXR

**BrowseXR** is a possible future browser extension for using the whole browser in VR, with
**StreamXR** (this project) as a component plugged into it for video. It's on hold: the
hard parts are in "Before building BrowseXR" below. Meanwhile StreamXR is being shaped so
that it could fit in later without a rewrite. Keep this page up to date as StreamXR
changes: tick off what's done, and add anything new that BrowseXR would need or break.

## Done

- **0.49.0: rename to StreamXR** (from StreamVR): namespace `__streamxr`, logs `[streamxr]`,
  events and CSS classes `streamxr-*`, storage `streamxr.*` (old `streamvr.*` / `ytvr.*`
  values move over when first read; the language choice too), the Skip Helper's messages.
- **0.49.0: folder split**: `src/core/` (the VR interface, BrowseXR's future core),
  `src/streamxr/` (the video player on it), `src/sites/` (site code). See ARCHITECTURE.md.
- **0.49.0: site registry** (`src/core/page/sites.js`): sites register their provider;
  StreamXR's generic `main.js` picks the page's site. Nothing outside `src/sites/` names a
  site.
- **0.49.0: media source** (`src/streamxr/page/media-source.js`): the session gets its
  frames from a source (today the site's `<video>`) instead of the provider's element
  directly. A captured tab's stream can play in a `<video>` too, so the renderers would
  work with it unchanged. Since 0.78.0 a source can have no video yet (VR starts without one;
  the screen shows the site's logo, `provider.logo()`), which a BrowseXR session would need too.

## Still to do for BrowseXR (not started)

### 1. A message-shaped provider boundary (the big one)

BrowseXR would host the VR session in its own page and show tabs from there, so the shell
(core + StreamXR) and a site's code would run in **different pages**. Today they share one
page and call each other directly. Everything that crosses between them would have to
become plain messages (JSON), sent and answered.

**Already fine:** panel models (what a controller gives a style to draw) and events (what
a style reports back) are plain data.

**What has to change, and what it would break while being converted** (nothing visible
if done right; the risk is regressions in these places):

- **`actions` become named commands**, e.g. `run('seekBy', 10)` instead of
  `actions.seekBy(10)`. Every caller changes: `vr-panel.js` (controls panel buttons, seek
  bar, menus, volume), `xr-session.js` (controller buttons, ads, format), the panel
  controllers.
- **`state()` must be pure JSON.** It's polled every frame; across pages it would arrive a
  frame late (fine for display).
- **Calls that return live objects or answers later** become a request and a reply:
  - pagers (`channelList`, `browseHome`, `browseSubscriptions`, `browseFeed`, `browsePlaylist`, `search`,
    `comments`, `replies`, `postComments`: objects with `items` and `next()`). The
    controllers that use them run with the site, so they'd stay on the site's side;
    only their models cross.
  - promises: `setSubscribed`, `rateComment`, `ratePost`, `votePoll`, `channelHeader`,
    `channelAbout`, `channelFeatured`, `suggest`.
  - `storyboard(seconds)` returns the preview frame at once (would become a reply, or the
    storyboard sheet's details sent once).
- **Controllers ask the style directly** how far it scrolls: `host.style.maxScroll()` and
  `host.style.shownScroll()` in `description.js` (loading more comments as you scroll,
  remembering each tab's position) and `channel.js` (loading more posts, the About page's
  scroll). These would have to arrive as data with each frame (`frame.scroll`). Risk:
  endless comments / posts stop loading, or scroll positions are lost between tabs.
- **Site code drawing on the shell's canvas:** the `threadFooter` slot (`comments.footer`
  in `provider/comments.js`) draws a comment's 👍 count, 👎 and ❤ straight onto the
  style's canvas. It would become data the style draws (a list of footer buttons, as post
  buttons already are). Risk: the comment footer's look or its like / dislike clicks.
- **Other functions in the registry:** `label` (functions for translation; could become
  keys), `create(host)` (stays on the site's side), `site.onControl(id, panel)` (becomes a
  message).

Watching 2D / 180° / 360° wouldn't change: the video still plays from the site's own
`<video>` in its own tab when StreamXR runs there.

### 2. Split the session loop

`src/streamxr/page/xr-session.js` mixes what any VR page needs (WebXR setup, the frame loop,
controllers and input, the VR panel, the global reset, recentering) with what video needs
(the media source, the video view, formats, ads, the pause overlay). For BrowseXR the
first part moves to the core (a "scene" the video is one layer of).

### 3. The controls panel belongs to StreamXR

The controls panel (`src/core/page/vr-panel/panel-ui.js`: play, seek bar, quality, speed,
volume, format) is a video player's panel, but it lives in the core because `vr-panel.js`
drives it (laser hits, scrubbing, menus). In BrowseXR it would become a panel StreamXR
adds, and `vr-panel.js` would host any number of panels.

### 4. The isolated world

The registry is page-world only. The isolated world still has one set of files per site
in the manifest (`site.js`, the page button, `main.js`). BrowseXR would need its own
launcher instead of the "Watch in VR" popup (`src/streamxr/ui/panel.js`).

### 5. Other media sources

A source for a captured tab (`chrome.tabCapture` stream in a `<video>`), next to the
site-`<video>` one.

## Before building BrowseXR: four experiments

Each is a small throwaway prototype with a yes / no answer (October 2026 analysis):

1. Can a WebXR session run from an extension page (`chrome-extension://`)?
2. How sharp and smooth is a background tab through `chrome.tabCapture` (resolution,
   frame rate, delay)?
3. Do clicks and typing sent through `chrome.debugger` (`Input.dispatchMouseEvent` /
   `dispatchKeyEvent`) feel usable? (It shows a "being debugged" bar.)
4. Can you switch from browsing to a site's own immersive session (StreamXR's native
   video) and back without taking the headset off, or does it need a click on the desktop?

Known costs whatever the answers: permissions (`tabCapture`, `debugger`, likely all sites:
a big install warning, against the "youtube.com only" rule today), DRM video shows black
when captured, captured text is only as sharp as the tab's window. SteamVR's own desktop
view already shows Chrome in the headset; BrowseXR's case is a VR-native interface and
StreamXR's native video quality.