Compare commits

..
Author SHA1 Message Date
toorux 7d9b5a23f4 release: 0.8.2 2026-10-16 00:42:32 +08:00
toorux b885b8f56a test(cast): add opt-in wearer framing visibility diagnostic 2026-10-16 00:40:27 +08:00
toorux 53ddf07f18 fix(cast): compensate stereo framing with live eye and UI depth 2026-10-16 00:40:10 +08:00
toorux b3c692e4a2 fix(ui): vibrate when hovering framing corner handles 2026-10-16 00:39:48 +08:00
toorux cbd13e7db2 fix(cast): restart streams and close framing editors reliably 2026-10-16 00:39:13 +08:00
toorux 9cf05cde09 feat(cast): add calibrated recording frame editor and region guide
Constrain frames to capture-eye visibility, restore menu sizing and add reset. Fix centered resize and trigger release handling, and prevent large native poll responses from losing framing commands.
2026-10-15 23:32:43 +08:00
toorux b23b39fc00 Record MP4 independently of casting with shared 4:3 defaults 2026-10-15 20:26:41 +08:00
46 changed files with 2098 additions and 114 deletions

No files matched your search

Generated
+1 -1
View File
@@ -425,7 +425,7 @@ dependencies = [
[[package]]
name = "framely"
version = "0.8.2-preview.2"
version = "0.8.2"
dependencies = [
"anyhow",
"base64",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "framely"
version = "0.8.2-preview.2"
version = "0.8.2"
edition = "2021"
license = "AGPL-3.0-only"
+25 -1
View File
@@ -4,7 +4,9 @@
## 采集与网页观看
“网页观看”和快捷面板的串流卡片均有 **开始/停止录制** 按钮,状态同步。录制复用当前原始 H.264/H.265 + 可选 Opus 串流,不进行二次编码,文件保存到 Steam 用户的 `Videos/framely`(头显默认 `/home/steamos/Videos/framely`),使用唯一文件名和 MKV 容器,不覆盖已有视频。未串流时开始录制会先启动串流;停止录制不停止串流,停止串流会先结束录制并写入文件尾。调整取景前必须停止录制。
“网页观看”和快捷面板的串流卡片均有 **开始/停止录制** 按钮,状态同步。录制与串流共用画面来源、取景、分辨率、帧率、码率、视频编码和声音设置;默认输出为 1920×1440(4:3),已有自定义设置保持不变。录制复用原始 H.264/H.265 视频,不进行二次视频编码,声音转为 AAC。文件保存到 Steam 用户的 `Videos/framely`(头显默认 `/home/steamos/Videos/framely`),使用唯一文件名和分片 MP4 容器,不覆盖已有视频。
录制与网页串流独立启停:点击录制直接启动本地采集和录制,不开启网页观看或 HLS 服务;之后开启串流会复用正在录制的采集编码,不重启录制。停止串流不停止录制,停止录制不停止串流,两者都停止时释放采集资源。录制期间不能修改采集参数或调整取景;接收投屏设置不受影响。
录制期间头显右上角显示红点、计时和停止图标。指向指示器并点击可停止当前录制;显示指示器不请求场景输入焦点或开启全局激光模式。录制进程异常退出时取消指示并在面板显示错误,已经写入的视频保留。正常关闭会话时尝试完成文件收尾;突然断电无法保证最后一段数据完整。
@@ -77,4 +79,26 @@ DISPLAY=:0 FRAMELY_MEDIA_BIN=/绝对路径/media/bin \
自动通过的协议/编解码测试不能替代真实发送端兼容性、头显内视觉比例和用户感知延迟验收。
快捷面板的录屏模块提供 **画幅调整**:透明空间窗口中的框可以移动、等比例缩放,并修改画幅比例(包括 3:4)及录制分辨率。应用前使用实际采集 LUT 检查镜片可见范围,超出范围时拒绝保存;录制中不能修改取景。**录制区域显示** 独立持久保存,只在实际录制或串流时显示无交互的淡色虚线框,停止后隐藏;空闲时开启开关不要求 SteamVR 已启动。框的内边缘对应录制边界,颜色读取 SteamVR 边界颜色设置。
画幅调整窗口和虚线框保持双眼可见,但几何范围由实际录制眼决定;跟随主力眼时同样使用采集进程实际报告的眼睛。双眼显示不是双眼录制,边界检查需分别覆盖左右眼。
调整窗口以实际录制眼的完整 render bounds 为固定参考,不能按当前取景大小缩小参考范围,否则缩小后重新打开会无法恢复放大。拖动四角以矩形中心等比例缩放;最小尺寸由中间菜单的实际布局尺寸决定,保存和改变比例时同样执行该限制。网页在松开、取消、失去指针捕获、窗口失焦时结束拖拽;原生层保留移动事件的按下标志,并使用现有物理扳机状态在丢失松开事件时补发释放,避免悬停继续缩放。采集标记对齐不能替代佩戴时的可见范围验收。
菜单的视觉角尺寸独立于参考画布,扩大画布时不能同步放大菜单。采集 LUT 输出由同一个 `shown` 判定生成的有效区域网格,前端使用该网格实时限制拖动、缩放和比例变化,不仅在应用时拒绝越界。网格检查保留一格外侧余量,使外绘边线不会跨入无效区;这个保守上限不改变取景边界与实际录制画面的投影对应关系。**重置大小** 保留画幅比例和分辨率,恢复居中的最大有效框;后端应用时仍做精确有效区域检查。元数据读取必须并发排空 stdout,不能等待进程退出后再读取,以免有效区域数据塞满较小的管道。
提交调整时携带打开窗口时的实际 `expectedEye`,后端重新读取采集几何后进行核对;主力眼在编辑期间改变时拒绝应用,要求重新打开,不能用旧眼的有效范围验证新眼。
应用时以当前串流运行状态为准:正在串流则用保存后的取景重启,未串流则不自动开启。保存成功即结束编辑会话;后续重启失败仍保持窗口关闭并明确报告“取景已保存”,不能让过期窗口恢复编辑。原生宿主通过每次轮询返回的 `framingEditor` 状态补偿可能丢失的关闭命令;后台忙时返回 `null`,宿主保留当前状态。普通轮询的空关闭/停止回执不得等待串流锁。四角悬停复用界面的振动设置。
空间框与采集共用工厂面板旋转、相机内参及 `grow_for_undistort` 投影;不能用通用 OpenVR render frustum 替代该投影。运行中的框只接受当前采集进程输出的几何信息,缺少有效校准时隐藏并报错,不显示近似框。
可见范围同时检查工厂畸变 UV 中的面板覆盖和 OpenVR render projection 中的 HiddenAreaMesh;两套坐标不可混用,也不能跟随面板扫描方向翻转隐藏网格。左右眼分别检查,边界标记测试覆盖最外侧像素。正式框和校准标记使用相同的前景层级,避免被 Steam 空间窗口遮挡。
上述检查证明的是软件渲染/采样有效范围,不是当前佩戴者的完整光学可见范围。OpenVR 的 HiddenAreaMesh 保证被遮区域不可见,但补集并不保证在所有瞳位、眼距下可见;不得把标记录制对齐测试当作鼻侧可见问题已经通过验收。鼻侧边界需要闭另一眼进行真实佩戴验证,不能通过任意翻转坐标或猜测裁剪半径来宣称修复。
双眼辅助框采用实时 `GetEyeToHeadTransform` 的实际选眼原点(含当前瞳距),而不是固定或上次启动时的眼距。保持取景角度不变,按中心射线与当前可见 Steam/Framely 窗口的真实交点计算参考深度,框宽度同步为 `2 × depth × tanHalfFov`。因此录制眼里的边界不变,另一眼的视差由实测窗口距离决定,减少原固定 2 米平面对近处窗口的错位。拖动时冻结深度以保持激光交互稳定;暂时移出窗口时保留最近实测深度。首次没有可测窗口时仍使用 2 米近似,不能宣称读取了任意游戏的场景深度;倾斜/曲面窗口在中心处匹配,整个边框仍是平面近似。改变参考深度不会扩大录制眼的光学可见范围。
仅在用户同意短暂显示校准标记后,运行 `cast::region_device_test::recording_region_device_alignment` 忽略测试。指定 `FRAMELY_REGION_TEST_MEDIA` 和 `FRAMELY_REGION_TEST_MARKER`,后者由 `tests/recording_region_device.cpp` 编译。测试覆盖左右眼、4:3、16:9、竖屏、偏移及高分辨率,以实际录制帧内四角和中心标记的位置检查空间框与输出是否一致;使用隔离临时目录,不替换正式安装文件。
`tests/cast_frame.cpp` 验证原生视频与透明控件的预乘 Alpha 合成、控件纹理保留和收起后的完整画面,需要 GLX 环境。
+83 -19
View File
@@ -27,6 +27,10 @@ const PANELS_ROTATED: bool = true;
pub struct Lut {
pub view: Option<(f64, f64, f64)>,
pub capture_to_head: Option<[[f64; 4]; 3]>,
pub region_fully_visible: bool,
pub editor_reference_tan: Option<f64>,
pub visibility: Option<serde_json::Value>,
pub width: u32,
pub height: u32,
pub step: f32,
@@ -49,6 +53,10 @@ impl Lut {
let height = out_h.div_ceil(STEP) + 1;
Self {
view: None,
capture_to_head: None,
region_fully_visible: false,
editor_reference_tan: None,
visibility: None,
width,
height,
step: STEP as f32,
@@ -164,12 +172,9 @@ impl EyeModel {
}
}
let mut triangles = vr.hidden_area(eye);
if panel_geometry.is_some() {
for triangle in &mut triangles {
for point in triangle { *point = [1. - point[0], 1. - point[1]]; }
}
}
// HiddenAreaMesh belongs to the OpenVR render projection, not the
// factory distortion UV space. Never rotate it with panel scanout.
let triangles = vr.hidden_area(eye);
let hidden = rasterize(&triangles);
if log::log_enabled!(log::Level::Debug) {
let rows: Vec<String> = (0..32)
@@ -199,25 +204,29 @@ impl EyeModel {
})
}
fn render_target(&self, tx:f64, ty:f64) -> [f64;2] {
let (s,c)=(-self.roll).sin_cos();
[(c*tx-s*ty-self.l)/(self.r-self.l),(s*tx+c*ty-self.t)/(self.b-self.t)]
}
/// Render target UV for a view direction given as tangents, with the
/// eye's roll undone so the video stays level with the head.
fn target(&self, tx: f64, ty: f64) -> [f64; 2] {
let [rx, ry] = if let Some(geometry) = self.panel_geometry {
geometry.tangents(tx, ty)
} else {
let (s, c) = (-self.roll).sin_cos();
[tx * c - ty * s, tx * s + ty * c]
};
[(rx - self.l) / (self.r - self.l), (ry - self.t) / (self.b - self.t)]
if let Some(geometry)=self.panel_geometry {return geometry.render_uv(tx,ty);}
self.render_target(tx,ty)
}
/// Whether a render target spot is on the panel and actually drawn.
fn shown(&self, uv: [f64; 2]) -> bool {
fn shown(&self, tx:f64, ty:f64) -> bool {
let uv=self.target(tx,ty);
if !(0.0..1.0).contains(&uv[0]) || !(0.0..1.0).contains(&uv[1]) {
return false;
}
let i = (uv[1] * MASK_SIZE as f64) as usize * MASK_SIZE + (uv[0] * MASK_SIZE as f64) as usize;
self.covered[i] && !self.hidden[i]
if !self.covered[i] {return false;}
let render=self.render_target(tx,ty);
if !(0.0..1.0).contains(&render[0]) || !(0.0..1.0).contains(&render[1]) {return false;}
let i=(render[1]*MASK_SIZE as f64) as usize*MASK_SIZE+(render[0]*MASK_SIZE as f64) as usize;
!self.hidden[i]
}
/// Panel position (eye local) that shows this render target spot in channel `c`.
@@ -233,8 +242,7 @@ impl EyeModel {
/// horizontal half-angle tangent plus a center offset (also tangents).
/// The center may drift a few degrees off straight ahead when that buys
/// a noticeably wider view, since the lens isn't symmetric.
fn widest(&self, aspect: f64) -> (f64, f64, f64) {
let fits = |tan_h: f64, cx: f64, cy: f64| {
fn region_fits(&self, tan_h: f64, cx: f64, cy: f64, aspect: f64) -> bool {
let tan_v = tan_h * aspect;
(0..=EDGE_SAMPLES).all(|k| {
let f = k as f64 / EDGE_SAMPLES as f64 * 2.0 - 1.0;
@@ -245,9 +253,27 @@ impl EyeModel {
(cx + tan_h, cy + f * tan_v),
]
.iter()
.all(|&(x, y)| self.shown(self.target(x, y)))
.all(|&(x, y)| self.shown(x, y))
})
};
}
fn visibility_grid(&self, extent:f64) -> serde_json::Value {
const N:usize=256;
let mut cells=String::with_capacity(N*N/4);
let mut nibble=0u8;
for i in 0..N*N {
let x=i%N;let y=i/N;
let visible=[(0.,0.),(1.,0.),(0.,1.),(1.,1.),(0.5,0.5)].iter().all(|&(dx,dy)| {
self.shown(((x as f64+dx)/N as f64*2.-1.)*extent,((y as f64+dy)/N as f64*2.-1.)*extent)
});
if visible {nibble|=1<<(i%4);}
if i%4==3 {cells.push(char::from_digit(nibble as u32,16).unwrap());nibble=0;}
}
serde_json::json!({"size":N,"extent":extent,"cells":cells})
}
fn widest(&self, aspect: f64) -> (f64, f64, f64) {
let fits = |tan_h, cx, cy| self.region_fits(tan_h, cx, cy, aspect);
let limit = (-self.l).max(self.r).max((-self.t).max(self.b) / aspect);
let search = |cx: f64, cy: f64| {
let (mut lo, mut hi) = (0.0, limit);
@@ -380,6 +406,23 @@ pub fn undistorted(vr: &OpenVr, eye: usize, fov_deg: Option<f64>, out_w: u32, ou
let mut lut = Lut::new(out_w, out_h);
let eye_offset = model.eye as f64 * 0.5;
lut.view = Some((tan_h, cx, cy));
// The editor canvas is the eye's full render bounds, never a multiple of
// the current crop: reopening a smaller crop must not lower its zoom limit.
lut.editor_reference_tan = Some((-model.l).max(model.r).max((-model.t).max(model.b)));
lut.visibility = Some(model.visibility_grid(lut.editor_reference_tan.unwrap()));
lut.region_fully_visible = model.region_fits(tan_h, cx, cy, aspect);
// The verified factory mapping consumes head-relative output rays.
// Without calibration, keep capture working but do not claim an accurate
// spatial guide: the generic OpenVR projection is not equivalent.
let eye = vr.eye_to_head(eye);
// With factory projection, output rays are already head-relative. The
// compositor applies panel orientation itself; applying it again to the
// guide would double the correction.
lut.capture_to_head = if model.panel_geometry.is_some() {
eye.map(|eye|std::array::from_fn(|i|std::array::from_fn(|j| {
if j == 3 {eye[i][3]} else {if i==j {1.} else {0.}}
})))
} else {None};
let mut guess = [[0.5f64, 0.5]; 3];
// Widest color fringe, measured in output pixels.
let mut max_split = 0.0f64;
@@ -494,3 +537,24 @@ fn nearest(forward: &[[[f64; 2]; 3]], n: usize, c: usize, target: [f64; 2]) -> O
.min_by(|a, b| a.1.total_cmp(&b.1))?;
Some([(best % n) as f64 / (n - 1) as f64, (best / n) as f64 / (n - 1) as f64])
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn recording_region_checks_hidden_mesh_in_render_not_distortion_space() {
let geometry=PanelGeometry::parse(&serde_json::json!({"model_number":"Steam Frame","tracking_to_eye_transform":[{"eye_to_head":[[-1,0,0],[0,-1,0],[0,0,1]],"intrinsics":[[1,0,0],[0,1,0],[0,0,-1]],"grow_for_undistort":0}]}).to_string(),0).unwrap();
let mut model=EyeModel {eye:0,l:-2.,r:1.,t:-2.,b:1.,roll:0.,panel_geometry:Some(geometry),forward:vec![],hidden:vec![false;MASK_SIZE*MASK_SIZE],covered:vec![true;MASK_SIZE*MASK_SIZE]};
let index=|uv:[f64;2]|(uv[1]*MASK_SIZE as f64) as usize*MASK_SIZE+(uv[0]*MASK_SIZE as f64) as usize;
let render=index(model.render_target(0.,0.));
let distortion=index(model.target(0.,0.));
assert_ne!(render,distortion);
model.hidden[render]=true;
assert!(!model.shown(0.,0.));
model.hidden[render]=false;
model.hidden[distortion]=true;
assert!(model.shown(0.,0.));
model.covered[distortion]=false;
assert!(!model.shown(0.,0.));
}
}
+10 -5
View File
@@ -14,11 +14,12 @@ use std::{io::{self, Write}, os::fd::AsRawFd, path::Path, time::{Duration, Insta
#[derive(Parser)]
struct Args {
#[arg(long, default_value_t = 1920)] width: u32,
#[arg(long, default_value_t = 1080)] height: u32,
#[arg(long, default_value_t = 1440)] height: u32,
#[arg(long, default_value_t = 30)] fps: u32,
#[arg(long, default_value_t = 8)] bitrate: u32,
#[arg(long, default_value_t = 0)] eye: usize,
#[arg(long)] raw: bool,
#[arg(long)] describe_view: bool,
#[arg(long, value_enum, default_value_t = encoder::Codec::H264)] codec: encoder::Codec,
#[arg(long)] fov: Option<f64>,
#[arg(long, default_value_t = 0.)] center_x: f64,
@@ -32,14 +33,18 @@ fn run(a: Args) -> Result<()> {
"Invalid output dimensions, rate or eye");
anyhow::ensure!(a.fov.is_none_or(|v| v.is_finite() && (10.0..=150.0).contains(&v))
&& [a.center_x, a.center_y].into_iter().all(|v| v.is_finite() && (-60.0..=60.0).contains(&v)), "Invalid field of view");
let kms = kms::Kms::open(Path::new("/dev/dri/card0"))?;
let height = if a.raw { ((a.width as f64 * kms.mode_size.1 as f64 / kms.mode_size.0 as f64).round() as u32 + 1) & !1 } else { a.height };
anyhow::ensure!(!a.describe_view || !a.raw, "Raw view has no single-eye region");
let kms = if a.describe_view {None} else {Some(kms::Kms::open(Path::new("/dev/dri/card0"))?)};
let height = if a.raw { let kms=kms.as_ref().unwrap(); ((a.width as f64 * kms.mode_size.1 as f64 / kms.mode_size.0 as f64).round() as u32 + 1) & !1 } else { a.height };
anyhow::ensure!(height>=160&&height<=4096&&u64::from(a.width)*u64::from(height)<=8_847_360,"Raw output exceeds supported dimensions");
let map = if a.raw { lut::raw(a.width, height, kms.mode_size.0, kms.mode_size.1) }
let map = if a.raw { let kms=kms.as_ref().unwrap(); lut::raw(a.width, height, kms.mode_size.0, kms.mode_size.1) }
else { lut::undistorted(&OpenVr::connect()?, a.eye, a.fov, a.width, height, a.center_x, a.center_y)? };
if let Some((tan_h, cx, cy)) = map.view {
eprintln!("{}", serde_json::json!({"event":"view","tanHalfHorizontal":tan_h,"centerTanX":cx,"centerTanY":cy,"aspect":a.width as f64/height as f64}));
let description=serde_json::json!({"event":"view","tanHalfHorizontal":tan_h,"centerTanX":cx,"centerTanY":cy,"aspect":a.width as f64/height as f64,"captureToHead":map.capture_to_head,"eye":a.eye,"projection":"factory-panel-v1","regionFullyVisible":map.region_fully_visible,"editorReferenceTan":map.editor_reference_tan,"visibility":map.visibility});
if a.describe_view { anyhow::ensure!(map.capture_to_head.is_some(),"SteamVR cannot provide an accurate eye transform");println!("{description}"); return Ok(()); }
eprintln!("{description}");
}
let kms=kms.unwrap();
let mut enc = encoder::Encoder::open(&encoder::find_device()?, encoder::Config {
codec: a.codec, width: a.width, height, fps: a.fps,
bitrate: a.bitrate * 1_000_000, qp: None,
+8
View File
@@ -201,6 +201,14 @@ impl OpenVr {
ok.then_some([out.red, out.green, out.blue])
}
pub fn eye_to_head(&self, eye: usize) -> Option<[[f64; 4]; 3]> {
if !self.full_layout { return None; }
let t = unsafe {
let f: unsafe extern "C" fn(i32) -> HmdMatrix34 = self.slot(SLOT_EYE_TO_HEAD);
f(eye as i32)
};
Some(t.m.map(|row|row.map(f64::from)))
}
/// How far the eye's view is rolled relative to the head, in radians.
/// Canted displays show up here.
pub fn eye_roll(&self, eye: usize) -> f64 {
+32 -3
View File
@@ -6,16 +6,25 @@ use anyhow::{ensure, Context, Result};
#[derive(Clone, Copy)]
pub struct PanelGeometry {
eye_to_head: [[f64; 3]; 3],
intrinsics: [f64; 4],
render_scale: f64,
}
impl PanelGeometry {
/// ComputeDistortion's render UVs use the factory camera, expanded by
/// grow_for_undistort, not the parallel GetProjectionRaw bounding frustum.
pub fn render_uv(&self, tx: f64, ty: f64) -> [f64; 2] {
let [x,y]=self.tangents(tx,ty);
let [fx,fy,cx,cy]=self.intrinsics;
[0.5+(fx*x+cx)/(2.*self.render_scale),0.5+(fy*y-cy)/(2.*self.render_scale)]
}
pub fn load(eye: usize) -> Result<Self> {
let path = "/persist/device_config.json";
ensure!(std::fs::metadata(path)?.len() <= 1024 * 1024, "oversized panel calibration");
Self::parse(&std::fs::read_to_string(path)?, eye)
}
fn parse(json: &str, eye: usize) -> Result<Self> {
pub(crate) fn parse(json: &str, eye: usize) -> Result<Self> {
let config: serde_json::Value = serde_json::from_str(json)?;
// Factory JSON retains the internal model name on retail devices.
ensure!(matches!(config["model_number"].as_str(), Some("Steam Frame" | "Deckard MP")), "not Steam Frame calibration");
@@ -39,7 +48,14 @@ impl PanelGeometry {
- rotation[0][1] * (rotation[1][0] * rotation[2][2] - rotation[1][2] * rotation[2][0])
+ rotation[0][2] * (rotation[1][0] * rotation[2][1] - rotation[1][1] * rotation[2][0]);
ensure!((determinant - 1.).abs() < 0.01 && rotation[2][2] > 0.5, "invalid panel basis");
Ok(Self { eye_to_head: rotation })
let calibration=&config["tracking_to_eye_transform"][eye];
let k=&calibration["intrinsics"];
let intrinsics=[k[0][0].as_f64(),k[1][1].as_f64(),k[0][2].as_f64(),k[1][2].as_f64()];
ensure!(intrinsics.iter().all(|v|v.is_some_and(f64::is_finite)),"missing or invalid panel intrinsics");
let intrinsics=intrinsics.map(Option::unwrap);
let render_scale=1.+calibration["grow_for_undistort"].as_f64().context("missing overscan calibration")?;
ensure!(intrinsics[0]>0. && intrinsics[1]>0. && render_scale.is_finite() && (1.0..=4.0).contains(&render_scale),"invalid panel projection");
Ok(Self { eye_to_head: rotation,intrinsics,render_scale })
}
pub fn tangents(&self, tx: f64, ty: f64) -> [f64; 2] {
@@ -61,13 +77,14 @@ mod tests {
use super::*;
fn config(matrix: serde_json::Value) -> String {
serde_json::json!({"model_number":"Steam Frame", "tracking_to_eye_transform":[{"eye_to_head":matrix}]}).to_string()
serde_json::json!({"model_number":"Steam Frame", "tracking_to_eye_transform":[{"eye_to_head":matrix,"intrinsics":[[1,0,0],[0,1,0],[0,0,-1]],"grow_for_undistort":0}]}).to_string()
}
#[test]
fn panel_geometry_removes_scanout_rotation() {
let p = PanelGeometry::parse(&config(serde_json::json!([[-1,0,0],[0,-1,0],[0,0,1]])), 0).unwrap();
assert_eq!(p.tangents(0.2, -0.3), [0.2, -0.3]);
assert_eq!(p.render_uv(0.2,-0.3),[0.6,0.35]);
}
#[test]
@@ -79,6 +96,18 @@ mod tests {
assert!((t[0] - 0.5*c).abs() < 1e-12);
assert!((t[1] - 0.5*s).abs() < 1e-12);
}
#[test]
fn panel_geometry_projects_with_intrinsics_and_overscan() {
let mut json:serde_json::Value=serde_json::from_str(&config(serde_json::json!([[-1,0,0],[0,-1,0],[0,0,1]]))).unwrap();
json["tracking_to_eye_transform"][0]["intrinsics"]=serde_json::json!([[1.2,0,0.04],[0,1.1,-0.02],[0,0,-1]]);
json["tracking_to_eye_transform"][0]["grow_for_undistort"]=serde_json::json!(0.6);
let geometry=PanelGeometry::parse(&json.to_string(),0).unwrap();
let uv=geometry.render_uv(0.2,-0.3);
assert!((uv[0]-(0.5+(1.2*0.2+0.04)/3.2)).abs()<1e-12);
assert!((uv[1]-(0.5+(-1.1*0.3+0.02)/3.2)).abs()<1e-12);
json["tracking_to_eye_transform"][0]["grow_for_undistort"]=serde_json::json!(-1.);
assert!(PanelGeometry::parse(&json.to_string(),0).is_err());
}
#[test]
fn panel_geometry_rejects_missing_and_invalid_calibration() {
+11
View File
@@ -0,0 +1,11 @@
#pragma once
#include "vendor/json.hpp"
#include <string>
// Native close commands are an optimization; this durable state also repairs
// a missed command without letting an old close affect a newly opened editor.
inline bool framing_editor_should_close(const nlohmann::json& state,const std::string& editor_id){
if(!state.is_object()||!state.contains("open")||!state["open"].is_boolean()||!state.contains("id")||!state["id"].is_string())return false;
auto id=state["id"].get<std::string>();
return state["open"].get<bool>()?id!=editor_id:id==editor_id;
}
+107 -10
View File
@@ -32,6 +32,9 @@
#include "ui_visibility.h"
#include "ui_feedback.h"
#include "recording_indicator.h"
#include "recording_region.h"
#include "recording_region_depth.h"
#include "framing_lifecycle.h"
#include "plugin_navigation.h"
#include <GL/gl.h>
#include <GL/glx.h>
@@ -104,6 +107,7 @@ static int gaze_feedback_hand=1;
static std::chrono::steady_clock::time_point gaze_sent{},gaze_feedback_at{};
static EntryTrigger entry_buttons[2][2];
static EntryAnalogTrigger entry_analog[2];
static bool framing_trigger_known[2]{},framing_trigger_down[2]{};
static EntryPointer entry_pointer;
static bool menu_open=false,keyboard_open=false,keyboard_close_pending=false;
static float notification_curve_radius=1.3f;
@@ -127,7 +131,8 @@ template<class T> struct Handler {
#include "cast_frame.h"
struct View {
CastFrame cast_frame;
std::string key,plugin,title;int width=600,height=840;float render_scale=1.f;bool dashboard=false,shown=false,dirty=false,closing=false,closed=false;
json region_geometry=json::object();
std::string key,plugin,title,region_editor_id;int width=600,height=840;float render_scale=1.f;bool dashboard=false,shown=false,dirty=false,closing=false,closed=false,framing_left_down=false;
OverlayPlacement placement;NotificationClick notification_click;LauncherPointer launcher_pointer;std::chrono::steady_clock::time_point haptic_at{},activation_at{};
cef_mouse_event_t pointer{};bool pointer_valid=false,stick_scrolling=false;vr::TrackedDeviceIndex_t pointer_device=vr::k_unTrackedDeviceIndexInvalid;std::chrono::steady_clock::time_point pointer_at{};
cef_file_dialog_callback_t* file_dialog=nullptr;std::string file_dialog_id;bool file_multiple=false,file_directory=false;
@@ -171,8 +176,12 @@ static std::vector<std::string> closed_casts;
// Protected by visibility_mutex (outgoing request) / queue_mutex (status).
static std::string pending_recording_stop;
static json recording_state=json::object();
static json recording_region_state=json::object();
static json framing_editor_state=nullptr;
static std::string pending_region_close;
static RecordingRegionOverlay recording_region_overlay;
static RecordingIndicator recording_indicator;
static json poll_http(int port){int fd=socket(AF_INET,SOCK_STREAM,0);if(fd<0)throw std::runtime_error("HTTP socket failed");timeval timeout{1,0};setsockopt(fd,SOL_SOCKET,SO_RCVTIMEO,&timeout,sizeof(timeout));setsockopt(fd,SOL_SOCKET,SO_SNDTIMEO,&timeout,sizeof(timeout));sockaddr_in addr{};addr.sin_family=AF_INET;addr.sin_port=htons(port);inet_pton(AF_INET,"127.0.0.1",&addr.sin_addr);if(connect(fd,reinterpret_cast<sockaddr*>(&addr),sizeof(addr))){close(fd);throw std::runtime_error("UI agent unavailable");}std::string payload;size_t closed_count=0;std::string stop_id;{std::lock_guard<std::mutex> lock(visibility_mutex);closed_count=closed_casts.size();stop_id=pending_recording_stop;payload=json{{"views",visible_views},{"closedCasts",closed_casts},{"stopRecording",stop_id}}.dump();}std::string req="POST /host/poll HTTP/1.1\r\nHost: 127.0.0.1:"+std::to_string(port)+"\r\nX-Framely-Native: "+token+"\r\nContent-Type: application/json\r\nContent-Length: "+std::to_string(payload.size())+"\r\nConnection: close\r\n\r\n"+payload;size_t sent=0;while(sent<req.size()){auto n=send(fd,req.data()+sent,req.size()-sent,MSG_NOSIGNAL);if(n<=0){close(fd);throw std::runtime_error("HTTP send failed");}sent+=n;}std::string response;char buffer[4096];ssize_t n;while((n=recv(fd,buffer,sizeof(buffer),0))>0){response.append(buffer,n);if(response.size()>4*1024*1024){close(fd);throw std::runtime_error("HTTP body limit");}}close(fd);auto body=response.find("\r\n\r\n");if(body==std::string::npos||response.find("200 OK")==std::string::npos)throw std::runtime_error("Invalid agent response");auto parsed=json::parse(response.substr(body+4));{std::lock_guard<std::mutex> lock(visibility_mutex);if(pending_recording_stop==stop_id)pending_recording_stop.clear();closed_casts.erase(closed_casts.begin(),closed_casts.begin()+std::min(closed_count,closed_casts.size()));}return parsed;}
static json poll_http(int port){int fd=socket(AF_INET,SOCK_STREAM,0);if(fd<0)throw std::runtime_error("HTTP socket failed");timeval timeout{1,0};setsockopt(fd,SOL_SOCKET,SO_RCVTIMEO,&timeout,sizeof(timeout));setsockopt(fd,SOL_SOCKET,SO_SNDTIMEO,&timeout,sizeof(timeout));sockaddr_in addr{};addr.sin_family=AF_INET;addr.sin_port=htons(port);inet_pton(AF_INET,"127.0.0.1",&addr.sin_addr);if(connect(fd,reinterpret_cast<sockaddr*>(&addr),sizeof(addr))){close(fd);throw std::runtime_error("UI agent unavailable");}std::string payload;size_t closed_count=0;std::string stop_id,region_closed;{std::lock_guard<std::mutex> lock(visibility_mutex);closed_count=closed_casts.size();stop_id=pending_recording_stop;region_closed=pending_region_close;payload=json{{"views",visible_views},{"closedCasts",closed_casts},{"stopRecording",stop_id},{"framingClosed",region_closed}}.dump();}std::string req="POST /host/poll HTTP/1.1\r\nHost: 127.0.0.1:"+std::to_string(port)+"\r\nX-Framely-Native: "+token+"\r\nContent-Type: application/json\r\nContent-Length: "+std::to_string(payload.size())+"\r\nConnection: close\r\n\r\n"+payload;size_t sent=0;while(sent<req.size()){auto n=send(fd,req.data()+sent,req.size()-sent,MSG_NOSIGNAL);if(n<=0){close(fd);throw std::runtime_error("HTTP send failed");}sent+=n;}std::string response;char buffer[4096];ssize_t n;while((n=recv(fd,buffer,sizeof(buffer),0))>0){response.append(buffer,n);if(response.size()>4*1024*1024){close(fd);throw std::runtime_error("HTTP body limit");}}close(fd);auto body=response.find("\r\n\r\n");if(body==std::string::npos||response.find("200 OK")==std::string::npos)throw std::runtime_error("Invalid agent response");auto parsed=json::parse(response.substr(body+4));{std::lock_guard<std::mutex> lock(visibility_mutex);if(pending_region_close==region_closed)pending_region_close.clear();if(pending_recording_stop==stop_id)pending_recording_stop.clear();closed_casts.erase(closed_casts.begin(),closed_casts.begin()+std::min(closed_count,closed_casts.size()));}return parsed;}
// Frame renders the native laser cursor in the default overlay sort group.
// Raising a panel above that group can obscure the runtime cursor.
// OpenVR owns and renders the laser cursor. Coordinates must remain in the
@@ -219,6 +228,71 @@ static vr::VROverlayHandle_t active_main_window(){
overlays->FindOverlay("valve.steam.gamepadui.main",&selected);
return selected;
}
struct RecordingRegionPresentation {json view;double distance=2.;};
static RecordingRegionPresentation recording_region_presentation(const json& original,bool freeze_depth=false){
RecordingRegionPresentation result{original,2.};
// This changes stereo presentation only, never the recorded angular crop.
// A center-ray hit measures real UI depth, including tilted/curved windows.
static auto sampled=std::chrono::steady_clock::time_point{};
static vr::HmdMatrix34_t left{},right{};static bool eyes_valid=false,measured=false;
static int previous_eye=-1;static double distance=2.;
try{
int eye=original.at("eye");if(eye<0||eye>1)return result;
vr::HmdMatrix34_t capture{};
for(int i=0;i<3;i++)for(int j=0;j<4;j++)capture.m[i][j]=original.at("captureToHead").at(i).at(j).get<float>();
auto now=std::chrono::steady_clock::now();bool probe=now-sampled>=std::chrono::milliseconds(60);
if(probe){left=vr_system->GetEyeToHeadTransform(vr::Eye_Left);right=vr_system->GetEyeToHeadTransform(vr::Eye_Right);eyes_valid=true;sampled=now;}
if(eyes_valid&&recording_region_refresh_eye_origin(static_cast<vr::EVREye>(eye),left,right,capture)){
for(int i=0;i<3;i++)result.view["captureToHead"][i][3]=capture.m[i][3];
}
if(eye!=previous_eye){previous_eye=eye;distance=2.;measured=false;probe=true;}
if(probe&&!freeze_depth){
vr::TrackedDevicePose_t poses[1]{};
vr_system->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding,0,poses,1);
if(poses[0].bPoseIsValid&&poses[0].bDeviceIsConnected){
using namespace recording_region_depth_detail;
const auto& head=poses[0].mDeviceToAbsoluteTracking;
const double cx=original.at("centerTanX"),cy=original.at("centerTanY");
auto origin=add(translation(head),rotate(head,translation(capture)));
auto direction=rotate(head,rotate(capture,{cx,-cy,-1.}));
double length=std::sqrt(dot(direction,direction));
if(std::isfinite(length)&&length>1e-6){
vr::VROverlayIntersectionParams_t ray{};ray.eOrigin=vr::TrackingUniverseStanding;
for(int i=0;i<3;i++){ray.vSource.v[i]=origin[i];ray.vDirection.v[i]=direction[i]/length;}
std::set<vr::VROverlayHandle_t> candidates;
for(const char* key:{"valve.steam.gamepadui.main","valve.steam.gamepadui.fallback"}){vr::VROverlayHandle_t handle=0;if(!overlays->FindOverlay(key,&handle))candidates.insert(handle);}
for(const auto& key:dashboard_keys)if(key.rfind("valve.steam.",0)==0&&key!="valve.steam.gamepadui.bar"){vr::VROverlayHandle_t handle=0;if(!overlays->FindOverlay(key.c_str(),&handle))candidates.insert(handle);}
for(const auto& [key,v]:views)if(!v->closing&&v->shown&&(v->dashboard||key.rfind("framely.window.",0)==0||key.rfind("framely.cast.",0)==0)&&key!="launcher")candidates.insert(v->overlay);
std::optional<double> nearest;
for(auto handle:candidates){
if(!overlays->IsOverlayVisible(handle))continue;
vr::VROverlayIntersectionResults_t hit{};
if(!overlays->ComputeOverlayIntersection(handle,&ray,&hit)||!std::isfinite(hit.vUVs.v[0])||!std::isfinite(hit.vUVs.v[1])||hit.vUVs.v[0]<0||hit.vUVs.v[0]>1||hit.vUVs.v[1]<0||hit.vUVs.v[1]>1)continue;
auto depth=recording_region_hit_depth(head,capture,cx,cy,hit.vPoint);
if(depth&&(!nearest||*depth<*nearest))nearest=depth;
}
if(nearest){
if(!measured){std::cout<<"Recording frame measured UI depth "<<*nearest<<" m, IPD "<<std::sqrt(dot(subtract(translation(right),translation(left)),subtract(translation(right),translation(left))))*1000<<" mm\n";distance=*nearest;measured=true;}
else distance+=(*nearest-distance)*.4;
}
// If a center ray temporarily leaves a window, retain its last measured
// depth rather than snapping back to 2 m. Without any UI measurement,
// 2 m remains an explicit approximation for arbitrary game scenes.
}
}
}
result.distance=distance;
}catch(...){return {original,2.};}
return result;
}
static void update_framing_placement(View& v){
if(v.closing||!v.region_geometry.is_object()||!v.region_geometry.contains("view"))return;
auto presentation=recording_region_presentation(v.region_geometry["view"],v.framing_left_down);
RecordingRegionPlane plane;
if(!recording_region_plane(presentation.view,v.region_geometry.at("referenceTan"),0,0,1,plane,presentation.distance))return;
overlays->SetOverlayWidthInMeters(v.overlay,plane.width);
overlays->SetOverlayTransformTrackedDeviceRelative(v.overlay,vr::k_unTrackedDeviceIndex_Hmd,&plane.transform);
}
static void hover_haptic(vr::VROverlayHandle_t overlay,std::chrono::steady_clock::time_point& last,bool activation=false){
auto now=std::chrono::steady_clock::now();if(now-last<std::chrono::milliseconds(80)||!overlays->IsOverlayVisible(overlay)||!overlays->IsHoverTargetOverlay(overlay))return;
last=now;if(ui_sounds)ui_audio.play(activation);
@@ -275,7 +349,7 @@ static bool keyboard_event(const vr::VREvent_t& e,vr::VROverlayHandle_t source_o
}else {keyboard_done=keyboard_done||e.eventType==vr::VREvent_KeyboardDone;keyboard_close_pending=true;}
return true;
}
static void close_view(View& v){if(v.closing)return;if(v.key.rfind("framely.cast.",0)==0){std::lock_guard<std::mutex> lock(visibility_mutex);closed_casts.push_back(v.key.substr(13));}cancel_file_dialog(v);if(keyboard_view==v.key)close_keyboard();v.closing=true;if(v.key=="menu"||v.key=="framely.apk.permission")overlays->SetOverlayFlag(v.overlay,vr::VROverlayFlags_WantsModalBehavior,false);overlays->ClearOverlayCursorPositionOverride(v.overlay);overlays->HideOverlay(v.overlay);v.host->close_browser(v.host,1);}
static void close_view(View& v){if(v.closing)return;if(v.key.rfind("framely.cast.",0)==0){std::lock_guard<std::mutex> lock(visibility_mutex);closed_casts.push_back(v.key.substr(13));}if(v.key=="framely.framing"){std::lock_guard<std::mutex> lock(visibility_mutex);pending_region_close=v.region_editor_id;}cancel_file_dialog(v);if(keyboard_view==v.key)close_keyboard();v.closing=true;if(v.key=="menu"||v.key=="framely.apk.permission")overlays->SetOverlayFlag(v.overlay,vr::VROverlayFlags_WantsModalBehavior,false);overlays->ClearOverlayCursorPositionOverride(v.overlay);overlays->HideOverlay(v.overlay);v.host->close_browser(v.host,1);}
static void anchor_to_head(View& v,float width,float x=0,float y=0,float z=-1.35f){vr::TrackedDevicePose_t poses[vr::k_unMaxTrackedDeviceCount]{};vr_system->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding,0,poses,vr::k_unMaxTrackedDeviceCount);if(!poses[0].bPoseIsValid)return;auto m=poses[0].mDeviceToAbsoluteTracking;for(int r=0;r<3;r++)m.m[r][3]+=m.m[r][0]*x+m.m[r][1]*y+m.m[r][2]*z;overlays->SetOverlayWidthInMeters(v.overlay,width);overlays->SetOverlayTransformAbsolute(v.overlay,vr::TrackingUniverseStanding,&m);}
static void anchor_permission(View& v){
vr::TrackedDevicePose_t poses[vr::k_unMaxTrackedDeviceCount]{};
@@ -312,9 +386,9 @@ static View& create_view(const std::string& key,const std::string& title,const s
if(views.count(key)){auto& v=*views.at(key);if(v.dashboard&&key!="launcher")overlays->ShowDashboard(key.c_str());else if(key!="notifications"&&key!="launcher")show(v,true);return v;}
// Use 1.5x for launcher artwork and 2x for scrolling panels. CEF readback,
// texture upload and launcher mipmap generation scale with the pixel count.
if(views.size()>=34)throw std::runtime_error("Window limit exceeded");auto v=std::make_unique<View>();v->key=key;v->plugin=plugin;v->title=title;v->dashboard=dashboard;v->width=width;v->height=height;v->render_scale=key=="launcher"?1.5f:(key=="framely.manager"||key=="menu"||key=="notifications")?2:1;
if(views.size()>=34)throw std::runtime_error("Window limit exceeded");auto v=std::make_unique<View>();v->key=key;v->plugin=plugin;v->title=title;v->dashboard=dashboard;v->width=width;v->height=height;v->render_scale=key=="launcher"?1.5f:(key=="framely.manager"||key=="menu"||key=="notifications"||key=="framely.framing")?2:1;
auto error=dashboard?overlays->CreateDashboardOverlay(key.c_str(),title.c_str(),&v->overlay,&v->thumbnail):overlays->CreateOverlay(key.c_str(),title.c_str(),&v->overlay);if(error)throw std::runtime_error("Overlay creation failed: "+std::to_string(error));
overlays->SetOverlayInputMethod(v->overlay,key=="notifications"?vr::VROverlayInputMethod_None:vr::VROverlayInputMethod_Mouse);vr::HmdVector2_t scale{{float(width),float(height)}};overlays->SetOverlayMouseScale(v->overlay,&scale);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_HideLaserIntersection,false);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_SendVRSmoothScrollEvents,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_VisibleInDashboard,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_SortWithNonSceneOverlays,key=="framely.apk.permission");overlays->SetOverlaySortOrder(v->overlay,key=="framely.apk.permission"?100:0);if(key=="framely.apk.permission")overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_WantsModalBehavior,true);
overlays->SetOverlayInputMethod(v->overlay,key=="notifications"?vr::VROverlayInputMethod_None:vr::VROverlayInputMethod_Mouse);vr::HmdVector2_t scale{{float(width),float(height)}};overlays->SetOverlayMouseScale(v->overlay,&scale);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_HideLaserIntersection,false);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_SendVRSmoothScrollEvents,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_VisibleInDashboard,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,true);overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_SortWithNonSceneOverlays,key=="framely.apk.permission");overlays->SetOverlaySortOrder(v->overlay,key=="framely.apk.permission"?100:0);if(key=="framely.framing")recording_region_foreground(overlays,v->overlay);if(key=="framely.apk.permission")overlays->SetOverlayFlag(v->overlay,vr::VROverlayFlags_WantsModalBehavior,true);
if(key=="framely.apk.permission"){
vr::VROverlayIntersectionMaskPrimitive_t mask{};
mask.m_nPrimitiveType=vr::OverlayIntersectionPrimitiveType_Rectangle;
@@ -506,6 +580,7 @@ static void launcher_gaze(vr::IVRInput* input,bool allowed,const bool (&connecte
gaze_active=true;
}
static void entry_actions(vr::VROverlayHandle_t button,bool available){
for(int hand=0;hand<2;hand++){framing_trigger_known[hand]=false;framing_trigger_down[hand]=false;}
const bool pointer_available=available||recording_indicator.active();
if(!entry_actions_ready){recording_indicator.cancel_input();return;}auto* input=vr::VRInput();vr::VRActiveActionSet_t sets[2]{};sets[0].ulActionSet=entry_set;sets[1].ulActionSet=gaze_set;
// Only the eye action participates in overlay priority. Controller bindings
@@ -564,6 +639,7 @@ static void entry_actions(vr::VROverlayHandle_t button,bool available){
if(kind==0&&!active)entry_analog[hand].held=false;
if(kind==0&&active)gaze_buttons[hand]|=down;
if(kind==0){
framing_trigger_known[hand]=active;framing_trigger_down[hand]=down||gaze_buttons[hand];
// Reuse the same physical tip / analog-trigger fallback as the Dock.
// No second action poll or input-focus request for the recording HUD.
auto stop_recording=recording_indicator.pointer(hand,pose.pose.mDeviceToAbsoluteTracking,active&&pose_ok,down);
@@ -613,9 +689,28 @@ static void entry_actions(vr::VROverlayHandle_t button,bool available){
if(gaze_allowed&&gaze_active)launcher_outside_at={};
if(launcher_open&&launcher_outside_at.time_since_epoch().count()&&now>=launcher_outside_at){launcher_outside_at={};if(!keyboard_open&&!launcher_dock_interaction())launcher_back();}
}
static void commands(){std::deque<json> commands;{std::lock_guard<std::mutex> lock(queue_mutex);commands.swap(queue);}for(const auto& c:commands){try{auto kind=c.value("kind","");if(kind=="feedback.settings"){ui_haptics=c.value("haptics",true);ui_sounds=c.value("sounds",false);if(!ui_sounds)ui_audio.silence();}else if(kind=="file.dialog.complete"){for(auto& [key,view]:views){auto& v=*view;if(!v.file_dialog||v.file_dialog_id!=c.value("id",std::string{}))continue;auto paths=c.value("paths",json::array());if(paths.empty()){cancel_file_dialog(v);break;}if((!v.file_multiple&&paths.size()!=1)||paths.size()>64)break;auto* callback=v.file_dialog;v.file_dialog=nullptr;v.file_dialog_id.clear();auto list=cef_string_list_alloc();for(const auto& path:paths){cef_string_t value{};str(value,path.get<std::string>());cef_string_list_append(list,&value);cef_string_utf16_clear(&value);}callback->cont(callback,list);cef_string_list_free(list);callback->base.release(&callback->base);break;}}else if(kind=="gaze.correction"){gaze_correction=c.value("calibration",json(nullptr));}else if(kind=="gaze.target"){gaze_target_id=c.value("id",std::string{});gaze_target_u=c.value("x",.5f);gaze_target_v=1-c.value("y",.5f);gaze_preview=c.value("preview",json(nullptr));gaze_target_cached=false;}else if(kind=="launcher.calibrate"){gaze_target_id.clear();gaze_target_cached=false;gaze_preview=nullptr;launcher_calibrating=c.value("active",true);if(launcher_calibrating&&!launcher_open){if(!views.count("launcher"))create_view("launcher","Framely launcher",origin+"/launcher#calibrate",true,1440,800);toggle_entry(true);}if(views.count("launcher")){auto& v=*views.at("launcher");auto* frame=v.browser->get_main_frame(v.browser);execute(frame,std::string("location.hash=")+(launcher_calibrating?"'#calibrate'":"''")+";window.dispatchEvent(new Event('hashchange'))");frame->base.release(&frame->base);}}else if(kind=="launcher.close"){if(c.contains("launcherSession")&&!c["launcherSession"].is_null()&&c["launcherSession"]!=launcher_session)continue;bool handoff=c.value("handoff",false);hide_launcher(!handoff,handoff);}else if(kind=="menu.open"){hide_launcher(false);auto& menu=create_view("menu","Framely",origin+"/",false,600,840);navigate(menu,"#plugin:"+c.at("plugin").get<std::string>());menu_open=true;launcher_open=false;menu_button.opened(std::chrono::steady_clock::now());}else if(kind=="haptic"){auto key=c.value("view","");if(views.count(key)&&views.at(key)->shown&&!views.at(key)->closing){auto& view=*views.at(key);bool activation=c.value("feedback","hover")=="activate";if(activation&&key=="launcher"&&gaze_active&&launcher_gaze_allowed()){auto now=std::chrono::steady_clock::now();if(now-view.activation_at>=std::chrono::milliseconds(80)){view.activation_at=now;if(ui_sounds)ui_audio.play(true);}}else hover_haptic(view.overlay,activation?view.activation_at:view.haptic_at,activation);}}else if(kind=="manager.open"){hide_launcher(false);if(views.count("framely.manager")&&views.at("framely.manager")->closing){std::lock_guard<std::mutex> lock(queue_mutex);queue.push_back(c);continue;}auto page=c.value("page",std::string{});auto& manager=create_view("framely.manager","Framely",origin+"/manager"+((manager_navigation_page(page)||page.rfind("plugin:",0)==0)?"#"+page:""),true,1440,810);if(page.rfind("plugin:",0)==0)navigate(manager,"#"+page);if(manager_navigation_page(page)){auto* frame=manager.browser->get_main_frame(manager.browser);execute(frame,"location.hash="+json(page).dump()+";window.dispatchEvent(new Event(\"hashchange\"))");frame->base.release(&frame->base);}menu_open=false;launcher_open=false;}else if(kind=="agreement.decline"){auto key=c.value("view","");if(key=="menu"){menu_open=false;if(views.count("menu")){show(*views.at("menu"),false);views.at("menu")->browser->reload(views.at("menu")->browser);}}else if(views.count(key))close_view(*views.at(key));}else if(kind=="manager.close"){if(views.count("framely.manager"))close_view(*views.at("framely.manager"));}else if(kind=="menu.close"){menu_open=false;}else if(kind=="apk.permission.open"){auto key="framely.apk.permission";if(views.count(key)&&views.at(key)->closing){std::lock_guard<std::mutex> lock(queue_mutex);queue.push_back(c);continue;}auto& v=create_view(key,"Android permission",origin+"/apk/permission",false,1000,560,"",.85f);anchor_permission(v);}else if(kind=="apk.permission.close"){if(views.count("framely.apk.permission"))close_view(*views.at("framely.apk.permission"));}else if(kind=="cast.window"){auto id=c.at("id").get<std::string>();auto key="framely.cast."+id;if(views.count(key)&&views.at(key)->closing){std::lock_guard<std::mutex> lock(queue_mutex);queue.push_back(c);continue;}int width=c.value("width",640),height=c.value("height",240);if(width<160||width>1920||height<112||height>8192)continue;auto& v=create_view(key,"Framely casting",origin+"/cast/window/"+id,false,width,height,"",2.f);v.cast_frame.path=c.at("frame").get<std::string>();if(v.width!=width||v.height!=height){v.width=width;v.heighLine truncated
static void commands(){std::deque<json> commands;{std::lock_guard<std::mutex> lock(queue_mutex);commands.swap(queue);}for(const auto& c:commands){try{auto kind=c.value("kind","");if(kind=="cast.framing.open"){auto key="framely.framing";if(views.count(key)&&views.at(key)->closing){std::lock_guard<std::mutex> lock(queue_mutex);queue.push_back(c);continue;}auto geometry=c.at("geometry");RecordingRegionPlane plane;auto view=geometry.at("view");if(!recording_region_plane(view,geometry.at("referenceTan"),0,0,1,plane))continue;auto& v=create_view(key,"Recording frame",origin+"/cast/framing",false,1600,1600);v.region_editor_id=geometry.at("id");v.region_geometry=geometry;update_framing_placement(v);menu_open=false;hide_launcher(false);}else if(kind=="cast.framing.reference"){auto found=views.find("framely.framing");if(found!=views.end()&&!found->second->closing&&found->second->region_editor_id==c.value("id",std::string{})){double cx=c.at("centerTanX"),cy=c.at("centerTanY");if(std::isfinite(cx)&&std::isfinite(cy)&&std::abs(cx)<=1.733&&std::abs(cy)<=1.733){found->second->region_geometry["view"]["centerTanX"]=cx;found->second->region_geometry["view"]["centerTanY"]=cy;}}}else if(kind=="cast.framing.close"){if(views.count("framely.framing")&&views.at("framely.framing")->region_editor_id==c.value("id",std::string{}))close_view(*views.at("framely.framing"));}else if(kind=="feedback.settings"){ui_haptics=c.value("haptics",true);ui_sounds=c.value("sounds",false);if(!ui_sounds)ui_audio.silence();}else if(kind=="file.dialog.complete"){for(auto& [key,view]:views){auto& v=*view;if(!v.file_dialog||v.file_dialog_id!=c.value("id",std::string{}))continue;auto paths=c.value("paths",json::array());if(paths.empty()){cancel_file_dialog(v);break;}if((!v.file_multiple&&paths.size()!=1)||paths.size()>64)break;auto* callback=v.file_dialog;v.file_dialog=nullptr;v.file_dialog_id.clear();auto list=cef_string_list_alloc();for(const auto& path:paths){cef_string_t value{};str(value,path.get<std::string>());cef_string_list_append(list,&value);cef_string_utf16_clear(&value);}callback->cont(callback,list);cef_string_list_free(list);callback->base.release(&callback->base);break;}}else if(kind=="gaze.correction"){gaze_correction=c.value("calibration",json(nullptr));}else if(kind=="gaze.target"){gaze_target_id=c.value("id",std::string{});gaze_target_u=c.value("x",.5f);gaze_target_v=1-c.value("y",.5f);gaze_preview=c.value("preview",json(nullptr));gaze_target_cached=false;}else if(kind=="launcher.calibrate"){gaze_target_id.clear();gaze_target_cached=false;gaze_preview=nullptr;launcher_calibrating=c.value("active",true);if(launcher_calibrating&&!launcher_open){if(!views.count("launcher"))create_view("launcher","Framely launcher",origin+"/launcher#calibrate",true,1440,800);toggle_entry(true);}if(views.count("launcher")){auto& v=*views.at("launcher");auto* frame=v.browser->get_main_frame(v.browser);execute(frame,std::string("location.hash=")+(launcher_calibrating?"'#calibrate'":"''")+";window.dispatchEvent(new Event('hashchange'))");frame->base.release(&frame->base);}}else if(kind=="launcher.close"){if(c.contains("launcherSession")&&!c["launcherSession"].is_null()&&c["launcherSession"]!=launcher_session)continue;bool handoff=c.value("handoff",false);hide_launcher(!handoff,handoff);}else if(kind=="menu.open"){hide_launcher(false);auto& menu=create_view("menu","Framely",origin+"/",false,600,840);navigate(menu,"#plugin:"+c.at("plugin").get<std::string>());menu_open=true;launcher_open=false;menu_button.opened(std::chrono::steady_clock::now());}else if(kind=="haptic"){auto key=c.value("view","");if(views.count(key)&&views.at(key)->shown&&!views.at(key)->closing){auto& view=*views.at(key);bool activation=c.value("feedback","hover")=="activate";if(activation&&key=="launcher"&&gaze_active&&launcher_gaze_allowed()){auto now=std::chrono::steady_clock::now();if(now-view.activation_at>=std::chrono::milliseconds(80)){view.activation_at=now;if(ui_sounds)ui_audio.play(true);}}else hover_haptic(view.overlay,activation?view.activation_at:view.haptic_at,activation);}}else if(kind=="manager.open"){hide_launcher(false);if(views.count("framely.manager")&&views.at("framely.manager")->closing){std::lock_guard<std::mutex> lock(queue_mutex);queue.push_back(c);continue;}auto page=c.value("page",std::string{});auto& manager=create_view("framely.manager","Framely",origin+"/manager"+((manager_navigation_page(page)||page.rfind("plugin:",0)==0)?"#"+page:""),true,1440,810);if(page.rfind("plugin:",0)==0)navigate(manager,"#"+page);if(manager_navigation_page(page)){auto* frame=manager.browser->get_main_frame(manager.browser);execute(frame,"location.hash="+json(page).dump()+";window.dispatchEvent(new Event(\"hashchange\"))");frame->base.release(&frame->base);}menu_open=false;launcher_open=false;}else if(kind=="agreement.decline"){auto key=c.value("view","");if(key=="menu"){menu_open=false;if(views.count("menu")){show(*views.at("menu"),false);views.at("menu")->browser->reload(views.at("menu"Line truncated
}
static void overlay_input(View& v,const vr::VREvent_t& e){if(v.closing)return;if((v.key=="notifications"||v.key=="launcher"||v.key=="menu")&&!v.shown&&e.eventType!=vr::VREvent_OverlayClosed)return;if(v.key=="launcher"&&!launcher_open&&e.eventType!=vr::VREvent_OverlayClosed)return;if(v.key=="launcher"&&!launcher_accepts(e)&&(e.eventType==vr::VREvent_Modal_Cancel||e.eventType==vr::VREvent_ButtonPress||e.eventType==vr::VREvent_MouseButtonDown||e.eventType==vr::VREvent_MouseButtonUp))return;if(keyboard_event(e,v.overlay))return;if(menu_open&&v.key!="menu"&&e.eventType==vr::VREvent_MouseButtonDown&&!keyboard_open&&menu_button.can_cancel(std::chrono::steady_clock::now()))menu_open=false;laser_cursor(v.overlay,e);cef_mouse_event_t mouse{};mouse.x=int(e.data.mouse.x);mouse.y=v.height-int(e.data.mouse.y);if(e.eventType==vr::VREvent_MouseMove||e.eventType==vr::VREvent_MouseButtonDown||e.eventType==vr::VREvent_MouseButtonUp){v.pointer=mouse;v.pointer_valid=true;v.pointer_device=e.trackedDeviceIndex;v.pointer_at=std::chrono::steady_clock::now();}switch(e.eventType){case vr::VREvent_Modal_Cancel:if(v.key=="launcher"&&!keyboard_open&&!gaze_active&&!launcher_dock_interaction()&&!overlays->IsHoverTargetOverlay(v.overlay))launcher_back();else if(v.key=="menu"&&!keyboard_open&&!keyboard_menu.holds(std::chrono::steady_clock::now())&&menu_button.can_cancel(std::chrono::steady_clock::now())){menu_open=false;std::cout<<"Menu closed: outside click"<<std::endl;}break;case vr::VREvent_OverlayClosed:if(v.key=="launcher")hide_launcher();else if(v.key!="menu"&&v.key!="notifications")close_view(v);break;case vr::VREvent_ButtonPress:if(e.data.controller.button==vr::k_EButton_ApplicationMenu){if(v.key=="launcher"){launcher_back();break;}auto* frame=v.browser->get_main_frame(v.browser);execute(frame,"window.dispatchEvent(new Event('framely.back'))");frame->base.release(&frame->base);}break;case vr::VREvent_FocusLeave:if(v.key=="launcher"&&v.launcher_pointer.holding(std::chrono::steady_clock::now()))break;if(v.key!="notifications"||!v.notification_click.active)v.host->send_mouse_move_event(v.host,&mouse,1);break;case vr::VREvent_MouseMove:if(v.key=="launcher")v.launcher_pointer.constrain(mouse.x,mouse.y,std::chrono::steady_clock::now());if(v.key!="notifications"||!v.notification_click.hold(mouse.x,mouse.y,std::chrono::steady_clock::now()))v.host->send_mouse_move_event(v.host,&mouse,0);break;case vr::VREvent_MouseButtonDown:case vr::VREvent_MouseButtonUp:{v.host->set_focus(v.host,1);auto button=e.data.mouse.button==vr::VRMouseButton_Right?MBT_RIGHT:MBT_LEFT;if(v.key=="launcher"&&button==MBT_LEFT){auto now=std::chrono::steady_clock::now();if(e.eventType==vr::VREvent_MouseButtonDown)v.launcher_pointer.begin(mouse.x,mouse.y,now);else{v.launcher_pointer.constrain(mouse.x,mouse.y,now);v.launcher_pointer.clear();}}if(v.key=="notifications"&&button==MBT_LEFT){auto now=std::chrono::steady_clock::now();if(e.eventType==vr::VREvent_MouseButtonDown)v.notification_click.begin(mouse.x,mouse.y,now);else v.notification_click.release(mouse.x,mouse.y,now);}mouse.modifiers=e.eventType==vr::VREvent_MouseButtonDown?EVENTFLAG_LEFT_MOUSE_BUTTON:0;v.host->send_mouse_click_event(v.host,&mouse,button,e.eventType==vr::VREvent_MouseButtonUp,1);break;}case vr::VREvent_ScrollSmooth:case vr::VREvent_ScrollDiscrete:if(v.stick_scrolling||keyboard_open||!v.pointer_valid)break;mouse=v.pointer;mouse.modifiers=0;v.host->send_mouse_wheel_event(v.host,&mouse,int(e.data.scroll.xdelta*120),int(e.data.scroll.ydelta*120));break;case vr::VREvent_OverlayGamepadFocusLost:if(!keyboard_open&&!v.dashboard&&v.key!="menu"&&v.key!="notifications"&&v.key!="launcher"&&v.key.rfind("framely.cast.",0)!=0&&v.key!="framely.apk.permission")close_view(v);break;default:break;}}
static void release_framing_pointer(View& v){
if(!v.framing_left_down)return;v.framing_left_down=false;
auto mouse=v.pointer;mouse.modifiers=0;v.host->send_mouse_click_event(v.host,&mouse,MBT_LEFT,1,1);
auto* frame=v.browser->get_main_frame(v.browser);execute(frame,"window.dispatchEvent(new Event('framely.pointerRelease'))");frame->base.release(&frame->base);
}
static void reconcile_framing_pointer(View& v){
if(v.key!="framely.framing"||!v.framing_left_down)return;
bool known=true,held=false,found=false,owner=false;
for(int hand=0;hand<2;hand++){
auto device=vr_system->GetTrackedDeviceIndexForControllerRole(hand?vr::TrackedControllerRole_RightHand:vr::TrackedControllerRole_LeftHand);
if(device==vr::k_unTrackedDeviceIndexInvalid)continue;
if(device==v.pointer_device){known=framing_trigger_known[hand];held=framing_trigger_down[hand];owner=true;break;}
}
if(!owner)for(int hand=0;hand<2;hand++){
auto device=vr_system->GetTrackedDeviceIndexForControllerRole(hand?vr::TrackedControllerRole_RightHand:vr::TrackedControllerRole_LeftHand);
if(device==vr::k_unTrackedDeviceIndexInvalid)continue;found=true;known&=framing_trigger_known[hand];held|=framing_trigger_down[hand];
}
if((owner||found)&&known&&!held)release_framing_pointer(v);
}
static void overlay_input(View& v,const vr::VREvent_t& e){if(v.closing)return;if((v.key=="notifications"||v.key=="launcher"||v.key=="menu")&&!v.shown&&e.eventType!=vr::VREvent_OverlayClosed)return;if(v.key=="launcher"&&!launcher_open&&e.eventType!=vr::VREvent_OverlayClosed)return;if(v.key=="launcher"&&!launcher_accepts(e)&&(e.eventType==vr::VREvent_Modal_Cancel||e.eventType==vr::VREvent_ButtonPress||e.eventType==vr::VREvent_MouseButtonDown||e.eventType==vr::VREvent_MouseButtonUp))return;if(keyboard_event(e,v.overlay))return;if(menu_open&&v.key!="menu"&&e.eventType==vr::VREvent_MouseButtonDown&&!keyboard_open&&menu_button.can_cancel(std::chrono::steady_clock::now()))menu_open=false;laser_cursor(v.overlay,e);cef_mouse_event_t mouse{};mouse.x=int(e.data.mouse.x);mouse.y=v.height-int(e.data.mouse.y);if(e.eventType==vr::VREvent_MouseMove||e.eventType==vr::VREvent_MouseButtonDown||e.eventType==vr::VREvent_MouseButtonUp){v.pointer=mouse;v.pointer_valid=true;v.pointer_device=e.trackedDeviceIndex;v.pointer_at=std::chrono::steady_clock::now();}switch(e.eventType){case vr::VREvent_Modal_Cancel:if(v.key=="launcher"&&!keyboard_open&&!gaze_active&&!launcher_dock_interaction()&&!overlays->IsHoverTargetOverlay(v.overlay))launcher_back();else if(v.key=="menu"&&!keyboard_open&&!keyboard_menu.holds(std::chrono::steady_clock::now())&&menu_button.can_cancel(std::chrono::steady_clock::now())){menu_open=false;std::cout<<"Menu closed: outside click"<<std::endl;}break;case vr::VREvent_OverlayClosed:if(v.key=="launcher")hide_launcher();else if(v.key!="menu"&&v.key!="notifications")close_view(v);break;case vr::VREvent_ButtonPress:if(e.data.controller.button==vr::k_EButton_ApplicationMenu){if(v.key=="launcher"){launcher_back();break;}auto* frame=v.browser->get_main_frame(v.browser);execute(frame,"window.dispatchEvent(new Event('framely.back'))");frame->base.release(&frame->base);}break;case vr::VREvent_FocusLeave:if(v.key=="framely.framing")release_framing_pointer(v);if(v.key=="launcher"&&v.launcher_pointer.holding(std::chrono::steady_clock::now()))break;if(v.key!="notifications"||!v.notification_click.active)v.host->send_mouse_move_event(v.host,&mouse,1);break;case vr::VREvent_MouseMove:if(v.key=="framely.framing")mouse.modifiers=v.framing_left_down?EVENTFLAG_LEFT_MOUSE_BUTTON:0;if(v.key=="launcher")v.launcher_pointer.constrain(mouse.x,mouse.y,std::chrono::steady_clock::now());if(v.key!="notifications"||!v.notification_click.hold(mouse.x,mouse.y,std::chrono::steady_clock::now()))v.host->send_mouse_move_event(v.host,&mouse,0);break;case vr::VREvent_MouseButtonDown:case vr::VREvent_MouseButtonUp:{v.host->set_focus(v.host,1);auto button=e.data.mouse.button==vr::VRMouseButton_Right?MBT_RIGHT:MBT_LEFT;if(v.key=="launcher"&&button==MBT_LEFT){auto now=std::chrono::steady_clock::now();if(e.eventType==vr::VREvent_MouseButtonDown)v.launcher_pointer.begin(mouse.x,mouse.y,now);else{v.launcher_pointer.constrain(mouse.x,mouse.y,now);v.launcher_pointer.clear();}}if(v.key=="notifications"&&button==MBT_LEFT){auto now=std::chrono::steady_clock::now();if(e.eventType==vr::VREvent_MouseButtonDown)v.notification_click.begin(mouse.x,mouse.y,now);else v.notification_click.release(mouse.x,mouse.y,now);}mouse.modifiers=e.eventType==vr::VREvent_MouseButtonDown?EVENTFLAG_LEFT_MOUSE_BUTTON:0;if(v.key=="framely.framing"&&button==MBT_LEFT)v.framing_left_down=e.eventType==vr::VREvent_MouseButtonDown;v.host->send_mouse_click_event(v.host,&mouse,button,e.eventType==vr::VREvent_MouseButtonUp,1);break;}case vr::VREvent_ScrollSmooth:case vr::VREvent_ScrollDiscrete:if(v.stick_scrolling||keyboard_open||!v.pointer_valid)break;mouse=v.pointer;mouse.modifiers=0;v.host->send_mouse_wheel_event(v.host,&mouse,int(e.data.scroll.xdelta*120),int(e.data.scroll.ydelta*120));break;case vr::VREvent_OverlayGamepadFocusLost:if(v.key=="framely.framing")release_framing_pointer(v);if(!keyboard_open&&!v.dashboard&&v.key!="menu"&&v.key!="notifications"&&v.key!="launcher"&&v.key.rfind("framely.cast.",0)!=0&&v.key!="framely.apk.permission"&&v.key!="framely.framing")close_view(v);break;default:break;}}
int main(int argc,char** argv){
std::cout.setf(std::ios::unitbuf);std::string initial;int port=0;for(int i=1;i+1<argc;i++){std::string s=argv[i];if(s=="--url")initial=argv[++i];else if(s=="--port")port=std::stoi(argv[++i]);else if(s=="--runtime")runtime=argv[++i];}
@@ -654,10 +749,10 @@ int main(int argc,char** argv){
vr::VROverlayHandle_t button=0;auto err=overlays->CreateOverlay("framely.dock.button","Framely",&button);if(err){cef_shutdown();vr::VR_Shutdown();return 6;}auto pixels=dock_icon(false,false);DockIconTexture icon_texture;icon_texture.init();if(!icon_texture.submit(button,pixels)){icon_texture.destroy();overlays->DestroyOverlay(button);cef_shutdown();vr::VR_Shutdown();return 6;}overlays->SetOverlayInputMethod(button,vr::VROverlayInputMethod_Mouse);vr::HmdVector2_t scale{{dock_canvas_width,dock_canvas_height}};overlays->SetOverlayMouseScale(button,&scale);vr::VROverlayIntersectionMaskPrimitive_t mask{};mask.m_nPrimitiveType=vr::OverlayIntersectionPrimitiveType_Rectangle;mask.m_Primitive.m_Rectangle={dock_icon_left,0,128,128};if(overlays->SetOverlayIntersectionMask(button,&mask,1)){std::cerr<<"Dock input mask unavailable\n";overlays->DestroyOverlay(button);icon_texture.destroy();cef_shutdown();vr::VR_Shutdown();return 6;}overlays->SetOverlayFlag(button,vr::VROverlayFlags_VisibleInDashboard,true);overlays->SetOverlayFlag(button,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,true);overlays->SetOverlayFlag(button,vr::VROverlayFlags_SortWithNonSceneOverlays,false);overlays->SetOverlaySortOrder(button,0);overlays->SetOverlayFlag(button,vr::VROverlayFlags_HideLaserIntersection,false);
try{create_view("menu","Framely",initial,false,600,840);}catch(const std::exception& e){std::cerr<<e.what()<<"\n";running=0;}
const char* steam_root=std::getenv("FRAMELY_STEAM_ROOT");if(!ui_audio.prepare(steam_root?steam_root:""))std::cerr<<"Steam UI sounds unavailable\n";
std::thread poller([port]{int failures=0;json last_feedback;json last_gaze;bool gaze_loaded=false;while(running){try{auto data=poll_http(port);notification_badge_count.store(data.value("notificationBadge",0u));launcher_prioritize_ui.store(data.value("launcher",json::object()).value("prioritizeUi",true));std::lock_guard<std::mutex> lock(queue_mutex);recording_state=data.value("recording",json::object());if(!recording_state.is_object())recording_state=json::object();auto feedback=data.value("feedback",json::object());if(!feedback.is_object())feedback=json::object();if(feedback!=last_feedback){queue.push_back({{"kind","feedback.settings"},{"haptics",feedback.value("haptics",true)},{"sounds",feedback.value("sounds",false)}});last_feedback=feedback;}auto calibration=data.value("gazeCalibration",json(nullptr));if(!gaze_loaded||calibration!=last_gaze){queue.push_back({{"kind","gaze.correction"},{"calibration",calibration}});last_gaze=calibration;gaze_loaded=true;}for(const auto& c:data.at("commands")){if(queue.size()<256)queue.push_back(c);}failures=0;}catch(const std::exception& e){if(++failures>=10){std::cerr<<"Agent connection lost: "<<e.what()<<"\n";running=0;}}std::this_thread::sleep_for(std::chrono::milliseconds(100));}});
std::thread poller([port]{int failures=0;json last_feedback;json last_gaze;bool gaze_loaded=false;while(running){try{auto data=poll_http(port);notification_badge_count.store(data.value("notificationBadge",0u));launcher_prioritize_ui.store(data.value("launcher",json::object()).value("prioritizeUi",true));std::lock_guard<std::mutex> lock(queue_mutex);recording_state=data.value("recording",json::object());recording_region_state=data.value("recordingRegion",json::object());framing_editor_state=data.value("framingEditor",json(nullptr));if(!recording_state.is_object())recording_state=json::object();auto feedback=data.value("feedback",json::object());if(!feedback.is_object())feedback=json::object();if(feedback!=last_feedback){queue.push_back({{"kind","feedback.settings"},{"haptics",feedback.value("haptics",true)},{"sounds",feedback.value("sounds",false)}});last_feedback=feedback;}auto calibration=data.value("gazeCalibration",json(nullptr));if(!gaze_loaded||calibration!=last_gaze){queue.push_back({{"kind","gaze.correction"},{"calibration",calibration}});last_gaze=calibration;gaze_loaded=true;}for(const auto& c:data.at("commands")){if(queue.size()<256)queue.push_back(c);}failures=0;}catch(const std::exception& e){if(++failures>=10){std::cerr<<"Agent connection lost: "<<e.what()<<"\n";running=0;}}std::this_thread::sleep_for(std::chrono::milliseconds(100));}});
std::thread priority_worker([]{LauncherPriority priority;while(running){priority.update(launcher_priority_active.load());std::this_thread::sleep_for(std::chrono::milliseconds(100));}});
DockUVState uv{};OverlayPlacement button_placement;bool icon_active=false,icon_hover=false;unsigned icon_badge=0;int icon_progress=0;vr::VROverlayHandle_t last_dock=0;bool was_visible=false;vr::HmdMatrix34_t saved_anchor{},saved_frame{};float saved_size=0;DockCurveEvidence saved_evidence{};auto button_haptic_at=std::chrono::steady_clock::time_point{};auto anchored_at=std::chrono::steady_clock::time_point{};auto sampled_at=std::chrono::steady_clock::time_point{};
while(running){auto frame_start=std::chrono::steady_clock::now();cef_do_message_loop_work();commands();json recording;{std::lock_guard<std::mutex> lock(queue_mutex);recording=recording_state;}recording_indicator.tick(overlays,recording.value("active",false)?recording.value("id",std::string{}):std::string{},recording.value("elapsedSeconds",uint64_t(0)));vr::VREvent_t global{};while(vr_system->PollNextEvent(&global,sizeof(global))){if(global.eventType==vr::VREvent_Quit){vr_system->AcknowledgeQuit_Exiting();running=0;}observe_dashboard(global);keyboard_event(global);}launcher_priority_active.store(launcher_open&&launcher_prioritize_ui.load());vr::VROverlayHandle_t dock=0;vr::HmdVector2_t mouse_scale{};bool visible=!overlays->FindOverlay("valve.steam.gamepadui.bar",&dock)&&overlays->IsDashboardVisible()&&overlays->IsOverlayVisible(dock)&&!overlays->GetOverlayMouseScale(dock,&mouse_scale);
while(running){auto frame_start=std::chrono::steady_clock::now();cef_do_message_loop_work();commands();json recording,region,framing;{std::lock_guard<std::mutex> lock(queue_mutex);recording=recording_state;region=recording_region_state;framing=framing_editor_state;}if(auto editor=views.find("framely.framing");editor!=views.end()&&framing_editor_should_close(framing,editor->second->region_editor_id))close_view(*editor->second);recording_indicator.tick(overlays,recording.value("active",false)?recording.value("id",std::string{}):std::string{},recording.value("elapsedSeconds",uint64_t(0)));if(auto editor=views.find("framely.framing");editor!=views.end())update_framing_placement(*editor->second);if(region.value("visible",false)&&region.contains("view")){auto presentation=recording_region_presentation(region["view"]);recording_region_overlay.tick(overlays,region,presentation.distance,&presentation.view);}else recording_region_overlay.tick(overlays,region);vr::VREvent_t global{};while(vr_system->PollNextEvent(&global,sizeof(global))){if(global.eventType==vr::VREvent_Quit){vr_system->AcknowledgeQuit_Exiting();running=0;}observe_dashboard(global);keyboard_event(global);}launcher_priority_active.store(launcher_open&&launcher_prioritize_ui.load());vr::VROverlayHandle_t dock=0;vr::HmdVector2_t mouse_scale{};bool visible=!overlays->FindOverlay("valve.steam.gamepadui.bar",&dock)&&overlays->IsDashboardVisible()&&overlays->IsOverlayVisible(dock)&&!overlays->GetOverlayMouseScale(dock,&mouse_scale);
if(visible&&(!was_visible||dock!=last_dock)){uv={};anchored_at={};sampled_at={};}last_dock=dock;
// Width and aspect are only ray-solver seeds; physical size comes from intersections.
float width=0,height=0;bool fresh=visible&&!overlays->GetOverlayWidthInMeters(dock,&width)&&std::isfinite(width)&&width>0&&width<10&&std::isfinite(mouse_scale.v[0])&&std::isfinite(mouse_scale.v[1])&&mouse_scale.v[0]>0&&mouse_scale.v[1]>0;
@@ -737,6 +832,7 @@ int main(int argc,char** argv){
}
if(launcher_open){bool held=false;for(auto& hand:entry_buttons)for(auto& trigger:hand)held|=trigger.held;launcher_dismiss.observe(held);}
for(auto&[key,v]:views){if(v->closing)continue;if(v->dashboard){bool active=overlays->IsDashboardVisible()&&overlays->IsActiveDashboardOverlay(v->overlay)&&overlays->IsOverlayVisible(v->overlay);if(active!=v->shown){if(!active&&key=="launcher"){if(v->launcher_pointer.active){cef_mouse_event_t outside{};outside.modifiers=EVENTFLAG_LEFT_MOUSE_BUTTON;v->host->send_mouse_move_event(v->host,&outside,0);outside.modifiers=0;v->host->send_mouse_click_event(v->host,&outside,MBT_LEFT,1,1);}v->launcher_pointer.clear();auto* frame=v->browser->get_main_frame(v->browser);execute(frame,"window.dispatchEvent(new Event('framely.launcher.cancelInput'))");frame->base.release(&frame->base);}v->host->was_hidden(v->host,!active);v->shown=active;}if(!active)v->active_seen=false;if(active&&!v->active_seen){v->active_seen=true;if(key!="launcher")for(auto&[other,t]:views)if(!t->dashboard&&other!="menu"&&other!="notifications"&&other!="launcher"&&other.rfind("framely.cast.",0)!=0&&other!="framely.apk.permission"&&!keyboard_open)close_view(*t);}}
reconcile_framing_pointer(*v);
while(overlays->PollNextOverlayEvent(v->overlay,&event,sizeof(event)))overlay_input(*v,event);
if(!v->cast_frame.path.empty()){glXMakeContextCurrent(xdisplay,glsurface,glsurface,glcontext);if(v->cast_frame.update())v->dirty=true;}
if(v->dirty&&!v->closing&&v->pixels.size()==size_t(v->width*v->render_scale)*size_t(v->height*v->render_scale)*4){
@@ -752,7 +848,7 @@ int main(int argc,char** argv){
if(!v->texture_ready[slot]){
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,mipmapped?GL_LINEAR_MIPMAP_LINEAR:GL_LINEAR);glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE);glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA8,v->width*v->render_scale,v->height*v->render_scale,0,GL_BGRA,GL_UNSIGNED_BYTE,nullptr);
glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA8,pixel_width,pixel_height,0,GL_BGRA,GL_UNSIGNED_BYTE,nullptr);
v->texture_ready[slot]=true;
damage.include(0,0,pixel_width,pixel_height);
}
@@ -813,5 +909,6 @@ int main(int argc,char** argv){
if(sync!=vr::VROverlayError_None&&sync!=vr::VROverlayError_TimedOut)std::this_thread::sleep_until(frame_start+std::chrono::milliseconds(8));
}
recording_indicator.destroy();
recording_region_overlay.destroy();
close_keyboard();for(auto&[key,v]:views)close_view(*v);auto deadline=std::chrono::steady_clock::now()+std::chrono::seconds(5);while(std::chrono::steady_clock::now()<deadline){cef_do_message_loop_work();bool all=true;for(auto&[key,v]:views)all&=v->closed;if(all)break;std::this_thread::sleep_for(std::chrono::milliseconds(5));}poller.join();priority_worker.join();for(auto&[key,v]:views)if(!v->closed){std::cerr<<"Browser shutdown timed out\n";_Exit(7);}for(auto&[key,v]:views){overlays->DestroyOverlay(v->overlay);if(v->thumbnail)overlays->DestroyOverlay(v->thumbnail);glDeleteTextures(2,v->textures);}views.clear();overlays->DestroyOverlay(button);icon_texture.destroy();cef_shutdown();vr::VR_Shutdown();glXMakeContextCurrent(xdisplay,None,None,nullptr);glXDestroyContext(xdisplay,glcontext);glXDestroyPbuffer(xdisplay,glsurface);XCloseDisplay(xdisplay);return 0;
}
+85
View File
@@ -0,0 +1,85 @@
#pragma once
#include <cmath>
#include <algorithm>
#include <vector>
#include <array>
#include <chrono>
#include "vendor/json.hpp"
using RegionJson=nlohmann::json;
struct RecordingRegionPlane {vr::HmdMatrix34_t transform{};float width=0;};
inline void recording_region_foreground(vr::IVROverlay* overlays,vr::VROverlayHandle_t handle){
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_SortWithNonSceneOverlays,true);
overlays->SetOverlaySortOrder(handle,1000);
}
inline bool recording_region_plane(const RegionJson& view,double tan,double cx,double cy,double aspect,RecordingRegionPlane& plane,double distance=2.){
try{
if(!std::isfinite(tan)||tan<=0||!std::isfinite(cx)||!std::isfinite(cy)||!std::isfinite(aspect)||aspect<=0||!std::isfinite(distance)||distance<=0)return false;
const auto& basis=view.at("captureToHead");
for(int i=0;i<3;i++){
double translation=basis.at(i).at(3).get<double>();
for(int j=0;j<3;j++){
double value=basis.at(i).at(j).get<double>();if(!std::isfinite(value))return false;
plane.transform.m[i][j]=value;
translation+=distance*value*(j==0?cx:j==1?-cy:-1.);
}
if(!std::isfinite(translation))return false;plane.transform.m[i][3]=translation;
}
plane.width=2*tan*distance;return std::isfinite(plane.width)&&plane.width>0;
}catch(...){return false;}
}
inline std::array<unsigned char,3> recording_region_color(){
static std::array<unsigned char,3> cached{0,255,255};static auto checked=std::chrono::steady_clock::time_point{};
auto now=std::chrono::steady_clock::now();if(now-checked<std::chrono::seconds(1))return cached;checked=now;
std::array<unsigned char,3> color{0,255,255};auto* settings=vr::VRSettings();
if(!settings)return cached;
const char* keys[]={vr::k_pch_CollisionBounds_ColorGammaR_Int32,vr::k_pch_CollisionBounds_ColorGammaG_Int32,vr::k_pch_CollisionBounds_ColorGammaB_Int32};
for(int i=0;i<3;i++){vr::EVRSettingsError error{};auto value=settings->GetInt32(vr::k_pch_CollisionBounds_Section,keys[i],&error);if(error!=vr::VRSettingsError_None||value<0||value>255)return cached;color[i]=static_cast<unsigned char>(value);}
cached=color;return cached;
}
inline std::vector<unsigned char> recording_region_pixels(int w,int h,const std::array<unsigned char,3>& color={0,255,255}){
std::vector<unsigned char> pixels(w*h*4);
for(int y=0;y<h;y++)for(int x=0;x<w;x++){
bool edge=x<2||x>=w-2||y<2||y>=h-2;
if(edge&&((x<2||x>=w-2?y:x)/14)%2==0){auto i=(y*w+x)*4;for(int c=0;c<3;c++)pixels[i+c]=color[c];pixels[i+3]=72;}
}
return pixels;
}
class RecordingRegionOverlay {
vr::IVROverlay* overlays=nullptr;vr::VROverlayHandle_t handle=0;std::string last,last_pixels;
public:
void destroy(){if(handle&&overlays)overlays->DestroyOverlay(handle);handle=0;last.clear();last_pixels.clear();}
void tick(vr::IVROverlay* api,const RegionJson& state,double distance=2.,const RegionJson* live_view=nullptr){
overlays=api;
if(!state.value("visible",false)){destroy();return;}
auto color=recording_region_color();auto serialized=state.dump()+std::string(color.begin(),color.end())+std::to_string(distance)+(live_view?live_view->dump():"");if(serialized==last)return;
try{
const auto& view=live_view?*live_view:state.at("view");double aspect=view.at("aspect");
RecordingRegionPlane plane;if(!recording_region_plane(view,view.at("tanHalfHorizontal"),view.at("centerTanX"),view.at("centerTanY"),aspect,plane,distance)){destroy();return;}
if(!handle){
if(overlays->CreateOverlay("framely.recording.region","Framely recording region",&handle))return;
overlays->SetOverlayInputMethod(handle,vr::VROverlayInputMethod_None);
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,false);
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_HideLaserIntersection,true);
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_VisibleInDashboard,true);
recording_region_foreground(overlays,handle);
}
int width=state.at("settings").at("width"),height=state.at("settings").at("height");
if(width<160||height<160||width>4096||height>4096||std::abs(double(width)/height-aspect)>1e-6){destroy();return;}
// The INNER edge is the capture boundary. Keep the guide outside it so
// it does not cover pixels inside the intended recorded image.
// Bound raw-overlay traffic independently of recording resolution.
// Square texels are explicitly stretched to the exact capture aspect.
width=height=512;plane.width*=512.f/508.f;
auto error=overlays->SetOverlayWidthInMeters(handle,plane.width);
if(!error)error=overlays->SetOverlayTexelAspect(handle,aspect);
if(!error)error=overlays->SetOverlayTransformTrackedDeviceRelative(handle,vr::k_unTrackedDeviceIndex_Hmd,&plane.transform);
const auto pixel_key=std::string(color.begin(),color.end());
if(!error&&(last_pixels.empty()||pixel_key!=last_pixels)){
auto pixels=recording_region_pixels(width,height,color);
error=overlays->SetOverlayRaw(handle,pixels.data(),width,height,4);
if(!error)last_pixels=pixel_key;
}
if(error){destroy();return;}overlays->ShowOverlay(handle);last=serialized;
}catch(...){destroy();}
}
};
+90
View File
@@ -0,0 +1,90 @@
#pragma once
#include <array>
#include <cmath>
#include <optional>
#include "openvr.h"
// Pure geometry only. Pose validity/freshness is owned by the caller: a
// numerically rigid matrix cannot reveal whether its tracking sample is old.
namespace recording_region_depth_detail {
using Point=std::array<double,3>;
inline double dot(const Point& a,const Point& b){return a[0]*b[0]+a[1]*b[1]+a[2]*b[2];}
inline Point translation(const vr::HmdMatrix34_t& m){return {m.m[0][3],m.m[1][3],m.m[2][3]};}
inline Point rotate(const vr::HmdMatrix34_t& m,const Point& p){
Point result{};for(int i=0;i<3;i++)for(int j=0;j<3;j++)result[i]+=m.m[i][j]*p[j];return result;
}
inline Point add(const Point& a,const Point& b){return {a[0]+b[0],a[1]+b[1],a[2]+b[2]};}
inline Point subtract(const Point& a,const Point& b){return {a[0]-b[0],a[1]-b[1],a[2]-b[2]};}
inline bool rigid(const vr::HmdMatrix34_t& m){
for(const auto& row:m.m)for(float v:row)if(!std::isfinite(v))return false;
for(int i=0;i<3;i++)for(int j=0;j<3;j++){
double product=0;for(int k=0;k<3;k++)product+=double(m.m[k][i])*m.m[k][j];
if(std::abs(product-(i==j?1.:0.))>0.01)return false;
}
const double determinant=m.m[0][0]*(double(m.m[1][1])*m.m[2][2]-double(m.m[1][2])*m.m[2][1])
-m.m[0][1]*(double(m.m[1][0])*m.m[2][2]-double(m.m[1][2])*m.m[2][0])
+m.m[0][2]*(double(m.m[1][0])*m.m[2][1]-double(m.m[1][1])*m.m[2][0]);
return std::abs(determinant-1.)<=0.01;
}
}
// Refresh just the eye origin. Capture's basis already encodes the LUT's ray
// convention and must not receive the physical-panel rotation a second time.
inline bool recording_region_refresh_eye_origin(vr::EVREye eye,
const vr::HmdMatrix34_t& left_to_head,const vr::HmdMatrix34_t& right_to_head,
vr::HmdMatrix34_t& capture_to_head){
using namespace recording_region_depth_detail;
if((eye!=vr::Eye_Left&&eye!=vr::Eye_Right)||!rigid(left_to_head)||!rigid(right_to_head)||!rigid(capture_to_head))return false;
const auto left=translation(left_to_head),right=translation(right_to_head),baseline=subtract(right,left);
const double ipd=std::sqrt(dot(baseline,baseline));
// Broad plausibility guards, not a replacement for live tracking validity.
if(ipd<0.04||ipd>0.09||baseline[0]<0.8*ipd||std::abs(baseline[1])>0.02||std::abs(baseline[2])>0.02
||dot(left,left)>0.25*0.25||dot(right,right)>0.25*0.25)return false;
const auto& selected=eye==vr::Eye_Left?left_to_head:right_to_head;
for(int i=0;i<3;i++)capture_to_head.m[i][3]=selected.m[i][3];
return true;
}
// Intersect the selected capture-eye center ray with a known UI window plane.
// The returned parameter D is measured along capture -Z, not Euclidean ray
// length: X = eye + D * basis * (centerTanX,-centerTanY,-1).
// It therefore plugs directly into a guide whose width is 2*tanHalfFov*D.
inline std::optional<double> recording_region_window_depth(
const vr::HmdMatrix34_t& hmd_to_world,const vr::HmdMatrix34_t& window_to_world,
const vr::HmdMatrix34_t& capture_to_head,double center_tan_x,double center_tan_y){
using namespace recording_region_depth_detail;
if(!rigid(hmd_to_world)||!rigid(window_to_world)||!rigid(capture_to_head)
||!std::isfinite(center_tan_x)||!std::isfinite(center_tan_y)
||std::abs(center_tan_x)>10.||std::abs(center_tan_y)>10.
||dot(translation(capture_to_head),translation(capture_to_head))>0.25*0.25)return std::nullopt;
const Point origin=add(translation(hmd_to_world),rotate(hmd_to_world,translation(capture_to_head)));
const Point ray=rotate(hmd_to_world,rotate(capture_to_head,{center_tan_x,-center_tan_y,-1.}));
const Point normal={window_to_world.m[0][2],window_to_world.m[1][2],window_to_world.m[2][2]};
const double denominator=dot(normal,ray);
if(std::abs(denominator)<1e-5*std::sqrt(dot(ray,ray)))return std::nullopt;
const double distance=dot(normal,subtract(translation(window_to_world),origin))/denominator;
if(!std::isfinite(distance)||distance<0.25||distance>6.)return std::nullopt;
return distance;
}
// ComputeOverlayIntersection can supply an exact curved/tilted UI hit. Recover
// the same capture -Z parameter, rejecting unrelated/off-ray intersection data.
inline std::optional<double> recording_region_hit_depth(
const vr::HmdMatrix34_t& hmd_to_world,const vr::HmdMatrix34_t& capture_to_head,
double center_tan_x,double center_tan_y,const vr::HmdVector3_t& world_hit){
using namespace recording_region_depth_detail;
if(!rigid(hmd_to_world)||!rigid(capture_to_head)
||!std::isfinite(center_tan_x)||!std::isfinite(center_tan_y)
||std::abs(center_tan_x)>10.||std::abs(center_tan_y)>10.
||dot(translation(capture_to_head),translation(capture_to_head))>0.25*0.25)return std::nullopt;
Point hit{};for(int i=0;i<3;i++){hit[i]=world_hit.v[i];if(!std::isfinite(hit[i]))return std::nullopt;}
const Point origin=add(translation(hmd_to_world),rotate(hmd_to_world,translation(capture_to_head)));
const Point ray=rotate(hmd_to_world,rotate(capture_to_head,{center_tan_x,-center_tan_y,-1.}));
const Point delta=subtract(hit,origin);
const double squared_length=dot(ray,ray),distance=dot(delta,ray)/squared_length;
if(!std::isfinite(distance)||distance<0.25||distance>6.)return std::nullopt;
const Point residual={delta[0]-distance*ray[0],delta[1]-distance*ray[1],delta[2]-distance*ray[2]};
const double tolerance=1e-4+1e-4*distance*std::sqrt(squared_length);
if(dot(residual,residual)>tolerance*tolerance)return std::nullopt;
return distance;
}
+3
View File
@@ -3,6 +3,9 @@
pub mod settings;
pub mod runtime;
mod recording;
pub mod region;
#[cfg(test)]
mod region_device_test;
pub mod receiver;
pub mod watch_access;
+90 -11
View File
@@ -1,4 +1,4 @@
//! Mux the existing stream without another capture or video encoder.
//! MP4 recording shares the capture encoder, with AAC audio for compatibility.
use anyhow::{ensure, Context, Result};
use serde_json::{json, Value};
use std::{
@@ -30,19 +30,25 @@ fn has_media(progress: &str) -> bool {
positive("out_time_us=") && positive("total_size=")
}
impl Recording {
pub fn start(rtsp_port: u16, directory: &Path, output: &Path) -> Result<Self> {
Self::start_command(Command::new("ffmpeg"), rtsp_port, directory, output)
pub fn start(
rtsp_port: u16,
directory: &Path,
output: &Path,
codec: super::settings::Codec,
) -> Result<Self> {
Self::start_command(Command::new("ffmpeg"), rtsp_port, directory, output, codec)
}
fn start_command(
mut cmd: Command,
rtsp_port: u16,
directory: &Path,
output: &Path,
codec: super::settings::Codec,
) -> Result<Self> {
fs::create_dir_all(output).context("无法创建录制目录")?;
let id = hex::encode(rand::random::<[u8; 16]>());
let path = output.join(format!(
"Framely-{}-{}.mkv",
"Framely-{}-{}.mp4",
crate::service::now_ms(),
&id[..8]
));
@@ -75,8 +81,12 @@ impl Recording {
"0:v:0",
"-map",
"0:a?",
"-c",
"-c:v",
"copy",
"-c:a",
"aac",
"-b:a",
"128k",
"-avoid_negative_ts",
"make_zero",
"-stats_period",
@@ -84,7 +94,21 @@ impl Recording {
"-progress",
])
.arg(&progress)
.args(["-flush_packets", "1", "-f", "matroska"])
// Fragmented MP4 remains usable without a final global index, and
// avoids buffering an entire recording until it is stopped.
.args([
"-flush_packets",
"1",
"-movflags",
"+frag_keyframe+empty_moov+default_base_moof",
"-f",
"mp4",
])
.args(if codec == super::settings::Codec::H265 {
vec!["-tag:v", "hvc1"]
} else {
vec![]
})
.arg(&path)
.stdin(Stdio::piped())
.stdout(Stdio::null())
@@ -227,7 +251,7 @@ printf 'trailer\n' >> "$previous"
command
.arg(script)
.env("FAIL_LATER", if fail { "1" } else { "0" });
Recording::start_command(command, 1, directory, output).unwrap()
Recording::start_command(command, 1, directory, output, super::settings::Codec::H264).unwrap()
}
#[cfg(test)]
@@ -249,6 +273,7 @@ mod tests {
let output = dir.path().join("videos");
let mut first = test_recording(dir.path(), &output, false);
let path = first.finish().unwrap();
assert_eq!(path.extension().unwrap(), "mp4");
assert_eq!(fs::read_to_string(&path).unwrap(), "header\ntrailer\n");
assert_eq!(
fs::metadata(&path).unwrap().permissions().mode() & 0o777,
@@ -277,7 +302,8 @@ mod tests {
Command::new("/framely-no-such-ffmpeg"),
1,
dir.path(),
&output
&output,
super::super::settings::Codec::H264
)
.is_err());
assert_eq!(fs::read_dir(output).unwrap().count(), 0);
@@ -304,8 +330,10 @@ mod tests {
let output = dir.path().join("videos");
let rtsp = port();
let api = port();
let whep = port();
let hls = port();
let config = dir.path().join("mediamtx.yml");
fs::write(&config,format!("logLevel: warn\nrtspAddress: 127.0.0.1:{rtsp}\nrtspTransports: [tcp]\napi: yes\napiAddress: 127.0.0.1:{api}\nrtmp: no\nhls: no\nwebrtc: no\nsrt: no\nmoq: no\npaths:\n headset:\n source: publisher\n")).unwrap();
fs::write(&config,format!("logLevel: warn\nrtspAddress: 127.0.0.1:{rtsp}\nrtspTransports: [tcp]\napi: yes\napiAddress: 127.0.0.1:{api}\nrtmp: no\nhls: no\nhlsAddress: 127.0.0.1:{hls}\nwebrtc: no\nwebrtcAddress: 127.0.0.1:{whep}\nwebrtcLocalUDPAddress: :0\nwebrtcLocalTCPAddress: :0\nsrt: no\nmoq: no\npaths:\n headset:\n source: publisher\n")).unwrap();
let binary = std::env::var("FRAMELY_MEDIA_MTX").unwrap();
let _server = Worker(
Command::new(binary)
@@ -392,9 +420,45 @@ mod tests {
assert!(Instant::now() < end, "publisher not ready");
std::thread::sleep(Duration::from_millis(50));
}
let mut recording = Recording::start(rtsp, dir.path(), &output).unwrap();
let mut recording = Recording::start(
rtsp,
dir.path(),
&output,
if codec == "hevc" {
super::super::settings::Codec::H265
} else {
super::super::settings::Codec::H264
},
)
.unwrap();
// Toggling viewing services must leave the RTSP publisher/recorder
// intact. Record-only initially has no WebRTC or HLS listeners.
assert!(std::net::TcpStream::connect(("127.0.0.1", whep)).is_err());
for enabled in [true, false] {
ureq::patch(&format!("http://127.0.0.1:{api}/v3/config/global/patch"))
.timeout(Duration::from_secs(3))
.send_json(json!({"webrtc":enabled,"hls":enabled}))
.unwrap();
let end = Instant::now() + Duration::from_secs(3);
while [whep, hls].iter().any(|port| {
std::net::TcpStream::connect(("127.0.0.1", *port)).is_ok() != enabled
}) {
assert!(Instant::now() < end, "viewing listeners did not switch");
std::thread::sleep(Duration::from_millis(20));
}
let path: Value =
ureq::get(&format!("http://127.0.0.1:{api}/v3/paths/get/headset"))
.timeout(Duration::from_secs(2))
.call()
.unwrap()
.into_json()
.unwrap();
assert_eq!(path["ready"], true, "viewing toggle interrupted capture");
assert!(!recording.exited().unwrap());
}
std::thread::sleep(Duration::from_millis(800));
let path = recording.finish().unwrap();
assert_eq!(path.extension().unwrap(), "mp4");
let probe = Command::new("ffprobe")
.args([
"-v",
@@ -414,6 +478,21 @@ mod tests {
);
let data: Value = serde_json::from_slice(&probe.stdout).unwrap();
let streams = data["streams"].as_array().unwrap();
assert!(
data["format"]["format_name"]
.as_str()
.unwrap()
.contains("mp4"),
"{data}"
);
if codec == "hevc" {
assert!(
streams
.iter()
.any(|s| s["codec_name"] == "hevc" && s["codec_tag_string"] == "hvc1"),
"{data}"
);
}
assert!(
streams
.iter()
@@ -421,7 +500,7 @@ mod tests {
"{data}"
);
assert_eq!(
streams.iter().any(|s| s["codec_name"] == "opus"),
streams.iter().any(|s| s["codec_name"] == "aac"),
audio,
"{data}"
);
+271
View File
@@ -0,0 +1,271 @@
//! Projection metadata comes from the exact LUT used by panel capture.
use super::settings::{Eye, Output, Settings, Source};
use anyhow::{ensure, Context, Result};
use serde_json::{json, Value};
use std::{
io::Read,
path::Path,
process::{Command, Stdio},
time::{Duration, Instant},
};
#[derive(Default)]
pub struct Region {
pub editing: bool,
pub editor_id: String,
enabled: bool,
settings: Option<Settings>,
description: Option<Value>,
pub error: Option<String>,
}
pub fn supported(settings: &Settings) -> bool {
settings.source == Source::Screen && settings.output == Output::Eye
}
pub fn validate_description(value: &Value) -> Result<()> {
ensure!(value["projection"]=="factory-panel-v1" && matches!(value["eye"].as_u64(),Some(0|1)), "缺少有效的面板校准数据,无法显示准确录制区域");
let tan = value["tanHalfHorizontal"]
.as_f64()
.context("Missing capture FOV")?;
let aspect = value["aspect"].as_f64().context("Missing capture aspect")?;
ensure!(
tan.is_finite() && tan > 0. && aspect.is_finite() && aspect > 0.,
"Invalid capture view"
);
for name in ["centerTanX", "centerTanY"] {
ensure!(
value[name].as_f64().is_some_and(f64::is_finite),
"Invalid capture center"
);
}
let matrix: [[f64; 4]; 3] = serde_json::from_value(value["captureToHead"].clone())?;
ensure!(
matrix.iter().flatten().all(|v| v.is_finite()),
"Invalid capture transform"
);
for i in 0..3 {
for j in 0..3 {
let dot: f64 = (0..3).map(|k| matrix[i][k] * matrix[j][k]).sum();
ensure!(
(dot - if i == j { 1. } else { 0. }).abs() < 0.01,
"Invalid capture basis"
);
}
}
Ok(())
}
impl Region {
/// Last editor identity remains available after closing so the native host
/// can reconcile it on every poll, even if a one-shot close command is lost.
pub fn editor_state(&self) -> Value {
json!({"id":self.editor_id,"open":self.editing})
}
pub fn acknowledge_editor_closed(&mut self, id: &str) {
// A delayed acknowledgement for the previous window must not close a
// newer editor that has already been opened.
if self.editor_id == id {
self.editing = false;
}
}
pub fn require_visible(view: &Value) -> Result<()> {
ensure!(view["regionFullyVisible"] == true, "取景范围超出头显可见区域,请缩小或调整位置");
Ok(())
}
pub fn invalidate(&mut self) {
self.description = None;
self.settings = None;
}
pub fn refresh(
&mut self,
settings: &Settings,
binaries: &Path,
active: Option<Value>,
) -> Result<()> {
self.enabled = settings.recording_region_visible;
if !supported(settings) || (!self.enabled && !self.editing) || (active.is_none() && !self.editing) {
self.description = None;
self.settings = None;
self.error = None;
return Ok(());
}
if let Some(value) = active {
// Never substitute configured geometry for a live recording.
validate_description(&value)?;
if !self.editing { Self::require_visible(&value)?; }
self.description = Some(value);
self.settings = Some(settings.clone());
self.error = None;
return Ok(());
}
let eye = match settings.eye {
Eye::Left => 0,
Eye::Right => 1,
Eye::SteamVR => super::runtime::dominant_eye()?,
};
if self
.settings
.as_ref()
.is_some_and(|s| s.same_capture(settings))
&& self.description.as_ref().is_some_and(|d| d["eye"] == eye)
{
if !self.editing { Self::require_visible(self.description.as_ref().unwrap())?; }
return Ok(());
}
self.description = None;
self.settings = None;
let mut command = Command::new(binaries.join("framely-capture"));
command.args([
"--describe-view",
"--width",
&settings.width.to_string(),
"--height",
&settings.height.to_string(),
"--eye",
&eye.to_string(),
]);
command.args([
format!("--center-x={}", settings.center_x),
format!("--center-y={}", settings.center_y),
]);
if let Some(fov) = settings.horizontal_fov {
command.arg(format!("--fov={fov}"));
}
let mut child = command
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
.context("无法读取实际录制区域,请更新采集程序")?;
// Drain metadata while the worker runs: the visibility grid can be
// larger than a small pipe's capacity. Waiting first would deadlock.
let stdout=child.stdout.take().context("Missing geometry output")?;
let output=std::thread::spawn(move || -> std::io::Result<Vec<u8>> {
let mut bytes=Vec::new();stdout.take(65537).read_to_end(&mut bytes)?;Ok(bytes)
});
let deadline = Instant::now() + Duration::from_secs(12);
let status = loop {
match child.try_wait() {
Ok(Some(status))=>break status,
Ok(None)=>{},
Err(error)=>{let _=child.kill();let _=child.wait();let _=output.join();return Err(error.into());}
}
if Instant::now() >= deadline {
let _ = child.kill();
let _ = child.wait();
let _ = output.join();
anyhow::bail!("读取录制区域超时");
}
std::thread::sleep(Duration::from_millis(20));
};
let bytes=output.join().map_err(|_|anyhow::anyhow!("Invalid geometry output reader"))??;
ensure!(
status.success(),
"无法读取实际录制区域,请确认 SteamVR 已启动"
);
ensure!(bytes.len() <= 65536, "Invalid capture geometry output");
let value: Value = serde_json::from_slice(&bytes)?;
validate_description(&value)?;
if !self.editing { Self::require_visible(&value)?; }
self.description = Some(value);
self.settings = Some(settings.clone());
self.error = None;
Ok(())
}
pub fn fail(&mut self, error: &anyhow::Error) {
self.description = None;
self.error = Some(format!("{error:#}"));
}
pub fn snapshot(&self, active:bool) -> Value {
json!({"visible":active && self.enabled && !self.editing && self.description.is_some(),"editing":self.editing,"view":self.description,"settings":self.settings,"error":self.error})
}
pub fn editor(&self) -> Result<Value> {
let view = self
.description
.as_ref()
.context("无法读取实际录制区域,请确认 SteamVR 已启动")?;
let reference=view["editorReferenceTan"].as_f64().filter(|v|v.is_finite()&&*v>0.).context("无法读取实际录制区域,请更新采集程序")?;
let visibility=&view["visibility"];
let n=visibility["size"].as_u64().filter(|n|(8..=256).contains(n)&&n%4==0).context("无法读取实际录制区域,请更新采集程序")?;
ensure!(visibility["extent"].as_f64()==Some(reference) && visibility["cells"].as_str().is_some_and(|s|s.len()==(n*n/4) as usize&&s.bytes().all(|c|c.is_ascii_hexdigit())),"无法读取实际录制区域,请更新采集程序");
Ok(
json!({"id":self.editor_id,"settings":self.settings,"view":view,"referenceTan":reference}),
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn recording_region_editor_close_state_survives_poll_and_rejects_stale_ack() {
let mut region=Region::default();
region.editor_id="first".into();region.editing=true;
assert_eq!(region.editor_state(),json!({"id":"first","open":true}));
region.acknowledge_editor_closed("first");
assert_eq!(region.editor_state(),json!({"id":"first","open":false}));
// Reading a state response must not consume the required close state.
assert_eq!(region.editor_state(),json!({"id":"first","open":false}));
region.editor_id="second".into();region.editing=true;
region.acknowledge_editor_closed("first");
assert_eq!(region.editor_state(),json!({"id":"second","open":true}));
region.acknowledge_editor_closed("second");
assert!(!region.editing);
}
#[test]
fn recording_region_drains_large_metadata_before_waiting() {
use std::os::unix::fs::PermissionsExt;
let directory=tempfile::tempdir().unwrap();
let view=json!({"projection":"factory-panel-v1","eye":0,"regionFullyVisible":true,"editorReferenceTan":1.7,"visibility":{"size":8,"extent":1.7,"cells":"f".repeat(16)},"tanHalfHorizontal":0.5,"aspect":4./3.,"centerTanX":0.,"centerTanY":0.,"captureToHead":[[1,0,0,0],[0,1,0,0],[0,0,1,0]],"padding":"x".repeat(48000)});
let worker=directory.path().join("framely-capture");
std::fs::write(&worker,format!("#!/bin/sh\nprintf '%s' '{}'\n",view)).unwrap();
std::fs::set_permissions(&worker,std::fs::Permissions::from_mode(0o700)).unwrap();
let mut region=Region::default();region.editing=true;
let started=Instant::now();
region.refresh(&Settings {eye:Eye::Left,..Settings::default()},directory.path(),None).unwrap();
assert!(started.elapsed()<Duration::from_secs(3));
assert_eq!(region.editor().unwrap()["referenceTan"],1.7);
}
#[test]
fn recording_region_visibility_persists_independently_of_capture() {
let root=tempfile::tempdir().unwrap();
let mut core=crate::tests::accepted_service(root.path(),1000).unwrap();
let before=core.handle("status",json!({})).unwrap()["database"]["casting"].clone();
core.handle("cast.region.visible.save",json!({"enabled":true})).unwrap();
let mut enabled=core.handle("status",json!({})).unwrap()["database"]["casting"].clone();
assert_eq!(enabled["recordingRegionVisible"],true);enabled["recordingRegionVisible"]=json!(false);assert_eq!(enabled,before);
drop(core);
let mut restored=crate::service::Service::load(root.path(),1000).unwrap();
assert_eq!(restored.handle("status",json!({})).unwrap()["database"]["casting"]["recordingRegionVisible"],true);
assert!(!serde_json::from_value::<Settings>(json!({})).unwrap().recording_region_visible);
}
#[test]
fn recording_region_rejects_missing_invalid_or_stale_geometry() {
let view = json!({"projection":"factory-panel-v1","eye":0,"regionFullyVisible":true,"editorReferenceTan":1.7,"visibility":{"size":8,"extent":1.7,"cells":"f".repeat(16)},"tanHalfHorizontal":1.,"aspect":4./3.,"centerTanX":0.1,"centerTanY":-0.1,"captureToHead":[[1,0,0,-0.032],[0,1,0,0],[0,0,1,0]]});
assert!(Region::require_visible(&json!({"regionFullyVisible":false})).is_err());
assert!(Region::require_visible(&json!({})).is_err());
validate_description(&view).unwrap();
let mut region = Region::default();
let mut settings = Settings::default();
settings.recording_region_visible = true;
region
.refresh(&settings, Path::new("/missing"), Some(view))
.unwrap();
assert_eq!(region.snapshot(true)["visible"], true);
assert_eq!(region.editor().unwrap()["referenceTan"],1.7);
let mut smaller=region.description.clone().unwrap();smaller["tanHalfHorizontal"]=json!(0.1);
region.refresh(&settings,Path::new("/missing"),Some(smaller)).unwrap();
assert_eq!(region.editor().unwrap()["referenceTan"],1.7);
assert_eq!(region.snapshot(false)["visible"], false);
region.refresh(&settings,Path::new("/missing"),None).unwrap();
assert_eq!(region.snapshot(true)["visible"], false);
assert!(region
.refresh(&settings, Path::new("/missing"), Some(json!({})))
.is_err());
region.fail(&anyhow::anyhow!("invalid"));
assert_eq!(region.snapshot(true)["visible"], false);
settings.output = Output::Raw;
region
.refresh(&settings, Path::new("/missing"), None)
.unwrap();
assert_eq!(region.snapshot(true)["visible"], false);
}
}
+168
View File
@@ -0,0 +1,168 @@
use super::{
runtime::Casting,
settings::{Eye, Settings},
};
use serde_json::Value;
use std::{
fs,
path::{Path, PathBuf},
process::{Command, Stdio},
time::{Duration, Instant},
};
#[test]
#[ignore = "displays calibration markers on Steam Frame; requires explicit user approval"]
fn recording_region_device_alignment() {
let media = PathBuf::from(std::env::var("FRAMELY_REGION_TEST_MEDIA").unwrap());
let marker = std::env::var("FRAMELY_REGION_TEST_MARKER").unwrap();
let directory = tempfile::tempdir().unwrap();
for (eye, width, height, fov, cx, cy) in [
(Eye::Left, 640, 480, None, 0., 0.),
(Eye::Right, 640, 360, Some(55.), 8., -5.),
(Eye::Left, 360, 640, Some(45.), -6., 4.),
(Eye::Right, 1920, 1440, Some(70.), 0., 0.),
(Eye::Right, 2560, 1600, Some(76.47219658546469), 0., -1.504517427660815),
(Eye::Left, 2560, 1600, Some(76.47219658546469), 0., -1.504517427660815),
] {
let mut command = Command::new(media.join("framely-capture"));
command.args([
"--describe-view",
"--eye",
if eye == Eye::Left { "0" } else { "1" },
"--width",
&width.to_string(),
"--height",
&height.to_string(),
]);
command.args([format!("--center-x={cx}"), format!("--center-y={cy}")]);
if let Some(fov) = fov {
command.arg(format!("--fov={fov}"));
}
let output = command.output().unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let mut view: Value = serde_json::from_slice(&output.stdout).unwrap();
super::region::validate_description(&view).unwrap();
view["width"] = width.into();
view["height"] = height.into();
let geometry = directory.path().join("geometry.json");
let ready = directory.path().join("ready");
let stop = directory.path().join("stop");
let _ = fs::remove_file(&ready);
let _ = fs::remove_file(&stop);
fs::write(&geometry, view.to_string()).unwrap();
let mut overlay = Command::new(&marker)
.arg(&geometry)
.arg(&ready)
.arg(&stop)
.stdout(Stdio::null())
.spawn()
.unwrap();
let end = Instant::now() + Duration::from_secs(5);
while !ready.exists() {
assert!(overlay.try_wait().unwrap().is_none());
assert!(Instant::now() < end);
std::thread::sleep(Duration::from_millis(20));
}
let helper = crate::ipc::call(
Path::new("/run/framely/control.sock"),
"cast.panel.start",
serde_json::json!({}),
)
.unwrap();
let socket = PathBuf::from(helper["socket"].as_str().unwrap());
let mut cast = Casting::new(directory.path().join("capture"), media.clone());
let settings = Settings {
eye,
width,
height,
horizontal_fov: fov,
center_x: cx,
center_y: cy,
system_audio: false,
..Settings::default()
};
let result = (|| -> anyhow::Result<Vec<u8>> {
let status=cast.start(&settings, Some(&socket))?;
super::region::validate_description(&status["view"])?;
for key in ["tanHalfHorizontal","centerTanX","centerTanY","aspect"] {
anyhow::ensure!((status["view"][key].as_f64().unwrap()-view[key].as_f64().unwrap()).abs()<1e-10,"described and active capture differ: {key}");
}
let frame = Command::new("ffmpeg")
.args([
"-v",
"error",
"-rtsp_transport",
"tcp",
"-i",
&format!("rtsp://127.0.0.1:{}/headset", cast.test_rtsp_port()),
"-frames:v",
"1",
"-pix_fmt",
"rgb24",
"-f",
"rawvideo",
"pipe:1",
])
.output()?;
anyhow::ensure!(
frame.status.success(),
"{}",
String::from_utf8_lossy(&frame.stderr)
);
Ok(frame.stdout)
})();
cast.stop();
fs::write(&stop, b"stop").unwrap();
let _ = overlay.wait();
let rgb = result.unwrap();
assert_eq!(rgb.len(), width as usize * height as usize * 3);
let positions = [
(16, 16),
(width - 16, 16),
(16, height - 16),
(width - 16, height - 16),
(width / 2, height / 2),
];
for (index, (x, y)) in positions.into_iter().enumerate() {
let mut n = 0.;
let mut sx = 0.;
let mut sy = 0.;
for dy in -24..24 {
for dx in -24..24 {
let px = x as i32 + dx;
let py = y as i32 + dy;
if px < 0 || py < 0 || px >= width as i32 || py >= height as i32 { continue; }
let i = (py as usize * width as usize + px as usize) * 3;
let (r, g, b) = (rgb[i], rgb[i + 1], rgb[i + 2]);
let hit = match index {
0 => r > 160 && g < 100 && b < 100,
1 => g > 160 && r < 100 && b < 100,
2 => b > 160 && r < 100 && g < 100,
3 => r > 160 && b > 160 && g < 100,
_ => r > 200 && g > 200 && b > 200,
};
if hit {
n += 1.;
sx += px as f64;
sy += py as f64;
}
}
}
assert!(n > 400., "fiducial {index} missing for {eye:?}: {n} pixels");
let error = (sx / n - (x as f64 - 0.5))
.abs()
.max((sy / n - (y as f64 - 0.5)).abs());
println!(
"ALIGNMENT {eye:?} {width}x{height} marker={index} error={error:.2}px pixels={n}"
);
assert!(
error <= 3.,
"recording border does not match sampled image: {error:.2}px"
);
}
}
}
+139 -9
View File
@@ -15,6 +15,7 @@ pub struct Casting {
pub binaries: PathBuf,
workers: Vec<Child>,
started: Option<Instant>,
streaming: bool,
pub whep_port: u16,
rtsp_port: u16,
api_port: u16,
@@ -29,6 +30,7 @@ pub struct Casting {
recording: Option<super::recording::Recording>,
recording_error: Option<String>,
recording_path: Option<PathBuf>,
pub region: super::region::Region,
}
fn port() -> Result<u16> {
Ok(std::net::TcpListener::bind("127.0.0.1:0")?
@@ -42,6 +44,7 @@ impl Casting {
binaries,
workers: Vec::new(),
started: None,
streaming: false,
whep_port: 0,
rtsp_port: 0,
api_port: 0,
@@ -56,17 +59,31 @@ impl Casting {
recording: None,
recording_error: None,
recording_path: None,
region: Default::default(),
}
}
pub fn status(&mut self) -> Value {
self.poll_recording();
self.poll_workers();
let view = fs::File::open(self.directory.join("stream.log")).ok().and_then(|file| { let mut log = String::new(); file.take(65536).read_to_string(&mut log).ok()?; Some(log) }).and_then(|log| log.lines().filter_map(|line|serde_json::from_str::<Value>(line).ok()).find(|v|v["event"]=="view"));
json!({"running":self.started.is_some(),"error":self.error,"preview":self.preview_restore.is_some(),"view":view,
"watchPath":if self.started.is_some(){Some("/cast/watch")}else{None},
"elapsedSeconds":self.started.map(|t|t.elapsed().as_secs()),
json!({"running":self.streaming && self.started.is_some(),"error":self.error,"preview":self.preview_restore.is_some(),"view":view,
"watchPath":if self.streaming && self.started.is_some(){Some("/cast/watch")}else{None},
"elapsedSeconds":if self.streaming {self.started.map(|t|t.elapsed().as_secs())} else {None},
"codec":self.active_settings.as_ref().map(|s|s.codec.name()),"dominantEye":dominant_eye().ok(),"recording":self.recording_status()})
}
pub fn refresh_region(&mut self, configured: &Settings) -> Result<()> {
let mut effective=self.active_settings.clone().unwrap_or_else(||configured.clone());
effective.recording_region_visible=configured.recording_region_visible;
let active=if self.started.is_some() && super::region::supported(&effective) && (effective.recording_region_visible || self.region.editing) {
Some(self.status()["view"].clone())
} else {None};
let result=self.region.refresh(&effective,&self.binaries,active);
if let Err(error)=&result {self.region.fail(error);}
result
}
pub fn region_status(&self) -> Value {
self.region.snapshot((self.streaming && self.started.is_some()) || self.recording.is_some())
}
fn poll_workers(&mut self) {
let exited = self
.workers
@@ -88,18 +105,40 @@ impl Casting {
let _ = self.stop_recording();
}
}
pub fn start_recording(&mut self, output: &Path) -> Result<Value> {
pub fn validate_recording_settings(&self, settings: &Settings) -> Result<()> {
ensure!(self.recording.is_none() || self.active_settings.as_ref().is_some_and(|s|s.same_capture(settings)), "请先停止录制再调整取景");
Ok(())
}
pub fn record(&mut self, settings: &Settings, panel_socket: Option<&Path>, output: &Path) -> Result<Value> {
ensure!(!self.region.editing, "请先关闭画幅调整窗口");
self.status();
ensure!(self.preview_restore.is_none(), "取景预览期间不能录制");
ensure!(self.recording.is_none(), "录制已在运行");
let capture_started = self.started.is_none();
if capture_started {
self.start_capture(settings, panel_socket, false)?;
}
let result = self.start_recording(output);
if result.is_err() && capture_started { self.stop_workers(); }
result
}
fn start_recording(&mut self, output: &Path) -> Result<Value> {
self.status();
ensure!(self.started.is_some(), "请先开始串流");
ensure!(self.preview_restore.is_none(), "取景预览期间不能录制");
ensure!(self.recording.is_none(), "录制已在运行");
self.recording_error = None;
match super::recording::Recording::start(self.rtsp_port, &self.directory, output) {
match super::recording::Recording::start(self.rtsp_port, &self.directory, output, self.codec()) {
Ok(recording) => {self.recording = Some(recording); Ok(self.recording_status())},
Err(error) => {self.recording_error = Some(format!("{error:#}")); Err(error)}
}
}
pub fn stop_recording(&mut self) -> Result<Value> {
let result = self.finish_recording();
if !self.streaming {self.stop_workers();}
result
}
fn finish_recording(&mut self) -> Result<Value> {
if let Some(mut recording) = self.recording.take() {
match recording.finish() {
Ok(path) => self.recording_path = Some(path),
@@ -112,7 +151,7 @@ impl Casting {
if self.recording.as_ref().is_some_and(|r|r.id == id) {let _ = self.stop_recording();}
}
fn stop_workers(&mut self) {
let _ = self.stop_recording();
let _ = self.finish_recording();
for child in &mut self.workers {
if child.try_wait().ok().flatten().is_some() {
continue;
@@ -138,7 +177,9 @@ impl Casting {
self.dlna_started = false;
self.stop_vp8();
self.active_settings = None;
self.region.invalidate();
self.started = None;
self.streaming = false;
self.stream_key.clear();
}
/// Negotiate with browsers that omit H.264 (including the bundled CEF).
@@ -148,6 +189,7 @@ impl Casting {
#[cfg(test)]
pub(crate) fn test_whep_server(&mut self, port: u16) {
self.started = Some(Instant::now());
self.streaming = true;
self.active_settings = Some(Settings::default());
self.whep_port = port;
}
@@ -330,7 +372,25 @@ impl Casting {
self.error = None;
json!(true)
}
fn configure_streaming(&mut self, enabled: bool) -> Result<()> {
ureq::AgentBuilder::new().redirects(0).timeout(Duration::from_secs(3)).build()
.patch(&format!("http://127.0.0.1:{}/v3/config/global/patch", self.api_port))
.send_json(json!({"webrtc":enabled,"hls":enabled}))
.context("无法切换串流服务")?;
self.streaming = enabled;
if !enabled {self.stream_key.clear();self.stop_vp8();}
else {self.stream_key = crate::session::cast_random_key();}
Ok(())
}
pub fn stop_streaming(&mut self) -> Result<Value> {
self.status();
if self.recording.is_some() {
if self.streaming {self.configure_streaming(false)?;}
} else {self.stop();}
Ok(json!(true))
}
pub fn start_preview(&mut self, settings: &Settings, socket: &Path) -> Result<Value> {
ensure!(!self.region.editing, "请先关闭画幅调整窗口");
ensure!(self.recording.is_none(), "请先停止录制再调整取景");
ensure!(self.preview_restore.is_none(), "取景预览已在运行");
let restore = self.active_settings.clone();
@@ -372,12 +432,27 @@ impl Casting {
Ok(())
}
pub fn start(&mut self, settings: &Settings, panel_socket: Option<&Path>) -> Result<Value> {
ensure!(!self.region.editing, "请先关闭画幅调整窗口");
self.status();
ensure!(!self.streaming, "串流已在运行");
if self.started.is_some() {
// Recording already owns the encoder. Enabling viewing must not
// restart it or change the recording's resolution/bitrate.
ensure!(self.active_settings.as_ref().is_some_and(|s|s.same_capture(settings)), "请先停止录制再调整取景");
self.configure_streaming(true)?;
return Ok(self.status());
}
self.start_capture(settings, panel_socket, true)
}
fn start_capture(&mut self, settings: &Settings, panel_socket: Option<&Path>, streaming: bool) -> Result<Value> {
self.region.invalidate();
settings.validate()?;
ensure!(self.started.is_none(), "串流已在运行");
fs::create_dir_all(&self.directory)?;
fs::set_permissions(&self.directory, fs::Permissions::from_mode(0o700))?;
// A new session does not accumulate unbounded log history.
fs::write(self.directory.join("stream.log"), [])?;
self.streaming = streaming;
let result = self.start_inner(settings, panel_socket);
if let Err(e) = &result {
use std::io::{Read, Seek, SeekFrom};
@@ -407,8 +482,8 @@ impl Casting {
self.api_port = port()?;
self.whep_port = port()?;
self.hls_port = port()?;
self.stream_key = crate::session::cast_random_key();
let configuration = format!("logLevel: warn\nrtspAddress: 127.0.0.1:{rtsp}\nrtspTransports: [tcp]\nrtmp: no\nmoq: no\nsrt: no\napi: yes\napiAddress: 127.0.0.1:{}\nmetrics: no\nplayback: no\nwebrtc: yes\nwebrtcAddress: 127.0.0.1:{}\nwebrtcLocalUDPAddress: :0\nwebrtcLocalTCPAddress: :0\nhls: yes\nhlsAddress: 127.0.0.1:{}\nhlsVariant: mpegts\nhlsSegmentCount: 3\nhlsSegmentDuration: 1s\npaths:\n headset:\n source: publisher\n dlna:\n source: publisher\n headset_vp8:\n source: publisher\n", self.api_port, self.whep_port, self.hls_port);
self.stream_key = if self.streaming {crate::session::cast_random_key()} else {String::new()};
let configuration = format!("logLevel: warn\nrtspAddress: 127.0.0.1:{rtsp}\nrtspTransports: [tcp]\nrtmp: no\nmoq: no\nsrt: no\napi: yes\napiAddress: 127.0.0.1:{}\nmetrics: no\nplayback: no\nwebrtc: {}\nwebrtcAddress: 127.0.0.1:{}\nwebrtcLocalUDPAddress: :0\nwebrtcLocalTCPAddress: :0\nhls: {}\nhlsAddress: 127.0.0.1:{}\nhlsVariant: mpegts\nhlsSegmentCount: 3\nhlsSegmentDuration: 1s\npaths:\n headset:\n source: publisher\n dlna:\n source: publisher\n headset_vp8:\n source: publisher\n", self.api_port, self.streaming, self.whep_port, self.streaming, self.hls_port);
let config_path = self.directory.join("mediamtx.yml");
fs::write(&config_path, configuration)?;
self.spawn(Command::new(mediamtx).arg(config_path))?;
@@ -723,11 +798,66 @@ mod tests {
);
}
#[test]
fn cast_recording_only_stays_private_and_releases_capture_on_stop() {
let dir=tempfile::tempdir().unwrap();let output=dir.path().join("videos");
let mut cast=Casting::new(dir.path().into(),dir.path().into());
cast.started=Some(Instant::now());cast.active_settings=Some(Settings::default());
cast.recording=Some(super::super::recording::test_recording(dir.path(),&output,false));
let status=cast.status();
assert_eq!(status["running"],false);assert_eq!(status["recording"]["active"],true);
assert!(status["watchPath"].is_null());assert!(cast.stream_key.is_empty());
cast.stop_streaming().unwrap();assert!(cast.recording.is_some());
let id=cast.recording.as_ref().unwrap().id.clone();cast.stop_recording_id(&id);
assert!(cast.started.is_none());assert!(cast.active_settings.is_none());
assert_eq!(cast.recording_path.as_ref().unwrap().extension().unwrap(),"mp4");
}
#[test]
fn cast_recording_stream_toggle_keeps_encoder_and_recording() {
use tiny_http::{Response,Server};
let server=Server::http("127.0.0.1:0").unwrap();
let port=server.server_addr().to_ip().unwrap().port();
let worker=std::thread::spawn(move || {
for enabled in [true,false,true] {
let mut req=server.recv().unwrap();assert_eq!(req.method(),&tiny_http::Method::Patch);
assert_eq!(req.url(),"/v3/config/global/patch");
let value:Value=serde_json::from_reader(req.as_reader()).unwrap();
assert_eq!(value,json!({"webrtc":enabled,"hls":enabled}));
req.respond(Response::from_string("{}")).unwrap();
}
});
let dir=tempfile::tempdir().unwrap();let output=dir.path().join("videos");
let mut cast=Casting::new(dir.path().into(),dir.path().into());
let settings=Settings::default();cast.api_port=port;
cast.started=Some(Instant::now());cast.active_settings=Some(settings.clone());
cast.recording=Some(super::super::recording::test_recording(dir.path(),&output,false));
let started=cast.started;let id=cast.recording.as_ref().unwrap().id.clone();
assert_eq!(cast.start(&settings,None).unwrap()["running"],true);
cast.stop_streaming().unwrap();assert_eq!(cast.status()["running"],false);
assert_eq!(cast.recording.as_ref().unwrap().id,id);assert_eq!(cast.started,started);
assert_eq!(cast.start(&settings,None).unwrap()["running"],true);
cast.stop_recording().unwrap();assert!(cast.streaming);assert_eq!(cast.started,started);
worker.join().unwrap();cast.stop();
}
#[test]
fn cast_recording_capture_settings_are_shared_and_locked() {
let dir=tempfile::tempdir().unwrap();let output=dir.path().join("videos");
let mut cast=Casting::new(dir.path().into(),dir.path().into());
let mut settings=Settings::default();assert_eq!(settings.width*3,settings.height*4);
cast.started=Some(Instant::now());cast.active_settings=Some(settings.clone());
cast.recording=Some(super::super::recording::test_recording(dir.path(),&output,false));
settings.height=1080;assert!(cast.validate_recording_settings(&settings).is_err());
assert!(cast.start(&settings,None).is_err());
settings.height=1440;settings.bitrate_mbps+=1;assert!(cast.validate_recording_settings(&settings).is_err());
settings.bitrate_mbps-=1;settings.airplay=true;settings.receiver_name="Another name".into();
assert!(cast.validate_recording_settings(&settings).is_ok());
}
#[test]
fn cast_recording_stale_indicator_does_not_stop_new_recording_and_stream_stop_finishes() {
let dir=tempfile::tempdir().unwrap();let output=dir.path().join("videos");
let mut cast=Casting::new(dir.path().into(),dir.path().into());
assert!(cast.start_recording(&output).is_err());
cast.started=Some(Instant::now());
cast.streaming=true;
cast.recording=Some(super::super::recording::test_recording(dir.path(),&output,false));
assert!(cast.start_recording(&output).is_err());
assert!(cast.start_preview(&Settings::default(),dir.path()).is_err());
@@ -741,7 +871,7 @@ mod tests {
#[test]
fn cast_recording_process_failure_keeps_stream_and_reports_error() {
let dir=tempfile::tempdir().unwrap();let output=dir.path().join("videos");
let mut cast=Casting::new(dir.path().into(),dir.path().into());cast.started=Some(Instant::now());
let mut cast=Casting::new(dir.path().into(),dir.path().into());cast.started=Some(Instant::now());cast.streaming=true;
cast.recording=Some(super::super::recording::test_recording(dir.path(),&output,true));
let end=Instant::now()+Duration::from_secs(3);
while cast.recording.is_some()&&Instant::now()<end {cast.maintain();std::thread::sleep(Duration::from_millis(25));}
+11 -1
View File
@@ -56,6 +56,7 @@ pub struct Settings {
pub airplay: bool,
pub dlna: bool,
pub receiver_name: String,
pub recording_region_visible: bool,
}
impl Default for Settings {
fn default() -> Self {
@@ -65,7 +66,7 @@ impl Default for Settings {
output: Output::Eye,
eye: Eye::SteamVR,
width: 1920,
height: 1080,
height: 1440,
fps: 30,
bitrate_mbps: 8,
horizontal_fov: None,
@@ -76,10 +77,19 @@ impl Default for Settings {
airplay: false,
dlna: false,
receiver_name: "Framely".into(),
recording_region_visible: false,
}
}
}
impl Settings {
pub fn same_capture(&self, other: &Self) -> bool {
let mut capture = self.clone();
capture.airplay = other.airplay;
capture.dlna = other.dlna;
capture.receiver_name.clone_from(&other.receiver_name);
capture.recording_region_visible = other.recording_region_visible;
capture == *other
}
pub fn validate(&self) -> Result<()> {
ensure!(
(160..=4096).contains(&self.width)
+7
View File
@@ -1133,6 +1133,13 @@ impl Service {
self.save()?;
Ok(json!(true))
}
"cast.region.visible.save" => {
let enabled=p["enabled"].as_bool().context("Missing region visibility")?;
let previous=self.db.casting.recording_region_visible;
self.db.casting.recording_region_visible=enabled;
if let Err(error)=self.save() {self.db.casting.recording_region_visible=previous;return Err(error);}
Ok(json!(true))
}
"cast.panel.start" => {
let executable = self.root.join("current/bin/framely-panel-grab");
ensure!(executable.is_file(), "缺少内置屏幕采集助手");
+202 -21
View File
@@ -37,6 +37,7 @@ struct Agent {
files: Arc<crate::manager_files::Files>,
catalog_cache: Arc<Mutex<std::collections::BTreeMap<String, Value>>>,
casting: Mutex<crate::cast::runtime::Casting>,
recording_overlay_cache: Mutex<Value>,
watch_access: Mutex<crate::cast::watch_access::Access>,
receivers: Mutex<crate::cast::receiver::Receivers>,
}
@@ -408,9 +409,15 @@ impl Agent {
r.as_reader().take(32769).read_to_end(&mut bytes)?;
ensure!(bytes.len() <= 32768, "Visibility report too large");
let views: Value = serde_json::from_slice(&bytes)?;
if let Some(id) = views["stopRecording"].as_str() {
// Empty acknowledgements are sent on ordinary native polls.
// Do not wait for the casting lock while describe/start is
// running: the native HTTP timeout is intentionally short.
if let Some(id) = views["stopRecording"].as_str().filter(|id|!id.is_empty()) {
self.casting.lock().unwrap().stop_recording_id(id);
}
if let Some(id)=views["framingClosed"].as_str().filter(|id|!id.is_empty()) {
self.casting.lock().unwrap().region.acknowledge_editor_closed(id);
}
if views["views"].is_object() {
self.core("host.ui.visibility",views["views"].clone())?;
if let Some(ids)=views["closedCasts"].as_array(){ensure!(ids.len()<=32,"Too many closed casts");for id in ids{if let Some(id)=id.as_str(){let _=self.receivers.lock().unwrap().control(id,"stop",json!({}));}}}
@@ -440,7 +447,7 @@ impl Agent {
.unwrap_or(json!({}));
return send_json(
r,
json!({"feedback":database["feedback"],"commands":commands,"notificationBadge":badge,"gazeCalibration":self.core("launcher.gaze.calibration.get",json!({})).unwrap_or(Value::Null),"launcher":database["launcher"],"recording":self.casting.lock().unwrap().recording_status()}),
{let (recording,region,framing)=if let Ok(cast)=self.casting.try_lock() {let status=cast.recording_status();*self.recording_overlay_cache.lock().unwrap()=status.clone();(status,cast.region_status(),cast.region.editor_state())}else{(self.recording_overlay_cache.lock().unwrap().clone(),json!({"visible":false}),Value::Null)};json!({"feedback":database["feedback"],"commands":commands,"notificationBadge":badge,"gazeCalibration":self.core("launcher.gaze.calibration.get",json!({})).unwrap_or(Value::Null),"launcher":database["launcher"],"recording":recording,"recordingRegion":region,"framingEditor":framing})},
);
}
if let Some(rel) = path.strip_prefix("/plugin-assets/") {
@@ -941,6 +948,7 @@ impl Agent {
}
if path == "/"
|| path == "/cast/watch"
|| path == "/cast/framing"
|| path.starts_with("/cast/window/")
|| path == "/apk/permission"
|| path == "/manager"
@@ -1021,6 +1029,7 @@ impl Agent {
match method {
"cast.status" => {
let mut value = self.casting.lock().unwrap().status();
value["recordingRegion"]=self.casting.lock().unwrap().region_status();
let access = self.watch_access.lock().unwrap();
let watch_path = access.path();
value["watchAccessEnabled"] = json!(access.token.is_some());
@@ -1043,10 +1052,100 @@ impl Agent {
"cast.settings.save" => {
let settings: crate::cast::settings::Settings = serde_json::from_value(p.clone())?;
settings.validate()?;
let cast = self.casting.lock().unwrap();
ensure!(!cast.region.editing, "请先关闭画幅调整窗口");
cast.validate_recording_settings(&settings)?;
let result = self.core(method,p)?;
drop(cast);
self.receivers.lock().unwrap().configure(&settings)?;
Ok(result)
},
"cast.region.visible.save" => {
let enabled=p["enabled"].as_bool().context("Missing region visibility")?;
let mut settings:crate::cast::settings::Settings=serde_json::from_value(self.core("status",json!({}))?["database"]["casting"].clone())?;
ensure!(!enabled || crate::cast::region::supported(&settings),"录制区域显示仅支持屏幕采集的单眼画面");
let previous=settings.clone();
settings.recording_region_visible=enabled;
let mut cast=self.casting.lock().unwrap();
// Resolve geometry before persisting an enabled switch.
cast.refresh_region(&settings)?;
if let Err(error)=self.core("cast.region.visible.save",json!({"enabled":enabled})) {let _=cast.refresh_region(&previous);return Err(error);}
Ok(json!({"enabled":enabled}))
},
"cast.framing.open" => {
let settings:crate::cast::settings::Settings=serde_json::from_value(self.core("status",json!({}))?["database"]["casting"].clone())?;
ensure!(crate::cast::region::supported(&settings),"画幅调整仅支持屏幕采集的单眼画面");
let mut cast=self.casting.lock().unwrap();
ensure!(cast.recording_status()["active"]!=true,"请先停止录制再调整取景");
ensure!(!cast.region.editing && cast.status()["preview"]!=true,"请先关闭画幅调整窗口");
cast.region.editing=true;
cast.region.editor_id=hex::encode(rand::random::<[u8;16]>());
if let Err(error)=cast.refresh_region(&settings) {cast.region.editing=false;return Err(error);}
let geometry=match cast.region.editor() {
Ok(geometry)=>geometry,
Err(error)=>{cast.region.editing=false;return Err(error);}
};
self.commands.lock().unwrap().push_back(json!({"kind":"cast.framing.open","geometry":geometry}));
Ok(geometry)
},
"cast.framing.get" => {
let cast=self.casting.lock().unwrap();ensure!(cast.region.editing,"画幅调整窗口已关闭");cast.region.editor()
},
"cast.framing.reference" => {
let cast=self.casting.lock().unwrap();
ensure!(cast.region.editing && p["id"]==cast.region.editor_id,"画幅调整窗口已关闭");
let cx=p["centerX"].as_f64().context("Invalid reference center")?;
let cy=p["centerY"].as_f64().context("Invalid reference center")?;
ensure!([cx,cy].into_iter().all(|v|v.is_finite()&&(-60.0..=60.0).contains(&v)),"Invalid reference center");
// Presentation-only depth probe; never persist a draft crop.
self.commands.lock().unwrap().push_back(json!({"kind":"cast.framing.reference","id":cast.region.editor_id,"centerTanX":cx.to_radians().tan(),"centerTanY":cy.to_radians().tan()}));
Ok(json!(true))
},
"cast.framing.cancel" => {
let mut cast=self.casting.lock().unwrap();
ensure!(p["id"]==cast.region.editor_id,"画幅调整窗口已关闭");
cast.region.editing=false;
self.commands.lock().unwrap().push_back(json!({"kind":"cast.framing.close","id":cast.region.editor_id}));Ok(json!(true))
},
"cast.framing.apply" => {
let mut settings:crate::cast::settings::Settings=serde_json::from_value(self.core("status",json!({}))?["database"]["casting"].clone())?;
let mut cast=self.casting.lock().unwrap();ensure!(cast.region.editing,"画幅调整窗口已关闭");
ensure!(p["id"]==cast.region.editor_id,"画幅调整窗口已关闭");
ensure!(cast.recording_status()["active"]!=true,"请先停止录制再调整取景");
settings.width=p["width"].as_u64().and_then(|v|u32::try_from(v).ok()).context("Invalid width")?;
settings.height=p["height"].as_u64().and_then(|v|u32::try_from(v).ok()).context("Invalid height")?;
settings.horizontal_fov=Some(p["horizontalFov"].as_f64().context("Invalid FOV")?);
settings.center_x=p["centerX"].as_f64().context("Invalid center")?;
settings.center_y=p["centerY"].as_f64().context("Invalid center")?;
settings.validate()?;
let mut candidate=crate::cast::region::Region::default();
candidate.editing=true;
candidate.refresh(&settings,&cast.binaries,None)?;
ensure!(p["expectedEye"]==candidate.editor()?["view"]["eye"],"录制眼已改变,请重新打开画幅调整");
crate::cast::region::Region::require_visible(&candidate.editor()?["view"])?;
// Opening the editor does not pause streaming. Check the live
// state now rather than replaying a stale state from opening.
let running=cast.status()["running"]==true;
self.core("cast.settings.save",serde_json::to_value(&settings)?)?;
cast.region.editing=false;
// Close before starting workers, which can take several polls.
// The persistent framingEditor state also retries lost closes.
self.commands.lock().unwrap().push_back(json!({"kind":"cast.framing.close","id":cast.region.editor_id}));
if running {
cast.stop();
let restart=(|| -> Result<()> {
let panel=self.core("cast.panel.start",json!({}))?;
let socket=PathBuf::from(panel["socket"].as_str().context("Missing panel socket")?);
cast.start(&settings,Some(&socket))?;
Ok(())
})();
restart.context("取景已保存,但重新启动串流失败")?;
}
// Saved settings and a closed editor are committed even if the
// optional passive guide cannot refresh; never revive a stale UI.
cast.refresh_region(&settings).context("取景已保存,但录制区域显示刷新失败")?;
Ok(json!(true))
},
"cast.devices" => self.receivers.lock().unwrap().devices(),
"cast.send" => {
let network=self.network_status.lock().unwrap().clone();ensure!(network["enabled"]==true,"请先开启网络管理面板,以便设备获取投屏画面");
@@ -1100,22 +1199,16 @@ impl Agent {
} else { None };
self.casting.lock().unwrap().start(&settings, panel.as_deref())
}
"cast.stop" => Ok(self.casting.lock().unwrap().stop()),
"cast.stop" => self.casting.lock().unwrap().stop_streaming(),
"cast.record.start" => {
let mut cast = self.casting.lock().unwrap();
let auto_started = cast.status()["running"] != true;
if auto_started {
let status = self.core("status", json!({}))?;
let settings: crate::cast::settings::Settings = serde_json::from_value(status["database"]["casting"].clone())?;
let panel = if settings.source == crate::cast::settings::Source::Screen {
let response = self.core("cast.panel.start", json!({}))?;
Some(PathBuf::from(response["socket"].as_str().context("Missing panel socket")?))
} else {None};
cast.start(&settings, panel.as_deref())?;
}
let result = cast.start_recording(&crate::steam::home()?.join("Videos/framely"));
if result.is_err() && auto_started {cast.stop();}
result
let status = self.core("status", json!({}))?;
let settings: crate::cast::settings::Settings = serde_json::from_value(status["database"]["casting"].clone())?;
let panel = if settings.source == crate::cast::settings::Source::Screen {
let response = self.core("cast.panel.start", json!({}))?;
Some(PathBuf::from(response["socket"].as_str().context("Missing panel socket")?))
} else {None};
cast.record(&settings, panel.as_deref(), &crate::steam::home()?.join("Videos/framely"))
},
"cast.record.stop" => self.casting.lock().unwrap().stop_recording(),
"launcher.search.index" => {
@@ -1547,7 +1640,7 @@ impl Agent {
view.len() <= 160
&& (matches!(
view,
"menu" | "launcher" | "framely.manager" | "notifications"
"menu" | "launcher" | "framely.manager" | "notifications" | "framely.framing"
) || view.starts_with("framely.window.") || view.starts_with("framely.cast.")),
"Invalid haptic view"
);
@@ -1848,6 +1941,9 @@ fn fetch_catalogs_progress(
fn send_json(r: Request, v: Value) -> Result<()> {
r.respond(
Response::from_string(serde_json::to_string(&v)?)
// The native poller reads a length-delimited JSON body, not HTTP
// chunks. Geometry metadata can exceed tiny_http's 32 KiB default.
.with_chunked_threshold(usize::MAX)
.with_header(header("Content-Type", "application/json"))
.with_header(header("Cache-Control", "no-store")),
)?;
@@ -2008,6 +2104,7 @@ pub fn serve(
..Default::default()
}),
casting: Mutex::new(crate::cast::runtime::Casting::new(runtime.join("cast"), crate::cast::runtime::binary_directory(&assets))),
recording_overlay_cache: Mutex::new(json!({"active":false})),
receivers: Mutex::new(crate::cast::receiver::Receivers::new(runtime.join("receive"), crate::cast::runtime::binary_directory(&assets))),
socket,
state,
@@ -2029,7 +2126,7 @@ pub fn serve(
let cast_agent=agent.clone();
std::thread::spawn(move || { let mut configured=std::time::Instant::now()-Duration::from_secs(10);loop {
if configured.elapsed()>Duration::from_secs(2) {
if let Ok(status)=cast_agent.core("status",json!({})) {if status["agreement"]["accepted"]==true {if let Ok(settings)=serde_json::from_value(status["database"]["casting"].clone()){let _=cast_agent.receivers.lock().unwrap().configure(&settings);}}}
if let Ok(status)=cast_agent.core("status",json!({})) {if status["agreement"]["accepted"]==true {if let Ok(settings)=serde_json::from_value(status["database"]["casting"].clone()){let _=cast_agent.casting.lock().unwrap().refresh_region(&settings);let _=cast_agent.receivers.lock().unwrap().configure(&settings);}}}
configured=std::time::Instant::now();
}
cast_agent.maintain_cast();std::thread::sleep(Duration::from_millis(100));
@@ -2542,6 +2639,7 @@ mod tests {
Arc::new(Agent {
watch_access: Mutex::default(),
casting: Mutex::new(crate::cast::runtime::Casting::new(root.join("cast"), root.join("media"))),
recording_overlay_cache: Mutex::new(json!({"active":false})),
receivers: Mutex::new(crate::cast::receiver::Receivers::new(root.join("receive"), root.join("media"))),
socket: root.join("control.sock"),
state: root.join("state"),
@@ -2577,6 +2675,62 @@ mod tests {
})
}
#[test]
fn framing_apply_restarts_live_stream_and_closes_even_when_restart_fails() {
use std::os::unix::fs::PermissionsExt;
for running in [false,true] {
let root=tempfile::tempdir().unwrap();
let server=Server::http("127.0.0.1:0").unwrap();
let agent=agent(&server,root.path());
let mut core=crate::tests::accepted_service(&agent.state,1000).unwrap();
let mut settings=crate::cast::settings::Settings::default();
settings.eye=crate::cast::settings::Eye::Left;
core.handle("cast.settings.save",serde_json::to_value(&settings).unwrap()).unwrap();
fs::create_dir(root.path().join("media")).unwrap();
let view=json!({"projection":"factory-panel-v1","eye":0,"regionFullyVisible":true,"editorReferenceTan":1.7,"visibility":{"size":8,"extent":1.7,"cells":"f".repeat(16)},"tanHalfHorizontal":0.5,"aspect":4./3.,"centerTanX":0.,"centerTanY":0.,"captureToHead":[[1,0,0,0],[0,1,0,0],[0,0,1,0]]});
let capture=root.path().join("media/framely-capture");
fs::write(&capture,format!("#!/bin/sh\nprintf '%s' '{}'\n",view)).unwrap();
fs::set_permissions(&capture,fs::Permissions::from_mode(0o700)).unwrap();
{
let mut cast=agent.casting.lock().unwrap();
if running {cast.test_whep_server(1);}
cast.region.editor_id="apply-test".into();cast.region.editing=true;
}
let listener=UnixListener::bind(&agent.socket).unwrap();
listener.set_nonblocking(true).unwrap();
let stop=Arc::new(AtomicBool::new(false));let core_stop=stop.clone();
let panel_calls=Arc::new(std::sync::atomic::AtomicUsize::new(0));let calls=panel_calls.clone();
let worker=std::thread::spawn(move || {
while !core_stop.load(Ordering::Relaxed) {
if let Ok((mut stream,_))=listener.accept() {
let request=ipc::read(&mut stream).unwrap();
let method=request["method"].as_str().unwrap();
let result=if method=="cast.panel.start" {
calls.fetch_add(1,Ordering::Relaxed);
Err(anyhow::anyhow!("test restart failure"))
} else {core.handle(method,request["params"].clone())};
ipc::write(&mut stream,&ipc::response(result)).unwrap();
} else {std::thread::sleep(Duration::from_millis(5));}
}
Ok::<_,anyhow::Error>(())
});
assert!(agent.api(json!({"method":"cast.framing.reference","params":{"id":"old-test","centerX":0.,"centerY":0.}})).is_err());
assert!(agent.api(json!({"method":"cast.framing.reference","params":{"id":"apply-test","centerX":61.,"centerY":0.}})).is_err());
agent.api(json!({"method":"cast.framing.reference","params":{"id":"apply-test","centerX":4.5,"centerY":-3.}})).unwrap();
let reference=agent.commands.lock().unwrap().back().unwrap().clone();
assert_eq!(reference["kind"],"cast.framing.reference");
assert!((reference["centerTanX"].as_f64().unwrap()-4.5_f64.to_radians().tan()).abs()<1e-12);
assert_eq!(agent.core("status",json!({})).unwrap()["database"]["casting"]["centerX"],0.);
let result=agent.api(json!({"method":"cast.framing.apply","params":{"id":"apply-test","expectedEye":0,"width":1280,"height":960,"horizontalFov":60.,"centerX":0.,"centerY":0.}}));
assert_eq!(result.is_err(),running);
if let Err(error)=result {assert!(format!("{error:#}").contains("取景已保存,但重新启动串流失败"));}
assert_eq!(panel_calls.load(Ordering::Relaxed),usize::from(running));
assert_eq!(agent.core("status",json!({})).unwrap()["database"]["casting"]["width"],1280);
assert_eq!(agent.casting.lock().unwrap().region.editor_state(),json!({"id":"apply-test","open":false}));
assert_eq!(agent.commands.lock().unwrap().back().unwrap(),&json!({"kind":"cast.framing.close","id":"apply-test"}));
stop.store(true,Ordering::Relaxed);worker.join().unwrap().unwrap();
}
}
#[test]
fn calibration_preview_allows_unvalidated_error_but_rejects_invalid_geometry() {
let root = tempfile::tempdir().unwrap();
let server = Server::http("127.0.0.1:0").unwrap();
@@ -3365,10 +3519,22 @@ mod tests {
let root = tempfile::tempdir().unwrap();
let server = Server::http("127.0.0.1:0").unwrap();
let agent = agent(&server, root.path());
// The native host parses the raw HTTP body as JSON. Exercise a
// framing command larger than tiny_http's default chunk threshold.
let framing=json!({"kind":"cast.framing.open","geometry":{"visibility":"f".repeat(48000)}});
agent.commands.lock().unwrap().push_back(framing.clone());
agent.casting.lock().unwrap().region.editor_id="closed-test".into();
let listener = std::os::unix::net::UnixListener::bind(&agent.socket).unwrap();
let socket_worker = std::thread::spawn(move || {
let mut core = crate::service::Service::load(&root.path().join("state"), 1000).unwrap();
for expected in [
"host.ui.visibility",
"host.notification.badge",
"status",
"launcher.gaze.calibration.get",
"host.notification.badge",
"status",
"launcher.gaze.calibration.get",
"host.ui.visibility",
"host.notification.badge",
"status",
@@ -3400,15 +3566,30 @@ mod tests {
.set("Cookie", &format!("framely={}", agent.web_key))
.send_json(json!({"menu":true}))
.is_err());
let poll: Value = ureq::post(&url)
let response = ureq::post(&url)
.set("X-Framely-Native", &agent.native_key)
.send_json(json!({"menu":true}))
.unwrap()
.into_json()
.unwrap();
assert!(response.header("Transfer-Encoding").is_none());
assert!(response.header("Content-Length").unwrap().parse::<usize>().unwrap()>48000);
let poll:Value=response.into_json().unwrap();
assert_eq!(poll["commands"],json!([framing]));
assert_eq!(poll["feedback"], json!({"haptics":true,"sounds":false}));
assert!(poll.get("gazeCalibration").is_some());
assert!(poll["launcher"].is_object());
assert_eq!(poll["framingEditor"],json!({"id":"closed-test","open":false}));
let retry:Value=ureq::get(&url).set("X-Framely-Native",&agent.native_key).call().unwrap().into_json().unwrap();
assert_eq!(retry["commands"],json!([]));
assert_eq!(retry["framingEditor"],poll["framingEditor"]);
// The real native host includes empty acknowledgement fields on every
// poll. An applying editor may own this lock for several seconds.
let cast=agent.casting.lock().unwrap();
let busy:Value=ureq::post(&url).timeout(Duration::from_secs(1))
.set("X-Framely-Native",&agent.native_key)
.send_json(json!({"views":{"menu":true},"stopRecording":"","framingClosed":""}))
.unwrap().into_json().unwrap();
assert!(busy["framingEditor"].is_null());
drop(cast);
assert!(agent
.api(json!({"method":"host.ui.visibility","params":{"menu":false}}))
.is_err());
+26
View File
@@ -0,0 +1,26 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
const listeners=new Map(),windowListeners=new Map();
Object.defineProperty(globalThis,'navigator',{value:{languages:['zh-CN']},configurable:true});
globalThis.document={documentElement:{lang:''},addEventListener(name,fn){listeners.set(name,fn);}};
globalThis.window={addEventListener(name,fn){windowListeners.set(name,fn);}};
class Element {
constructor(attributes={},parentElement=null){this.attributes=attributes;this.parentElement=parentElement;}
closest(selector){for(let node=this;node;node=node.parentElement){const a=node.attributes;if(selector.startsWith('[inert]')){if(a.inert||a.disabled||a['aria-disabled']==='true')return node;}else if(a['data-framely-interactive']!==undefined||a.tabindex!==undefined)return node;}return null;}
}
globalThis.Element=Element;globalThis.getComputedStyle=()=>({cursor:'default'});
const result=await build({stdin:{contents:`import React from 'react';import {renderToStaticMarkup} from 'react-dom/server';import {FramingEditor} from './ui/src/cast-framing-editor';export {installHoverFeedback} from './sdk/src/hover';export const render=()=>renderToStaticMarkup(<FramingEditor spatial settings={{width:1200,height:900,horizontalFov:60,centerX:0,centerY:0}} view={{tanHalfHorizontal:1.4,centerTanX:0,centerTanY:0,aspect:1}} onChange={()=>{}} onError={()=>{}}/>);`,loader:'tsx',resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',write:false});
const module={exports:{}};new Function('module','exports','require',result.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const markup=module.exports.render();
assert.equal((markup.match(/data-framely-interactive=""/g)||[]).length,4,'all four corner markers use the shared haptic hook');
const pulses=[];module.exports.installHoverFeedback(kind=>pulses.push(kind));
const frame=new Element({tabindex:0}),nw=new Element({'data-framely-interactive':''},frame),ne=new Element({'data-framely-interactive':''},frame),child=new Element({},nw);
const over=target=>listeners.get('pointerover')({target});
const out=(target,relatedTarget)=>listeners.get('pointerout')({target,relatedTarget});
over(frame);out(frame,nw);over(nw);out(nw,child);over(child);out(child,ne);over(ne);
assert.deepEqual(pulses,['hover','hover','hover'],'entering each corner pulses once; children do not repeat');
frame.attributes.inert=true;out(ne,nw);over(nw);listeners.get('click')({target:nw});assert.equal(pulses.length,3,'pending/closed controls cannot pulse');
frame.attributes.inert=false;windowListeners.get('blur')();over(nw);listeners.get('click')({target:nw});assert.deepEqual(pulses.slice(-2),['hover','activate']);
console.log('PASS: four framing corners use existing preference-aware feedback, one hover pulse per corner, inert editors have no feedback');
+28
View File
@@ -0,0 +1,28 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
const result=await build({stdin:{contents:`export * from './ui/src/cast-framing-gesture';export {normalizeRegion} from './ui/src/cast-region';export {frameFromSettings} from './ui/src/cast-framing';`,resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',write:false});
const module={exports:{}};new Function('module','exports','require',result.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const {FrameGesture,fitMinimumFrame,minimumFrameWidth,normalizeRegion,frameFromSettings}=module.exports;
const close=(a,b)=>assert.ok(Math.abs(a-b)<1e-10,`${a} != ${b}`);
for(const ratio of [4/3,3/4,16/9,9/16,1]){
const minimum=minimumFrameWidth(ratio,{width:.24,height:.28},.05);
const rect=fitMinimumFrame({x:.3,y:.3,width:.4,height:.4/ratio},ratio,minimum),cx=rect.x+rect.width/2,cy=rect.y+rect.height/2;
for(const corner of ['nw','ne','sw','se']){
const sx=corner.includes('w')?-1:1,sy=corner.includes('n')?-1:1,gesture=new FrameGesture();
gesture.begin(1,500,500,rect,corner);
const smaller=gesture.move(1,500-sx*40,500-sy*40,1,{width:1000,height:1000},ratio,minimum);
const larger=gesture.move(1,500+sx*40,500+sy*40,1,{width:1000,height:1000},ratio,minimum);
assert.ok(smaller.width<=rect.width&&larger.width>=rect.width);
for(const next of [smaller,larger]){close(next.x+next.width/2,cx);close(next.y+next.height/2,cy);assert.ok(next.width>=.24-1e-10&&next.height>=.28-1e-10);close(next.width/next.height,ratio);}
assert.equal(gesture.move(1,800,800,0,{width:1000,height:1000},ratio,minimum),null);
assert.equal(gesture.active(),false); // lost pointerup cannot leave a drag armed
assert.equal(gesture.move(1,300,300,1,{width:1000,height:1000},ratio,minimum),null);
gesture.begin(2,500,500,rect,corner);assert.equal(gesture.finish(1),false);assert.equal(gesture.finish(2),true);
assert.equal(gesture.move(2,300,300,0,{width:1000,height:1000},ratio,minimum),null);
}
const reference={tanHalfHorizontal:1.4,centerTanX:0,centerTanY:0,aspect:1};
const saved=normalizeRegion({width:Math.round(1600*ratio),height:1600,horizontalFov:10,centerX:0,centerY:0},reference,{width:.24,height:.28});
const displayed=frameFromSettings(saved,saved.width/saved.height,reference);assert.ok(displayed.width>=.24-1e-10&&displayed.height>=.28-1e-10);
}
console.log('PASS: all four corners grow/shrink around center, menu minimum survives save/ratio changes, release/cancel/hover cannot continue dragging');
+42
View File
@@ -0,0 +1,42 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
import {readFile} from 'node:fs/promises';
const bundle=await build({stdin:{contents:`export * from './ui/src/cast-framing-limits';export {normalizeRegion} from './ui/src/cast-region';export {frameFromSettings,settingsFromFrame} from './ui/src/cast-framing';`,resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',write:false});
const module={exports:{}};new Function('module','exports','require',bundle.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const {FramingLimits,framingMenuScale,normalizeRegion,frameFromSettings,settingsFromFrame}=module.exports;
const close=(a,b)=>assert.ok(Math.abs(a-b)<1e-8,`${a} != ${b}`);
for(const extent of [.9,1.695,1.734])close(framingMenuScale(extent)*extent,.9);
for(const eye of [0,1]){
const size=64,bits=[];for(let y=0;y<size;y++)for(let x=0;x<size;x++){
const u=(x+.5)/size,v=(y+.5)/size,lower=.1+Math.abs(v-.5)*.2,upper=.9-Math.abs(v-.5)*.5;
bits.push(v>.12&&v<.88&&(eye===0?u>lower&&u<upper:1-u>lower&&1-u<upper));
}
let cells='';for(let i=0;i<bits.length;i+=4)cells+=(+bits[i]+(+bits[i+1]<<1)+(+bits[i+2]<<2)+(+bits[i+3]<<3)).toString(16);
const limit=new FramingLimits({size,extent:1.7,cells},1.7);
for(const ratio of [4/3,3/4,16/9,9/16,1]){
const min=Math.max(.10,.12*ratio),reset=limit.reset(ratio,min);
assert.ok(limit.contains(reset));close(reset.width/reset.height,ratio);
const grown=limit.fit({...reset,x:reset.x-.2,y:reset.y-.2/ratio,width:reset.width+.4,height:reset.height+.4/ratio},ratio,min,reset);
assert.ok(limit.contains(grown));close(grown.x+grown.width/2,reset.x+reset.width/2);close(grown.y+grown.height/2,reset.y+reset.height/2);
const moved=limit.fit({...reset,x:-2},ratio,min,reset);assert.ok(limit.contains(moved));close(moved.width,reset.width);
const smaller=limit.fit({...reset,x:reset.x+reset.width/4,y:reset.y+reset.height/4,width:reset.width/2,height:reset.height/2},ratio,min,reset);
assert.ok(limit.contains(smaller));assert.ok(smaller.width>=min-1e-10);
const reference={tanHalfHorizontal:1.7,centerTanX:0,centerTanY:0,aspect:1};
const sizeSettings={width:Math.round(ratio*1600),height:1600,...settingsFromFrame(grown,reference)};
const saved=normalizeRegion(sizeSettings,reference,{width:.1,height:.12},limit.fit.bind(limit));
const displayed=frameFromSettings(saved,saved.width/saved.height,reference);assert.ok(limit.contains(displayed));
}
assert.throws(()=>limit.reset(1,.95));
}
if(process.env.FRAMELY_VIEW_FIXTURE){
const views=JSON.parse(await readFile(process.env.FRAMELY_VIEW_FIXTURE,'utf8'));
const checks=[];
for(const view of views){const limit=new FramingLimits(view.visibility,view.editorReferenceTan),reference={tanHalfHorizontal:view.editorReferenceTan,centerTanX:0,centerTanY:0,aspect:1};
for(const [width,height] of [[2560,1920],[1920,2560],[2560,1440],[1440,2560],[1920,1920]]){
const ratio=width/height,rect=limit.reset(ratio,Math.max(.1,.1*ratio));assert.ok(limit.contains(rect));
checks.push({eye:view.eye,width,height,...settingsFromFrame(rect,reference)});
}
}
console.log(JSON.stringify(checks));
}else console.log('PASS: menu angular size fixed; both-eye masks clamp growth/movement/reset and saved crops stay inside actual visibility');
+2 -2
View File
@@ -7,8 +7,8 @@ globalThis.document={documentElement:{lang:''}};
const bundle=await build({stdin:{contents:`import React from 'react';import {renderToStaticMarkup} from 'react-dom/server';import {RecordingButton} from './ui/src/cast-recording';export const render=(recording,disabled=false)=>renderToStaticMarkup(<RecordingButton recording={recording} disabled={disabled} onClick={()=>{}}/>);`,resolveDir:process.cwd(),loader:'tsx'},bundle:true,platform:'node',format:'cjs',write:false});
const module={exports:{}};new Function('module','exports','require',bundle.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const idle=module.exports.render({active:false});assert.ok(idle.includes('开始录制'));assert.ok(!idle.includes('停止录制'));
const active=module.exports.render({active:true,elapsedSeconds:125,path:'/home/steamos/Videos/framely/test.mkv'});
assert.ok(active.includes('停止录制'));assert.ok(active.includes('02:05'));assert.ok(active.includes('test.mkv'));assert.ok(active.includes(' recording'));
const active=module.exports.render({active:true,elapsedSeconds:125,path:'/home/steamos/Videos/framely/test.mp4'});
assert.ok(active.includes('停止录制'));assert.ok(active.includes('02:05'));assert.ok(active.includes('test.mp4'));assert.ok(active.includes(' recording'));
assert.ok(module.exports.render({active:true},true).includes('disabled=""'));
for(const locale of ['en-US','zh-CN']){
const {messages}=JSON.parse(await readFile(`ui/locales/${locale}.json`,'utf8'));
+17
View File
@@ -0,0 +1,17 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
const bundle=await build({stdin:{contents:`export {normalizeRegion} from './ui/src/cast-region';export {frameFromSettings,settingsFromFrame,framePresets,frameRatios} from './ui/src/cast-framing';`,resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',write:false});
const module={exports:{}};new Function('module','exports','require',bundle.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const {normalizeRegion,frameFromSettings,settingsFromFrame,framePresets}=module.exports;
assert.ok(module.exports.frameRatios.some(r=>r.label==='3:4'&&r.value===3/4));
const reference={tanHalfHorizontal:1.4,centerTanX:0,centerTanY:0,aspect:1};
for(const aspect of [4/3,3/4,16/9,9/16,1])for(const size of framePresets(aspect)){
const input={...size,horizontalFov:72,centerX:12,centerY:-8};
const saved=normalizeRegion(input,reference),rect=frameFromSettings(saved,saved.width/saved.height,reference),effective=settingsFromFrame(rect,reference);
for(const key of ['horizontalFov','centerX','centerY'])assert.ok(Math.abs(saved[key]-effective[key])<1e-9,`${key}: displayed box differs from saved capture`);
assert.ok(Math.abs(rect.width/rect.height-saved.width/saved.height)<1e-10);
}
const fitted=normalizeRegion({width:1080,height:1920,horizontalFov:150,centerX:60,centerY:60},reference);
assert.ok(fitted.horizontalFov<150);assert.ok(fitted.centerX<=60&&fitted.centerY<=60);
console.log('PASS: frame ratio/resolution changes save exactly the displayed crop');
+73
View File
@@ -0,0 +1,73 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
import {resolve} from 'node:path';
Object.defineProperty(globalThis,'navigator',{value:{languages:['zh-CN']},configurable:true});
globalThis.document={documentElement:{lang:'',classList:{add(){},remove(){}}}};
globalThis.window={addEventListener(){},removeEventListener(){}};
const reactPath=resolve('node_modules/react/index.js');
// A small hook runner exercises this page's async lifecycle without mounting
// video/CEF or adding a DOM-test dependency. Children remain React elements.
const hooks=`
import React from ${JSON.stringify(reactPath)};
export * from ${JSON.stringify(reactPath)};
export default React;
let values=[],cursor=0,effects=[];
const same=(a,b)=>a&&b&&a.length===b.length&&a.every((v,i)=>Object.is(v,b[i]));
export function useState(initial){const i=cursor++;if(!(i in values))values[i]=typeof initial==='function'?initial():initial;return [values[i],value=>{values[i]=typeof value==='function'?value(values[i]):value;}];}
export function useRef(initial){const i=cursor++;return values[i]??(values[i]={current:initial});}
export function useMemo(fn,deps){const i=cursor++;if(!same(values[i]?.deps,deps))values[i]={deps,value:fn()};return values[i].value;}
export function useEffect(fn,deps){const i=cursor++;if(!same(values[i]?.deps,deps)){const old=values[i];values[i]={deps};effects.push(()=>{old?.cleanup?.();values[i].cleanup=fn();});}}
export const useLayoutEffect=useEffect;
export function render(Component){cursor=0;return Component();}
export function flush(){const pending=effects;effects=[];for(const effect of pending)effect();}
export function reset(){for(const value of values)value?.cleanup?.();values=[];cursor=0;effects=[];}
`;
const api=`let handler;export const calls=[];export function setHandler(next){handler=next;calls.length=0;}export function api(method,params,signal){calls.push({method,params,signal});return Promise.resolve().then(()=>handler(method,params));}`;
const result=await build({stdin:{contents:`export {SpatialFraming} from './ui/src/cast-spatial-framing';export {render,flush,reset} from 'react';export {calls,setHandler} from './ui/src/api';`,resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',loader:{'.css':'empty'},write:false,plugins:[{name:'page-harness',setup(build){build.onResolve({filter:/^react$/},()=>({path:'hooks',namespace:'test'}));build.onLoad({filter:/^hooks$/,namespace:'test'},()=>({contents:hooks,loader:'js',resolveDir:process.cwd()}));build.onLoad({filter:/\/ui\/src\/api\.ts$/},()=>({contents:api,loader:'js'}));}}]});
const module={exports:{}};new Function('module','exports','require',result.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const {SpatialFraming,render,flush,reset,calls,setHandler}=module.exports;
const geometry={id:'editor-1',referenceTan:1.4,settings:{width:1200,height:900,horizontalFov:60,centerX:0,centerY:0},view:{tanHalfHorizontal:.7,centerTanX:0,centerTanY:0,eye:1,visibility:{size:32,extent:1.4,cells:'f'.repeat(256)}}};
const tick=()=>new Promise(resolve=>setImmediate(resolve));
function walk(node,visit){if(!node||typeof node!=='object')return;visit(node);for(const child of [node.props?.children].flat(Infinity))walk(child,visit);}
function draw(){const tree=render(SpatialFraming);walk(tree,node=>{if(node.props?.ref){const panel=node.type==='section';node.props.ref.current={getBoundingClientRect:()=>({width:panel?340:1600,height:panel?300:1600})};}});flush();return tree;}
function button(tree,label){let found;walk(tree,node=>{if(node.type==='button'&&node.props.children===label)found=node;});assert.ok(found,label);return found;}
async function mount(finish){reset();setHandler((method,params)=>method==='status'?{database:{language:'zh-CN'}}:method==='language.list'?{packs:[]}:method==='cast.framing.get'?geometry:method==='cast.framing.reference'?true:finish(method,params));for(let i=0;i<4;i++){draw();await tick();}return draw();}
let resolveApply;
let tree=await mount(method=>{assert.equal(method,'cast.framing.apply');return new Promise(resolve=>{resolveApply=resolve;});});
const apply=button(tree,'应用取景').props.onClick;
apply();apply();await tick();tree=draw();
assert.equal(tree.props.inert,true);
assert.equal(calls.filter(call=>call.method==='cast.framing.apply').length,1,'double click cannot apply twice');
resolveApply(true);await tick();assert.equal(draw(),null,'successful apply immediately removes the stale editor');
apply();await tick();assert.equal(calls.filter(call=>call.method==='cast.framing.apply').length,1);
tree=await mount(()=>{throw Error('Fixture save rejected');});
button(tree,'应用取景').props.onClick();await tick();tree=draw();
assert.ok(tree,'save failure leaves the editor usable');assert.equal(tree.props.inert,false);
let error;walk(tree,node=>{if(node.props?.role==='alert')error=node.props.children;});assert.equal(error,'Fixture save rejected');
assert.equal(button(tree,'应用取景').props.disabled,false);
tree=await mount(()=>true);button(tree,'取消').props.onClick();await tick();assert.equal(draw(),null,'cancel has the same terminal state');
tree=await mount(()=>{throw Error('Saved, but restart failed');});
setHandler(method=>{if(method==='cast.framing.apply')throw Error('Saved, but restart failed');if(method==='cast.framing.get')throw Error('画幅调整窗口已关闭');throw Error(method);});
button(tree,'应用取景').props.onClick();await tick();assert.equal(draw(),null,'post-save restart failure cannot revive a closed editor');
const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));
function frame(tree){let found;walk(tree,node=>{if(node.props?.spatial&&node.props?.settings&&node.props?.onChange)found=node;});assert.ok(found,'framing editor');return found;}
tree=await mount(()=>true);await delay(120);
let references=calls.filter(call=>call.method==='cast.framing.reference');
assert.equal(references.length,1);assert.deepEqual(references[0].params,{id:geometry.id,centerX:0,centerY:0});
frame(tree).props.onChange({centerX:5,centerY:-3});tree=draw();
frame(tree).props.onChange({centerX:6,centerY:-4});tree=draw();await delay(120);
references=calls.filter(call=>call.method==='cast.framing.reference');
assert.equal(references.length,2,'rapid committed centers are debounced');
assert.ok(Math.abs(references[1].params.centerX-6)<1e-8&&Math.abs(references[1].params.centerY+4)<1e-8,'live draft is sent in degrees');
assert.ok(references[0].signal.aborted,'previous reference request is canceled after a new center');
frame(tree).props.onChange({centerX:7});tree=draw();
button(tree,'取消').props.onClick();await tick();assert.equal(draw(),null);await delay(120);
assert.equal(calls.filter(call=>call.method==='cast.framing.reference').length,2,'closing cancels the pending depth-probe update');
console.log('PASS: terminal editor lifecycle, error recovery, debounced live-center depth updates and closing cancellation');
+24
View File
@@ -0,0 +1,24 @@
import {build} from 'esbuild';
import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
import {readFile} from 'node:fs/promises';
Object.defineProperty(globalThis,'navigator',{value:{languages:['en-US']},configurable:true});
globalThis.document={documentElement:{lang:''},addEventListener(){}};
globalThis.window={addEventListener(){}};
const bundle=await build({stdin:{contents:`import React from 'react';import {renderToStaticMarkup} from 'react-dom/server';import {SpatialFraming} from './ui/src/cast-spatial-framing';export {loadSpatialFraming} from './ui/src/cast-spatial-framing';export const render=()=>renderToStaticMarkup(<SpatialFraming/>);`,loader:'tsx',resolveDir:process.cwd()},bundle:true,platform:'node',format:'cjs',loader:{'.css':'empty'},write:false});
const module={exports:{}};new Function('module','exports','require',bundle.outputFiles[0].text)(module,module.exports,createRequire(import.meta.url));
const calls=[];
const geometry={id:'test'};
await module.exports.loadSpatialFraming(async method=>{calls.push(method);return method==='status'?{database:{language:'zh-CN'}}:method==='language.list'?{packs:[]}:geometry;});
assert.deepEqual(calls.sort(),['cast.framing.get','language.list','status']);
assert.equal(document.documentElement.lang,'zh-CN');
const chinese=module.exports.render();
for(const text of ['画幅调整','画面比例','录制分辨率','应用取景','重置大小'])assert.ok(chinese.includes(text),text);
assert.ok(!chinese.includes('<select'));
assert.equal((chinese.match(/aria-haspopup="listbox"/g)||[]).length,2);
await module.exports.loadSpatialFraming(async method=>method==='status'?{database:{language:'en-US'}}:method==='language.list'?{packs:[]}:geometry);
assert.ok(module.exports.render().includes('Adjust recording frame'));
const css=await readFile('ui/src/casting.css','utf8');
assert.match(css,/\.cast-quick-framing-row\{display:flex;align-items:center;justify-content:space-between/);
assert.match(css,/\.cast-quick-framing-row \.switch-row\{width:auto;min-width:0/);
console.log('PASS: spatial frame loads configured language and uses two in-page dropdowns; quick controls share one row');
+13
View File
@@ -0,0 +1,13 @@
#include "framing_lifecycle.h"
#include <cassert>
#include <iostream>
int main(){
using nlohmann::json;
assert(!framing_editor_should_close(nullptr,"current"));
assert(!framing_editor_should_close(json::object(),"current"));
assert(!framing_editor_should_close({{"id","current"},{"open",true}},"current"));
assert(framing_editor_should_close({{"id","current"},{"open",false}},"current"));
assert(!framing_editor_should_close({{"id","old"},{"open",false}},"current"));
assert(framing_editor_should_close({{"id","new"},{"open",true}},"current"));
std::cout<<"PASS: persistent framing state repairs lost closes and ignores stale acknowledgements\n";
}
+25
View File
@@ -0,0 +1,25 @@
#include "openvr.h"
#include "recording_region.h"
#include <cassert>
#include <iostream>
int main(){
for(double distance:{.5,1.,2.,4.})for(double roll:{-.2,0.,.2})for(double yaw:{-.3,0.,.3})for(double aspect:{4./3,16./9,9./16,1.}){
double c=cos(roll),s=sin(roll),cy=cos(yaw),sy=sin(yaw);
RegionJson view={{"captureToHead",{{c*cy,-s, c*sy,-.032},{s*cy,c,s*sy,.003},{-sy,0,cy,.01}}}};
const double tan=.8,cx=.15,center_y=-.08;
RecordingRegionPlane plane;assert(recording_region_plane(view,tan,cx,center_y,aspect,plane,distance));
// Undo the exported head basis: each overlay corner must be exactly the
// tangent ray consumed at the corresponding output-LUT corner.
for(double u:{0.,.25,1.})for(double v:{0.,.75,1.}){
double world[3];for(int i=0;i<3;i++)world[i]=plane.transform.m[i][3]+plane.transform.m[i][0]*(u-.5)*plane.width+plane.transform.m[i][1]*(.5-v)*plane.width/aspect-view["captureToHead"][i][3].get<double>();
double ray[3]{};for(int j=0;j<3;j++)for(int i=0;i<3;i++)ray[j]+=view["captureToHead"][i][j].get<double>()*world[i]/distance;
assert(std::abs(ray[0]-(cx+(2*u-1)*tan))<1e-6);
assert(std::abs(ray[1]-(-center_y+(1-2*v)*tan/aspect))<1e-6);
assert(std::abs(ray[2]+1.)<1e-6);
}
}
auto pixels=recording_region_pixels(100,80);assert(pixels[(40*100+50)*4+3]==0);assert(pixels[3]>0&&pixels[3]<100);
auto colored=recording_region_pixels(100,80,{12,34,56});assert(colored[0]==12&&colored[1]==34&&colored[2]==56&&colored[3]==72);
RecordingRegionPlane bad;assert(!recording_region_plane(RegionJson::object(),1,0,0,1,bad));
std::cout<<"PASS: recording border projection matches capture rays, including roll/yaw/eye offset and aspect ratios\n";
}
+86
View File
@@ -0,0 +1,86 @@
#include "recording_region_depth.h"
#include <cassert>
#include <iostream>
#include <limits>
using namespace recording_region_depth_detail;
static vr::HmdMatrix34_t pose(double yaw=0,double pitch=0,double roll=0,Point t={0,0,0}){
const double cy=cos(yaw),sy=sin(yaw),cp=cos(pitch),sp=sin(pitch),cr=cos(roll),sr=sin(roll);
vr::HmdMatrix34_t m{{
{float(cy*cr+sy*sp*sr),float(-cy*sr+sy*sp*cr),float(sy*cp),float(t[0])},
{float(cp*sr),float(cp*cr),float(-sp),float(t[1])},
{float(-sy*cr+cy*sp*sr),float(sy*sr+cy*sp*cr),float(cy*cp),float(t[2])}}};return m;
}
static vr::HmdMatrix34_t multiply(const vr::HmdMatrix34_t& a,const vr::HmdMatrix34_t& b){
vr::HmdMatrix34_t m{};for(int i=0;i<3;i++){
for(int j=0;j<3;j++)for(int k=0;k<3;k++)m.m[i][j]+=a.m[i][k]*b.m[k][j];
m.m[i][3]=a.m[i][3];for(int k=0;k<3;k++)m.m[i][3]+=a.m[i][k]*b.m[k][3];
}return m;
}
static void near(double a,double b,double tolerance=1e-5){assert(std::abs(a-b)<tolerance);}
int main(){
for(double ipd:{0.055,0.064,0.075})for(auto eye:{vr::Eye_Left,vr::Eye_Right}){
const auto left=pose(0,0,0,{-ipd/2,0,0.012}),right=pose(0,0,0,{ipd/2,0,0.012});
auto capture=pose(0.12,-0.08,0.04,{0,0,0});const auto before=capture;
assert(recording_region_refresh_eye_origin(eye,left,right,capture));
for(int i=0;i<3;i++)for(int j=0;j<3;j++)assert(capture.m[i][j]==before.m[i][j]);
const auto selected=eye==vr::Eye_Left?left:right;
for(int i=0;i<3;i++)near(capture.m[i][3],selected.m[i][3]);
for(double depth:{0.5,1.,2.,4.})for(double yaw:{-0.35,0.,0.35})for(double pitch:{-0.25,0.,0.25}){
const auto hmd=pose(0.4,-0.1,0.05,{0.5,1.6,-0.2});
const double cx=0.16,cy=-0.09;
const auto origin=add(translation(hmd),rotate(hmd,translation(capture)));
const auto direction=rotate(hmd,rotate(capture,{cx,-cy,-1.}));
const Point intersection={origin[0]+depth*direction[0],origin[1]+depth*direction[1],origin[2]+depth*direction[2]};
auto window=multiply(hmd,pose(yaw,pitch,0.07));for(int i=0;i<3;i++)window.m[i][3]=float(intersection[i]);
const auto result=recording_region_window_depth(hmd,window,capture,cx,cy);
assert(result);near(*result,depth);
const vr::HmdVector3_t hit{{float(intersection[0]),float(intersection[1]),float(intersection[2])}};
const auto hit_depth=recording_region_hit_depth(hmd,capture,cx,cy,hit);
assert(hit_depth);near(*hit_depth,depth);
}
// Any finite plane distance preserves selected-eye ray bounds. The other
// eye's shift is baseline/D, symmetric for left and right recordings.
for(double depth:{0.5,1.,2.,4.})for(double tx:{-0.7,0.,0.7}){
auto parallel=pose(0,0,0,{0,0,0});assert(recording_region_refresh_eye_origin(eye,left,right,parallel));
const auto origin=translation(parallel),other=translation(eye==vr::Eye_Left?right:left);
const Point point={origin[0]+depth*tx,origin[1],origin[2]-depth};
near((point[0]-origin[0])/(origin[2]-point[2]),tx);
const double baseline=origin[0]-other[0];
const double actual_other=(point[0]-other[0])/(other[2]-point[2]);
near(actual_other,tx+baseline/depth);
const double old_other=tx+baseline/2.;
near(actual_other-old_other,baseline*(1./depth-0.5));
if(depth!=2.)assert(std::abs(actual_other-old_other)>0.001);
}
}
// Changing IPD refreshes a stale exported origin without altering ray basis.
auto stale=pose(0,0,0,{0.032,0,0});
assert(recording_region_refresh_eye_origin(vr::Eye_Right,pose(0,0,0,{-0.0375,0,0}),pose(0,0,0,{0.0375,0,0}),stale));
near(stale.m[0][3],0.0375);
const auto identity=pose(),left=pose(0,0,0,{-0.032,0,0}),right=pose(0,0,0,{0.032,0,0});
auto unchanged=identity;
assert(!recording_region_refresh_eye_origin(vr::Eye_Right,pose(0,0,0,{-0.1,0,0}),pose(0,0,0,{0.1,0,0}),unchanged));
assert(!recording_region_refresh_eye_origin(vr::Eye_Right,right,left,unchanged));
assert(unchanged.m[0][3]==0);
auto invalid=identity;invalid.m[0][0]=std::numeric_limits<float>::quiet_NaN();
assert(!recording_region_refresh_eye_origin(vr::Eye_Right,invalid,right,unchanged));
assert(!recording_region_window_depth(invalid,pose(0,0,0,{0,0,-2}),identity,0,0));
assert(!recording_region_window_depth(vr::HmdMatrix34_t{},pose(0,0,0,{0,0,-2}),identity,0,0));
assert(!recording_region_window_depth(identity,pose(acos(-1.)/2,0,0,{0,0,-2}),identity,0,0));
for(double z:{-0.1,-8.,1.})assert(!recording_region_window_depth(identity,pose(0,0,0,{0,0,z}),identity,0,0));
assert(!recording_region_window_depth(identity,pose(0,0,0,{0,0,-2}),identity,std::numeric_limits<double>::infinity(),0));
// Eye Z matters: head-relative depth alone would return 2, but D=2.012.
const auto zdepth=recording_region_window_depth(identity,pose(0,0,0,{0,0,-2}),right,0,0);
assert(zdepth);near(*zdepth,2.0); // This fixture's right eye has Z=0.
const auto elevated=pose(0,0,0,{0.032,0,0.012});
const auto exact=recording_region_window_depth(identity,pose(0,0,0,{0,0,-2}),elevated,0,0);
assert(exact);near(*exact,2.012);
for(double depth:{0.25,0.5,1.,2.,4.,6.}){
const vr::HmdVector3_t hit{{0,0,float(-depth)}};
const auto result=recording_region_hit_depth(identity,identity,0,0,hit);assert(result);near(*result,depth);
}
for(const auto& hit:std::array<vr::HmdVector3_t,4>{{{{0.02f,0,-2}},{{0,0,2}},{{0,0,-8}},{{std::numeric_limits<float>::quiet_NaN(),0,-2}}}})
assert(!recording_region_hit_depth(identity,identity,0,0,hit));
assert(!recording_region_hit_depth(invalid,identity,0,0,vr::HmdVector3_t{{0,0,-2}}));
std::cout<<"PASS: current eye origin, exact center-ray UI depth, both-eye parallax and invalid-pose guards\n";
}
+37
View File
@@ -0,0 +1,37 @@
// Isolated device fixture: visible fiducials, never part of the installed app.
#include "openvr.h"
#include "recording_region.h"
#include <fstream>
#include <thread>
#include <chrono>
#include <filesystem>
#include <iostream>
int main(int argc,char** argv){
if(argc!=4)return 2;RegionJson view;std::ifstream(argv[1])>>view;
vr::EVRInitError error{};vr::VR_Init(&error,vr::VRApplication_Overlay);if(error)return 3;
auto* overlays=vr::VROverlay();vr::VROverlayHandle_t handle=0;
if(overlays->CreateOverlay("framely.region.alignment.fixture","Recording alignment fixture",&handle))return 4;
RecordingRegionPlane plane;if(!recording_region_plane(view,view.at("tanHalfHorizontal"),view.at("centerTanX"),view.at("centerTanY"),view.at("aspect"),plane))return 5;
const int w=view.at("width"),h=view.at("height"),texture=512;std::vector<unsigned char> pixels(texture*texture*4);
const int positions[5][2]={{16,16},{w-16,16},{16,h-16},{w-16,h-16},{w/2,h/2}};
const int colors[5][3]={{255,0,0},{0,255,0},{0,0,255},{255,0,255},{255,255,255}};
for(int y=0;y<texture;y++)for(int x=0;x<texture;x++)for(int p=0;p<5;p++){
double ox=(x+.5)*w/texture,oy=(y+.5)*h/texture;
if(ox<positions[p][0]-16||ox>=positions[p][0]+16||oy<positions[p][1]-16||oy>=positions[p][1]+16)continue;
auto i=(y*texture+x)*4;for(int c=0;c<3;c++)pixels[i+c]=colors[p][c];pixels[i+3]=255;
}
overlays->SetOverlayInputMethod(handle,vr::VROverlayInputMethod_None);
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,false);
overlays->SetOverlayFlag(handle,vr::VROverlayFlags_VisibleInDashboard,true);
recording_region_foreground(overlays,handle);
overlays->SetOverlayWidthInMeters(handle,plane.width);
overlays->SetOverlayTexelAspect(handle,view.at("aspect"));
overlays->SetOverlayTransformTrackedDeviceRelative(handle,vr::k_unTrackedDeviceIndex_Hmd,&plane.transform);
overlays->SetOverlayRaw(handle,pixels.data(),texture,texture,4);overlays->ShowOverlay(handle);
std::ofstream(argv[2])<<"ready";
auto until=std::chrono::steady_clock::now()+std::chrono::seconds(40);
while(std::chrono::steady_clock::now()<until&&!std::filesystem::exists(argv[3])){
std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
overlays->DestroyOverlay(handle);vr::VR_Shutdown();return 0;
}
+116
View File
@@ -0,0 +1,116 @@
// Isolated, opt-in wearer diagnostic. Never part of the installed app.
// Usage: helper geometry.json ready-file stop-file
#include "openvr.h"
#include "recording_region.h"
#include <fstream>
#include <thread>
#include <chrono>
#include <filesystem>
#include <iostream>
#include <stdexcept>
namespace {
constexpr int texture_size=512;
constexpr double aspect=4./3.;
constexpr int fovs[]{80,75,70,65,60};
constexpr std::array<unsigned char,3> colors[]{
{255,48,48},{255,230,32},{32,255,255},{48,255,64},{255,255,255}
};
// Five columns, seven rows; each row's most significant bit is the left pixel.
constexpr unsigned char digits[10][7]{
{14,17,19,21,25,17,14},{4,12,4,4,4,4,14},
{14,17,1,2,4,8,31},{30,1,1,14,1,1,30},
{2,6,10,18,31,2,2},{31,16,16,30,1,1,30},
{14,16,16,30,17,17,14},{31,1,2,4,8,8,8},
{14,17,17,14,17,17,14},{14,17,17,15,1,1,14}
};
std::vector<unsigned char> pixels(int fov,const std::array<unsigned char,3>& color){
std::vector<unsigned char> image(texture_size*texture_size*4);
auto put=[&](int x,int y){
auto index=(y*texture_size+x)*4;
for(int c=0;c<3;c++)image[index+c]=color[c];
image[index+3]=255;
};
constexpr int stroke=3;
for(int y=0;y<texture_size;y++)for(int x=0;x<texture_size;x++){
if(x<stroke||x>=texture_size-stroke||y<stroke||y>=texture_size-stroke)put(x,y);
}
// Center the FOV number just inside the top edge. Large block digits remain
// readable without CEF or a font installation; all other pixels are clear.
// Keep the label shorter than the angular gap to the next rectangle, so
// an inner horizontal border cannot strike through the outer FOV number.
constexpr int scale=2,digit_width=5*scale,gap=2*scale;
constexpr int start_x=(texture_size-(2*digit_width+gap))/2,start_y=5;
for(int digit=0;digit<2;digit++){
int value=digit==0?fov/10:fov%10;
for(int row=0;row<7;row++)for(int column=0;column<5;column++){
if(!(digits[value][row]&(1<<(4-column))))continue;
for(int dy=0;dy<scale;dy++)for(int dx=0;dx<scale;dx++)
put(start_x+digit*(digit_width+gap)+column*scale+dx,start_y+row*scale+dy);
}
}
return image;
}
void check(vr::IVROverlay* api,vr::EVROverlayError error,const char* operation){
if(error!=vr::VROverlayError_None)
throw std::runtime_error(std::string(operation)+": "+api->GetOverlayErrorNameFromEnum(error));
}
struct Cleanup {
vr::IVROverlay* api=nullptr;
std::vector<vr::VROverlayHandle_t> handles;
bool initialized=false;
~Cleanup(){
for(auto handle:handles)if(api){
auto hidden=api->HideOverlay(handle),destroyed=api->DestroyOverlay(handle);
if(hidden||destroyed)std::cerr<<"Diagnostic overlay cleanup error: "<<hidden<<", "<<destroyed<<"\n";
}
if(initialized)vr::VR_Shutdown();
}
};
}
int main(int argc,char** argv){
if(argc!=4){std::cerr<<"Usage: recording-visibility-device geometry.json ready-file stop-file\n";return 2;}
Cleanup cleanup;
try{
RegionJson view;std::ifstream input(argv[1]);if(!input)throw std::runtime_error("Cannot open geometry");input>>view;
const double cx=view.at("centerTanX"),cy=view.at("centerTanY");
vr::EVRInitError error{};vr::VR_Init(&error,vr::VRApplication_Overlay);
if(error)throw std::runtime_error(vr::VR_GetVRInitErrorAsEnglishDescription(error));
cleanup.initialized=true;cleanup.api=vr::VROverlay();
if(!cleanup.api)throw std::runtime_error("OpenVR overlay interface unavailable");
auto* api=cleanup.api;
for(int i=0;i<5;i++){
RecordingRegionPlane plane;
const double tangent=std::tan(fovs[i]*std::acos(-1.)/360.);
if(!recording_region_plane(view,tangent,cx,cy,aspect,plane))throw std::runtime_error("Invalid capture geometry");
vr::VROverlayHandle_t handle=0;
const std::string key="framely.recording.visibility.fixture."+std::to_string(fovs[i]);
check(api,api->CreateOverlay(key.c_str(),("Recording visibility "+std::to_string(fovs[i])).c_str(),&handle),"CreateOverlay");
cleanup.handles.push_back(handle);
check(api,api->SetOverlayInputMethod(handle,vr::VROverlayInputMethod_None),"SetOverlayInputMethod");
check(api,api->SetOverlayFlag(handle,vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible,false),"Disable interactive overlay");
check(api,api->SetOverlayFlag(handle,vr::VROverlayFlags_HideLaserIntersection,true),"Hide laser intersection");
check(api,api->SetOverlayFlag(handle,vr::VROverlayFlags_VisibleInDashboard,true),"VisibleInDashboard");
check(api,api->SetOverlayFlag(handle,vr::VROverlayFlags_SortWithNonSceneOverlays,true),"Foreground sort flag");
check(api,api->SetOverlaySortOrder(handle,1000+i),"SetOverlaySortOrder");
check(api,api->SetOverlayWidthInMeters(handle,plane.width),"SetOverlayWidthInMeters");
check(api,api->SetOverlayTexelAspect(handle,aspect),"SetOverlayTexelAspect");
check(api,api->SetOverlayTransformTrackedDeviceRelative(handle,vr::k_unTrackedDeviceIndex_Hmd,&plane.transform),"SetOverlayTransformTrackedDeviceRelative");
auto image=pixels(fovs[i],colors[i]);
check(api,api->SetOverlayRaw(handle,image.data(),texture_size,texture_size,4),"SetOverlayRaw");
check(api,api->ShowOverlay(handle),"ShowOverlay");
}
std::ofstream ready(argv[2]);if(!ready)throw std::runtime_error("Cannot open ready file");
ready<<"ready\n";ready.close();if(!ready)throw std::runtime_error("Cannot write ready file");
std::cout<<"Visibility diagnostic ready: eye="<<view.at("eye")<<", center="<<cx<<","<<cy
<<", 4:3 FOVs red80 yellow75 cyan70 green65 white60; auto-hide after 90 seconds\n"<<std::flush;
const auto until=std::chrono::steady_clock::now()+std::chrono::seconds(90);
while(std::chrono::steady_clock::now()<until&&!std::filesystem::exists(argv[3]))
std::this_thread::sleep_for(std::chrono::milliseconds(20));
return 0;
}catch(const std::exception& error){std::cerr<<"Recording visibility diagnostic: "<<error.what()<<"\n";return 1;}
}
+1 -1
View File
@@ -3,7 +3,7 @@ set -euo pipefail
base=$(cd -- "$(dirname -- "$0")/.." && pwd)
cd "$base"
mkdir -p target/tests
for test in gamepad_mapping gamepad_steam gamepad_direct entry_hold menu_geometry keyboard_state notification_geometry notification_badge ui_visibility plugin_navigation launcher_input launcher_gaze gaze_geometry launcher_fade launcher_pointer page_scroll paint_buffer render_timing launcher_priority recording_indicator; do
for test in gamepad_mapping gamepad_steam gamepad_direct entry_hold menu_geometry keyboard_state notification_geometry notification_badge ui_visibility plugin_navigation launcher_input launcher_gaze gaze_geometry launcher_fade launcher_pointer page_scroll paint_buffer render_timing launcher_priority recording_indicator recording_region framing_lifecycle recording_region_depth; do
"${CXX:-g++}" -std=c++17 -O2 -pthread -Inative -Inative/vendor/openvr "tests/$test.cpp" -ldl -o "target/tests/$test"
"target/tests/$test"
done
+18
View File
@@ -3,6 +3,24 @@
"locale": "en-US",
"name": "English",
"messages": {
"重置大小": "Reset size",
"录制眼已改变,请重新打开画幅调整": "The recording eye has changed. Reopen frame adjustment.",
"保留比例和分辨率,恢复居中有效画幅": "Keep the aspect ratio and resolution, and restore the centered valid frame.",
"菜单尺寸超过当前可录制范围": "The menu is larger than the available recording area.",
"缺少有效的面板校准数据,无法显示准确录制区域": "Valid panel calibration is unavailable; an accurate recording region cannot be displayed.",
"取景范围超出头显可见区域,请缩小或调整位置": "The recording region extends beyond the headset's visible area. Reduce its size or adjust its position.",
"自定义": "Custom",
"画幅调整": "Adjust recording frame",
"录制区域显示": "Show recording region",
"框内即录制区域,拖动移动,拖动四角等比例缩放。": "The area inside the frame is recorded. Drag to move; drag a corner to resize proportionally.",
"录制分辨率": "Recording resolution",
"录制区域显示仅支持屏幕采集的单眼画面": "Recording region display requires single-eye screen capture.",
"画幅调整仅支持屏幕采集的单眼画面": "Frame adjustment requires single-eye screen capture.",
"请先关闭画幅调整窗口": "Close frame adjustment first.",
"画幅调整窗口已关闭": "The frame adjustment window has closed.",
"无法读取实际录制区域,请更新采集程序": "Cannot read the actual recording region. Update the capture worker.",
"无法读取实际录制区域,请确认 SteamVR 已启动": "Cannot read the actual recording region. Make sure SteamVR is running.",
"读取录制区域超时": "Reading the recording region timed out.",
"安卓权限请求": "Android permission request",
"正在读取权限请求": "Reading permission request",
"安卓权限选择": "Android permission selection",
+18
View File
@@ -3,6 +3,24 @@
"locale": "zh-CN",
"name": "简体中文",
"messages": {
"重置大小": "重置大小",
"录制眼已改变,请重新打开画幅调整": "录制眼已改变,请重新打开画幅调整",
"保留比例和分辨率,恢复居中有效画幅": "保留比例和分辨率,恢复居中有效画幅",
"菜单尺寸超过当前可录制范围": "菜单尺寸超过当前可录制范围",
"缺少有效的面板校准数据,无法显示准确录制区域": "缺少有效的面板校准数据,无法显示准确录制区域",
"取景范围超出头显可见区域,请缩小或调整位置": "取景范围超出头显可见区域,请缩小或调整位置",
"自定义": "自定义",
"画幅调整": "画幅调整",
"录制区域显示": "录制区域显示",
"框内即录制区域,拖动移动,拖动四角等比例缩放。": "框内即录制区域,拖动移动,拖动四角等比例缩放。",
"录制分辨率": "录制分辨率",
"录制区域显示仅支持屏幕采集的单眼画面": "录制区域显示仅支持屏幕采集的单眼画面",
"画幅调整仅支持屏幕采集的单眼画面": "画幅调整仅支持屏幕采集的单眼画面",
"请先关闭画幅调整窗口": "请先关闭画幅调整窗口",
"画幅调整窗口已关闭": "画幅调整窗口已关闭",
"无法读取实际录制区域,请更新采集程序": "无法读取实际录制区域,请更新采集程序",
"无法读取实际录制区域,请确认 SteamVR 已启动": "无法读取实际录制区域,请确认 SteamVR 已启动",
"读取录制区域超时": "读取录制区域超时",
"安卓权限请求": "安卓权限请求",
"正在读取权限请求": "正在读取权限请求",
"安卓权限选择": "安卓权限选择",
+1 -1
View File
@@ -2,7 +2,7 @@ import {t} from './i18n';
import {installKeyboard} from '../../sdk/src/keyboard';
import {installHoverFeedback} from '../../sdk/src/hover';
export async function api<T=any>(method:string,params:unknown={},signal?:AbortSignal):Promise<T>{const r=await fetch('/api',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({method,params}),signal});if(r.status===401&&r.headers.get('X-Framely-Reauthenticate')==='1')location.replace('/manager');if(!r.ok)throw new Error(t('服务请求失败:{0}',{0:r.status}));const v=await r.json();if(v.error)throw new Error(t(v.error));return v.result;}
export function viewKey(){const p=location.pathname.split('/');return p[1]==='manager'?'framely.manager':p[1]==='window'?`framely.window.${p[2]}.${p[3]}`:p[1]==='cast'&&p[2]==='window'?`framely.cast.${p[3]}`:p[1]==='apk'&&p[2]==='permission'?'framely.apk.permission':p[1]==='notifications'?'notifications':p[1]==='launcher'?'launcher':'menu';}
export function viewKey(){const p=location.pathname.split('/');return p[1]==='cast'&&p[2]==='framing'?'framely.framing':p[1]==='manager'?'framely.manager':p[1]==='window'?`framely.window.${p[2]}.${p[3]}`:p[1]==='cast'&&p[2]==='window'?`framely.cast.${p[3]}`:p[1]==='apk'&&p[2]==='permission'?'framely.apk.permission':p[1]==='notifications'?'notifications':p[1]==='launcher'?'launcher':'menu';}
installKeyboard(params=>api('host.keyboard',{view:viewKey(),...params}));
+21 -21
View File
@@ -1,42 +1,42 @@
import React,{useEffect,useRef} from 'react';
import {IconMinus,IconPlus,IconRestore} from '@tabler/icons-react';
import {CastVideo} from './cast-viewer';
import {clamp,fitFrame,frameFromSettings,settingsFromFrame,type FrameRect,type ReferenceView} from './cast-framing';
import {fitFrame,frameFromSettings,settingsFromFrame,type FrameRect,type ReferenceView} from './cast-framing';
import type {CastSettings} from './casting';
import {CastRange,updateRangeFill} from './cast-range';
import {t} from './i18n';
import {FrameGesture,fitMinimumFrame,minimumFrameWidth,resizeFrameCentered,type MinimumFrame} from './cast-framing-gesture';
import type {FrameLimiter} from './cast-region';
export function FramingEditor({settings,view,onChange,onError}:{settings:CastSettings;view:ReferenceView|null;onChange:(patch:Partial<CastSettings>)=>void;onError:(error:string)=>void}){
export function FramingEditor({settings,view,onChange,onError,spatial=false,minimumSize={width:0,height:0},limit}:{settings:CastSettings;view:ReferenceView|null;onChange:(patch:Partial<CastSettings>)=>void;onError:(error:string)=>void;spatial?:boolean;minimumSize?:MinimumFrame;limit?:FrameLimiter}){
const stage=useRef<HTMLDivElement>(null),box=useRef<HTMLDivElement>(null),zoom=useRef<HTMLInputElement>(null),readout=useRef<HTMLOutputElement>(null);
const current=useRef<FrameRect>({x:0,y:0,width:1,height:1}),gesture=useRef<{x:number;y:number;rect:FrameRect;corner:string|null}|null>(null);
const current=useRef<FrameRect>({x:0,y:0,width:1,height:1}),gesture=useRef(new FrameGesture()),pointer=useRef<number|null>(null),release=useRef<(id?:number)=>void>(()=>{});
const aspect=settings.width/settings.height;
const maxWidth=view?Math.min(1,aspect/view.aspect):1,minWidth=view?Math.min(maxWidth,Math.tan(5*Math.PI/180)/view.tanHalfHorizontal):.1;
const ratio=aspect/(view?.aspect??1),maxWidth=Math.min(1,ratio),minWidth=minimumFrameWidth(ratio,minimumSize,view?Math.tan(5*Math.PI/180)/view.tanHalfHorizontal:.1);
function draw(rect:FrameRect){
try{if(limit)rect=limit(rect,ratio,minWidth,current.current);}catch(e){if(box.current)box.current.style.visibility='hidden';onError((e as Error).message);return null;}
current.current=rect;
if(box.current){box.current.style.left=`${rect.x*100}%`;box.current.style.top=`${rect.y*100}%`;box.current.style.width=`${rect.width*100}%`;box.current.style.height=`${rect.height*100}%`;}
if(box.current){box.current.style.visibility='visible';box.current.style.left=`${rect.x*100}%`;box.current.style.top=`${rect.y*100}%`;box.current.style.width=`${rect.width*100}%`;box.current.style.height=`${rect.height*100}%`;}
if(view){const fov=settingsFromFrame(rect,view).horizontalFov;if(zoom.current){zoom.current.value=String(maxWidth>minWidth?(rect.width-minWidth)/(maxWidth-minWidth)*100:100);updateRangeFill(zoom.current);}if(readout.current)readout.current.textContent=`${Math.round(fov)}°`;}
return rect;
}
function commit(){if(view)onChange(settingsFromFrame(current.current,view));}
useEffect(()=>{if(view)draw(frameFromSettings(settings,aspect,view));},[view?.tanHalfHorizontal,view?.centerTanX,view?.centerTanY,aspect,settings.horizontalFov,settings.centerX,settings.centerY]);
function scale(width:number){if(!view)return;const r=current.current;const next=fitFrame({...r,width,x:r.x+(r.width-width)/2,y:r.y+(r.height-width*view.aspect/aspect)/2},aspect,view.aspect,Math.tan(5*Math.PI/180)/view.tanHalfHorizontal);draw(next);}
function down(e:React.PointerEvent<HTMLDivElement>){if(!view||e.button!==0)return;e.preventDefault();e.currentTarget.setPointerCapture(e.pointerId);gesture.current={x:e.clientX,y:e.clientY,rect:{...current.current},corner:(e.target as HTMLElement).closest<HTMLElement>('[data-corner]')?.dataset.corner??null};}
function move(e:React.PointerEvent<HTMLDivElement>){const g=gesture.current,bounds=stage.current?.getBoundingClientRect();if(!g||!bounds||!view)return;const dx=(e.clientX-g.x)/bounds.width,dy=(e.clientY-g.y)/bounds.height;
if(!g.corner){draw(fitFrame({...g.rect,x:g.rect.x+dx,y:g.rect.y+dy},aspect,view.aspect));return;}
const left=g.corner.includes('w'),top=g.corner.includes('n'),ratio=aspect/view.aspect;
const delta=Math.abs(dx)>Math.abs(dy*ratio)?dx*(left?-1:1):dy*ratio*(top?-1:1);
const anchorX=left?g.rect.x+g.rect.width:g.rect.x,anchorY=top?g.rect.y+g.rect.height:g.rect.y;
const maximum=Math.min(left?anchorX:1-anchorX,(top?anchorY:1-anchorY)*ratio);
const width=clamp(g.rect.width+delta,Math.min(maximum,Math.tan(5*Math.PI/180)/view.tanHalfHorizontal),maximum),height=width/ratio;
draw({x:left?anchorX-width:anchorX,y:top?anchorY-height:anchorY,width,height});
function commit(){if(view&&box.current?.style.visibility!=='hidden')onChange(settingsFromFrame(current.current,view));}
useEffect(()=>{if(view){const old=frameFromSettings(settings,aspect,view),next=draw(fitMinimumFrame(old,ratio,minWidth));if(next&&(Math.abs(next.width-old.width)>1e-9||Math.abs(next.x-old.x)>1e-9||Math.abs(next.y-old.y)>1e-9))commit();}},[view?.tanHalfHorizontal,view?.centerTanX,view?.centerTanY,aspect,settings.horizontalFov,settings.centerX,settings.centerY,minWidth]);
function scale(width:number){if(view)draw(resizeFrameCentered(current.current,width,ratio,minWidth));}
function finish(id?:number){if(!gesture.current.finish(id))return;const captured=pointer.current;pointer.current=null;try{if(captured!==null&&box.current?.hasPointerCapture(captured))box.current.releasePointerCapture(captured);}catch{}commit();}
release.current=finish;
useEffect(()=>{const up=(e:PointerEvent)=>release.current(e.pointerId),cancel=()=>release.current();document.addEventListener('pointerup',up,true);document.addEventListener('pointercancel',up,true);window.addEventListener('blur',cancel);window.addEventListener('framely.pointerRelease',cancel);return()=>{document.removeEventListener('pointerup',up,true);document.removeEventListener('pointercancel',up,true);window.removeEventListener('blur',cancel);window.removeEventListener('framely.pointerRelease',cancel);gesture.current.finish();};},[]);
function down(e:React.PointerEvent<HTMLDivElement>){if(!view||e.button!==0)return;e.preventDefault();finish();gesture.current.begin(e.pointerId,e.clientX,e.clientY,current.current,(e.target as HTMLElement).closest<HTMLElement>('[data-corner]')?.dataset.corner??null);pointer.current=e.pointerId;try{e.currentTarget.setPointerCapture(e.pointerId);}catch{}}
function move(e:React.PointerEvent<HTMLDivElement>){if(!gesture.current.active(e.pointerId))return;if(!(e.buttons&1)){finish(e.pointerId);return;}const bounds=stage.current?.getBoundingClientRect();if(!bounds||!view)return;const next=gesture.current.move(e.pointerId,e.clientX,e.clientY,e.buttons,bounds,ratio,minWidth);if(next)draw(next);
}
function end(e:React.PointerEvent<HTMLDivElement>){if(!gesture.current)return;gesture.current=null;if(e.currentTarget.hasPointerCapture(e.pointerId))e.currentTarget.releasePointerCapture(e.pointerId);commit();}
function end(e:React.PointerEvent<HTMLDivElement>){finish(e.pointerId);}
function keyboard(e:React.KeyboardEvent<HTMLDivElement>){if(!view)return;const delta=e.shiftKey?.05:.01,r=current.current;const changes:Record<string,Partial<FrameRect>>={ArrowLeft:{x:r.x-delta},ArrowRight:{x:r.x+delta},ArrowUp:{y:r.y-delta},ArrowDown:{y:r.y+delta}};if(changes[e.key]){e.preventDefault();draw(fitFrame({...r,...changes[e.key]},aspect,view.aspect));commit();}}
return <div className="cast-framing-editor">
<div className="cast-framing-stage" ref={stage}>
<CastVideo active controls={false} aspectRatio={1} onError={onError}/>
{view&&<div ref={box} className="cast-crop-box" tabIndex={0} role="group" aria-label={t('取景框,使用方向键移动')} onKeyDown={keyboard} onPointerDown={down} onPointerMove={move} onPointerUp={end} onPointerCancel={end}>
{!spatial&&<CastVideo active controls={false} aspectRatio={1} onError={onError}/>}
{view&&<div ref={box} className="cast-crop-box" tabIndex={0} role="group" aria-label={t('取景框,使用方向键移动')} onKeyDown={keyboard} onPointerDown={down} onPointerMove={move} onPointerUp={end} onPointerCancel={end} onLostPointerCapture={end}>
<div className="cast-crop-grid" aria-hidden="true"/>
{['nw','ne','sw','se'].map(corner=><span key={corner} data-corner={corner} className={`cast-crop-handle ${corner}`} aria-hidden="true"/>)}
{['nw','ne','sw','se'].map(corner=><span key={corner} data-corner={corner} data-framely-interactive="" className={`cast-crop-handle ${corner}`} aria-hidden="true"/>)}
</div>}
</div>
<div className="cast-zoom-controls">
+33
View File
@@ -0,0 +1,33 @@
import {clamp,fitFrame,type FrameRect} from './cast-framing';
export type MinimumFrame={width:number;height:number};
export function minimumFrameWidth(ratio:number,minimum:MinimumFrame,fovMinimum=0){
return Math.min(1,ratio,Math.max(fovMinimum,minimum.width,minimum.height*ratio));
}
export function fitMinimumFrame(rect:FrameRect,ratio:number,minimum:number):FrameRect{
const width=clamp(rect.width,minimum,Math.min(1,ratio)),height=width/ratio;
const cx=clamp(rect.x+rect.width/2,width/2,1-width/2),cy=clamp(rect.y+rect.height/2,height/2,1-height/2);
return {x:cx-width/2,y:cy-height/2,width,height};
}
export function resizeFrameCentered(rect:FrameRect,width:number,ratio:number,minimum:number):FrameRect{
const cx=rect.x+rect.width/2,cy=rect.y+rect.height/2;
const maximum=Math.min(2*cx,2*(1-cx),2*cy*ratio,2*(1-cy)*ratio);
width=clamp(width,Math.min(minimum,maximum),maximum);
return {x:cx-width/2,y:cy-width/ratio/2,width,height:width/ratio};
}
export class FrameGesture {
private drag:{id:number;x:number;y:number;rect:FrameRect;corner:string|null}|null=null;
begin(id:number,x:number,y:number,rect:FrameRect,corner:string|null){this.drag={id,x,y,rect:{...rect},corner};}
active(id?:number){return !!this.drag&&(id===undefined||this.drag.id===id);}
finish(id?:number){if(!this.active(id))return false;this.drag=null;return true;}
move(id:number,x:number,y:number,buttons:number,stage:{width:number;height:number},ratio:number,minimum:number):FrameRect|null{
const g=this.drag;if(!g||g.id!==id||stage.width<=0||stage.height<=0)return null;
if(!(buttons&1)){this.finish(id);return null;}
const dx=(x-g.x)/stage.width,dy=(y-g.y)/stage.height;
if(!g.corner)return fitFrame({...g.rect,x:g.rect.x+dx,y:g.rect.y+dy},ratio,1,minimum);
const sx=g.corner.includes('w')?-1:1,sy=g.corner.includes('n')?-1:1;
// Project pointer travel onto the corner's aspect-constrained diagonal.
// Both sides move equally: no opposite-corner/top-left anchoring.
const delta=2*(sx*dx+sy*dy/ratio)/(1+1/(ratio*ratio));
return resizeFrameCentered(g.rect,g.rect.width+delta,ratio,minimum);
}
}
+53
View File
@@ -0,0 +1,53 @@
import type {FrameRect} from './cast-framing';
import {fitMinimumFrame,resizeFrameCentered} from './cast-framing-gesture';
export type VisibilityGrid={size:number;extent:number;cells:string};
export const framingMenuScale=(referenceTan:number)=>0.9/referenceTan;
export class FramingLimits {
private readonly prefix:Int32Array;
readonly size:number;
constructor(grid:VisibilityGrid,referenceTan:number){
const n=grid?.size;
if(!Number.isInteger(n)||n<8||n>256||n%4||grid.extent!==referenceTan||grid.cells?.length!==n*n/4||!/^[0-9a-f]+$/i.test(grid.cells))throw Error('无法读取实际录制区域,请更新采集程序');
this.size=n;this.prefix=new Int32Array((n+1)*(n+1));
for(let y=0;y<n;y++)for(let x=0;x<n;x++){
const i=y*n+x,invalid=(parseInt(grid.cells[i>>2],16)&(1<<(i&3)))?0:1,p=(y+1)*(n+1)+x+1;
this.prefix[p]=invalid+this.prefix[p-1]+this.prefix[p-n-1]-this.prefix[p-n-2];
}
}
contains(rect:FrameRect){
const n=this.size;
// Include a one-cell halo: the visible outline is outside the capture
// boundary, and must not straddle an invalid lens/panel cell.
const x0=Math.floor(rect.x*n+1e-9)-1,y0=Math.floor(rect.y*n+1e-9)-1,x1=Math.ceil((rect.x+rect.width)*n-1e-9)+1,y1=Math.ceil((rect.y+rect.height)*n-1e-9)+1;
if(![rect.x,rect.y,rect.width,rect.height].every(Number.isFinite)||rect.width<=0||rect.height<=0||x0<0||y0<0||x1>n||y1>n)return false;
const stride=n+1,p=this.prefix;
return p[y1*stride+x1]-p[y0*stride+x1]-p[y1*stride+x0]+p[y0*stride+x0]===0;
}
fit(rect:FrameRect,ratio:number,minimum:number,previous?:FrameRect):FrameRect{
rect=fitMinimumFrame(rect,ratio,minimum);
if(this.contains(rect))return rect;
const centered=(cx:number,cy:number,width:number):FrameRect=>({x:cx-width/2,y:cy-width/ratio/2,width,height:width/ratio});
let cx=rect.x+rect.width/2,cy=rect.y+rect.height/2;
if(previous&&Math.abs(previous.width/previous.height-ratio)<1e-9&&previous.width>=minimum-1e-9&&this.contains(previous)){
if(Math.abs(rect.width-previous.width)<1e-9){
let lo=0,hi=1;for(let k=0;k<36;k++){const t=(lo+hi)/2,next={...rect,x:previous.x+(rect.x-previous.x)*t,y:previous.y+(rect.y-previous.y)*t};if(this.contains(next))lo=t;else hi=t;}
return {...rect,x:previous.x+(rect.x-previous.x)*lo,y:previous.y+(rect.y-previous.y)*lo};
}
}
if(!this.contains(centered(cx,cy,minimum))){
const targetX=cx,targetY=cy;
let distance=Infinity,found=false;
for(let y=0;y<this.size;y++)for(let x=0;x<this.size;x++){
const nx=(x+.5)/this.size,ny=(y+.5)/this.size,d=(nx-targetX)**2+(ny-targetY)**2;
if(d<distance&&this.contains(centered(nx,ny,minimum))){distance=d;found=true;cx=nx;cy=ny;}
}
if(!found)throw Error('菜单尺寸超过当前可录制范围');
}
let lo=minimum,hi=rect.width;
for(let k=0;k<36;k++){const width=(lo+hi)/2;if(this.contains(centered(cx,cy,width)))lo=width;else hi=width;}
return centered(cx,cy,lo);
}
reset(ratio:number,minimum:number){
return this.fit(resizeFrameCentered({x:0,y:0,width:1,height:1},Math.min(1,ratio),ratio,minimum),ratio,minimum);
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
export type FrameRect={x:number;y:number;width:number;height:number};
export type ReferenceView={tanHalfHorizontal:number;centerTanX:number;centerTanY:number;aspect:number};
export const clamp=(v:number,min:number,max:number)=>Math.max(min,Math.min(max,v));
export const frameRatios=[{label:'16:9',value:16/9},{label:'16:10',value:16/10},{label:'4:3',value:4/3},{label:'1:1',value:1},{label:'9:16',value:9/16}];
export const frameRatios=[{label:'16:9',value:16/9},{label:'16:10',value:16/10},{label:'4:3',value:4/3},{label:'3:4',value:3/4},{label:'1:1',value:1},{label:'9:16',value:9/16}];
export function framePresets(aspect:number){
return [1280,1920,2560,3840].map(edge=>aspect>=1?{width:edge,height:Math.round(edge/aspect/2)*2}:{width:Math.round(edge*aspect/2)*2,height:edge});
}
@@ -13,7 +13,7 @@ export function fitFrame(rect:FrameRect,aspect:number,referenceAspect=1,minWidth
}
export function frameFromSettings(settings:{horizontalFov:number|null;centerX:number;centerY:number},aspect:number,view:ReferenceView):FrameRect{
const maxWidth=Math.min(1,aspect/view.aspect);
const width=settings.horizontalFov===null?maxWidth:Math.tan(settings.horizontalFov*Math.PI/360)/view.tanHalfHorizontal;
const width=clamp(settings.horizontalFov===null?maxWidth:Math.tan(settings.horizontalFov*Math.PI/360)/view.tanHalfHorizontal,Math.min(maxWidth,Math.tan(5*Math.PI/180)/view.tanHalfHorizontal),maxWidth);
const height=width*view.aspect/aspect;
const x=.5+(Math.tan(settings.centerX*Math.PI/180)-view.centerTanX)/(2*view.tanHalfHorizontal)-width/2;
const y=.5+(Math.tan(settings.centerY*Math.PI/180)-view.centerTanY)*view.aspect/(2*view.tanHalfHorizontal)-height/2;
+8 -3
View File
@@ -1,12 +1,12 @@
import React,{useEffect,useState} from 'react';
import {IconCast,IconPlayerPlay,IconPlayerStop,IconArrowUpRight,IconDeviceVisionPro} from '@tabler/icons-react';
import {IconCast,IconPlayerPlay,IconPlayerStop,IconArrowUpRight,IconDeviceVisionPro,IconCrop} from '@tabler/icons-react';
import {api} from './api';
import {t} from './i18n';
import {Switch} from './switch';
import {Switch,SwitchRow} from './switch';
import type {CastSettings} from './casting';
import {RecordingButton,type RecordingStatus} from './cast-recording';
type Runtime={running:boolean;recording?:RecordingStatus;preview?:boolean;error?:string;receiverError?:string;watchAddresses?:string[]};
type Runtime={running:boolean;recording?:RecordingStatus;recordingRegion?:{editing:boolean;error?:string};preview?:boolean;error?:string;receiverError?:string;watchAddresses?:string[]};
export function CastQuick({config,refresh}:{config:CastSettings;refresh:()=>Promise<void>}){
const [runtime,setRuntime]=useState<Runtime|null>(null),[busy,setBusy]=useState(false),[error,setError]=useState('');
useEffect(()=>{let alive=true;const poll=()=>void api<Runtime>('cast.status').then(v=>{if(alive)setRuntime(v);}).catch(e=>{if(alive)setError(e.message);});poll();const timer=setInterval(poll,1000);return()=>{alive=false;clearInterval(timer);};},[]);
@@ -16,6 +16,11 @@ export function CastQuick({config,refresh}:{config:CastSettings;refresh:()=>Prom
<header><IconCast size={25} stroke={1.8}/><h1>{t('串流')}</h1><button aria-label={t('投屏设置')} onClick={()=>void run(()=>api('host.manager.open',{page:'casting'}))}><IconArrowUpRight size={22}/></button></header>
<div className="cast-quick-stream"><IconDeviceVisionPro size={40} stroke={1.5}/><h2>{running?t('正在串流'):t('头显画面')}</h2><p>{runtime?`${config.width} × ${config.height} / ${config.fps} FPS`:t('正在连接')}</p><div className="cast-stream-actions"><button className={running?'':'primary'} disabled={busy||!runtime} onClick={()=>void run(()=>api(running?'cast.stop':'cast.start'))}>{running?<IconPlayerStop size={21}/>:<IconPlayerPlay size={21}/>}<span>{running?t('停止串流'):t('开始串流')}</span></button><RecordingButton recording={runtime?.recording} disabled={busy||!runtime||!!runtime.preview} onClick={()=>void run(()=>api(runtime?.recording?.active?'cast.record.stop':'cast.record.start'))}/></div><p className="cast-record-path">{t('录制保存到 /home/steamos/Videos/framely')}</p></div>
{runtime?.recording?.error&&<p className="error" role="alert">{runtime.recording.error}</p>}
<div className="cast-quick-framing"><div className="cast-quick-framing-row"><button disabled={busy||!runtime||!!runtime.recording?.active||!!runtime.preview||!!runtime.recordingRegion?.editing||config.source!=='screen'||config.output!=='eye'} onClick={()=>void run(()=>api('cast.framing.open'))}><IconCrop size={21}/>{t('画幅调整')}</button>
<SwitchRow label={t('录制区域显示')} checked={config.recordingRegionVisible??false} disabled={busy||!runtime||(!(config.source==='screen'&&config.output==='eye')&&!config.recordingRegionVisible)} onChange={enabled=>void run(()=>api('cast.region.visible.save',{enabled}))}/>
</div>{(config.source!=='screen'||config.output!=='eye')&&<p className="cast-help">{t('录制区域显示仅支持屏幕采集的单眼画面')}</p>}
{runtime?.recordingRegion?.error&&<p className="error" role="alert">{runtime.recordingRegion.error}</p>}
</div>
{running&&runtime?.watchAddresses?.[0]&&<div className="cast-quick-address"><span>{t('网页观看')}</span><code>{runtime.watchAddresses[0]}</code></div>}
<div className="cast-quick-receive"><h2>{t('接收投屏')}</h2>{(['airplay','dlna'] as const).map(protocol=><label key={protocol}><span>{protocol==='airplay'?'AirPlay':'DLNA'}</span><Switch label={protocol==='airplay'?'AirPlay':'DLNA'} checked={config[protocol]} disabled={busy} onChange={value=>void run(()=>api('cast.settings.save',{...config,[protocol]:value}))}/></label>)}</div>
{(error||runtime?.error||runtime?.receiverError)&&<p className="error" role="alert">{error||runtime?.error||runtime?.receiverError}</p>}
+15
View File
@@ -0,0 +1,15 @@
import {frameFromSettings,settingsFromFrame,clamp,type ReferenceView,type FrameRect} from './cast-framing';
import type {CastSettings} from './casting';
import {fitMinimumFrame,minimumFrameWidth,type MinimumFrame} from './cast-framing-gesture';
// Canonicalize the displayed box BEFORE saving; ratio changes must not leave
// a clamped visual rectangle paired with different capture parameters.
export type FrameLimiter=(rect:FrameRect,ratio:number,minimum:number,previous?:FrameRect)=>FrameRect;
export function normalizeRegion(settings:CastSettings,reference:ReferenceView,minimum:MinimumFrame={width:0,height:0},limit?:FrameLimiter,previous?:CastSettings):CastSettings{
const bounded={...settings,horizontalFov:clamp(settings.horizontalFov??90,10,150),centerX:clamp(settings.centerX,-60,60),centerY:clamp(settings.centerY,-60,60)};
const ratio=settings.width/settings.height/reference.aspect;
const minWidth=minimumFrameWidth(ratio,minimum,Math.tan(5*Math.PI/180)/reference.tanHalfHorizontal);
let rect=fitMinimumFrame(frameFromSettings(bounded,settings.width/settings.height,reference),ratio,minWidth);
if(limit)rect=limit(rect,ratio,minWidth,previous?frameFromSettings(previous,previous.width/previous.height,reference):undefined);
const next=settingsFromFrame(rect,reference);
return {...bounded,...next,centerX:clamp(next.centerX,-60,60),centerY:clamp(next.centerY,-60,60)};
}
+11
View File
@@ -0,0 +1,11 @@
html.cast-framing-view,html.cast-framing-view body,html.cast-framing-view #root{background:transparent!important;width:100%;height:100%;overflow:hidden;margin:0;padding:0}
.cast-native-framing{position:fixed;inset:0;background:transparent;isolation:isolate}
.cast-native-framing .cast-framing-stage{position:absolute;inset:0;width:100%;height:100%;max-width:none;border-radius:0;background:transparent}
.cast-native-framing .cast-crop-box{border:0;outline:2px solid #c1e0ef;outline-offset:0;background:rgb(80 110 130/.035);box-shadow:0 0 0 2000px rgb(0 0 0/.10);border-radius:0}
.cast-native-framing .cast-crop-grid{opacity:.14}
.cast-native-framing .cast-zoom-controls,.cast-native-framing .cast-help{display:none}
.cast-native-framing-controls{position:absolute;left:50%;top:50%;transform:translate(-50%,-50%) scale(var(--framing-menu-scale,1));width:340px;padding:24px;border:1px solid rgb(255 255 255/.22);border-radius:14px;background:rgb(20 24 28/.78);color:#f4f7fa;box-shadow:0 8px 32px rgb(0 0 0/.16)}
.cast-native-framing-controls .cast-framing-reset{width:100%;margin:12px 0 0;min-height:44px}
.cast-native-framing-controls h1{font-size:23px;margin:0 0 10px}.cast-native-framing-controls p{font-size:14px;line-height:1.6;color:#c2ced6}
.cast-native-framing-field{display:flex;align-items:center;justify-content:space-between;gap:14px;font-size:15px;margin:16px 0}.cast-native-framing-field .framely-select{width:175px}.cast-native-framing-field .framely-select>button{margin:0;background:#252c32}.cast-native-framing-field [role=listbox]{z-index:20!important;background:#252c32!important}.cast-native-framing-field [role=option]{margin:0;min-height:44px}
.cast-native-framing-controls footer{display:flex;gap:12px;margin-top:20px}.cast-native-framing-controls footer button{flex:1}
+79
View File
@@ -0,0 +1,79 @@
import React,{useEffect,useMemo,useRef,useState} from 'react';
import {api} from './api';
import {t,configureLanguage,type LanguagePack} from './i18n';
import {Select} from './localized-select';
import type {CastSettings} from './casting';
import {FramingEditor} from './cast-framing-editor';
import {framePresets,frameRatios,settingsFromFrame,type ReferenceView} from './cast-framing';
import {normalizeRegion} from './cast-region';
import {FramingLimits,framingMenuScale,type VisibilityGrid} from './cast-framing-limits';
import {minimumFrameWidth} from './cast-framing-gesture';
import './casting.css';
import './cast-spatial-framing.css';
type Geometry={id:string;settings:CastSettings;referenceTan:number;view:ReferenceView&{visibility:VisibilityGrid;eye:0|1}};
export async function loadSpatialFraming(request:typeof api=api){
const [geometry,status,language]=await Promise.all([request<Geometry>('cast.framing.get'),request<{database:{language?:string}}>('status'),request<{packs?:LanguagePack[]}>('language.list')]);
configureLanguage(status.database.language??'auto',language.packs??[]);
return geometry;
}
export function SpatialFraming(){
const [geometry,setGeometry]=useState<Geometry|null>(null),[draft,setDraft]=useState<CastSettings|null>(null),[error,setError]=useState(''),[busy,setBusy]=useState(false),[closed,setClosed]=useState(false);
const finishing=useRef(false),finished=useRef(false);
const root=useRef<HTMLElement>(null),menu=useRef<HTMLElement>(null),[minimum,setMinimum]=useState({width:0,height:0,referenceTan:0});
useEffect(()=>{const measure=()=>{const area=root.current?.getBoundingClientRect(),panel=menu.current?.getBoundingClientRect();if(area?.width&&area.height&&panel)setMinimum({width:(panel.width+32)/area.width,height:(panel.height+32)/area.height,referenceTan:geometry?.referenceTan??1});};measure();const observer=typeof ResizeObserver==='undefined'?null:new ResizeObserver(measure);if(menu.current)observer?.observe(menu.current);if(root.current)observer?.observe(root.current);window.addEventListener('resize',measure);return()=>{observer?.disconnect();window.removeEventListener('resize',measure);};},[geometry?.referenceTan]);
const reference:ReferenceView={tanHalfHorizontal:geometry?.referenceTan??1,centerTanX:0,centerTanY:0,aspect:1};
const limits=useMemo(()=>geometry?new FramingLimits(geometry.view.visibility,geometry.referenceTan):null,[geometry]);
const limit=useMemo(()=>limits?.fit.bind(limits),[limits]);
useEffect(()=>{let alive=true;document.documentElement.classList.add('cast-framing-view');
void loadSpatialFraming().then(g=>{if(!alive)return;setGeometry(g);setDraft({...g.settings,horizontalFov:2*Math.atan(g.view.tanHalfHorizontal)*180/Math.PI,centerX:Math.atan(g.view.centerTanX)*180/Math.PI,centerY:Math.atan(g.view.centerTanY)*180/Math.PI});}).catch(e=>{if(alive)setError(String(e.message??e));});
return()=>{alive=false;document.documentElement.classList.remove('cast-framing-view');};
},[]);
function change(patch:Partial<CastSettings>){if(!draft||finishing.current||finished.current||minimum.referenceTan!==geometry?.referenceTan)return false;try{setDraft(normalizeRegion({...draft,...patch},reference,minimum,limit,draft));return true;}catch(e){setError(t((e as Error).message));return false;}}
function resetSize(){if(!draft||!limits)return;try{const ratio=draft.width/draft.height,minWidth=minimumFrameWidth(ratio,minimum,Math.tan(5*Math.PI/180)/reference.tanHalfHorizontal);if(change(settingsFromFrame(limits.reset(ratio,minWidth),reference)))setError('');}catch(e){setError(t((e as Error).message));}}
function closeEditor(){finished.current=true;setClosed(true);setGeometry(null);setDraft(null);}
async function finish(apply:boolean){if(!geometry||finishing.current||finished.current)return;finishing.current=true;setBusy(true);setError('');try{
await api(apply?'cast.framing.apply':'cast.framing.cancel',apply&&draft?{id:geometry.id,expectedEye:geometry.view.eye,...normalizeRegion(draft,reference,minimum,limit,draft)}:{id:geometry.id});
// The native host closes asynchronously. Never leave the completed editor
// interactive while it is waiting for that close command or status update.
closeEditor();
}catch(e){setError(String((e as Error).message??e));
// Saving can succeed even if restarting the stream fails afterwards. The
// backend has already closed that editor, so do not revive its stale id.
try{const latest=await api<Geometry>('cast.framing.get');if(latest.id!==geometry.id)closeEditor();}
catch(stateError){if((stateError as Error).message===t('画幅调整窗口已关闭'))closeEditor();}
}finally{finishing.current=false;setBusy(false);}}
useEffect(()=>{const close=()=>void finish(false);window.addEventListener('framely.back',close);return()=>window.removeEventListener('framely.back',close);},[geometry,busy]);
const aspect=draft?draft.width/draft.height:4/3;
const ready=!!geometry&&minimum.referenceTan===geometry.referenceTan;
useEffect(()=>{
if(!ready||!geometry||!draft||busy||closed||finishing.current||finished.current)return;
const controller=new AbortController(),params={id:geometry.id,centerX:draft.centerX,centerY:draft.centerY};
// Draft centers commit after a drag ends. Update only the native depth probe,
// not capture settings, and coalesce rapid keyboard/ratio/reset adjustments.
const timer=setTimeout(()=>{
if(finishing.current||finished.current)return;
void api('cast.framing.reference',params,controller.signal).catch(()=>{});
},100);
// This is presentation-only, so aborted/stale updates must not overwrite a
// real save error or revive an editor whose native window is closing.
return()=>{clearTimeout(timer);controller.abort();};
},[ready,geometry?.id,draft?.centerX,draft?.centerY,busy,closed]);
if(closed)return null;
return <main ref={root} className="cast-native-framing" aria-label={t('画幅调整')} inert={busy}>
{draft&&geometry&&ready&&<FramingEditor spatial limit={limit} minimumSize={minimum} settings={draft} view={reference} onChange={change} onError={message=>setError(t(message))}/>}
<section ref={menu} className="cast-native-framing-controls" style={{'--framing-menu-scale':framingMenuScale(reference.tanHalfHorizontal)} as React.CSSProperties}>
<h1>{t('画幅调整')}</h1><p>{t('框内即录制区域,拖动移动,拖动四角等比例缩放。')}</p>
<div className="cast-native-framing-field"><span>{t('画面比例')}</span><Select label={t('画面比例')} disabled={busy||!draft} value={String(frameRatios.find(r=>Math.abs(r.value-aspect)<.004)?.value??aspect)} onChange={value=>{const ratio=Number(value),edge=Math.max(draft!.width,draft!.height);change(ratio>=1?{width:edge,height:Math.round(edge/ratio/2)*2}:{width:Math.round(edge*ratio/2)*2,height:edge});}} options={[
...(!frameRatios.some(r=>Math.abs(r.value-aspect)<.004)?[{value:String(aspect),label:t('自定义')}]:[]),
...frameRatios.map(r=>({value:String(r.value),label:r.label}))
]}/></div>
<div className="cast-native-framing-field"><span>{t('录制分辨率')}</span><Select label={t('录制分辨率')} disabled={busy||!draft} value={draft?`${draft.width}x${draft.height}`:''} onChange={value=>{const [width,height]=value.split('x').map(Number);change({width,height});}} options={[
...(draft&&!framePresets(aspect).some(s=>s.width===draft.width&&s.height===draft.height)?[{value:`${draft.width}x${draft.height}`,label:`${draft.width} × ${draft.height}`}]:[]),
...framePresets(aspect).filter(s=>s.width<=4096&&s.height<=4096&&s.width*s.height<=8847360).map(s=>({value:`${s.width}x${s.height}`,label:`${s.width} × ${s.height}`}))
]}/></div>
{error&&<p role="alert" className="error">{error}</p>}
<button className="cast-framing-reset" title={t('保留比例和分辨率,恢复居中有效画幅')} disabled={busy||!draft||!limits||!ready} onClick={resetSize}>{t('重置大小')}</button>
<footer><button disabled={busy||!geometry} onClick={()=>void finish(false)}>{t('取消')}</button><button className="primary" disabled={busy||!draft||!ready} onClick={()=>void finish(true)}>{t('应用取景')}</button></footer>
</section>
</main>;
}
+1
View File
@@ -17,3 +17,4 @@
.cast-stream-bar>.cast-stream-actions span{color:inherit;font-size:inherit;display:initial}.cast-quick-stream .cast-record-path{font-size:12px;margin:16px 0 0}
/* Viewer: one restrained floating toolbar; the video always remains complete. */
.cast-watch video{object-fit:contain!important}.cast-watch-top{padding-bottom:40px;gap:12px;background:linear-gradient(rgb(0 0 0/.55),transparent)}.cast-watch-title{font-size:14px;font-weight:500;color:#e9e9e9}.cast-watch-connection{font-size:12px;color:#c6c6c6}.cast-watch-bottom{padding-top:32px;background:transparent;display:flex;justify-content:center}.cast-watch-controls{width:auto;max-width:100%;padding:6px 12px;gap:8px;background:rgb(18 18 18/.88);border:1px solid rgb(255 255 255/.12);border-radius:999px;box-shadow:0 8px 28px #0005;backdrop-filter:blur(16px);-webkit-backdrop-filter:blur(16px)}.cast-watch .cast-watch-icon{width:40px;height:40px;min-height:40px;border:0;border-radius:50%;padding:8px;background:transparent;color:#ededed}.cast-watch .cast-watch-icon:hover{background:#ffffff12}.cast-watch-top .cast-watch-icon{background:transparent}.cast-watch-top .cast-watch-icon:hover{background:#ffffff12}.cast-watch-sound{gap:6px}.cast-watch-sound input{width:80px}.cast-watch-live{margin:0 12px 0 8px;font-size:11px;color:#bcbcbc;letter-spacing:.02em}.cast-watch-tools{gap:4px;padding-left:8px;border-left:1px solid #ffffff1f}.cast-watch .cast-range{--blue:#e7e7e7}.cast-watch .cast-range::-webkit-slider-thumb{width:20px;height:12px;margin-top:-3px}.cast-watch .cast-range::-moz-range-thumb{width:20px;height:12px}.cast-watch-message h1{font-size:22px;font-weight:500}.cast-watch-message p{font-size:13px}.cast-watch-message button{border-radius:999px;min-height:40px;font-size:13px;padding:10px 18px}@media(max-width:767px){.cast-watch-bottom{padding-bottom:calc(16px + env(safe-area-inset-bottom))}.cast-watch-controls{gap:4px;padding:5px 9px}.cast-watch-top{padding-top:calc(12px + env(safe-area-inset-top));padding-bottom:32px}.cast-watch-live{display:inline;margin:0 10px 0 4px}.cast-watch-title{font-size:13px}.cast-watch-tools{padding-left:6px;gap:2px}}@media(prefers-reduced-transparency:reduce){.cast-watch-controls{background:#181818;backdrop-filter:none;-webkit-backdrop-filter:none}}
.cast-quick-framing{padding:16px 0;border-top:1px solid var(--line)}.cast-quick-framing-row{display:flex;align-items:center;justify-content:space-between;gap:16px}.cast-quick-framing-row>button{display:flex;gap:10px;align-items:center;margin:0;flex:none}.cast-quick-framing-row .switch-row{width:auto;min-width:0;margin:0;gap:10px;padding:0;flex:1}.cast-quick-framing-row .switch-row b{font-size:16px}.cast-quick-framing-row .ui-switch{width:60px;min-width:60px;flex:none;margin:0}
+2 -2
View File
@@ -11,7 +11,7 @@ import {framePresets,frameRatios,type ReferenceView} from './cast-framing';
import './casting.css';
import {RecordingButton,type RecordingStatus} from './cast-recording';
export type CastSettings={source:'screen'|'steamVR';codec:'h264'|'h265';output:'eye'|'raw';eye:'steamVR'|'left'|'right';width:number;height:number;fps:number;bitrateMbps:number;horizontalFov:number|null;centerX:number;centerY:number;systemAudio:boolean;microphone:boolean;airplay:boolean;dlna:boolean;receiverName:string};
export type CastSettings={source:'screen'|'steamVR';codec:'h264'|'h265';output:'eye'|'raw';eye:'steamVR'|'left'|'right';width:number;height:number;fps:number;bitrateMbps:number;horizontalFov:number|null;centerX:number;centerY:number;systemAudio:boolean;microphone:boolean;airplay:boolean;dlna:boolean;receiverName:string;recordingRegionVisible?:boolean};
type CastSession={id:string;protocol:string;accepted:boolean;width?:number;height?:number};
type Device={id:string;name:string};
type Runtime={running:boolean;recording?:RecordingStatus;error:string|null;dominantEye:number|null;watchPath:string|null;watchAccessEnabled?:boolean;elapsedSeconds:number|null;preview?:boolean;view?:ReferenceView|null;receiverError?:string|null;sessions?:CastSession[];watchAddresses?:string[]};
@@ -24,7 +24,7 @@ export function Casting({config,refresh,network,page='casting'}:{config:CastSett
const [streamInfo,setStreamInfo]=useState<CastStreamInfo|null>(null);
const [draft,setDraft]=useState(config),[runtime,setRuntime]=useState<Runtime|null>(null),[pending,setPending]=useState(''),[error,setError]=useState(''),[saved,setSaved]=useState(false),[epoch,setEpoch]=useState(0),[devices,setDevices]=useState<Device[]>([]),[sending,setSending]=useState<string>(''),[copied,setCopied]=useState(false),[editing,setEditing]=useState(false),[searched,setSearched]=useState(false);
const beforeFraming=useRef<Pick<CastSettings,'horizontalFov'|'centerX'|'centerY'>|null>(null),editorActive=useRef(false),deviceScan=useRef(0),alive=useRef(true);
const captureSettings=({airplay,dlna,receiverName,...capture}:CastSettings)=>capture;
const captureSettings=({airplay,dlna,receiverName,recordingRegionVisible,...capture}:CastSettings)=>capture;
const captureDirty=JSON.stringify(captureSettings(draft))!==JSON.stringify(captureSettings(config)),dirty=JSON.stringify(draft)!==JSON.stringify(config),busy=!!pending,screen=draft.source==='screen',eye=screen&&draft.output==='eye';
const aspect=screen&&draft.output==='raw'?2:draft.width/draft.height;
const ratio=frameRatios.find(r=>Math.abs(r.value-aspect)<.004)?.label;
+2
View File
@@ -1,4 +1,5 @@
import {CastQuick} from './cast-quick';
import {SpatialFraming} from './cast-spatial-framing';
import {CastPlayer} from './cast-player';
import {Casting,type CastSettings,type CastPage} from './casting';
import {CastViewer} from './cast-viewer';
@@ -38,6 +39,7 @@ const managerMode=location.pathname==='/manager';
const empty:Status={version:'0.4.0',database:{plugins:{},sources:[],safeMode:false},notifications:[],running:[]};
function App(){
if(location.pathname==='/cast/framing')return <SpatialFraming/>;
if(location.pathname==='/apk/permission')return <ApkPermission/>;
if(location.pathname==='/cast/watch')return <CastViewer/>;
if(location.pathname.startsWith('/cast/window/'))return <CastPlayer/>;