diff --git a/README.md b/README.md index 99341c3..e14e232 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,125 @@ -This is a dev branch for vr support in Mocktail. +# Mocktail VR (experimental) -idk if I can do this. +Build this checkout, start WiVRn or SteamVR/ALVR, then connect your headset before launching: -But it will be fun :) +```sh +git submodule update --init --recursive +cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DMOCKTAIL_ENABLE_VR=ON +cmake --build build -j4 +./build/mocktail --vr +``` + +## Automatic runtime selection + +Normally just run `./build/mocktail --vr`. The default `auto` mode detects a +running SteamVR server (also used by ALVR) or a listening WiVRn IPC socket for +the current desktop user. A stale socket file does not count as a running +server. If exactly one server is detected, its runtime takes priority over an +old registration for the other server. If both or neither are detected, the +registered OpenXR runtime has priority. + +SteamVR is also found through `openvr/openvrpaths.vrpath`, standard Steam +locations, and the running server's executable path. WiVRn development builds +are discovered near the running server executable; a matching registered +WiVRn manifest is preferred, including a Flatpak registration. The selected +path and the reason for selecting it are printed at startup. + +Explicit `XR_RUNTIME_JSON` and `MOCKTAIL_VR_RUNTIME` overrides remain respected. +To return to automatic selection after manually choosing a runtime: + +```sh +unset XR_RUNTIME_JSON MOCKTAIL_VR_RUNTIME +./build/mocktail --vr +``` + +Detection does not start servers, connect a headset, or modify system runtime +registration. It observes the current process/user namespace; container and +host runtimes still need correctly shared libraries and IPC access. + +## WiVRn + +Start WiVRn and connect its headset client first. Run WiVRn and Mocktail as the +same desktop user, with the same runtime environment; do not start Mocktail +from a root shell while WiVRn runs in your user session. For containers/chroots, +the host runtime library and IPC socket must both be accessible inside. + +```sh +MOCKTAIL_VR_RUNTIME=wivrn ./build/mocktail --vr +``` + +If WiVRn disconnects and exits, reconnect/restart it before retrying Mocktail. +`Connection refused` on `wivrn/comp_ipc` means the selected server is not +accepting the connection. Installing `dbus-launch` alone does not fix that. + +## ALVR through SteamVR + +Install SteamVR and ALVR, register the ALVR driver using its dashboard, then +connect the headset and confirm SteamVR sees it. Follow the upstream +[ALVR Linux instructions](https://github.com/alvr-org/ALVR/wiki/Linux-Troubleshooting), +including the SteamVR launch-options workaround when required. + +```sh +env -u XR_RUNTIME_JSON MOCKTAIL_VR_RUNTIME=alvr \ + MOCKTAIL_GRAPHICS_BACKEND=direct-vulkan ./build/mocktail --vr +``` + +The command clears an inherited WiVRn manifest override. `steamvr` is an alias +for `alvr`. The selection uses SteamVR's OpenXR runtime; +ALVR handles the streaming. If automatic discovery cannot find a custom +installation, specify its actual manifest: + +```sh +XR_RUNTIME_JSON="/path/to/SteamVR/steamxr_linux64.json" ./build/mocktail --vr +``` + +`XR_RUNTIME_JSON` always wins. `MOCKTAIL_VR_RUNTIME=system` delegates selection +to the OpenXR loader. The default `auto` uses the server detection described above, then registered +and installed runtimes. Selection affects only this process. + +## OpenGL ES + +The `opengl`/`system` renderer uses the guest's real EGL/OpenGL ES context: + +```sh +MOCKTAIL_GRAPHICS_BACKEND=opengl ./build/mocktail --vr +``` + +The runtime must provide `XR_KHR_opengl_es_enable` and `XR_MNDX_egl_enable`, and +support sharing images with the guest EGL device. Both eye copies stay on the +GPU; no Python viewer or CPU readback is required. Camera tracking uses the +same OpenXR frame/pose pipeline as Vulkan. If the runtime cannot support this +EGL binding, use `MOCKTAIL_GRAPHICS_BACKEND=direct-vulkan`; the ALVR command +above deliberately selects Vulkan. + +## Current validation and limitations + +- Native VR camera/eye hooks are verified for Roblox build **2998**, Build ID + `ade08266c67aee88ec9c1d00902150e1684dad3a`. Approval of a newer payload for + ordinary rendering does **not** establish VR compatibility. Do not bypass + the Build-ID checks to run the old VR hooks on build 3092. +- GLES framebuffer copying, distinct eyes, state restoration, resource + invalidation, and an EGL OpenXR session with session-loss recovery have been + tested. A local unauthenticated Roblox run on hardware GLES submitted over + 240 projection frames through a simulated Monado headset. +- This host's software GLES/Vulkan combination could create XR texture names + but not usable framebuffer attachments. The copy failure is reported and + XR output stops; this is not counted as a successful rendering test. +- The local Monado test runtime reports `u_graphics_buffer_unref: Bad file + descriptor (fd: 0)` during GLES teardown, including successful hardware + runs. This remains an integration caveat; clean headset teardown is not + claimed from those simulator runs. During auto-selection validation, one + run also failed a later EGL session recreation with + `vkCreateFence: VK_ERROR_OUT_OF_HOST_MEMORY`; a fresh repeat passed. + Runtime selection and initial frame submission succeeded in both runs. +- Real headset acceptance through WiVRn and ALVR/SteamVR is still pending. + Controller input and canted eye orientations are not implemented. +- Use `./build/mocktail --no-vr` for ordinary rendering. No helper script is + needed for normal VR startup. + +The GLES OpenXR integration test is opt-in and needs an already connected +runtime (or a separately prepared simulator). Run from a graphical desktop: + +```sh +MOCKTAIL_TEST_XR_GLES=1 ./build/tests/gles_vr_transport_test \ + --gtest_filter='*OpenXrEglSession*' +``` diff --git a/cmake/MocktailVR.cmake b/cmake/MocktailVR.cmake index f71f7cb..f46ee95 100644 --- a/cmake/MocktailVR.cmake +++ b/cmake/MocktailVR.cmake @@ -14,7 +14,7 @@ add_library(mocktail_vr STATIC add_library(Mocktail::VR ALIAS mocktail_vr) target_include_directories(mocktail_vr PUBLIC "${CMAKE_SOURCE_DIR}/include") target_include_directories(mocktail_vr PRIVATE "${CMAKE_SOURCE_DIR}/src") -target_link_libraries(mocktail_vr PUBLIC Vulkan::Headers) +target_link_libraries(mocktail_vr PUBLIC Vulkan::Headers PRIVATE nlohmann_json::nlohmann_json) # shm_open for the desktop eye mirror lives in librt on glibc. find_library(MOCKTAIL_RT_LIBRARY rt) if(MOCKTAIL_RT_LIBRARY) @@ -47,12 +47,14 @@ if(MOCKTAIL_ENABLE_VR) endfunction() mocktail_add_openxr_sdk() target_sources(mocktail_vr PRIVATE src/vr/openxr_probe.cc src/vr/openxr_preview.cc - src/vr/openxr_backend.cc) + src/vr/openxr_backend.cc src/vr/gles_transport.cc) # Error unwinding is confined to these native session implementations; their # public entry points catch errors before returning to the guest runtime. set_source_files_properties(src/vr/openxr_preview.cc src/vr/openxr_backend.cc PROPERTIES COMPILE_OPTIONS -fexceptions) target_compile_definitions(mocktail_vr PRIVATE XR_USE_GRAPHICS_API_VULKAN) + set_source_files_properties(src/vr/openxr_backend.cc PROPERTIES COMPILE_DEFINITIONS "XR_USE_GRAPHICS_API_OPENGL_ES;XR_USE_PLATFORM_EGL") + target_include_directories(mocktail_vr PRIVATE ${MOCKTAIL_EGL_INCLUDE_DIR} ${MOCKTAIL_GLES3_INCLUDE_DIR}) target_link_libraries(mocktail_vr PRIVATE OpenXR::openxr_loader Vulkan::Headers ${CMAKE_DL_LIBS}) else() target_sources(mocktail_vr PRIVATE src/vr/openxr_disabled.cc) diff --git a/include/mocktail/vr/openxr_backend.h b/include/mocktail/vr/openxr_backend.h index bed4428..9d0fbb8 100644 --- a/include/mocktail/vr/openxr_backend.h +++ b/include/mocktail/vr/openxr_backend.h @@ -25,10 +25,21 @@ enum class VrBackendMode { kXrOutput, }; -// Explicit XR_RUNTIME_JSON wins. Otherwise prefer a native WiVRn manifest, -// then let the OpenXR loader use the registered runtime (including Flatpak WiVRn). +// Explicit XR_RUNTIME_JSON wins. Runtime preference can select SteamVR (ALVR), +// WiVRn, or the registered OpenXR runtime without changing system registration. std::string SelectVrRuntimeManifest(const std::string& explicit_manifest, const std::vector& candidates); +std::vector VrRuntimeManifestCandidates( + const std::string &preference, const std::string &home, + const std::string &config_home, const std::string &config_dirs); +struct VrRuntimeDiscovery { + std::vector candidates; + std::string reason; +}; +VrRuntimeDiscovery DiscoverVrRuntime( + const std::string& preference, const std::string& home, + const std::string& config_home, const std::string& config_dirs, + const std::string& runtime_dir, const std::string& proc_root = "/proc"); std::string VrRuntimeUnavailableHint(bool user_selected, const std::string& manifest); VrBackendMode ResolveVrBackendMode(bool vr_enabled); const char* VrBackendModeName(VrBackendMode mode); @@ -58,6 +69,10 @@ struct ScriptedPoseSample { // frame N+1 and publishing the scripted/runtime head pose that the device // bridge injects into Roblox before its eye passes run. // +// EGL/OpenGL ES uses XR_MNDX_egl_enable on the guest SDL context and GPU +// framebuffer blits into GLES swapchain textures. It shares the same frame +// scheduling, tracking, pose application and session recovery below. +// // Eye image provenance is recorded in the adapter during the guest eye // initializer (owner = DebugDeviceVR object) and bound to an eye index only // through the real per-frame eye-getter request plus the actual render-pass @@ -67,6 +82,9 @@ struct ScriptedPoseSample { // All public entry points are exception-free at their boundary; the // implementation is compiled with exceptions like openxr_preview.cc and // translates failures into Status/log outcomes. +class GlesTransport; +enum class VrGraphicsApi { kVulkan, kOpenGles }; + class OpenXrBackend final { public: OpenXrBackend() = default; @@ -75,7 +93,14 @@ class OpenXrBackend final { OpenXrBackend& operator=(const OpenXrBackend&) = delete; // Claims the process-wide backend slot and creates XrInstance/system. - Status Arm(); + Status Arm(VrGraphicsApi graphics_api = VrGraphicsApi::kVulkan); + // EGL context must be current on the calling thread. Resolver bypasses guest + // hooks. + Status AttachGlesContext(void *display, void *config, void *context, + void *egl_get_proc, void *(*resolve)(const char *)); + void DetachGlesContext(void *context); + void NoteGlesPresent(); + void *WrapGlesProcAddress(const char *name, void *raw); // Idempotent full teardown (XR resources first, then the instance). void Disarm(); bool armed() const { return armed_.load(std::memory_order_acquire); } @@ -161,6 +186,7 @@ class OpenXrBackend final { struct EyeSwapchain { void* swapchain = nullptr; // XrSwapchain (opaque to keep the header XR-free) std::vector images; + std::vector gl_images; std::uint32_t acquired_index = 0; bool image_acquired = false; std::uint32_t width = 0; @@ -172,6 +198,9 @@ class OpenXrBackend final { VkDevice device, const VkDeviceCreateInfo* create_info, std::string* error); + bool InitializeGlesSessionLocked(std::string *error); + bool InitializeSessionResourcesLocked(std::string *error); + bool CopyGlesEyesIntoSwapchains(); void DisarmLocked(); void* ResolveFunction(const char* name) const; void SetupDeviceAfterCreationLocked(VkPhysicalDevice physical_device, @@ -201,6 +230,11 @@ class OpenXrBackend final { void EnsureMirrorForExtent(std::uint32_t width, std::uint32_t height); MirrorVkProcs MirrorProcs() const; + VrGraphicsApi graphics_api_ = VrGraphicsApi::kVulkan; + GlesTransport *gles_ = nullptr; + std::uint64_t gl_min_version_ = 0; + std::uint64_t gl_max_version_ = 0; + mutable std::mutex mutex_; std::atomic armed_{false}; std::atomic recording_{false}; diff --git a/src/main.cc b/src/main.cc index 1e7a23c..ea2a2a3 100644 --- a/src/main.cc +++ b/src/main.cc @@ -351,7 +351,8 @@ int main(int argc, char* argv[]) { "debug device (exact Build ID 2998 only)"; if (vr_backend_mode == mocktail::vr::VrBackendMode::kXrOutput) { std::cerr << "; eye images are submitted to the active OpenXR runtime " - "as a projection layer. Start WiVRn and connect the headset " + "as a projection layer. Start WiVRn or SteamVR/ALVR and " + "connect the headset " "before launching.\n"; } else { std::cerr << "; OpenXR output is disabled in this diagnostic mode " @@ -894,14 +895,20 @@ int main(int argc, char* argv[]) { mocktail::vr::VrBackendMode::kXrOutput) { // Fail closed and loudly when no OpenXR runtime is reachable; the // native-stereo diagnostic mode must be requested explicitly. - const mocktail::Status xr_status = xr_backend.Arm(); + const mocktail::Status xr_status = + xr_backend.Arm(runtime_config.config.graphics_backend() == + mocktail::runtime::GraphicsBackend::kVulkan + ? mocktail::vr::VrGraphicsApi::kVulkan + : mocktail::vr::VrGraphicsApi::kOpenGles); if (!xr_status.ok()) { - std::cerr << "[FATAL] Cannot arm the OpenXR backend for --vr: " - << xr_status.message() << '\n' - << " Start WiVRn, connect your headset, then run mocktail -vr again.\n" - << " For a custom runtime set XR_RUNTIME_JSON to its manifest.\n" - << " Set MOCKTAIL_VR_BACKEND=native for the " - "native-stereo diagnostic mode without a runtime.\n"; + std::cerr + << "[FATAL] Cannot arm the OpenXR backend for --vr: " + << xr_status.message() << '\n' + << " Start WiVRn or SteamVR/ALVR, connect your headset, then " + "run mocktail -vr again.\n" + << " For a custom runtime set XR_RUNTIME_JSON to its manifest.\n" + << " Set MOCKTAIL_VR_BACKEND=native for the " + "native-stereo diagnostic mode without a runtime.\n"; return EXIT_FAILURE; } } diff --git a/src/vr/gles_transport.cc b/src/vr/gles_transport.cc new file mode 100644 index 0000000..f083c51 --- /dev/null +++ b/src/vr/gles_transport.cc @@ -0,0 +1,395 @@ +#include "vr/gles_transport.h" +#include "mocktail/vr/openxr_backend.h" + +#include +#include +#include +#include + +extern "C" void *mocktail_vr_initializing_object(); +extern "C" bool mocktail_vr_current_eye(void **, int *, void **); +extern "C" void mocktail_vr_consume_eye(); + +namespace mocktail::vr { +namespace { +// The window owns a single EGL context. Function pointers are loaded from +// that exact SDL EGL/GLES implementation, never from libGL or the guest shim. +#define GL_PROCS(X) \ + X(GetIntegerv, PFNGLGETINTEGERVPROC) \ + X(IsEnabled, PFNGLISENABLEDPROC) \ + X(Enable, PFNGLENABLEPROC) \ + X(Disable, PFNGLDISABLEPROC) \ + X(GenFramebuffers, PFNGLGENFRAMEBUFFERSPROC) X(DeleteFramebuffers, \ + PFNGLDELETEFRAMEBUFFERSPROC) \ + X(BindFramebuffer, PFNGLBINDFRAMEBUFFERPROC) \ + X(CheckFramebufferStatus, PFNGLCHECKFRAMEBUFFERSTATUSPROC) X( \ + GetFramebufferAttachmentParameteriv, \ + PFNGLGETFRAMEBUFFERATTACHMENTPARAMETERIVPROC) \ + X(FramebufferTexture2D, PFNGLFRAMEBUFFERTEXTURE2DPROC) X( \ + FramebufferTextureLayer, PFNGLFRAMEBUFFERTEXTURELAYERPROC) \ + X(FramebufferRenderbuffer, PFNGLFRAMEBUFFERRENDERBUFFERPROC) \ + X(TexImage2D, PFNGLTEXIMAGE2DPROC) X( \ + TexStorage2D, PFNGLTEXSTORAGE2DPROC) \ + X(RenderbufferStorage, PFNGLRENDERBUFFERSTORAGEPROC) \ + X(RenderbufferStorageMultisample, \ + PFNGLRENDERBUFFERSTORAGEMULTISAMPLEPROC) \ + X(DeleteTextures, PFNGLDELETETEXTURESPROC) \ + X(DeleteRenderbuffers, \ + PFNGLDELETERENDERBUFFERSPROC) \ + X(BlitFramebuffer, \ + PFNGLBLITFRAMEBUFFERPROC) \ + X(Flush, PFNGLFLUSHPROC) +struct Procs { +#define DECLARE(name, type) type name = nullptr; + GL_PROCS(DECLARE) +#undef DECLARE +} gl; +thread_local GlesTransport *current = nullptr; +GlesTransport *Current() { + return current && current->IsCurrent() ? current : nullptr; +} +void GL_APIENTRY GenFramebuffers(GLsizei n, GLuint *ids) { + gl.GenFramebuffers(n, ids); + if (auto *t = Current()) + t->Generated(n, ids); +} +void GL_APIENTRY DeleteFramebuffers(GLsizei n, const GLuint *ids) { + if (auto *t = Current()) + t->Deleted(n, ids); + gl.DeleteFramebuffers(n, ids); +} +void GL_APIENTRY BindFramebuffer(GLenum target, GLuint id) { + gl.BindFramebuffer(target, id); + if (auto *t = Current()) + t->Bind(target, id); +} +void GL_APIENTRY FramebufferTexture2D(GLenum target, GLenum attachment, + GLenum tex_target, GLuint texture, + GLint level) { + gl.FramebufferTexture2D(target, attachment, tex_target, texture, level); + if (auto *t = Current()) + t->AttachmentChanged(target); +} +void GL_APIENTRY FramebufferTextureLayer(GLenum target, GLenum attachment, + GLuint texture, GLint level, + GLint layer) { + gl.FramebufferTextureLayer(target, attachment, texture, level, layer); + if (auto *t = Current()) + t->AttachmentChanged(target); +} +void GL_APIENTRY FramebufferRenderbuffer(GLenum target, GLenum attachment, + GLenum rb_target, GLuint rb) { + gl.FramebufferRenderbuffer(target, attachment, rb_target, rb); + if (auto *t = Current()) + t->AttachmentChanged(target); +} +void GL_APIENTRY TexImage2D(GLenum target, GLint level, GLint internal_format, + GLsizei w, GLsizei h, GLint border, GLenum format, + GLenum type, const void *data) { + gl.TexImage2D(target, level, internal_format, w, h, border, format, type, + data); + if (target == GL_TEXTURE_2D && level == 0) + if (auto *t = Current()) + t->Storage(true, w, h, internal_format); +} +void GL_APIENTRY TexStorage2D(GLenum target, GLsizei levels, GLenum format, + GLsizei w, GLsizei h) { + gl.TexStorage2D(target, levels, format, w, h); + if (target == GL_TEXTURE_2D) + if (auto *t = Current()) + t->Storage(true, w, h, format); +} +void GL_APIENTRY RenderbufferStorage(GLenum target, GLenum format, GLsizei w, + GLsizei h) { + gl.RenderbufferStorage(target, format, w, h); + if (target == GL_RENDERBUFFER) + if (auto *t = Current()) + t->Storage(false, w, h, format); +} +void GL_APIENTRY RenderbufferStorageMultisample(GLenum target, GLsizei samples, + GLenum format, GLsizei w, + GLsizei h) { + gl.RenderbufferStorageMultisample(target, samples, format, w, h); + if (target == GL_RENDERBUFFER) + if (auto *t = Current()) + t->Storage(false, w, h, format, samples); +} +void GL_APIENTRY DeleteTextures(GLsizei n, const GLuint *ids) { + if (auto *t = Current()) + t->ObjectsDeleted(true, n, ids); + gl.DeleteTextures(n, ids); +} +void GL_APIENTRY DeleteRenderbuffers(GLsizei n, const GLuint *ids) { + if (auto *t = Current()) + t->ObjectsDeleted(false, n, ids); + gl.DeleteRenderbuffers(n, ids); +} +} // namespace + +bool GlesTransport::Initialize(void *display, void *config, void *context, + void *egl_get_proc, + void *(*resolve)(const char *), + std::string *error) { + if (!display || !config || !context || !egl_get_proc || !resolve) { + *error = + "OpenGL VR requires a real current EGL context, config and display"; + return false; + } +#define LOAD(name, type) \ + gl.name = reinterpret_cast(resolve("gl" #name)); \ + if (!gl.name) { \ + *error = "OpenGL VR is missing gl" #name; \ + return false; \ + } + GL_PROCS(LOAD) +#undef LOAD + using GetProc = __eglMustCastToProperFunctionPointerType (*)(const char *); + get_current_context_ = reinterpret_cast( + reinterpret_cast(egl_get_proc)("eglGetCurrentContext")); + if (!get_current_context_ || get_current_context_() != context) { + *error = "OpenGL VR must initialize on the current EGL render context"; + return false; + } + GLint major = 0, minor = 0; + gl.GetIntegerv(GL_MAJOR_VERSION, &major); + gl.GetIntegerv(GL_MINOR_VERSION, &minor); + if (major < 3) { + *error = "OpenGL VR requires OpenGL ES 3.0 or newer"; + return false; + } + version_ = (std::uint64_t(major) << 48) | (std::uint64_t(minor) << 32); + display_ = display; + config_ = config; + context_ = context; + egl_get_proc_ = egl_get_proc; + gl.GenFramebuffers(2, copy_fbos_); + if (!copy_fbos_[0] || !copy_fbos_[1]) { + *error = "Cannot allocate OpenGL VR copy framebuffers"; + return false; + } + MakeCurrent(); + return true; +} +bool GlesTransport::IsCurrent() const { + return get_current_context_ && get_current_context_() == context_; +} +void GlesTransport::MakeCurrent() { current = this; } +void GlesTransport::ReleaseCurrent() { current = nullptr; } +void GlesTransport::Destroy() { + if (IsCurrent()) + gl.DeleteFramebuffers(2, copy_fbos_); + copy_fbos_[0] = copy_fbos_[1] = 0; + if (current == this) + current = nullptr; + owners_.clear(); + textures_.clear(); + renderbuffers_.clear(); + ResetEyes(); + context_ = nullptr; +} +void *GlesTransport::Wrap(const char *name, void *raw) { + if (!raw || !name) + return raw; +#define WRAP(name_) \ + if (std::strcmp(name, "gl" #name_) == 0) \ + return reinterpret_cast(&name_); + WRAP(GenFramebuffers) + WRAP(DeleteFramebuffers) + WRAP(BindFramebuffer) WRAP(FramebufferTexture2D) WRAP(FramebufferRenderbuffer) + WRAP(FramebufferTextureLayer) WRAP(TexImage2D) WRAP(TexStorage2D) + WRAP(RenderbufferStorage) WRAP(RenderbufferStorageMultisample) + WRAP(DeleteTextures) WRAP(DeleteRenderbuffers) +#undef WRAP + return raw; +} +void GlesTransport::ResetEyes() { + eyes_[0] = {}; + eyes_[1] = {}; +} +void GlesTransport::Generated(int count, const unsigned *ids) { + void *owner = mocktail_vr_initializing_object(); + if (!owner || !ids) + return; + for (int i = 0; i < count; ++i) + if (ids[i]) + owners_[ids[i]] = owner; +} +void GlesTransport::Deleted(int count, const unsigned *ids) { + if (!ids) + return; + for (int i = 0; i < count; ++i) { + owners_.erase(ids[i]); + for (auto &eye : eyes_) + if (eye.framebuffer == ids[i]) + eye = {}; + } +} +void GlesTransport::ObjectsDeleted(bool texture, int count, + const unsigned *ids) { + if (!ids) + return; + for (int i = 0; i < count; ++i) { + (texture ? textures_ : renderbuffers_).erase(ids[i]); + for (auto &eye : eyes_) + if (eye.texture == texture && eye.object == ids[i]) + eye = {}; + } +} +void GlesTransport::Storage(bool texture, int w, int h, unsigned format, + int samples) { + GLint id = 0; + gl.GetIntegerv(texture ? GL_TEXTURE_BINDING_2D : GL_RENDERBUFFER_BINDING, + &id); + if (!id) + return; + // Storage redefinition invalidates previously rendered eye content. + const unsigned object = static_cast(id); + ObjectsDeleted(texture, 1, &object); + (texture ? textures_ : renderbuffers_)[object] = {w, h, format, samples}; +} +void GlesTransport::AttachmentChanged(unsigned target) { + GLint id = 0; + gl.GetIntegerv(target == GL_READ_FRAMEBUFFER ? GL_READ_FRAMEBUFFER_BINDING + : GL_DRAW_FRAMEBUFFER_BINDING, + &id); + for (auto &eye : eyes_) + if (eye.framebuffer == static_cast(id)) + eye = {}; +} +void GlesTransport::Bind(unsigned target, unsigned framebuffer, + std::uint64_t frame) { + if (!frame) + return; + if (target != GL_FRAMEBUFFER && target != GL_DRAW_FRAMEBUFFER) + return; + auto found = owners_.find(framebuffer); + if (found == owners_.end()) + return; + void *owner = nullptr; + void *guest_fb = nullptr; + int eye = -1; + if (!mocktail_vr_current_eye(&owner, &eye, &guest_fb) || eye < 0 || eye > 1 || + found->second != owner) + return; + if (frame == UINT64_MAX) { + auto *backend = ActiveVrBackend(); + const auto pose = + backend ? backend->PublishedHeadPose() : ScriptedPoseSample{}; + if (!pose.valid) + return; + frame = pose.frame; + } + GLint bound = 0; + gl.GetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &bound); + if (static_cast(bound) != framebuffer) + return; + if (gl.CheckFramebufferStatus(GL_DRAW_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) + return; + GLint type = 0, object = 0, level = 0, face = 0; + gl.GetFramebufferAttachmentParameteriv( + GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE, &type); + if (type != GL_TEXTURE && type != GL_RENDERBUFFER) + return; + gl.GetFramebufferAttachmentParameteriv( + GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME, &object); + if (type == GL_TEXTURE) { + gl.GetFramebufferAttachmentParameteriv( + GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_LEVEL, &level); + gl.GetFramebufferAttachmentParameteriv( + GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_CUBE_MAP_FACE, &face); + if (level != 0 || face != 0) + return; + } + const auto &storage = type == GL_TEXTURE ? textures_ : renderbuffers_; + auto it = storage.find(object); + if (it == storage.end()) + return; + const auto &size = it->second; + // DebugDeviceVR currently creates single-sample 500x500 RGBA eyes. + // Reject unsupported formats/extents instead of copying unrelated pixels. + if (size.width != 500 || size.height != 500 || size.samples != 0 || + (size.format != GL_RGBA8 && size.format != GL_RGBA && + size.format != GL_SRGB8_ALPHA8)) + return; + + if ((eyes_[0].owner && eyes_[0].owner != owner) || + (eyes_[1].owner && eyes_[1].owner != owner)) + ResetEyes(); + const bool changed = eyes_[eye].framebuffer != framebuffer; + eyes_[eye] = {framebuffer, + static_cast(object), + type == GL_TEXTURE, + size.width, + size.height, + owner, + frame}; + mocktail_vr_consume_eye(); + if (changed) + std::fprintf( + stderr, + " [vr-backend] GLES eye-bound eye=%d owner=%p guest_fb=%p gl_fb=%u " + "color=%u extent=%dx%d provenance=guest_initializer_and_eye_getter\n", + eye, owner, guest_fb, framebuffer, object, size.width, size.height); +} +bool GlesTransport::Copy(const unsigned textures[2], unsigned width, + unsigned height) { + if (!IsCurrent() || !textures[0] || !textures[1] || !eyes_[0].object || + !eyes_[1].object || + (eyes_[0].texture == eyes_[1].texture && + eyes_[0].object == eyes_[1].object)) + return false; + GLint read_fb = 0, draw_fb = 0; + gl.GetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &read_fb); + gl.GetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &draw_fb); + const bool scissor = gl.IsEnabled(GL_SCISSOR_TEST); + gl.Disable(GL_SCISSOR_TEST); + bool copied = true; + for (int i = 0; i < 2; ++i) { + gl.BindFramebuffer(GL_READ_FRAMEBUFFER, copy_fbos_[0]); + if (eyes_[i].texture) + gl.FramebufferTexture2D(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, eyes_[i].object, 0); + else + gl.FramebufferRenderbuffer(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_RENDERBUFFER, eyes_[i].object); + gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, copy_fbos_[1]); + gl.FramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, textures[i], 0); + const auto read_status = gl.CheckFramebufferStatus(GL_READ_FRAMEBUFFER); + const auto draw_status = gl.CheckFramebufferStatus(GL_DRAW_FRAMEBUFFER); + if (read_status != GL_FRAMEBUFFER_COMPLETE || + draw_status != GL_FRAMEBUFFER_COMPLETE) { + static bool reported = false; + if (!reported) + std::fprintf(stderr, + " [vr-backend] GLES copy framebuffer incomplete: " + "read=0x%x draw=0x%x XR texture=%u\n", + read_status, draw_status, textures[i]); + reported = true; + copied = false; + break; + } + gl.BlitFramebuffer(0, 0, eyes_[i].width, eyes_[i].height, 0, 0, width, + height, GL_COLOR_BUFFER_BIT, GL_NEAREST); + } + // Release references to runtime textures before releasing XR swapchain + // images. + gl.BindFramebuffer(GL_READ_FRAMEBUFFER, copy_fbos_[0]); + gl.FramebufferRenderbuffer(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_RENDERBUFFER, 0); + gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, copy_fbos_[1]); + gl.FramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, 0, 0); + gl.BindFramebuffer(GL_READ_FRAMEBUFFER, read_fb); + gl.BindFramebuffer(GL_DRAW_FRAMEBUFFER, draw_fb); + if (scissor) + gl.Enable(GL_SCISSOR_TEST); + // OpenXR's GL binding synchronizes the flushed work at image release. + gl.Flush(); + return copied; +} +} // namespace mocktail::vr diff --git a/src/vr/gles_transport.h b/src/vr/gles_transport.h new file mode 100644 index 0000000..30fc95d --- /dev/null +++ b/src/vr/gles_transport.h @@ -0,0 +1,69 @@ +#ifndef MOCKTAIL_VR_GLES_TRANSPORT_H_ +#define MOCKTAIL_VR_GLES_TRANSPORT_H_ + +#include +#include +#include + +namespace mocktail::vr { +// One guest EGL context. GL callbacks and copies run on its current thread. +// Only framebuffers created inside the verified DebugDeviceVR initializer +// can become eyes; a live eye-getter request supplies the eye index. +class GlesTransport { +public: + struct Eye { + unsigned framebuffer = 0; + unsigned object = 0; + bool texture = false; + int width = 0, height = 0; + void *owner = nullptr; + std::uint64_t frame = 0; + }; + bool Initialize(void *display, void *config, void *context, + void *egl_get_proc, void *(*resolve)(const char *), + std::string *error); + bool IsCurrent() const; + void Destroy(); // Requires this context current; call before EGL destruction. + void MakeCurrent(); + static void ReleaseCurrent(); + static void *Wrap(const char *name, void *raw); + void ResetEyes(); + bool Copy(const unsigned textures[2], unsigned width, unsigned height); + const Eye &eye(int index) const { return eyes_[index]; } + void *display() const { return display_; } + void *config() const { return config_; } + void *context() const { return context_; } + void *egl_get_proc() const { return egl_get_proc_; } + std::uint64_t version() const { return version_; } + + // Called after the real GL operation, never while the backend mutex is held. + void Generated(int count, const unsigned *ids); + void Deleted(int count, const unsigned *ids); + void Storage(bool texture, int width, int height, unsigned format, + int samples = 0); + void ObjectsDeleted(bool texture, int count, const unsigned *ids); + void AttachmentChanged(unsigned target); + void Bind(unsigned target, unsigned framebuffer, + std::uint64_t frame = UINT64_MAX); + +private: + friend struct GlesTransportTestAccess; + struct StorageInfo { + int width = 0, height = 0; + unsigned format = 0; + int samples = 0; + }; + void *display_ = nullptr; + void *config_ = nullptr; + void *context_ = nullptr; + void *egl_get_proc_ = nullptr; + std::uint64_t version_ = 0; + void *(*get_current_context_)() = nullptr; + unsigned copy_fbos_[2] = {}; + std::unordered_map owners_; + std::unordered_map textures_; + std::unordered_map renderbuffers_; + Eye eyes_[2]; +}; +} // namespace mocktail::vr +#endif diff --git a/src/vr/openxr_backend.cc b/src/vr/openxr_backend.cc index 89a3b84..e1c8bf7 100644 --- a/src/vr/openxr_backend.cc +++ b/src/vr/openxr_backend.cc @@ -2,10 +2,15 @@ #include "mocktail/vr/vr_perf.h" #include "mocktail/vr/vr_pose_math.h" +#include "vr/gles_transport.h" +#include +#include + #include #include #include +#include #include #include @@ -219,7 +224,7 @@ OpenXrBackend* ActiveVrBackend() { return g_backend; } -Status OpenXrBackend::Arm() { +Status OpenXrBackend::Arm(VrGraphicsApi graphics_api) { std::lock_guard lock(mutex_); if (armed_.load(std::memory_order_acquire)) { return Status::Ok(); @@ -234,36 +239,74 @@ Status OpenXrBackend::Arm() { } try { const std::string user_manifest = EnvString("XR_RUNTIME_JSON", ""); - const auto manifest = SelectVrRuntimeManifest(user_manifest, - {"/usr/local/share/openxr/1/openxr_wivrn.json", - "/usr/share/openxr/1/openxr_wivrn.json"}); + const auto preference = EnvString("MOCKTAIL_VR_RUNTIME", "auto"); + Require( + preference == "auto" || preference == "system" || + preference == "wivrn" || preference == "steamvr" || + preference == "alvr", + "MOCKTAIL_VR_RUNTIME must be auto, system, wivrn, steamvr, or alvr"); + const auto discovery = + user_manifest.empty() + ? DiscoverVrRuntime( + preference, EnvString("HOME", ""), + EnvString("XDG_CONFIG_HOME", ""), + EnvString("XDG_CONFIG_DIRS", ""), + EnvString("XDG_RUNTIME_DIR", + ("/run/user/" + std::to_string(getuid())).c_str())) + : VrRuntimeDiscovery{{}, "explicit XR_RUNTIME_JSON override"}; + const auto manifest = + SelectVrRuntimeManifest(user_manifest, discovery.candidates); + Log(" [vr-backend] runtime selection: %s\n", discovery.reason.c_str()); + Require(!manifest.empty() || + (preference != "steamvr" && preference != "alvr" && + preference != "wivrn"), + "Selected VR runtime was not found. For ALVR, install SteamVR and " + "set XR_RUNTIME_JSON " + "to its steamxr_linux64.json if automatic discovery cannot find " + "the installation. " + "For WiVRn, install its native runtime or select its manifest " + "explicitly."); if (!manifest.empty()) { Require(setenv("XR_RUNTIME_JSON", manifest.c_str(), 1) == 0, "Cannot select the OpenXR runtime"); Log(" [vr-backend] runtime manifest: %s\n", manifest.c_str()); } else { - Log(" [vr-backend] using registered OpenXR runtime; connect the headset in WiVRn first\n"); + Log(" [vr-backend] using registered OpenXR runtime; connect the headset " + "in WiVRn or SteamVR/ALVR first\n"); } - vk_loader_ = dlopen("libvulkan.so.1", RTLD_NOW | RTLD_LOCAL); - Require(vk_loader_ != nullptr, - "Cannot load the host Vulkan loader libvulkan.so.1"); - auto loader_gipa = reinterpret_cast( - dlsym(vk_loader_, "vkGetInstanceProcAddr")); - Require(loader_gipa != nullptr, "Host vkGetInstanceProcAddr is missing"); - - const char* extension = XR_KHR_VULKAN_ENABLE2_EXTENSION_NAME; + graphics_api_ = graphics_api; + if (graphics_api_ == VrGraphicsApi::kVulkan) { + vk_loader_ = dlopen("libvulkan.so.1", RTLD_NOW | RTLD_LOCAL); + Require(vk_loader_ != nullptr, + "Cannot load the host Vulkan loader libvulkan.so.1"); + auto loader_gipa = reinterpret_cast( + dlsym(vk_loader_, "vkGetInstanceProcAddr")); + Require(loader_gipa != nullptr, "Host vkGetInstanceProcAddr is missing"); + } + const char *vk_extensions[] = {XR_KHR_VULKAN_ENABLE2_EXTENSION_NAME}; + const char *gl_extensions[] = {XR_KHR_OPENGL_ES_ENABLE_EXTENSION_NAME, + XR_MNDX_EGL_ENABLE_EXTENSION_NAME}; auto info = XrInfo(XR_TYPE_INSTANCE_CREATE_INFO); std::strcpy(info.applicationInfo.applicationName, "Mocktail Roblox VR"); std::strcpy(info.applicationInfo.engineName, "Mocktail"); info.applicationInfo.apiVersion = XR_MAKE_VERSION(1, 0, 0); - info.enabledExtensionCount = 1; - info.enabledExtensionNames = &extension; + info.enabledExtensionCount = + graphics_api_ == VrGraphicsApi::kVulkan ? 1 : 2; + info.enabledExtensionNames = + graphics_api_ == VrGraphicsApi::kVulkan ? vk_extensions : gl_extensions; XrInstance instance = XR_NULL_HANDLE; const XrResult instance_result = xrCreateInstance(&info, &instance); if (instance_result == XR_ERROR_RUNTIME_UNAVAILABLE) { throw std::runtime_error("xrCreateInstance: " + VrRuntimeUnavailableHint(!user_manifest.empty(), manifest)); } + if (instance_result == XR_ERROR_EXTENSION_NOT_PRESENT && + graphics_api_ == VrGraphicsApi::kOpenGles) { + throw std::runtime_error( + "OpenGL VR requires runtime extensions XR_KHR_opengl_es_enable and " + "XR_MNDX_egl_enable; select direct-vulkan if the runtime does not " + "support EGL."); + } CheckXr(XR_NULL_HANDLE, instance_result, "xrCreateInstance"); xr_instance_ = instance; @@ -287,7 +330,8 @@ Status OpenXrBackend::Arm() { // state while the server runs with no headset connected. throw std::runtime_error( "The OpenXR runtime is reachable but reports no connected headset " - "(XR_ERROR_FORM_FACTOR_UNAVAILABLE). With WiVRn: start the server " + "(XR_ERROR_FORM_FACTOR_UNAVAILABLE). Start WiVRn or SteamVR with " + "ALVR " "and connect the headset first; VR applications can only run while " "the headset connection is established."); } @@ -295,23 +339,36 @@ Status OpenXrBackend::Arm() { Require(xr_system_ != XR_NULL_SYSTEM_ID, "The OpenXR runtime exposes no head-mounted display system"); - auto graphics_requirements = - XrInfo( - XR_TYPE_GRAPHICS_REQUIREMENTS_VULKAN2_KHR); - auto get_requirements = reinterpret_cast( - ResolveFunction("xrGetVulkanGraphicsRequirements2KHR")); - CheckXr(instance, - get_requirements(instance, xr_system_, &graphics_requirements), - "xrGetVulkanGraphicsRequirements2KHR"); - Log(" [vr-backend] runtime Vulkan requirement: %u.%u - %u.%u\n", - XR_VERSION_MAJOR(graphics_requirements.minApiVersionSupported), - XR_VERSION_MINOR(graphics_requirements.minApiVersionSupported), - XR_VERSION_MAJOR(graphics_requirements.maxApiVersionSupported), - XR_VERSION_MINOR(graphics_requirements.maxApiVersionSupported)); + if (graphics_api_ == VrGraphicsApi::kVulkan) { + auto graphics_requirements = XrInfo( + XR_TYPE_GRAPHICS_REQUIREMENTS_VULKAN2_KHR); + auto get_requirements = + reinterpret_cast( + ResolveFunction("xrGetVulkanGraphicsRequirements2KHR")); + CheckXr(instance, + get_requirements(instance, xr_system_, &graphics_requirements), + "xrGetVulkanGraphicsRequirements2KHR"); + Log(" [vr-backend] runtime Vulkan requirement: %u.%u - %u.%u\n", + XR_VERSION_MAJOR(graphics_requirements.minApiVersionSupported), + XR_VERSION_MINOR(graphics_requirements.minApiVersionSupported), + XR_VERSION_MAJOR(graphics_requirements.maxApiVersionSupported), + XR_VERSION_MINOR(graphics_requirements.maxApiVersionSupported)); + } else { + auto requirements = XrInfo( + XR_TYPE_GRAPHICS_REQUIREMENTS_OPENGL_ES_KHR); + auto get = reinterpret_cast( + ResolveFunction("xrGetOpenGLESGraphicsRequirementsKHR")); + CheckXr(instance, get(instance, xr_system_, &requirements), + "xrGetOpenGLESGraphicsRequirementsKHR"); + gl_min_version_ = requirements.minApiVersionSupported; + gl_max_version_ = requirements.maxApiVersionSupported; + } armed_.store(true, std::memory_order_release); - Log(" [vr-backend] armed: OpenXR instance/system ready; awaiting the " - "guest Vulkan device\n"); + Log(" [vr-backend] armed: OpenXR instance/system ready; awaiting guest " + "%s\n", + graphics_api_ == VrGraphicsApi::kVulkan ? "Vulkan device" + : "OpenGL ES / EGL context"); return Status::Ok(); } catch (const std::exception& error) { DisarmLocked(); @@ -326,6 +383,10 @@ void OpenXrBackend::Disarm() { void OpenXrBackend::DisarmLocked() { TeardownSessionLocked("disarm"); + if (gles_) + gles_->Destroy(); + delete gles_; + gles_ = nullptr; if (xr_instance_ != nullptr) { const auto instance = static_cast(xr_instance_); (void)xrDestroyInstance(instance); @@ -611,6 +672,141 @@ bool OpenXrBackend::InitializeSessionLocked( "xrCreateSession"); xr_session_ = session; + return InitializeSessionResourcesLocked(error); + } catch (const std::exception &exception) { + *error = exception.what(); + TeardownSessionLocked("session init failure", session_recovery_pending_); + return false; + } +} + +Status OpenXrBackend::AttachGlesContext(void *display, void *config, + void *context, void *egl_get_proc, + void *(*resolve)(const char *)) { + std::lock_guard lock(mutex_); + if (!armed_ || graphics_api_ != VrGraphicsApi::kOpenGles) + return Status::Ok(); + try { + if (gles_) { + Require(gles_->context() == context && gles_->display() == display && + gles_->IsCurrent(), + "OpenGL VR cannot switch to a different live EGL context"); + gles_->MakeCurrent(); + return Status::Ok(); + } + gles_ = new GlesTransport; + std::string error; + Require(gles_->Initialize(display, config, context, egl_get_proc, resolve, + &error), + error); + Require(gles_->version() >= gl_min_version_ && + gles_->version() <= gl_max_version_, + "EGL context version is outside the OpenXR runtime's OpenGL ES " + "requirements"); + Require(InitializeGlesSessionLocked(&error), error); + Log(" [vr-backend] bound OpenGL ES context to OpenXR; GPU eye copies " + "enabled, pose_source=%s\n", + EnvString("MOCKTAIL_VR_POSE_SOURCE", "runtime").c_str()); + return Status::Ok(); + } catch (const std::exception &error) { + TeardownSessionLocked("EGL binding failed"); + if (gles_) + gles_->Destroy(); + delete gles_; + gles_ = nullptr; + return Status::Error(StatusCode::kUnavailable, error.what()); + } +} + +bool OpenXrBackend::InitializeGlesSessionLocked(std::string *error) { + try { + Require(gles_ && gles_->IsCurrent(), + "OpenXR EGL session creation requires the current guest context"); + auto binding = + XrInfo(XR_TYPE_GRAPHICS_BINDING_EGL_MNDX); + binding.display = gles_->display(); + binding.config = gles_->config(); + binding.context = gles_->context(); + binding.getProcAddress = + reinterpret_cast(gles_->egl_get_proc()); + auto info = XrInfo(XR_TYPE_SESSION_CREATE_INFO); + info.next = &binding; + info.systemId = xr_system_; + XrSession session = XR_NULL_HANDLE; + CheckXr( + static_cast(xr_instance_), + xrCreateSession(static_cast(xr_instance_), &info, &session), + "xrCreateSession EGL"); + xr_session_ = session; + return InitializeSessionResourcesLocked(error); + } catch (const std::exception &exception) { + *error = exception.what(); + TeardownSessionLocked("EGL session init failure", + session_recovery_pending_); + return false; + } +} + +void OpenXrBackend::DetachGlesContext(void *context) { + std::lock_guard lock(mutex_); + if (!gles_ || gles_->context() != context) + return; + TeardownSessionLocked("EGL context destruction"); + gles_->Destroy(); + delete gles_; + gles_ = nullptr; +} + +void *OpenXrBackend::WrapGlesProcAddress(const char *name, void *raw) { + if (!armed_ || graphics_api_ != VrGraphicsApi::kOpenGles || !gles_) + return raw; + return GlesTransport::Wrap(name, raw); +} + +void OpenXrBackend::NoteGlesPresent() { + { + std::lock_guard lock(mutex_); + if (!armed_ || graphics_api_ != VrGraphicsApi::kOpenGles || !gles_ || + !gles_->IsCurrent()) + return; + for (int i = 0; i < 2; ++i) { + const auto &eye = gles_->eye(i); + eyes_[i].valid = eye.framebuffer != 0; + eyes_[i].frame = eye.frame; + eyes_[i].owner = eye.owner; + eyes_[i].width = eye.width; + eyes_[i].height = eye.height; + } + const auto &left = gles_->eye(0); + const auto &right = gles_->eye(1); + eyes_bound_ = + left.framebuffer && right.framebuffer && + left.framebuffer != right.framebuffer && left.owner == right.owner && + (left.texture != right.texture || left.object != right.object); + } + NoteHostPresent(VK_NULL_HANDLE, VK_NULL_HANDLE); +} + +bool OpenXrBackend::CopyGlesEyesIntoSwapchains() { + if (!gles_) + return false; + unsigned textures[2] = {}; + for (int i = 0; i < 2; ++i) { + const auto &slot = eye_swapchains_[i]; + if (!slot.image_acquired || slot.acquired_index >= slot.gl_images.size()) + return false; + textures[i] = slot.gl_images[slot.acquired_index]; + } + Require(gles_->Copy(textures, eye_swapchains_[0].width, + eye_swapchains_[0].height), + "Cannot copy GLES eyes into runtime textures; verify EGL and OpenXR " + "use the same GPU and support image sharing"); + return true; +} + +bool OpenXrBackend::InitializeSessionResourcesLocked(std::string *error) { + try { + const auto session = static_cast(xr_session_); auto space_info = XrInfo( XR_TYPE_REFERENCE_SPACE_CREATE_INFO); space_info.poseInReferenceSpace.orientation.w = 1; @@ -670,13 +866,12 @@ bool OpenXrBackend::InitializeSessionLocked( } recording_.store(true, std::memory_order_release); return true; - } catch (const std::exception& exception) { + } catch (const std::exception &exception) { *error = exception.what(); TeardownSessionLocked("session init failure", session_recovery_pending_); return false; } } - bool OpenXrBackend::CreateSwapchains(std::string* error) { try { const auto session = static_cast(xr_session_); @@ -694,10 +889,15 @@ bool OpenXrBackend::CreateSwapchains(std::string* error) { // The swapchain format must accept the guest eye content. Prefer the // common 8-bit formats; the final choice is cross-checked against the // recorded eye images before the first copy (blit converts if needed). - VkFormat chosen = VK_FORMAT_UNDEFINED; - for (const auto candidate : - {VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_R8G8B8A8_UNORM, - VK_FORMAT_B8G8R8A8_SRGB, VK_FORMAT_R8G8B8A8_SRGB}) { + std::int64_t chosen = 0; + const std::vector candidates = + graphics_api_ == VrGraphicsApi::kVulkan + ? std::vector{VK_FORMAT_B8G8R8A8_UNORM, + VK_FORMAT_R8G8B8A8_UNORM, + VK_FORMAT_B8G8R8A8_SRGB, + VK_FORMAT_R8G8B8A8_SRGB} + : std::vector{GL_RGBA8, GL_SRGB8_ALPHA8}; + for (const auto candidate : candidates) { if (std::find(formats.begin(), formats.end(), static_cast(candidate)) != formats.end()) { chosen = candidate; @@ -730,30 +930,48 @@ bool OpenXrBackend::CreateSwapchains(std::string* error) { CheckXr(static_cast(xr_instance_), xrCreateSwapchain(session, &create_info, &swapchain), "xrCreateSwapchain"); + EyeSwapchain &slot = eye_swapchains_[eye]; + slot.swapchain = + swapchain; // Own immediately, including enumeration failures. std::uint32_t image_count = 0; CheckXr(static_cast(xr_instance_), xrEnumerateSwapchainImages(swapchain, 0, &image_count, nullptr), "xrEnumerateSwapchainImages count"); Require(image_count > 0 && image_count <= 16, "the runtime returned an empty swapchain"); - std::vector images(image_count); - for (auto& image : images) { - image.type = XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR; - } - CheckXr(static_cast(xr_instance_), - xrEnumerateSwapchainImages( - swapchain, image_count, &image_count, - reinterpret_cast(images.data())), - "xrEnumerateSwapchainImages"); - EyeSwapchain& slot = eye_swapchains_[eye]; - slot.swapchain = swapchain; - slot.images.clear(); - for (const auto& image : images) { - slot.images.push_back(image.image); + if (graphics_api_ == VrGraphicsApi::kVulkan) { + std::vector images(image_count); + for (auto &image : images) { + image.type = XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR; + } + CheckXr( + static_cast(xr_instance_), + xrEnumerateSwapchainImages( + swapchain, image_count, &image_count, + reinterpret_cast(images.data())), + "xrEnumerateSwapchainImages"); + slot.images.clear(); + for (const auto &image : images) { + slot.images.push_back(image.image); + } + } else { + std::vector images(image_count); + for (auto &image : images) + image.type = XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR; + CheckXr( + static_cast(xr_instance_), + xrEnumerateSwapchainImages( + swapchain, image_count, &image_count, + reinterpret_cast(images.data())), + "xrEnumerateSwapchainImages GLES"); + slot.gl_images.clear(); + for (const auto &image : images) { + slot.gl_images.push_back(image.image); + } } slot.width = width; slot.height = height; - slot.format = chosen; + slot.format = static_cast(chosen); Log(" [vr-backend] eye%d swapchain: %ux%u format=%d images=%u\n", eye, width, height, static_cast(chosen), image_count); } @@ -767,6 +985,7 @@ bool OpenXrBackend::CreateSwapchains(std::string* error) { void OpenXrBackend::DestroySwapchains() { for (EyeSwapchain& slot : eye_swapchains_) { slot.images.clear(); + slot.gl_images.clear(); if (slot.swapchain != nullptr) { (void)xrDestroySwapchain(static_cast(slot.swapchain)); slot.swapchain = nullptr; @@ -817,6 +1036,8 @@ void OpenXrBackend::TeardownSessionLocked(const char* reason, bool preserve_devi canted_rejection_logged_ = false; eyes_[0] = {}; eyes_[1] = {}; + if (gles_) + gles_->ResetEyes(); applied_poses_.clear(); published_pose_.valid = false; if (preserve_device) { @@ -1264,12 +1485,19 @@ void OpenXrBackend::HandleReferenceSpaceChange(std::int64_t change_time) { } bool OpenXrBackend::RecoverSessionLocked(std::uint64_t now_ns) { - if (!session_recovery_pending_ || vk_device_ == VK_NULL_HANDLE || vk_ == nullptr || - xr_instance_ == nullptr || now_ns < next_session_retry_ns_) return false; + if (!session_recovery_pending_ || + (graphics_api_ == VrGraphicsApi::kVulkan + ? (vk_device_ == VK_NULL_HANDLE || vk_ == nullptr) + : gles_ == nullptr) || + xr_instance_ == nullptr || now_ns < next_session_retry_ns_) + return false; next_session_retry_ns_ = now_ns + 1'000'000'000ULL; std::string error; // nullptr means reuse the verified queue family of this existing VkDevice. - if (!InitializeSessionLocked(vk_physical_device_, vk_device_, nullptr, &error)) { + if (!(graphics_api_ == VrGraphicsApi::kVulkan + ? InitializeSessionLocked(vk_physical_device_, vk_device_, nullptr, + &error) + : InitializeGlesSessionLocked(&error))) { Log(" [vr-backend] waiting to restore XR session: %s\n", error.c_str()); return false; } @@ -1400,10 +1628,13 @@ void OpenXrBackend::RunFrameCycle(VkQueue queue, VkDevice device) { } const bool copied = [&] { const ScopedStageTimer copy_record(perf::Stage::kCopyRecord); - return CopyEyesIntoSwapchains(queue); + return graphics_api_ == VrGraphicsApi::kVulkan + ? CopyEyesIntoSwapchains(queue) + : CopyGlesEyesIntoSwapchains(); }() && [&] { const ScopedStageTimer evidence_timer(perf::Stage::kEvidence); - return CaptureEyeEvidence(queue, 0) && CaptureEyeEvidence(queue, 1); + return graphics_api_ == VrGraphicsApi::kOpenGles || + (CaptureEyeEvidence(queue, 0) && CaptureEyeEvidence(queue, 1)); }(); for (int eye = 0; eye < 2; ++eye) { if (images_held[eye]) { @@ -1454,14 +1685,23 @@ void OpenXrBackend::RunFrameCycle(VkQueue queue, VkDevice device) { (!EnvString("MOCKTAIL_VR_XR_EVIDENCE_DIR", "").empty() && published_pose_.frame % std::max(1, EnvUnsigned("MOCKTAIL_VR_XR_EVIDENCE_INTERVAL", 20)) == 0)) { - Log(" [vr-backend][evidence] projection submitted=%llu frame=%llu eye0_image=%p " - "eye1_image=%p eye0_swapchain_index=%u eye1_swapchain_index=%u\n", - static_cast(submitted), - static_cast(published_pose_.frame), - static_cast(eyes_[0].image), - static_cast(eyes_[1].image), - eye_swapchains_[0].acquired_index, - eye_swapchains_[1].acquired_index); + if (graphics_api_ == VrGraphicsApi::kOpenGles) { + Log(" [vr-backend][evidence] projection submitted=%llu frame=%llu " + "graphics=GLES eye0_texture=%u eye1_texture=%u\n", + static_cast(submitted), + static_cast(published_pose_.frame), + gles_->eye(0).object, gles_->eye(1).object); + } else { + Log(" [vr-backend][evidence] projection submitted=%llu frame=%llu " + "eye0_image=%p " + "eye1_image=%p eye0_swapchain_index=%u eye1_swapchain_index=%u\n", + static_cast(submitted), + static_cast(published_pose_.frame), + static_cast(eyes_[0].image), + static_cast(eyes_[1].image), + eye_swapchains_[0].acquired_index, + eye_swapchains_[1].acquired_index); + } } } else { perf::ProcessCollector().Count(perf::Counter::kFramesSkipped); @@ -2192,3 +2432,32 @@ void mocktail_vr_xr_note_present(VkQueue queue, VkDevice device) { } } // extern "C" + +extern "C" bool mocktail_vr_attach_egl(void *display, void *config, + void *context, void *egl_get_proc, + void *(*resolve)(const char *)) { + if (auto *backend = mocktail::vr::ActiveVrBackend()) { + const auto status = backend->AttachGlesContext(display, config, context, + egl_get_proc, resolve); + if (!status.ok()) + std::fprintf(stderr, " [vr-backend] %s\n", status.message().c_str()); + return status.ok(); + } + return true; +} +extern "C" void mocktail_vr_detach_egl(void *context) { + if (auto *backend = mocktail::vr::ActiveVrBackend()) + backend->DetachGlesContext(context); +} +extern "C" void mocktail_vr_release_egl() { + mocktail::vr::GlesTransport::ReleaseCurrent(); +} +extern "C" void *mocktail_vr_wrap_gles_proc(const char *name, void *raw) { + if (auto *backend = mocktail::vr::ActiveVrBackend()) + return backend->WrapGlesProcAddress(name, raw); + return raw; +} +extern "C" void mocktail_vr_gles_present() { + if (auto *backend = mocktail::vr::ActiveVrBackend()) + backend->NoteGlesPresent(); +} diff --git a/src/vr/openxr_backend_mode.cc b/src/vr/openxr_backend_mode.cc index c2764ce..007a831 100644 --- a/src/vr/openxr_backend_mode.cc +++ b/src/vr/openxr_backend_mode.cc @@ -1,36 +1,253 @@ #include "mocktail/vr/openxr_backend.h" +#include +#include #include #include +#include +#include +#include +#include +#define JSON_NOEXCEPTION 1 +#include namespace mocktail::vr { -std::string SelectVrRuntimeManifest(const std::string& explicit_manifest, - const std::vector& candidates) { - if (!explicit_manifest.empty()) return explicit_manifest; - for (const auto& path : candidates) { +std::string +SelectVrRuntimeManifest(const std::string &explicit_manifest, + const std::vector &candidates) { + if (!explicit_manifest.empty()) + return explicit_manifest; + for (const auto &path : candidates) { std::error_code error; - if (std::filesystem::is_regular_file(path, error)) return path; + if (std::filesystem::is_regular_file(path, error)) + return path; } return {}; } -std::string VrRuntimeUnavailableHint(bool user_selected, const std::string& manifest) { +std::vector VrRuntimeManifestCandidates( + const std::string &preference, const std::string &home, + const std::string &config_home, const std::string &config_dirs) { + if (preference == "steamvr" || preference == "alvr") { + if (home.empty()) + return {}; + std::vector paths; + const auto config = config_home.empty() ? home + "/.config" : config_home; + const auto registry = config + "/openvr/openvrpaths.vrpath"; + std::error_code error; + if (std::filesystem::file_size(registry, error) <= 1024 * 1024 && !error) { + std::ifstream input(registry); + const auto json = nlohmann::json::parse(input, nullptr, false); + if (json.is_object() && json.contains("runtime") && + json["runtime"].is_array()) { + for (const auto &item : json["runtime"]) { + if (item.is_string()) { + const auto root = std::filesystem::path(item.get()); + if (root.is_absolute()) + paths.push_back((root / "steamxr_linux64.json").string()); + } + } + } + } + const std::vector standard = { + home + + "/.local/share/Steam/steamapps/common/SteamVR/steamxr_linux64.json", + home + "/.steam/steam/steamapps/common/SteamVR/steamxr_linux64.json", + home + "/.steam/debian-installation/steamapps/common/SteamVR/" + "steamxr_linux64.json"}; + paths.insert(paths.end(), standard.begin(), standard.end()); + return paths; + } + const std::vector wivrn = { + "/usr/local/share/openxr/1/openxr_wivrn.json", + "/usr/share/openxr/1/openxr_wivrn.json"}; + if (preference == "wivrn") + return wivrn; + if (preference == "system") + return {}; // Use the loader's full discovery rules. + std::vector candidates; + const auto add_config = [&](const std::string &path) { + if (path.empty() || path[0] != '/') + return; + candidates.push_back(path + "/openxr/1/active_runtime.x86_64.json"); + candidates.push_back(path + "/openxr/1/active_runtime.json"); + }; + add_config(!config_home.empty() ? config_home + : (home.empty() ? "" : home + "/.config")); + std::istringstream directories(config_dirs.empty() ? "/etc/xdg" + : config_dirs); + std::string directory; + while (std::getline(directories, directory, ':')) + add_config(directory); + add_config("/etc"); + candidates.insert(candidates.end(), wivrn.begin(), wivrn.end()); + return candidates; +} + +VrRuntimeDiscovery DiscoverVrRuntime(const std::string &preference, + const std::string &home, + const std::string &config_home, + const std::string &config_dirs, + const std::string &runtime_dir, + const std::string &proc_root) { + VrRuntimeDiscovery result; + auto steam = + VrRuntimeManifestCandidates("steamvr", home, config_home, config_dirs); + auto wivrn = + VrRuntimeManifestCandidates("wivrn", home, config_home, config_dirs); + bool steam_running = false; + bool wivrn_listening = false; + // Read the kernel's socket table: a leftover socket pathname alone is not + // evidence of a listening server. Do not connect and create a dummy XR + // client. + const auto ipc = (std::filesystem::path(runtime_dir) / "wivrn/comp_ipc") + .lexically_normal(); + if (!runtime_dir.empty()) { + std::ifstream sockets(std::filesystem::path(proc_root) / "net/unix"); + std::string line; + while (std::getline(sockets, line)) { + std::istringstream row(line); + std::string num, refs, protocol, flags, type, state, inode, path; + if (!(row >> num >> refs >> protocol >> flags >> type >> state >> inode)) + continue; + std::getline(row >> std::ws, path); + if (flags == "00010000" && type == "0001" && + std::filesystem::path(path).lexically_normal() == ipc) { + struct stat socket_info{}; + if (stat(path.c_str(), &socket_info) == 0 && + S_ISSOCK(socket_info.st_mode) && socket_info.st_uid == getuid()) + wivrn_listening = true; + } + } + } + // Restrict process discovery to this user. In particular, never select a + // root server for a desktop client or another user's SteamVR installation. + std::error_code error; + std::filesystem::directory_iterator processes(proc_root, error); + for (; !error && processes != std::filesystem::directory_iterator(); + processes.increment(error)) { + const auto name = processes->path().filename().string(); + if (name.empty() || !std::all_of(name.begin(), name.end(), [](char c) { + return c >= '0' && c <= '9'; + })) + continue; + struct stat metadata{}; + if (stat(processes->path().c_str(), &metadata) != 0 || + metadata.st_uid != getuid()) + continue; + std::ifstream comm_file(processes->path() / "comm"); + std::string comm; + std::getline(comm_file, comm); + if (comm != "vrserver" && comm != "wivrn-server") + continue; + std::error_code exe_error; + const auto executable = + std::filesystem::read_symlink(processes->path() / "exe", exe_error); + if (exe_error || !executable.is_absolute() || executable.filename() != comm) + continue; + if (comm == "vrserver") { + steam_running = true; + steam.insert(steam.begin(), + (executable.parent_path().parent_path().parent_path() / + "steamxr_linux64.json") + .string()); + } else { + // Native developer builds keep this manifest at their build root. + auto directory = executable.parent_path(); + for (int i = 0; i < 3 && directory.has_relative_path(); + ++i, directory = directory.parent_path()) { + wivrn.insert(wivrn.begin(), + (directory / "openxr_wivrn-dev.json").string()); + } + } + } + if (preference == "steamvr" || preference == "alvr") { + result.candidates = steam; + result.reason = "explicit SteamVR/ALVR preference"; + } else if (preference == "wivrn") { + result.candidates = wivrn; + result.reason = "explicit WiVRn preference"; + } else if (preference == "system") { + result.reason = "system OpenXR loader selection"; + } else { + const auto registered = + VrRuntimeManifestCandidates("auto", home, config_home, config_dirs); + if (steam_running != wivrn_listening) { + const std::string runtime_tag = steam_running ? "steamvr" : "wivrn"; + for (const auto &path : registered) { + // Only registered entries, not native fallback paths. + if (std::filesystem::path(path).filename().string().find( + "active_runtime") != 0) + continue; + std::error_code manifest_error; + if (std::filesystem::file_size(path, manifest_error) > 1024 * 1024 || + manifest_error) + continue; + auto identity = + std::filesystem::weakly_canonical(path, manifest_error).string(); + std::ifstream input(path); + const auto json = nlohmann::json::parse(input, nullptr, false); + if (json.is_object() && json.contains("runtime") && + json["runtime"].is_object()) { + for (const auto *key : {"name", "library_path"}) { + const auto &runtime = json["runtime"]; + if (runtime.contains(key) && runtime[key].is_string()) + identity += " " + runtime[key].get(); + } + } + std::transform(identity.begin(), identity.end(), identity.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (identity.find(runtime_tag) != std::string::npos) + result.candidates.push_back(path); + } + const auto &active = steam_running ? steam : wivrn; + result.candidates.insert(result.candidates.end(), active.begin(), + active.end()); + result.reason = steam_running + ? "running SteamVR detected (including ALVR)" + : "listening WiVRn server detected"; + } else { + result.reason = + steam_running + ? "both servers detected; registered runtime has priority" + : "no single running server detected; registered runtime has " + "priority"; + } + result.candidates.insert(result.candidates.end(), registered.begin(), + registered.end()); + result.candidates.insert(result.candidates.end(), steam.begin(), + steam.end()); + result.candidates.insert(result.candidates.end(), wivrn.begin(), + wivrn.end()); + } + return result; +} + +std::string VrRuntimeUnavailableHint(bool user_selected, + const std::string &manifest) { if (user_selected) { return "User-selected XR_RUNTIME_JSON '" + manifest + - "' is unavailable. Check the manifest/library and that its runtime server " - "is running with a connected headset; this error alone does not distinguish them."; + "' is unavailable. Check the manifest/library and that its runtime " + "server " + "is running with a connected headset; this error alone does not " + "distinguish them."; } if (!manifest.empty()) { - return "Auto-selected WiVRn runtime is unavailable. Start WiVRn and connect " - "the headset, then retry. If it remains unavailable, check the runtime installation."; + return "Selected OpenXR runtime '" + manifest + + "' is unavailable. Start its server " + "(WiVRn, or SteamVR with ALVR) and connect the headset. Run the " + "server and " + "Mocktail as the same desktop user in the same runtime environment."; } - return "The registered OpenXR runtime is unavailable. Install/start WiVRn and " - "connect the headset, or explicitly select an installed runtime with XR_RUNTIME_JSON."; + return "The registered OpenXR runtime is unavailable. Start WiVRn or SteamVR " + "with ALVR and " + "connect the headset, or explicitly select an installed runtime with " + "XR_RUNTIME_JSON."; } VrBackendMode ResolveVrBackendMode(bool vr_enabled) { if (!vr_enabled) { return VrBackendMode::kDisabled; } - const char* value = std::getenv("MOCKTAIL_VR_BACKEND"); + const char *value = std::getenv("MOCKTAIL_VR_BACKEND"); const std::string mode = value ? value : "xr"; if (mode == "native" || mode == "native-stereo") { return VrBackendMode::kNativeOnly; @@ -38,16 +255,16 @@ VrBackendMode ResolveVrBackendMode(bool vr_enabled) { return VrBackendMode::kXrOutput; } -const char* VrBackendModeName(VrBackendMode mode) { +const char *VrBackendModeName(VrBackendMode mode) { switch (mode) { - case VrBackendMode::kDisabled: - return "disabled"; - case VrBackendMode::kNativeOnly: - return "native-stereo-diagnostic"; - case VrBackendMode::kXrOutput: - return "openxr-output"; + case VrBackendMode::kDisabled: + return "disabled"; + case VrBackendMode::kNativeOnly: + return "native-stereo-diagnostic"; + case VrBackendMode::kXrOutput: + return "openxr-output"; } return "disabled"; } -} +} // namespace mocktail::vr diff --git a/src/vr/openxr_disabled.cc b/src/vr/openxr_disabled.cc index f8dc382..3c75de6 100644 --- a/src/vr/openxr_disabled.cc +++ b/src/vr/openxr_disabled.cc @@ -4,7 +4,7 @@ namespace mocktail::vr { OpenXrBackend::~OpenXrBackend() = default; -Status OpenXrBackend::Arm() { +Status OpenXrBackend::Arm(VrGraphicsApi) { return Status::Error(StatusCode::kUnavailable, "OpenXR support is disabled in this build"); } void OpenXrBackend::Disarm() {} diff --git a/src/window/window.cc b/src/window/window.cc index c393edd..34abd80 100644 --- a/src/window/window.cc +++ b/src/window/window.cc @@ -40,6 +40,9 @@ namespace mocktail { namespace window { +namespace { +bool AttachVrEgl(); +} // Local EGL declarations avoid conflicts with the stub headers. @@ -1309,6 +1312,16 @@ bool Init(int width, int height, const char* title) { } return CreateSoftwareWaitingWindow(width, height, title); } + if (!AttachVrEgl()) { + SDL_GL_DestroyContext(sdl_context); + SDL_DestroyWindow(g_state.sdl_window); + g_state.sdl_window = nullptr; + g_state.egl_context = nullptr; + g_state.egl_display = nullptr; + g_state.egl_config = nullptr; + g_state.egl_surface = nullptr; + return false; + } fprintf(stderr, " [window] EGL context and surface initialized via SDL3\n"); if (WindowTraceEnabled()) { fprintf( @@ -1342,6 +1355,23 @@ bool Init(int width, int height, const char* title) { return true; } +namespace { +bool AttachVrEgl() { + using Attach = + bool (*)(void *, void *, void *, void *, void *(*)(const char *)); + auto attach = + reinterpret_cast(dlsym(RTLD_DEFAULT, "mocktail_vr_attach_egl")); + if (!attach) + return true; + return attach( + g_state.egl_display, g_state.egl_config, g_state.egl_context, + reinterpret_cast(SDL_EGL_GetProcAddress("eglGetProcAddress")), + +[](const char *name) -> void * { + return reinterpret_cast(SDL_GL_GetProcAddress(name)); + }); +} +} // namespace + bool MakeCurrentOnThread() { if (!g_state.initialised || !g_state.egl_context) { if (WindowTraceEnabled()) { @@ -1359,7 +1389,7 @@ bool MakeCurrentOnThread() { return false; } fprintf(stderr, " [window] EGL context now current on engine thread\n"); - return true; + return AttachVrEgl(); } bool ReleaseCurrentOnThread() { @@ -1374,6 +1404,11 @@ bool ReleaseCurrentOnThread() { SDL_GetError()); return false; } + using Release = void (*)(); + auto release = + reinterpret_cast(dlsym(RTLD_DEFAULT, "mocktail_vr_release_egl")); + if (release) + release(); if (WindowTraceEnabled()) { fprintf(stderr, " [window] EGL context released on current thread\n"); } @@ -1516,7 +1551,11 @@ void* GetGLProcAddress(const char* name) { if (!g_state.initialised || name == nullptr || name[0] == '\0') { return nullptr; } - return reinterpret_cast(SDL_GL_GetProcAddress(name)); + void *raw = reinterpret_cast(SDL_GL_GetProcAddress(name)); + using Wrap = void *(*)(const char *, void *); + static auto wrap = + reinterpret_cast(dlsym(RTLD_DEFAULT, "mocktail_vr_wrap_gles_proc")); + return wrap ? wrap(name, raw) : raw; } void ShowIfHidden() { @@ -1575,6 +1614,16 @@ bool SwapBuffers() { return false; } + using Notify = void (*)(); + static auto vr_begin = reinterpret_cast( + dlsym(RTLD_DEFAULT, "mocktail_vr_note_host_present_begin")); + static auto vr_present = + reinterpret_cast(dlsym(RTLD_DEFAULT, "mocktail_vr_gles_present")); + if (vr_begin) + vr_begin(); + if (vr_present) + vr_present(); + if (g_gles_text_overlay == nullptr) { graphics::GlesTextOverlaySource source; source.may_present = reinterpret_cast( @@ -2356,6 +2405,14 @@ void Shutdown() { // resources below. g_gles_text_overlay.reset(); if (!g_state.direct_vulkan && g_state.egl_context != nullptr) { + // Drain the XR session while its EGL context still exists and is current. + SDL_GL_MakeCurrent(g_state.sdl_window, + static_cast(g_state.egl_context)); + using Detach = void (*)(void *); + auto detach = + reinterpret_cast(dlsym(RTLD_DEFAULT, "mocktail_vr_detach_egl")); + if (detach) + detach(g_state.egl_context); SDL_GL_MakeCurrent(g_state.sdl_window, nullptr); } if (!g_state.direct_vulkan && g_state.egl_context) { diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 863b8e1..8487620 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -29,7 +29,7 @@ file(GENERATE } ]=]) -add_executable(vr_mirror_test vr/vr_mirror_test.cc) +add_executable(vr_mirror_test vr/vr_mirror_test.cc vr/runtime_detection_test.cc) target_link_libraries(vr_mirror_test PRIVATE Mocktail::VR GTest::gtest_main) mocktail_apply_compile_options(vr_mirror_test) add_executable(vr_pose_math_test vr/vr_pose_math_test.cc) @@ -57,6 +57,11 @@ if(MOCKTAIL_ENABLE_VR) target_include_directories(openxr_backend_test PRIVATE "${PROJECT_SOURCE_DIR}/third_party/OpenXR-SDK/include") target_link_libraries(openxr_backend_test PRIVATE Mocktail::VR GTest::gtest_main) gtest_discover_tests(openxr_backend_test) + add_executable(gles_vr_transport_test vr/gles_transport_test.cc) + target_include_directories(gles_vr_transport_test PRIVATE "${PROJECT_SOURCE_DIR}/src" "${PROJECT_SOURCE_DIR}/third_party/OpenXR-SDK/include" ${MOCKTAIL_GLES3_INCLUDE_DIR}) + target_link_libraries(gles_vr_transport_test PRIVATE Mocktail::VR SDL3::SDL3 GTest::gtest_main) + gtest_discover_tests(gles_vr_transport_test PROPERTIES ENVIRONMENT "SDL_VIDEODRIVER=offscreen") + add_library(mocktail_test_openxr_runtime MODULE vr/fake_openxr_runtime.cc vr/fake_openxr_preview.cc) target_link_libraries(mocktail_test_openxr_runtime PRIVATE diff --git a/tests/vr/gles_transport_test.cc b/tests/vr/gles_transport_test.cc new file mode 100644 index 0000000..60a960f --- /dev/null +++ b/tests/vr/gles_transport_test.cc @@ -0,0 +1,260 @@ +#include "mocktail/vr/openxr_backend.h" +#include "mocktail/vr/roblox_vr_device_bridge.h" +#include "vr/gles_transport.h" +#include +#include +#include +#include +#include +#include + +namespace mocktail::vr { +namespace internal { +struct VrBridgeTestAccess { + static void Enable(RobloxVrDeviceBridge &bridge) { bridge.active_ = true; } +}; +} // namespace internal +struct GlesTransportTestAccess { + static void Owner(GlesTransport &t, unsigned fb, void *owner) { + t.owners_[fb] = owner; + } +}; +struct OpenXrBackendTestAccess { + static GlesTransport &Transport(OpenXrBackend &b) { return *b.gles_; } + static void LoseSession(OpenXrBackend &b) { + b.HandleSessionState(XR_SESSION_STATE_LOSS_PENDING); + } +}; +namespace { +#define GL(name) \ + reinterpret_cast(SDL_GL_GetProcAddress("gl" #name)) +class GlesVrTransportTest : public ::testing::Test { +protected: + void SetUp() override { + SDL_SetHint(SDL_HINT_VIDEO_FORCE_EGL, "1"); + ASSERT_TRUE(SDL_Init(SDL_INIT_VIDEO)) << SDL_GetError(); + SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_ES); + SDL_GL_SetAttribute(SDL_GL_CONTEXT_MAJOR_VERSION, 3); + SDL_GL_SetAttribute(SDL_GL_CONTEXT_MINOR_VERSION, 0); + window = SDL_CreateWindow("Mocktail GLES VR test", 64, 64, + SDL_WINDOW_OPENGL | SDL_WINDOW_HIDDEN); + ASSERT_NE(window, nullptr) << SDL_GetError(); + context = SDL_GL_CreateContext(window); + ASSERT_NE(context, nullptr) << SDL_GetError(); + ASSERT_TRUE(SDL_GL_MakeCurrent(window, context)); + internal::VrBridgeTestAccess::Enable(bridge); + } + void TearDown() override { + transport.Destroy(); + if (context) + SDL_GL_DestroyContext(context); + if (window) + SDL_DestroyWindow(window); + SDL_Quit(); + } + void InitTransport(GlesTransport &t) { + std::string error; + ASSERT_TRUE(t.Initialize( + SDL_EGL_GetCurrentDisplay(), SDL_EGL_GetCurrentConfig(), context, + reinterpret_cast(SDL_EGL_GetProcAddress("eglGetProcAddress")), + Resolve, &error)) + << error; + } + static void *Resolve(const char *name) { + return reinterpret_cast(SDL_GL_GetProcAddress(name)); + } + GLuint Texture(GlesTransport *t, float r, float g, float b) { + GLuint texture = 0, fb = 0; + GL(GenTextures)(1, &texture); + GL(BindTexture)(GL_TEXTURE_2D, texture); + GL(TexStorage2D)(GL_TEXTURE_2D, 1, GL_RGBA8, 500, 500); + if (t) + t->Storage(true, 500, 500, GL_RGBA8); + GL(GenFramebuffers)(1, &fb); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + GL(FramebufferTexture2D)(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, texture, 0); + GL(ClearColor)(r, g, b, 1); + GL(Clear)(GL_COLOR_BUFFER_BIT); + GL(DeleteFramebuffers)(1, &fb); + return texture; + } + GLuint Eye(GlesTransport &t, int eye, GLuint texture, std::uint64_t frame = 7, + void *owner = reinterpret_cast(123)) { + GLuint fb = 0; + GL(GenFramebuffers)(1, &fb); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + GL(FramebufferTexture2D)(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, texture, 0); + GlesTransportTestAccess::Owner(t, fb, owner); + BindEye(t, eye, fb, frame, owner); + return fb; + } + void BindEye(GlesTransport &t, int eye, GLuint fb, std::uint64_t frame, + void *owner) { + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + bridge.OnEyeGetterCall(owner, eye, + reinterpret_cast(std::uintptr_t(fb))); + t.Bind(GL_FRAMEBUFFER, fb, frame); + } + std::array Pixel(GLuint texture, int x = 250, int y = 250) { + GLuint fb = 0; + GL(GenFramebuffers)(1, &fb); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + GL(FramebufferTexture2D)(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_TEXTURE_2D, texture, 0); + std::array rgba{}; + GL(ReadPixels)(x, y, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, rgba.data()); + GL(DeleteFramebuffers)(1, &fb); + return rgba; + } + SDL_Window *window = nullptr; + SDL_GLContext context = nullptr; + GlesTransport transport; + RobloxVrDeviceBridge bridge; +}; +TEST_F(GlesVrTransportTest, CopiesDistinctEyesAndRestoresGuestState) { + InitTransport(transport); + const GLuint red = Texture(&transport, 1, 0, 0), + green = Texture(&transport, 0, 1, 0); + Eye(transport, 0, red); + Eye(transport, 1, green); + const GLuint targets[2] = {Texture(nullptr, 0, 0, 0), + Texture(nullptr, 0, 0, 0)}; + // A tiny guest scissor must not crop the eye copy, and bindings must survive. + GLuint fbs[2]; + GL(GenFramebuffers)(2, fbs); + GL(BindFramebuffer)(GL_READ_FRAMEBUFFER, fbs[0]); + GL(BindFramebuffer)(GL_DRAW_FRAMEBUFFER, fbs[1]); + GL(Enable)(GL_SCISSOR_TEST); + GL(Scissor)(0, 0, 1, 1); + GL(Viewport)(2, 3, 17, 19); + ASSERT_TRUE(transport.Copy(targets, 500, 500)); + GLint read = 0, draw = 0, viewport[4]; + GL(GetIntegerv)(GL_READ_FRAMEBUFFER_BINDING, &read); + GL(GetIntegerv)(GL_DRAW_FRAMEBUFFER_BINDING, &draw); + GL(GetIntegerv)(GL_VIEWPORT, viewport); + EXPECT_EQ(static_cast(read), fbs[0]); + EXPECT_EQ(static_cast(draw), fbs[1]); + EXPECT_TRUE(GL(IsEnabled)(GL_SCISSOR_TEST)); + EXPECT_EQ(viewport[0], 2); + EXPECT_EQ(viewport[1], 3); + EXPECT_EQ(viewport[2], 17); + EXPECT_EQ(viewport[3], 19); + EXPECT_EQ(Pixel(targets[0]), (std::array{255, 0, 0, 255})); + EXPECT_EQ(Pixel(targets[1]), (std::array{0, 255, 0, 255})); + EXPECT_EQ(GL(GetError)(), GL_NO_ERROR); +} +TEST_F(GlesVrTransportTest, RejectsUnownedAndAliasedEyesAndTracksDeletion) { + InitTransport(transport); + GLuint texture = Texture(&transport, 1, 0, 0); + GLuint fbs[2]; + GL(GenFramebuffers)(2, fbs); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fbs[0]); + GL(FramebufferTexture2D)(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, + texture, 0); + BindEye(transport, 0, fbs[0], 7, reinterpret_cast(123)); + EXPECT_EQ(transport.eye(0).framebuffer, 0u); + Eye(transport, 0, texture); + Eye(transport, 1, texture); + const GLuint targets[2] = {Texture(nullptr, 0, 0, 0), + Texture(nullptr, 0, 0, 0)}; + EXPECT_FALSE(transport.Copy(targets, 500, 500)); + transport.ObjectsDeleted(true, 1, &texture); + EXPECT_EQ(transport.eye(0).framebuffer, 0u); + EXPECT_EQ(transport.eye(1).framebuffer, 0u); +} +TEST_F(GlesVrTransportTest, NewOwnerAndFramebufferDeletionInvalidateOldPair) { + InitTransport(transport); + const GLuint red = Texture(&transport, 1, 0, 0), + green = Texture(&transport, 0, 1, 0); + Eye(transport, 0, red); + Eye(transport, 1, green); + const auto fb = Eye(transport, 0, red, 8, reinterpret_cast(456)); + EXPECT_EQ(transport.eye(1).framebuffer, 0u); + EXPECT_EQ(transport.eye(0).frame, 8u); + transport.Deleted(1, &fb); + EXPECT_EQ(transport.eye(0).framebuffer, 0u); +} +TEST_F(GlesVrTransportTest, RenderbufferEyesAndAttachmentMutation) { + InitTransport(transport); + GLuint rb = 0, fb = 0; + GL(GenRenderbuffers)(1, &rb); + GL(BindRenderbuffer)(GL_RENDERBUFFER, rb); + GL(RenderbufferStorage)(GL_RENDERBUFFER, GL_RGBA8, 500, 500); + transport.Storage(false, 500, 500, GL_RGBA8); + GL(GenFramebuffers)(1, &fb); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + GL(FramebufferRenderbuffer)(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, + GL_RENDERBUFFER, rb); + GL(ClearColor)(0, 0, 1, 1); + GL(Clear)(GL_COLOR_BUFFER_BIT); + GlesTransportTestAccess::Owner(transport, fb, reinterpret_cast(123)); + BindEye(transport, 0, fb, 7, reinterpret_cast(123)); + EXPECT_EQ(transport.eye(0).object, rb); + EXPECT_FALSE(transport.eye(0).texture); + const GLuint green = Texture(&transport, 0, 1, 0); + Eye(transport, 1, green); + const GLuint targets[2] = {Texture(nullptr, 0, 0, 0), + Texture(nullptr, 0, 0, 0)}; + ASSERT_TRUE(transport.Copy(targets, 500, 500)); + EXPECT_EQ(Pixel(targets[0]), (std::array{0, 0, 255, 255})); + GL(BindFramebuffer)(GL_FRAMEBUFFER, fb); + transport.AttachmentChanged(GL_FRAMEBUFFER); + EXPECT_EQ(transport.eye(0).framebuffer, 0u); + EXPECT_EQ(GL(GetError)(), GL_NO_ERROR); +} +TEST_F(GlesVrTransportTest, OpenXrEglSessionSubmitsAndRecovers) { + if (!std::getenv("MOCKTAIL_TEST_XR_GLES")) + GTEST_SKIP() << "Opt-in: requires a connected or simulated OpenXR runtime"; + OpenXrBackend backend; + auto status = backend.Arm(VrGraphicsApi::kOpenGles); + ASSERT_TRUE(status.ok()) << status.message(); + status = backend.AttachGlesContext( + SDL_EGL_GetCurrentDisplay(), SDL_EGL_GetCurrentConfig(), context, + reinterpret_cast(SDL_EGL_GetProcAddress("eglGetProcAddress")), + Resolve); + ASSERT_TRUE(status.ok()) << status.message(); + auto &t = OpenXrBackendTestAccess::Transport(backend); + const GLuint red = Texture(&t, 1, 0, 0), green = Texture(&t, 0, 1, 0); + auto *owner = reinterpret_cast(123); + const GLuint fbs[2] = {Eye(t, 0, red), Eye(t, 1, green)}; + bool lost_session = false; + for (int i = 0; i < 200 && backend.submitted_frames() < 20; ++i) { + if (!lost_session && backend.submitted_frames() >= 10) { + OpenXrBackendTestAccess::LoseSession(backend); + EXPECT_FALSE(backend.session_running()); + EXPECT_FALSE(backend.PublishedHeadPose().valid); + lost_session = true; + } + backend.NoteGlesPresent(); + auto pose = backend.PublishedHeadPose(); + if (!pose.valid) { + SDL_Delay(10); + continue; + } + backend.NotePoseApplied(owner, pose.frame); + BindEye(t, 0, fbs[0], pose.frame, owner); + BindEye(t, 1, fbs[1], pose.frame, owner); + EXPECT_EQ(t.eye(0).frame, pose.frame); + EXPECT_EQ(t.eye(1).frame, pose.frame); + EXPECT_TRUE(t.IsCurrent()); + } + EXPECT_TRUE(lost_session); + EXPECT_GE(backend.submitted_frames(), 20u); + EXPECT_EQ(GL(GetError)(), GL_NO_ERROR); + backend.DetachGlesContext(context); + EXPECT_FALSE(backend.session_running()); + status = backend.AttachGlesContext( + SDL_EGL_GetCurrentDisplay(), SDL_EGL_GetCurrentConfig(), context, + reinterpret_cast(SDL_EGL_GetProcAddress("eglGetProcAddress")), + Resolve); + EXPECT_TRUE(status.ok()) << status.message(); + if (status.ok()) { + EXPECT_EQ(OpenXrBackendTestAccess::Transport(backend).eye(0).framebuffer, + 0u); + backend.DetachGlesContext(context); + } +} +} // namespace +} // namespace mocktail::vr diff --git a/tests/vr/runtime_detection_test.cc b/tests/vr/runtime_detection_test.cc new file mode 100644 index 0000000..b5eca65 --- /dev/null +++ b/tests/vr/runtime_detection_test.cc @@ -0,0 +1,141 @@ +#include "mocktail/vr/openxr_backend.h" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace mocktail::vr { +namespace { +class VrRuntimeDetectionTest : public ::testing::Test { +protected: + void SetUp() override { + char path[] = "/tmp/mocktail-xr-detect-XXXXXX"; + ASSERT_NE(mkdtemp(path), nullptr); + root = path; + std::filesystem::create_directories(root / "proc/net"); + std::filesystem::create_directories(root / "config/openxr/1"); + std::filesystem::create_directories(root / "run/wivrn"); + std::filesystem::create_symlink("/proc/net/unix", root / "proc/net/unix"); + } + void TearDown() override { + if (socket >= 0) + close(socket); + std::error_code error; + if (!root.empty()) + std::filesystem::remove_all(root, error); + } + void File(const std::filesystem::path &path, const std::string &data = "{}") { + std::filesystem::create_directories(path.parent_path()); + std::ofstream(path) << data; + } + std::string Steam() { + const auto install = root / "custom-library/SteamVR"; + File(install / "steamxr_linux64.json"); + File(install / "bin/linux64/vrserver", "test executable"); + File(root / "proc/100/comm", "vrserver\n"); + std::filesystem::create_symlink(install / "bin/linux64/vrserver", + root / "proc/100/exe"); + return (install / "steamxr_linux64.json").string(); + } + std::string WiVRn(bool listen) { + const auto install = root / "wivrn-build"; + File(install / "openxr_wivrn-dev.json"); + File(install / "server/wivrn-server", "test executable"); + File(root / "proc/200/comm", "wivrn-server\n"); + std::filesystem::create_symlink(install / "server/wivrn-server", + root / "proc/200/exe"); + const auto path = (root / "run/wivrn/comp_ipc").string(); + socket = ::socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0); + EXPECT_GE(socket, 0); + sockaddr_un address{}; + address.sun_family = AF_UNIX; + EXPECT_LT(path.size(), sizeof(address.sun_path)); + std::strcpy(address.sun_path, path.c_str()); + EXPECT_EQ( + bind(socket, reinterpret_cast(&address), sizeof(address)), + 0); + if (listen) + EXPECT_EQ(::listen(socket, 1), 0); + else { + close(socket); + socket = -1; + } // Leave a stale socket inode. + return (install / "openxr_wivrn-dev.json").string(); + } + VrRuntimeDiscovery Discover(const std::string &preference = "auto") { + return DiscoverVrRuntime(preference, (root / "home").string(), + (root / "config").string(), + (root / "etc/xdg").string(), + (root / "run").string(), (root / "proc").string()); + } + std::string Selected(const std::string &preference = "auto") { + return SelectVrRuntimeManifest("", Discover(preference).candidates); + } + std::filesystem::path root; + int socket = -1; +}; +TEST_F(VrRuntimeDetectionTest, RunningSteamOverridesOldRegisteredWiVRn) { + File(root / "config/openxr/1/active_runtime.json"); + EXPECT_EQ(Selected(), + (root / "config/openxr/1/active_runtime.json").string()); + const auto steam = Steam(); + EXPECT_EQ(Selected(), steam); + EXPECT_NE(Discover().reason.find("running SteamVR"), std::string::npos); +} +TEST_F(VrRuntimeDetectionTest, ListeningWiVRnFindsDevelopmentManifest) { + File(root / "config/openxr/1/active_runtime.json"); + const auto wivrn = WiVRn(true); + EXPECT_EQ(Selected(), wivrn); + EXPECT_NE(Discover().reason.find("listening WiVRn"), std::string::npos); +} +TEST_F(VrRuntimeDetectionTest, RegisteredWiVRnBuildWinsOverNativeFallback) { + const auto registered = + (root / "config/openxr/1/active_runtime.json").string(); + File( + registered, + R"({"runtime":{"library_path":"/var/lib/flatpak/app/wivrn/libwivrn.so"}})"); + WiVRn(true); + EXPECT_EQ(Selected(), registered); +} +TEST_F(VrRuntimeDetectionTest, StaleWiVRnSocketDoesNotOverrideSteam) { + const auto steam = Steam(); + WiVRn(false); + EXPECT_EQ(Selected(), steam); +} +TEST_F(VrRuntimeDetectionTest, + BothServersRespectRegisteredRuntimeAndExplicitOverrides) { + const auto registered = + (root / "config/openxr/1/active_runtime.json").string(); + File(registered); + const auto steam = Steam(); + const auto wivrn = WiVRn(true); + EXPECT_EQ(Selected(), registered); + EXPECT_EQ(Selected("alvr"), steam); + EXPECT_EQ(Selected("wivrn"), wivrn); + EXPECT_TRUE(Discover("system").candidates.empty()); + EXPECT_EQ( + SelectVrRuntimeManifest("/custom/override.json", Discover().candidates), + "/custom/override.json"); +} +TEST_F(VrRuntimeDetectionTest, + OpenVrRegistryFindsCustomSteamLibraryWithoutRunningServer) { + const auto install = root / "elsewhere/SteamVR"; + File(install / "steamxr_linux64.json"); + File(root / "config/openvr/openvrpaths.vrpath", + "{\"runtime\":[42,\"relative/path\",\"" + install.string() + "\"]}"); + EXPECT_EQ(Selected("alvr"), (install / "steamxr_linux64.json").string()); + // No external host registration or installed WiVRn is assumed in this + // assertion. + const auto candidates = Discover().candidates; + EXPECT_NE(std::find(candidates.begin(), candidates.end(), + (install / "steamxr_linux64.json").string()), + candidates.end()); + File(root / "config/openvr/openvrpaths.vrpath", "invalid json"); + EXPECT_NE(Selected("alvr"), (install / "steamxr_linux64.json").string()); +} +} // namespace +} // namespace mocktail::vr diff --git a/tests/vr/vr_mirror_test.cc b/tests/vr/vr_mirror_test.cc index f2a8e35..dfea199 100644 --- a/tests/vr/vr_mirror_test.cc +++ b/tests/vr/vr_mirror_test.cc @@ -66,7 +66,10 @@ class SegmentReader { TEST(VrRuntimeSelection, RuntimeErrorsPreserveSelectionProvenance) { EXPECT_NE(VrRuntimeUnavailableHint(true, "/custom.json").find("User-selected"), std::string::npos); - EXPECT_NE(VrRuntimeUnavailableHint(false, "/usr/share/openxr/1/openxr_wivrn.json").find("Auto-selected WiVRn"), std::string::npos); + EXPECT_NE( + VrRuntimeUnavailableHint(false, "/usr/share/openxr/1/openxr_wivrn.json") + .find("Selected OpenXR"), + std::string::npos); EXPECT_NE(VrRuntimeUnavailableHint(false, "").find("registered"), std::string::npos); } @@ -198,3 +201,28 @@ TEST(VrMirrorProducer, RejectsInvalidNameAndOversizedPublish) { } // namespace } // namespace mocktail::vr + +namespace mocktail::vr { +TEST(VrRuntimeSelection, AlvrSelectsSteamVrWithoutFallingBackToWiVRn) { + const auto alvr = VrRuntimeManifestCandidates("alvr", "/home/test", "", ""); + EXPECT_EQ(alvr, VrRuntimeManifestCandidates("steamvr", "/home/test", "", "")); + ASSERT_FALSE(alvr.empty()); + for (const auto &path : alvr) { + EXPECT_NE(path.find("SteamVR/steamxr_linux64.json"), std::string::npos); + EXPECT_EQ(path.find("wivrn"), std::string::npos); + } + EXPECT_TRUE( + VrRuntimeManifestCandidates("system", "/home/test", "", "").empty()); + EXPECT_EQ(SelectVrRuntimeManifest("/custom/steamxr_linux64.json", alvr), + "/custom/steamxr_linux64.json"); +} +TEST(VrRuntimeSelection, RegisteredRuntimePrecedesInstalledWiVRn) { + const auto paths = VrRuntimeManifestCandidates( + "auto", "/home/test", "/private/config", "/opt/xdg:/etc/xdg"); + ASSERT_GE(paths.size(), 4u); + EXPECT_EQ(paths[0], "/private/config/openxr/1/active_runtime.x86_64.json"); + EXPECT_EQ(paths[1], "/private/config/openxr/1/active_runtime.json"); + EXPECT_EQ(paths[2], "/opt/xdg/openxr/1/active_runtime.x86_64.json"); + EXPECT_EQ(paths.back(), "/usr/share/openxr/1/openxr_wivrn.json"); +} +} // namespace mocktail::vr