StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR!

Last update: Jan 4, 2023

Comments: 17

StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR! Inspired by libraries like XNA and Processing, StereoKit is meant to be fun to use and easy to develop with, yet still quite capable of creating professional and business ready software.

The getting started guide can be found here!

Interested in news and updates about StereoKit? Maybe just looking for some extra help?

Submit bugs on the Issues tab, and ask questions in the Discussions tab!
Follow koujaku on Twitter for development news.
Drop into the Discord channel for discussion, help and updates.
Check out this blog for occasional substantial updates!

StereoKit Features:

Platforms: HoloLens 2, Oculus Quest, Windows Mixed Reality, Oculus Desktop, SteamVR, Monado Linux, and eventually everywhere OpenXR is!
Flat screen mode with input emulation for easy development
Builds your application to device in seconds, not minutes
Mixed Reality inputs like hands and eyes are trivial to access
Easy and powerful UI and interactions
Model formats: .gltf, .glb, .fbx(partial), .obj, .stl, procedural
Texture formats: .jpg, .png, .tga, .bmp, .psd, .gif, .hdr, .pic, equirectangular cubemap, procedural
Runtime asset loading
Physics
Performance-by-default instanced render pipeline
Flexible shader/material system with built-in PBR
Documentation is generated directly from the source code, including screenshots

Getting started

Follow this guide for a detailed introduction! This repository is the raw source for those who wish to build StereoKit themselves, the Visul Studio templates and the NuGet packages referenced in the guide are how most people should build their applications!

StereoKit focuses on getting you productive with the least amount of code possible. You can actually do most tasks with a single line of code, including UI! Here's hello world with StereoKit, this is all you need to get up and running!

{ helmet.Draw(Matrix.TS(Vec3.Zero, 0.1f)); })); SK.Shutdown(); } } ">

using StereoKit;

class Program
{
	static void Main(string[] args)
	{
		SK.Initialize(new SKSettings{ appName = "Project" });

		Model helmet = Model.FromFile("Assets/DamagedHelmet.gltf");

		while (SK.Step(() => {
			helmet.Draw(Matrix.TS(Vec3.Zero, 0.1f));
		}));

		SK.Shutdown();
	}
}

Roadmap

Where is StereoKit going next? That depends on you! What do you need? Is StereoKit missing something that it really should have? File an issue and let us know!

In the short term, StereoKit will focus on the Core API, improving performance, and ensuring all basic features are present and work well! Some basic tooling, things like a visual shader editor might get built here. The Core of StereoKit is an Immediate Mode system that does not provide any framework or application state management.

Long term, we'd love to add StereoKit Framework! Framework is a higher-level layer that manages application state, so more complicated functionality can be taken care of. Features such as automatic multi-user capabilities, component systems, WYSIWYG UI design tools, visual scene editors and code-free design environments will become possible.

Dependencies

Just like all software, StereoKit is built on the shoulders of incredible people! Here's a list of the libraries StereoKit uses to get things done.

OpenXR
ReactPhysics3D - physics
cgltf - gltf format support
OpenFBX - fbx format support
miniz - ofbx dependency
Sean Barrett's stb libraries - image and font format support
miniaudio - audio playback
dr_wav - wav format support
sokol_time - cross platform high performance time

And some of my own libraries that I maintain separately from this repository.

sk_gpu.h - cross platform graphics API
ferr_hash.h - string and data hashing
array.h - dynamic arrays

Owner

Nick Klingensmith

Creative coder: procedural content, game development tools and VR/AR/MR!

https://github.com/maluoi/StereoKit https://stereokit.net/Pages/Guides/Getting-Started.html

Comments

Cannot build out of the box latest 0.3.6.
Description

I like to test an "out of the box" build before adding any changes to my projects. This one does not work, and I could not find the reasons why. Things done in the past when it would not build OOB did not work this time. I spent several hours trying to work out the issues with the build.

Trying to build out of the box following the instructions. There is a problem with the OpenXR version 1.0.22. The first error is a pathspec xxxxxxxx is not a known path in git. It also complains that the CMakelists.txt file is missing from the tools folder. (copying the file there does not fix the issue) After that error I get a collection of errors. Interestingly I had several problems with the shaderxxxxxx.h files having syntax errors. I tried downloading new clones and starting over from the beginning and the syntax errors seemed to move around, like the decompression was corrupt. There would be sequences like this in the files: ... 11, 116, , 23, ... and ... 11, 116, }; , 45, 123, ...

In any case I was not able to build successfully.

Platform / Environment

Windows 11 with Visual Studio 2022 both latest version. I am currently using SK 0.3.5 with no issues.

Logs or exception details

Attached the error log and the build raw output. _If you have an exception, details about that would also be essential._ [SK0-3-6_build errors list.zip](https://github.com/StereoKit/StereoKit/files/8883970/SK0-3-6_build.errors.list.zip)
Remote rendering on PC with StereoKit

Hi Nick,

Dose StereoKit support remote rendering on PC? I am wondering if I can use a PC to render the frames and send them back to HoloLens. It's kind of like Holographic remoting.

Thanks a lot.
BVH for ray-triangle intersection tests

Here's an initial implementation (C++ parts only, and 3 lines of C#) of using a bounding-volume hierarchy for ray-triangle intersection testing. This is based on existing BVH code I had laying around, plus some inspiration from Jacco Bikker's blog series on BVH construction (see bvh.cpp comments). It contains a fair amount of tracing printf()s, as a code such as this is challenging to debug otherwise, so for now those are best to leave in (I'm sure we'll find bugs ;-)).

I also added a small standalone C++ test for checking with different models and rays, but it's mostly for debugging right now. The next step would be to add the C# layer for easier testing, i.e. interactive ray casting in 3D, possibly as part of StereoKitTest.

This has only been tested with a few 3D models, but ranging from small ones to complex ones (up to 500k triangles, and even a 24M triangle model). Construction seems fairly quick, given that it's not very optimized (models of up to 50k triangles in less than 50ms). ~~Construction has had more testing than intersection testing, as the latter can be better done interactively (as mentioned above).~~

~~One thing I haven't figured out is why the added model_ray_intersect_bvh() does not return the same results as using mesh_ray_intersect() on each of the submeshes and taking the closest hit. In most models I tested the subparts do not have a non-identity transform, so that should not influence the outcome, yet it does.~~
Use correct LUID when creating OpenXR session

I tried StereoKit with the Oculus runtime and ran into the issue at openxr.cpp#L165 because the laptop I was using has Optimus and creates an integrated rather than discrete graphics device.

App on HoloLens 2 crashes with "[SK error] sk_gpu: CreateBuffer failed!"

Description

I start my UWP app on HoloLens 2. The app has one windows with text and one with an image looking like this: I picked the Windows with gestures to reposition. However, sometimes the error also happens without gestures. After some time (30 - 90sec) the error appears

Platform / Environment

UWP on HoloLens 2

Logs or exception details

2022-02-10 01:03:23.345 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveApplication(): created.
2022-02-10 01:03:23.755 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Using audio backend: ISAC
2022-02-10 01:03:23.836 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Initialization successful
2022-02-10 01:03:23.885 | FluoroImageAcquisitionServer | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionServer(): created.
2022-02-10 01:03:23.885 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionManager(): created.
2022-02-10 01:03:23.925 | RestServer[0.0.0.0:9005][/ImageAcquisition] | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Start(): started listening Listener[http://+:9005/ImageAcquisition/]
2022-02-10 01:03:23.925 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:03:23.925 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:03:23.925 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:06:31.925 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveMrSystem(): created.
2022-02-10 01:06:32.388 | AppSettings          | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  AppSettings.ReadString(): PelvisSiLiveMrConfigurationPath[PelvisSiLiveMr.xml]
2022-02-10 01:06:32.405 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  LoadConfiguration(): Data loaded successfully
Scene[LogWindow[Enabled[True] Pose[-1, 0.5, -1, 1, 1, 5] Size[100, 0] FontSize[] FontSize[0.012]] XrayImage[Enabled[True] Pose[1, 0.5, -1, -1, 1, 5] Size[100, 0]]]
ProNavigation[NavigationServerUri[http://0.0.0.0:9005/ProNavigation]]
FluoroImageAcquisition[ImageAcquisitionServerUri[http://0.0.0.0:9005/ImageAcquisition]]

2022-02-10 01:06:32.405 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  PelvisSiLiveApplication(): created.
2022-02-10 01:06:32.845 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Using audio backend: ISAC
2022-02-10 01:06:32.889 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] Initialization successful
2022-02-10 01:06:32.917 | FluoroImageAcquisitionServer | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionServer(): created.
2022-02-10 01:06:32.917 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  FluoroImageAcquisitionManager(): created.
2022-02-10 01:06:32.949 | RestServer[0.0.0.0:9005][/ImageAcquisition] | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Start(): started listening Listener[http://+:9005/ImageAcquisition/]
2022-02-10 01:06:32.949 | FluoroImageAcquisitionManager | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:06:32.949 | PelvisSiLiveMrApplication | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:06:32.949 | PelvisSiLiveMrSystem | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  Init(): successful.
2022-02-10 01:07:54.697 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:54.702 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!

The error keeps looping with some additions:

2022-02-10 01:07:58.906 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.906 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrWaitFrame [XR_ERROR_OUT_OF_MEMORY]
2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.977 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.987 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:58.997 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.002 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.002 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.005 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.005 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrEndFrame [XR_ERROR_RUNTIME_FAILURE]
2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.017 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.027 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.037 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.046 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Info      |  [SK info] xrBeginFrame [XR_ERROR_RUNTIME_FAILURE]
2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!
2022-02-10 01:07:59.057 | StereoKit            | TID[     1] | W:0/768/768 C:0/ 768/768 | Error     |  [SK error] sk_gpu: CreateBuffer failed!

The app finally "crashes". However, HoloLens 2 does not create a crashdump even if the app is marked to create one.

[Docs-Sprite.FromFile] Is it true, Atlased sprites are not implemented?

Getting this when trying to use an Atlas file. [SK diagnostic] sprite_create: Atlased sprites not implemented yet! Switching to single. [SK warning] A GPU asset is blocking its thread until the main thread is available, has async code accidentally shifted execution to a different thread since SK.Initialize?

I've spent days building all the supporting structures for atlased handling and get down to this line straight out of the documentation, Note: fi is a FileInfo object val.Key is a string with the value "00001" where this points to the atlas image 00001.png.

var sprite = Sprite.FromFile(fi.FullName, SpriteType.Atlased, val.Key);

It is in fact hanging the thread. The code never returns after the call to the method.

There are some other issues around this call. If the file name is invalid a hard exception is thrown somewhere down low in the C++ that is not trappable by the C#.

Unable to run example application inside an OpenXR overlay using XR_EXTX_overlay API layer

Description

What's happening? What are you expecting? More details are better.

I'm trying to get the sample StereoKit project to work inside an OpenXR overlay with the XR_EXTX_overlay API layer. When I try to run the project with SKSettings.overlayApp set to true, I'm getting an error that says:

[SK info] Couldn't create an OpenXR session, no MR device attached/ready? [XR_ERROR_INITIALIZATION_FAILED]

And the application then runs in flatscreen mode.

SystemInfo.overlayApp is returning True.

Platform / Environment

Windows/Linux/Android?
- Windows.
Which OpenXR runtime, or is this the flatscreen simulator?
- SteamVR, although logs suggest that Vive OpenXR: Vive SRanipal runtime is used even when SteamVR runtime is explicitly specified with the XR_RUNTIME_JSON environment variable.
What XR device is involved?
- HTC Vive Pro 2.
C# or C++?
- C#.
Are you using StereoKit templates, or are you building SK from scratch?
- I'm using the StereoKit .NET Core template from the Visual Studio marketplace.
Are your graphics drivers up-to-date? Etc.
- Yes, Nvidia driver version 472.12.

XR_EXTX_overlay compiled using the OpenXR SDK version 1.0.15 with the instructions on the repository

Logs or exception details

Running the example application with SKSettings.overlayApp set to true and the following environment variables:

XR_ENABLE_API_LAYERS=xr_extx_overlay
XR_API_LAYER_PATH=C:\Overlay_OpenXR
XR_LOADER_DEBUG=all
XR_RUNTIME_JSON=C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json

Contents of C:\Overlay_OpenXR folder (referred to as $WORKING_DIRECTORY in the LunarG XR_EXTX_overlay repo):

avatar1.png
avatar2.png
FreeImage.dll
FreeImagePlus.dll
highlighted1.png
openxr_loader.dll
openxr_loaderd.dll
openxr_loaderd.lib
OverlaySample.exe
selected1.png
xr_extx_overlay.dll
xr_extx_overlay.json
xr_extx_overlay.lib

Logs:

[SK diagnostic] Initializing StereoKit v0.3.6 Win32 x64...
[SK diagnostic] Initializing Assets
[SK diagnostic] Initializing Platform
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer xr_extx_overlay using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 4 layers enabled and runtime interface - created instance = 0x00000241646e6420
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
[SK diagnostic] sk_gpu: Using Direct3D 11: NVIDIA GeForce RTX 3080
[SK diagnostic] Starting mixed reality mode
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
[SK diagnostic] available: XR_HTC_facial_tracking
[SK diagnostic] available: XR_HTC_vive_srworks_pass_through
[SK diagnostic] available: XR_KHR_vulkan_enable
[SK diagnostic] available: XR_KHR_vulkan_enable2
[SK diagnostic] available: XR_KHR_D3D12_enable
[SK diagnostic] available: XR_KHR_opengl_enable
[SK diagnostic] available: XR_EXT_win32_appcontainer_compatible
[SK diagnostic] available: XR_KHR_visibility_mask
[SK diagnostic] available: XR_KHR_binding_modification
[SK diagnostic] available: XR_EXT_frame_composition_report
[SK diagnostic] available: XR_EXT_hand_joints_motion_range
[SK diagnostic] available: XR_HTC_vive_cosmos_controller_interaction
[SK diagnostic] available: XR_HTCX_vive_tracker_interaction
[SK diagnostic] available: XR_UNITY_hand_model_pose
[SK diagnostic] available: XR_VALVE_analog_threshold
[SK diagnostic] available: XR_EXT_dpad_binding
[SK diagnostic] available: XR_FB_display_refresh_rate
[SK diagnostic] available: XR_EXT_debug_utils
[SK diagnostic] REQUESTED: XR_EXT_hand_tracking
[SK diagnostic] REQUESTED: XR_EXT_eye_gaze_interaction
[SK diagnostic] REQUESTED: XR_MSFT_scene_understanding
[SK diagnostic] REQUESTED: XR_EXTX_overlay
[SK diagnostic] REQUESTED: XR_KHR_D3D11_enable
[SK diagnostic] REQUESTED: XR_KHR_win32_convert_performance_counter_time
[SK diagnostic] REQUESTED: XR_KHR_composition_layer_depth
[SK diagnostic] REQUESTED: XR_EXT_hp_mixed_reality_controller
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_hand_tracking
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_facial_tracking
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_srworks
[SK diagnostic] OpenXR layer found: xr_extx_overlay
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer xr_extx_overlay using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 4 layers enabled and runtime interface - created instance = 0x0000024165441ec0
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
[SK diagnostic] Using system: Vive OpenXR: Vive SRanipal
[SK diagnostic] Platform supports single-pass rendering
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
[SK diagnostic] Rejecting OpenXR's provided hand tracking extension due to the suspicion that it is inadequate for StereoKit.
[SK diagnostic] OpenXR depth LSR ext enabled!
[SK diagnostic] OpenXR world occlusion enabled! (Scene Understanding)
[SK info] Couldn't create an OpenXR session, no MR device attached/ready? [XR_ERROR_INITIALIZATION_FAILED]
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
[SK info] MixedReality display mode failed, falling back to Flatscreen
[SK diagnostic] Starting flatscreen mode
[SK diagnostic] Created swapchain: 1280x720 color:rgba32_linear depth:depth32
[SK diagnostic] Initializing Audio
[SK diagnostic] ISAC audio backend not available, falling back to miniaudio! It's likely the device doesn't have Windows Sonic enabled, which can be found under Settings->Sound->Device Properties->Spatial Sound.
[SK info] Using audio backend: WASAPI
[SK diagnostic] Initializing Defaults
[SK diagnostic] Initializing World
[SK diagnostic] Initializing Sprites
[SK diagnostic] Initializing Lines
[SK diagnostic] Initializing UI
[SK diagnostic] Initializing Renderer
[SK diagnostic] Initializing Physics
[SK diagnostic] Initializing Input
[SK info] Initialization successful

When running the application with exact same setup, but with SKSettings.overlayApp set to false, the following exception occurs in openxr.cpp:

Running the application without enabling the XR_EXTX_overlay API layer just with the following environment variables works as expected:

XR_LOADER_DEBUG=all
XR_RUNTIME_JSON=C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json

Here are the logs for when the application is running as expected and not in an OpenXR overlay:

[SK diagnostic] Initializing StereoKit v0.3.6 Win32 x64...
[SK diagnostic] Initializing Assets
[SK diagnostic] Initializing Platform
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 3 layers enabled and runtime interface - created instance = 0x000001f5b83d6240
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader terminator
Verbose [GENERAL | xrDestroyInstance | OpenXR-Loader] : Completed loader trampoline
Info [GENERAL | xrDestroyInstance | OpenXR-Loader] : RuntimeInterface::UnloadRuntime - Unloading RuntimeInterface
Info [GENERAL |  | OpenXR-Loader] : RuntimeInterface being destroyed.
[SK diagnostic] sk_gpu: Using Direct3D 11: NVIDIA GeForce RTX 3080
[SK diagnostic] Starting mixed reality mode
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - using environment variable override runtime file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::CreateIfValid - attempting to load C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
Info [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntime succeeded loading runtime defined in manifest file C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Entering loader trampoline
Verbose [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Completed loader trampoline
[SK diagnostic] available: XR_HTC_facial_tracking
[SK diagnostic] available: XR_HTC_vive_srworks_pass_through
[SK diagnostic] available: XR_KHR_vulkan_enable
[SK diagnostic] available: XR_KHR_vulkan_enable2
[SK diagnostic] available: XR_KHR_D3D12_enable
[SK diagnostic] available: XR_KHR_opengl_enable
[SK diagnostic] available: XR_EXT_win32_appcontainer_compatible
[SK diagnostic] available: XR_KHR_visibility_mask
[SK diagnostic] available: XR_KHR_binding_modification
[SK diagnostic] available: XR_EXT_frame_composition_report
[SK diagnostic] available: XR_EXT_hand_joints_motion_range
[SK diagnostic] available: XR_HTC_vive_cosmos_controller_interaction
[SK diagnostic] available: XR_HTCX_vive_tracker_interaction
[SK diagnostic] available: XR_UNITY_hand_model_pose
[SK diagnostic] available: XR_VALVE_analog_threshold
[SK diagnostic] available: XR_EXT_dpad_binding
[SK diagnostic] available: XR_FB_display_refresh_rate
[SK diagnostic] available: XR_EXT_debug_utils
[SK diagnostic] REQUESTED: XR_EXT_hand_tracking
[SK diagnostic] REQUESTED: XR_EXT_eye_gaze_interaction
[SK diagnostic] REQUESTED: XR_MSFT_scene_understanding
[SK diagnostic] REQUESTED: XR_KHR_D3D11_enable
[SK diagnostic] REQUESTED: XR_KHR_win32_convert_performance_counter_time
[SK diagnostic] REQUESTED: XR_KHR_composition_layer_depth
[SK diagnostic] REQUESTED: XR_EXT_hp_mixed_reality_controller
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_hand_tracking
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_facial_tracking
[SK diagnostic] OpenXR layer found: XR_APILAYER_VIVE_srworks
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_hand_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_facial_tracking using interface version 1 and OpenXR API version 1.0
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : ApiLayerInterface::LoadApiLayers succeeded loading layer XR_APILAYER_VIVE_srworks using interface version 1 and OpenXR API version 1.0
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering LoaderInstance::CreateInstance
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Entering loader terminator
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader terminator
Info [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance succeeded with 3 layers enabled and runtime interface - created instance = 0x000001f5b9207e90
Verbose [GENERAL | xrCreateInstance | OpenXR-Loader] : Completed loader trampoline
[SK diagnostic] Using system: Vive OpenXR: Vive SRanipal
[SK diagnostic] Platform supports single-pass rendering
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
Verbose [GENERAL | xrEnumerateApiLayerProperties | OpenXR-Loader] : Entering loader trampoline
Warning [GENERAL |  | OpenXR-Loader] : ReadLayerDataFilesInRegistry - failed to read registry location \ApiLayers\Explicit in either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER
[SK diagnostic] Rejecting OpenXR's provided hand tracking extension due to the suspicion that it is inadequate for StereoKit.
[SK diagnostic] OpenXR depth LSR ext enabled!
[SK diagnostic] OpenXR world occlusion enabled! (Scene Understanding)
[SK diagnostic] Creating view: PrimaryStereo color:rgba32_sRGB depth:depth32 blend:Opaque
[SK diagnostic] Setting view: PrimaryStereo to 3740x3740
[SK diagnostic] Initializing Audio
[SK diagnostic] ISAC audio backend not available, falling back to miniaudio! It's likely the device doesn't have Windows Sonic enabled, which can be found under Settings->Sound->Device Properties->Spatial Sound.
[SK info] Using audio backend: WASAPI
[SK diagnostic] Initializing Defaults
[SK diagnostic] Initializing World
[SK diagnostic] Initializing Sprites
[SK diagnostic] Initializing Lines
[SK diagnostic] Initializing UI
[SK diagnostic] Initializing Renderer
[SK diagnostic] Initializing Physics
[SK diagnostic] Initializing Input
[SK info] Initialization successful
[SK diagnostic] OpenXR session begin.
[SK diagnostic] Switched left controller profile to htc/vive_controller
[SK diagnostic] Switched right controller profile to htc/vive_controller

Providing Eye Gaze Sample Time
The OpenXR spec indicates that it is possible to query the actual sample time for a given eye gaze pose, which is useful for reasoning about high accuracy eye tracking, computing precise velocities over time, etc. It would be great if StereoKit could also provide that information.

I've implemented and locally tested one possible approach, which I was wondering if you might be willing to provide feedback on in a PR? I'm open to alternative approaches to surfacing this information, of course.

Basically, we just need a slightly modified version of openxr_get_space() to get the gaze space along with this extra timing information (it's specific to just gaze, and doesn't seem to be anything analogous for the head or hands).

bool32_t openxr_get_gaze_space(pose_t* out_pose, XrTime &out_sample_time, XrTime time) { if (time == 0) time = xr_time; XrEyeGazeSampleTimeEXT gaze_sample_time = { XR_TYPE_EYE_GAZE_SAMPLE_TIME_EXT }; XrSpaceLocation space_location = { XR_TYPE_SPACE_LOCATION, &gaze_sample_time }; XrResult res = xrLocateSpace(xr_gaze_space, xr_app_space, time, &space_location); if (XR_UNQUALIFIED_SUCCESS(res) && openxr_loc_valid(space_location)) { memcpy(&out_pose->position, &space_location.pose.position, sizeof(vec3)); memcpy(&out_pose->orientation, &space_location.pose.orientation, sizeof(quat)); out_sample_time = gaze_sample_time.time; return true; } return false; }

Then this sample time can be accessible in Input.cs as something like Input.EyesSampleTime.

Thoughts?
Add a `NotifyMode` to `UI.HSlider`

Describe the feature

This might be a bit too specific of a use case, but I'm looking for a way to determine if a slider is still being grabbed or not at the moment that e.g. HSlider() returns true.

An example use-case

The use case is for a slider value that controls a longer-running (remote) computation of the order of seconds, on-the-fly. I'd like to be able to have a user change the slider value with some precision, but only kick off the computation at the moment the user releases the slider grab, so as not to produce too many unnecessary recomputations while manipulating the slider, but still have reasonably fast updates. I can do it in the current API by starting a timer since the last value change and then force a recomputation after X milliseconds of no change, but it's a bit hacky and error-prone. However, I think adding this feature would not be easy to fit into the current API, as it would require an extra pass-by-ref value signaling the grab state.
Locatable camera, mediaframe
When I use MediaFrameSourceGroupto grab camera frames (together with their SpatialCoordinateSystem), how would I be able to use these frames in a Stereokit application? I looked at the QR code example where you get a pose for a SpatialNodeGuid, I am not clear though how this would apply for getting the pose of eg. the locatable camera with respect to the the world space (like the Unity CameraToWorldMatrix).

Eg. I would like to

Grab mediaframe

extract mediaframe spatialcoordinatesystem

get world coordinate system

calculate the CameraToWorldMatrix

I saw on your twitter that you planned to show an example, is that already possible with the current version of Stereokit together with the MediaFrame reference class?

Missing/mismatch in shared libs with VS Code on Linux

Description

I'm following the VSCode guide at https://stereokit.net/Pages/Guides/Getting-Started-VS-Code.html on Linux. It appears some native shared libs are missing, either because the StereoKit package expects them to be installed globally, or perhaps due to some project dependency that wasn't added.

When running dotnet run in a terminal window within VS Code I get:

[email protected] 15:50:~/concepts/skmap$ dotnet run
[SK error] Failed to load StereoKitC!
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'StereoKitC' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libStereoKitC: cannot open shared object file: No such file or directory
   at StereoKit.NativeAPI.sk_run(Action app_update, Action app_shutdown)
   at StereoKit.SK.Run(Action onStep, Action onShutdown) in C:\Data\Repositories\Stereo Kit\StereoKit\SK.cs:line 199
   at Program.Main(String[] args) in /home/melis/concepts/skmap/Program.cs:line 9

Looking a bit closer with LD_DEBUG=all dotnet run it seems /home/melis/concepts/skmap/bin/Debug/net6.0/runtimes/linux-x64/native/libStereoKitC.so is not finding two libs:

[email protected] 15:53:~$ ldd /home/melis/concepts/skmap/bin/Debug/net6.0/runtimes/linux-x64/native/libStereoKitC.so |grep not
	libGLEW.so.2.1 => not found
	libjsoncpp.so.1 => not found

I have both GLEW and jsoncpp installed as system packages, but the shared lib versions don't match. And I don't see them in the native code dir within the project:

[email protected] 15:55:~/concepts/skmap/bin/Debug/net6.0$ find . -name libGLEW\*
[email protected] 15:55:~/concepts/skmap/bin/Debug/net6.0$ find . -name \*jsoncpp\*
[email protected] 15:55:~/concepts/skmap/bin/Debug/net6.0$

Should these be installed separately through dotnet add package?

Platform / Environment

Arch Linux, VS Code 1.66.0, dotnet 6.0.110

SK Light Baking code crashes in `mesh_ray_intersect`

Description

mesh_ray_intersect crashes randomly when running SK Light Bake. Could be a multithreaded mesh update issue?

Yes, multithreading sounds like a plausible culprit.. since it's a 50/50 it crashes.. I also noted it's harder to make it crash on my slower notebook than it is on my main workstation. I noticed it's increasingly easy to get it to crash the heavier the mesh is.

Platform / Environment

SK v0.3.7-preview.7?

Logs or exception details
UI Handle overload with scaling

Hey Nick, I wrote a handle extension for scaling but noticed that you had previous and active pinch points in the source code, so I thought it could fit nicely here instead. I added the scaling to the clipboard for the DemoUI script for testing it out. Let me know what you think!
BVH Raycast in public API

BVH raycast was added, but still needs hooked properly into the public API! This may depend on some async mesh feature work #411, since the BVH build can take some non-trivial time on larger objects.
Android exit from app

Currently, when using SK.Quit on Android, the app exits just fine! But after two or three shutdowns, the OS will eventually pop up a warning about the app not closing properly.

Currently using Android.OS.Process.KillProcess(Android.OS.Process.MyPid()), but have also tried FinishAffinity and Finish, all seem to behave the same.
Calculating a `Pose` in front of the user

In HoloLens 2, at startup in SK Remote Rendering there is no way to begin a scene from a known position relative to the world coordinates. Problem: User is standing at a point. We want to start/go "home" the application from that point and we want to place all elements in the scene based on a line directly in front of the user's head position. The head position is completely arbitrary only have meaning to the user in the mixed reality. The user wants all the visible assets in front of him. The position of the elements in the world coordinates has not yet been decided by the user. Concrete example. User is at a conference demonstrating the HoloLens 2 with SK as the platform. The world location is unimportant. The person wants to stand at a point of beginning. Whatever direction his head is facing at the moment the app HoloLens 2 Remote Rendering Player connects the SK rendering should be the start position. And there is a need to repeat this where the user has a menu button that he pushes to "Start" and the scene begins at the current head position. How could I accomplish this?

Drawing with Text.Add does not show text in Window

Description

Using this simple window does not draw text:

		UI.WindowBegin(MenuHeaderText, ref infoPose, infoPoseSize * U.cm);
		StereoKit.Text.Add(DisplayText, Matrix.TR(new Vec3(0f, 0, -0.001f), Quat.LookDir(0, 0, -1)), 
			infoWindowSize, TextFit.Wrap, style,
			TextAlign.XLeft, TextAlign.CenterLeft, infoWindowCenter.x);
		UI.WindowEnd();

Where the infoPoseSize is a Vec2 30,16 and infoWindowsSize is a Vec2 = infoPoseSize * .999f. DisplayText is multiple lines of text with new line characters. The goal is to draw text inside the Window left aligned inside a margin box.

		UI.WindowBegin(MenuHeaderText, ref infoPose, infoPoseSize * U.cm);
		StereoKit.Text.Add(DisplayText, Matrix.TR(new Vec3(0f, 0, -0.001f), Quat.LookDir(0, 0, -1)), 
			infoWindowSize, TextFit.Overflow, style,
			TextAlign.Center, TextAlign.Center, infoWindowCenter.x);
		UI.WindowEnd();

Text center is drawn centered on the infoWindowSize top left corner.

Any setting of TextAlign other than Center, no text is visible.

Platform / Environment

Windows 11, VS 2022, SK 0.3.7 preview 7

Logs or exception details

Logs are super helpful, please include them if they're at all relevant!

If you have an exception, details about that would also be essential.

StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR!

StereoKit Features:

Getting started

Roadmap

Dependencies

Owner

Nick Klingensmith

Comments

Cannot build out of the box latest 0.3.6.

Description

Platform / Environment

Logs or exception details

Remote rendering on PC with StereoKit

BVH for ray-triangle intersection tests

Use correct LUID when creating OpenXR session

App on HoloLens 2 crashes with "[SK error] sk_gpu: CreateBuffer failed!"

Description

Platform / Environment

Logs or exception details

[Docs-Sprite.FromFile] Is it true, Atlased sprites are not implemented?

Unable to run example application inside an OpenXR overlay using XR_EXTX_overlay API layer

Description

Platform / Environment

Logs or exception details

Providing Eye Gaze Sample Time

Add a `NotifyMode` to `UI.HSlider`

Describe the feature

An example use-case

Locatable camera, mediaframe

Missing/mismatch in shared libs with VS Code on Linux

Description

Platform / Environment

SK Light Baking code crashes in `mesh_ray_intersect`

Description

Platform / Environment

Logs or exception details

UI Handle overload with scaling

BVH Raycast in public API

Android exit from app

Calculating a `Pose` in front of the user

Drawing with Text.Add does not show text in Window

Description

Platform / Environment

Logs or exception details

Related tags

A set of open c++ game development tools that are lightweight, easy-to-integrate and free to use. Currently hosting a magicavoxel .vox full scene loader.

A minimalist library with basic facilities for developing interactive real-time 3D applications, with a strong emphasis on simplicity and ease of use.

OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation

Open source Altium Database Library with over 147,000 high quality components and full 3d models.

Vizzu is a free, open-source Javascript/C++ library for animated data visualizations and data stories.

An Open-Source subdivision surface library.

A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

Modern C++14 library for the development of real-time graphical applications

SPIRV-Reflect is a lightweight library that provides a C/C++ reflection API for SPIR-V shader bytecode in Vulkan applications.

Open 3D Engine (O3DE) is an Apache 2.0-licensed multi-platform AAA Open 3D Engine

appleseed is an open source, physically-based global illumination rendering engine primarily designed for animation and visual effects.

An open-source implementation of Autodesk's FBX

The open-source tool for creating of 3D models

Cocos2d-x is a suite of open-source, cross-platform, game-development tools used by millions of developers all over the world.

Tesseract Open Source OCR Engine (main repository)

A completely free, open-source, 2D game engine built on proven torque technology.

Open-Source Vulkan C++ API

ZBar Bar Code Reader is an open source software suite for reading bar codes from various sources

Open-source, cross-platform, C++ game engine for creating 2D/3D games.