Stereokit – Open-source mixed reality library by Microsoft(github.com)
github.com
Stereokit – Open-source mixed reality library by Microsoft
https://github.com/StereoKit/StereoKit
47 comments
Somewhat meta, but why would Microsoft I use the Kit suffix which has been widely used by Apple adopted from NeXT?
That'd be my fault, me and my co-workers were working with a lot of tools that had the "Kit" suffix when we started on it :O Apple's definitely not the only one who does it!
>Somewhat meta, but why would Microsoft I use the Kit suffix which has been widely used by Apple adopted from NeXT?
The *Kit libraries from Apple are always capitalized. UIKit, SceneKit, WebKit, etc. The generic use of *kit for other libraries is pretty common though.
The *Kit libraries from Apple are always capitalized. UIKit, SceneKit, WebKit, etc. The generic use of *kit for other libraries is pretty common though.
Because the term is not allocated specifically to Apple.
Sure, but it has a 30 year lineage in their active product line. It would be like prefixing a library with Direct in the early 00s. It seems ignorant and derivative.
For the same reason they registered the domain typescriptlang.org instead of typescript.org: riding on the hype train created by others.
Besides, the project looks really interesting, I'll have a look into it.
Besides, the project looks really interesting, I'll have a look into it.
I think they probably used typescriptlang.org because typescript.org was registered in 2006 for a CMS called Typescript.
What is the hype train you associate with typescriptlang.org? "Lang"?
I associate it with "golang", the search-optimized way to refer to the language Go, from the world's largest search company.
Before golang there was rubylang, clang, racketlang, dlang, slang, etc. Seems like a pretty common convention for language websites not limited to a single organization. Be used “Kit”, but I’m not familiar with anyone else that did (and NeXT used it long before they did).
Sure. But would you refer to that as the "hype train"?
StereoKit seems like straight up cut to the fun. In my little free time all the other VR & MR platforms have so much cruft. Unity, endless configuration. Unreal, clone an fork of it and build it. Please I want to play & occasionally teach.
Any plans for AR support?
Any plans for AR support?
StereoKit was initially created with HoloLens 2 in mind, so it does plenty of AR! Also works on a North Star on Linux, as well as Varjo and Quest's passthrough AR. The only catch there is it doesn't do phone based AR like AR Core/Kit. Maybe someday, but for now StereoKit is focused on OpenXR compatible devices.
Thanks for the reply (and great work on Stereokit)
I was bucketing HL, Quest pass through, etc as MR and AR as being phone/screen based.
For example, I have been impressed by MRTK's AR support as a potential XR(?) framework (but Unity and the setup is the chore for experimentation)
Hypothetical on the "someday", are OpenXR and WebXR related? As in, could WebXR enable StereoKit to be AR "someday" before ARCore/Kit?
I was bucketing HL, Quest pass through, etc as MR and AR as being phone/screen based.
For example, I have been impressed by MRTK's AR support as a potential XR(?) framework (but Unity and the setup is the chore for experimentation)
Hypothetical on the "someday", are OpenXR and WebXR related? As in, could WebXR enable StereoKit to be AR "someday" before ARCore/Kit?
Haha, thanks! The terminology gradient gets pretty wild sometimes :)
On the "Someday" side of things, we have been actively working on WASM/WebXR support for StereoKit! It's a bigger feature that may still take some time, but the Simulator already runs in a browser from the C side, with the C# side mostly working. WebXR comes after that, and I've done some proof-of-concept work there to make sure it's feasible, which it totally is!
On the "Someday" side of things, we have been actively working on WASM/WebXR support for StereoKit! It's a bigger feature that may still take some time, but the Simulator already runs in a browser from the C side, with the C# side mostly working. WebXR comes after that, and I've done some proof-of-concept work there to make sure it's feasible, which it totally is!
Amazing. Just noticed I missed WASM in the github README (as I sit down to clone it).
This all sounds super exciting.
This all sounds super exciting.
I can recommend LÖVR as another "cut to the fun" VR framework. It's even easier to work with than StereoKit; no dependencies, just a binary. You program it in Lua.
https://lovr.org/
https://lovr.org/
How does this relate to or overlap with Microsoft MRTK? https://github.com/microsoft/MixedRealityToolkit-Unity
(@koujaku)
(@koujaku)
StereoKit and MRTK are currently part of the same team, so these people are my co-workers! :) We swap techniques, code, share design docs, etc. StereoKit joined the MR Middleware team towards the start of this year, and over time you'll likely see StereoKit move much closer to the Microsoft MRDL (Mixed Reality Design Language) and pick up similar features.
why build two toolkits that are similar?
MRTK is more of a library. This seems like a stand alone engine.
Yeah, this is correct! StereoKit is an XR App specific engine, which allows it to make API and architectural decisions that are very different from those a general purpose game engine would make. It's a completely separate alternative to Unity or Unreal.
MRTK for Unity benefits from the support of having the entire feature set of Unity with it, but it also comes with the complexity and knowledge requirements of a complex game engine. Game developers may like this a lot, but it is a lot to take in these days, and is way overkill for many applications.
StereoKit benefits from a simpler XR focused feature set, app focused design, and better integration with the normal C# ecosystem. It's basically just a Nuget package, so it's much more familiar to developers that come from a normal C# background, and can benefit from other libraries on Nuget! There's a lot of other details here too, I've done some pretty detailed talks that get into the weeds of the subject, should be able to find a handful on YouTube :)
MRTK for Unity benefits from the support of having the entire feature set of Unity with it, but it also comes with the complexity and knowledge requirements of a complex game engine. Game developers may like this a lot, but it is a lot to take in these days, and is way overkill for many applications.
StereoKit benefits from a simpler XR focused feature set, app focused design, and better integration with the normal C# ecosystem. It's basically just a Nuget package, so it's much more familiar to developers that come from a normal C# background, and can benefit from other libraries on Nuget! There's a lot of other details here too, I've done some pretty detailed talks that get into the weeds of the subject, should be able to find a handful on YouTube :)
You can use Nuget packages with Unity & MRTK, I have not had issues there
You can with some work but it's not really officially supported.
You end up with issues where the nuget package will not have meta files, will have multiple targets that unity does not support and will take tweaking to get things compiling again.
You end up with issues where the nuget package will not have meta files, will have multiple targets that unity does not support and will take tweaking to get things compiling again.
https://bobsutton.typepad.com/my_weblog/2012/07/dysfunctiona... (Historical Microsoft)
It's depressing how there's so little VR for Apple. Maybe they'll blow us all way with the iPhone of VR soon but at the moment my Macs have no VR love.
To make this related, I was prompted to this thought by the fact that the list of supported platforms does not include macOS
To make this related, I was prompted to this thought by the fact that the list of supported platforms does not include macOS
>It's depressing how there's so little VR for Apple. Maybe they'll blow us all way with the iPhone of VR soon but at the moment my Macs have no VR love.
There's probably no VR plans for macOS ever. Apple's version of VR, whatever it ends up being, will definitely not be a tethered experience. It's most likely we'll see a specific OS variant like with watchOS when/if they do it.
There's probably no VR plans for macOS ever. Apple's version of VR, whatever it ends up being, will definitely not be a tethered experience. It's most likely we'll see a specific OS variant like with watchOS when/if they do it.
Cross-platform has always been a pretty important element of StereoKit! Right now there's not really a Mac headset, so there's no rush to dive in just yet. When something does come out, we'll evaluate it and fit it somewhere on the priority list :)
I would be beyond shocked if Apple's VR set doesn't include a Thunderbolt port.
What’s the current landscape of mixed reality devices?
The time has never been better for a new “game engine” or 3D app framework. Unity is trying to be everything for everyone and hasn’t done a great job of taking care of its core audience: indie game developers.
Unity is the de facto choice for XR developers but it’s a poor application framework. Something like StereoKit that caters to non-game devs and can provide rendering, a flexible interaction model, UI, and a good multi-user app model could really steal Unity’s thunder for multi-platform development.
Unity is the de facto choice for XR developers but it’s a poor application framework. Something like StereoKit that caters to non-game devs and can provide rendering, a flexible interaction model, UI, and a good multi-user app model could really steal Unity’s thunder for multi-platform development.
I’d recommend you take a look at Godot. With in-built VR and C# support, a new major version peeking over the horizon, and many, many Unity devs coming over due to recent fallout in those circles, the time truly has never been better.
StereoKit started around 3 years ago, when the first pieces of OpenXR started coming out! OpenXR's native experience really showed just how good XR development _could_ be, and so we built StereoKit around that :)
And then yeah, if you're building apps, and not making a game engine, there's a lot of things you decide differently about the architecture! Like StereoKit's asset pipeline works at runtime, with user-space formats like GLTF. You can load up StereoKit's built-in file picker, and bring in an image or 3D model with 3 or 4 lines of code! Or grab it from a REST API, whatever.
And then yeah, if you're building apps, and not making a game engine, there's a lot of things you decide differently about the architecture! Like StereoKit's asset pipeline works at runtime, with user-space formats like GLTF. You can load up StereoKit's built-in file picker, and bring in an image or 3D model with 3 or 4 lines of code! Or grab it from a REST API, whatever.
I wonder if it can be embedded in the future in Unity.
StereoKit is an alternative to Unity that skips all the bloat, doesn't really work in concert with it. You could take some inspiration from StereoKit, and maybe port some code snippets, but you're better off using something like the MRTK if you're working in Unity!
Kudos for the Zig and V examples (or rather example skeletons), how very un-Microsoft-like to venture into such bleeding edge territory ;)
(the actually great thing is that the engine exposes a plain C API: https://github.com/StereoKit/StereoKit/blob/master/StereoKit...)
PS: double-kudos for making the API C99 designated-init-friendly, at least as far I can see from skimming over the source (e.g. "zero means default": https://github.com/StereoKit/StereoKit/blob/1bf86b281e883041...)
(the actually great thing is that the engine exposes a plain C API: https://github.com/StereoKit/StereoKit/blob/master/StereoKit...)
PS: double-kudos for making the API C99 designated-init-friendly, at least as far I can see from skimming over the source (e.g. "zero means default": https://github.com/StereoKit/StereoKit/blob/1bf86b281e883041...)
Thanks! Your work has actually been quite an inspiration for me personally, I've definitely picked up a few of your tricks here and there :)
Language bindings have always been a part of the long-term plan, C# is just the golden path we've optimized for! A good number of people do also use the C API, and there's some community driven effort to create rust bindings as well.
Language bindings have always been a part of the long-term plan, C# is just the golden path we've optimized for! A good number of people do also use the C API, and there's some community driven effort to create rust bindings as well.
I see font rendering is not adapted to 3D in the screenshot, so it is not yet ready for serious UI work. I presume curves are not rendered in an adapted way either, so everything will look pixelated around the edges.
StereoKit doesn't currently do SDF font rendering yet, no, but there is some shader work done to help make existing text a bit clearer than the naive case. It does support Unicode font rendering though, font glyphs are added to the font atlas on-demand to ensure that's a smooth and automatic experience!
The current implementation of SDF conversion is just a tad too slow to make on-demand glyph rendering comfortable right now, so there's a bit of additional optimization work that needs done before we can switch to that. We were just prioritizing language support over text clarity to start with.
The current implementation of SDF conversion is just a tad too slow to make on-demand glyph rendering comfortable right now, so there's a bit of additional optimization work that needs done before we can switch to that. We were just prioritizing language support over text clarity to start with.
Is it not possible to do serious UI work without fonts adapted to 3D?
I would say that the majority of people don't notice StereoKit uses regular raster for font rendering! However, that number is definitely not zero, and there's definitely people that can spot it right away, as evidenced. Designers in particular would probably spot it right away.
I'd say it's primarily a question of polish and feel! Those that can't spot it right away would still prefer SDF fonts in a side-by-side, and the difference becomes much more clear if you start getting your head really close to the text. I consider it to be an essential feature for a 1.0 version, but not a blocker for right now.
I'd say it's primarily a question of polish and feel! Those that can't spot it right away would still prefer SDF fonts in a side-by-side, and the difference becomes much more clear if you start getting your head really close to the text. I consider it to be an essential feature for a 1.0 version, but not a blocker for right now.
I actually worry more about looking at longer and smaller text in 3D. It is noticeably harder to read, which prevents many use cases. For instance, you would not read HN in VR using an RSS reader built with StereoKit.
Ah yeah I think there's a collection of issues related to small text within XR! The biggest one is really pixel density on XR headsets, which is always an issue for small text, or far-away text. XR also can't benefit from sub-pixel approaches to font rendering either. Since you can always move further from text, there's also the IRL issue that the text is just too far to make out!
The current raster technique should actually be fine for smaller text, there's some super-sampling going on in the shader that should help a bit here too! I'm not entirely certain what else could be improved on in this case besides font selection, I'd be open to suggestions or tips :)
The current raster technique should actually be fine for smaller text, there's some super-sampling going on in the shader that should help a bit here too! I'm not entirely certain what else could be improved on in this case besides font selection, I'd be open to suggestions or tips :)
I am not an expert in the area, just somebody who read up on it after realizing desktop in VR does not work out for me (and usually low res is not a problem, so it is about font aliasing).
Something like https://github.com/servo/pathfinder (newer) and as you mentioned SDF (older) are the latest approaches.
Something like https://github.com/servo/pathfinder (newer) and as you mentioned SDF (older) are the latest approaches.
Thanks for the link! The path based approach has been of interest to me, but I wasn't aware of an implementation other than Slug, which I have been avoiding. Guess I can finally do some real research on that topic now! :D
SDF is probably still the most realistic path for now, but I guess that may depend a bit on what comes up :)
SDF is probably still the most realistic path for now, but I guess that may depend a bit on what comes up :)
Stereokit is fantastic.
Ive been looking for a .net game engine to try - and love the runtime asset importing.
Are examples of its use for a traditional desktop use? Like screen space UI. And maybe mobile touch input? Or are both of those out of scope and XR is the focus. I am wanting to try a side project in it - but hoping to make it both VR+Desktop.
Are examples of its use for a traditional desktop use? Like screen space UI. And maybe mobile touch input? Or are both of those out of scope and XR is the focus. I am wanting to try a side project in it - but hoping to make it both VR+Desktop.
StereoKit very much prioritizes XR, I don't think I'd recommend doing traditional flat or hybrid XR/flat experiences with it! You might be able to do some flat 3d content, but the built-in UI won't be very helpful in this environment.