That was in some way the original motivation for the project!
I think if you are reconstructing your own data the algorithm better just work, without input, ideally.
But, imagine you could add in generated videos. Lay down a camera path, tell it what to generate, and add it to the reconstruciton. A brush stroke one might say ;)
This is definitely still a big blocker to adoption. The goal is to get to a more all-in-one system. The splatting optimization can also help align cameras, if they don't start out entirely random, so any system to quickly provide a good "initial guess" will help here. At least for mobile devices, initialization from ARCore / ARKit poses should be enough.
Not a dumb question! This first version is still mainly targetted at people who are in this area and generate some excitement, I do hope to make this more accesible though!
The output is a 3D model. Specifically a "Gaussian Splat", which is a sort of fuzzy point cloud. There are some tools out there to view & edit these (besides Brush), eg. https://playcanvas.com/supersplat/editor.
I'm a bit out of the loop on async runtimes. I know Tokio is of course the big on, but, that seemed much to heavy to just run some tasks, and isn't very WASM compatbile afaik.
Otherwise there's smol, and maybe others? Would love to hear what a good web WASM compatible async framework is nowadays!
Love to hear it!! Most viewers take some shortcuts, like only sorting every so often, it's good to hear the difference is noticable :)
Training a splat requires a lot less setup with this, but does still require running COLMAP(https://github.com/colmap/colmap) first, which is still a big barrier... one thing at a time!
Wonder if Rust should pursue this as a feature as well - might be hard to promise this as a fully fledged feature when Cranelift comes into the picture though. Still it could be possible to ship a linker that can handle cross-compiling like Zig and compile C code with LLVM irregardless of the Rust backend.
Either way, hope Zig and Rust continue to inspire each other a bit!
Hard at work to make performance better - the "main" kernels are at least as fast as gSplat, so now need to remove other overheads.
That, and make splatting train more efficiently in general, lots of compute is wasted on small steps.
Ps: the web version takes a minute to warm up and is generally slower, do try a native version if you haven't yet!