I'm reminded of the lyrics from Pink Floyd's Time:
"And then one day you find
Ten years have got behind you
no one told you when to run
you missed the starting gun"
The longer you live, the more tiny little mistakes you make. Things that at the time you could have done better, if you'd known, if you'd been a bit more careful. And these weigh on you, emotionally, pretty consistently.
And while it's pretty absurd, in the story, such tiny mistakes having such outsized consequences, the story reminds us that such severe consequences are well within the realm of possibility. People do lose limbs off of little, careless mistakes. Doubly so with all the incredibly concentrated sources of energy we have in the modern world - power tools, automobiles, explosives.
Would one really lose ten years trying to pick out a single Netflix show? No. But could one wake up one day and realize that they'd accomplished nothing of note for a decade, that all their free time was dumped into Netflix shows that weren't even that good?
So, what do you do with all that? Memento Mori, I guess.
I recently got into splatting. I looked for some good all-in-one tutorials, but didn't find any, and mostly muddled through through trial and error and LLM assistance. I present this workflow as a straight-line pipeline, though in practice it took a lot of iteration and backtracking and rework to get the final result. Here's what worked for me:
I captured a video on a smartphone camera, using the OpenCamera app. Specifically, this video was captured with exposure locked, framerate locked, focus locked, fairly high framerate and resolution. I walked slowly and carefully around an outdoor scene, trying to get fairly good coverage from multiple angles. I took roughly 20 minutes of video, weighing 19GB.
This video was sampled into individual image frames at about 5fps using ffmpeg. There's room for experimentation and improvement here, an adaptive, coverage-aware sampling strategy would be better. But fixed 5fps was Good Enough (tm). This resulted in roughly 8,000 images at 4k. This was a pretty hefty dataset for my limited 1080, but I made it work.
I then generated masks for these images, to ignore transient objects during the splat training. (i.e. to cut out people who transiently walked through the scene). For this I used Cutie (https://github.com/hkchengrex/Cutie). For outdoor scenes, it can also make sense to mask out low-parallax areas like faraway mountains or especially the sky, as these are difficult to train correctly. If masks are generated for some images, you'll need at least placeholder masks for the all of them. In the end I've got about 8,000 PNGs that are monochrome black/white masks.
Then the images are handed to COLMAP (https://github.com/colmap/colmap), using the 'global mapper' option. This registers the camera positions in 3D space, and generates a crude point cloud that's good for sanity-checking. This step required a fair bit of iteration to get right. The full reconstructed output from COLMAP is not necessary, only the pose-estimate .bin files. The output directory here was about 500MB for this step for me.
With COLMAP registration done, the next step is the actual training. I found two useful pieces of software for this, with different tradeoffs.
Brush (https://github.com/ArthurBrussee/brush). Was very straightforward to install and use, requiring very little in external dependencies and setup. It was also pretty speedy on training, and gave good results. Minor modifications to the training process were possible by editing source, though I didn't get too wild here. Brush takes the *.bin files from COLMAP, plus the original images directory, and the masks directory if it exists. Run on its own, this could produce gaussian splat .ply files, 500-800MB in size, containing 1-10M splats. More than that and my poor little 8GB of VRAM OOM'd.
nerfstudio (https://github.com/nerfstudio-project/nerfstudio) Was also useful, as many research papers get implemented in its framework. In particular, for this outdoor scene, I used wild-gaussians (https://github.com/jkulhanek/wild-gaussians/) to generate just a sky sphere (to help seed low-parallax areas in my particular dataset), stopped training, and used this as an init.ply to pass to brush.
I then set up a very simple viewer website, using SuperSplat (https://github.com/playcanvas/supersplat). I used supersplat's editor to align the splat's coordinate system with the rotation and scaling that I wanted, and then exported an optimized .sog file, roughly 1/10th the size. .sog is nominally open-standards, though I'm not aware of any other projects using the format. This gave fairly good framerates and adequate controls across a variety of platforms.
As a little bit extra, supersplat's splat-transform CLI tool was used to generate a crude collision mesh for the scene, enabling a walking mode that respected object boundaries.
If there's interest I can post my results, I got a bit sidetracked with other projects and other splats, and this particular one I got fiddling with some more cleanup. I can get it up with a few more hours work. But hopefully that's a good start, all of these are fully FOSS, and resulted in a good-looking splat.
I've lived in Japan for a few months. I was about halfway through the article, thinking about how it seemed to be a counter-example, before the author called out Japan specifically.
For all the other differences in culture, the attribute of "People Actually Care" seems to have a huge impact on how pleasant a place it is to visit or live.
I don't know why it seems to be the case there. I don't know how to replicate it. I don't think it's magic. I've heard people bandy about the theory of cultural homogeneity. That might be a _factor_, but I doubt it's the full story.
I suspect if you dig into it, differences in economics are a major factor. In the US, it feels like caring is actively punished, economically. Caring is nice, but someone can only _afford_ to care if their other needs are met.
I also wonder if density is a major factor - not so much for the difference in economy of scale, but the difference of "if my physical space is incredibly constrained, I'm both more incentivized to keep it looking nice, and there's less of it to keep looking nice."
And, of course, it's not like Japan is some kind of otherworldly utopia. There's serious tradeoffs and differences, there's negatives compared to other countries. But it does seem like almost everyone, everywhere, just... puts in a bit more effort. Takes a little bit more time.
The longer you live, the more tiny little mistakes you make. Things that at the time you could have done better, if you'd known, if you'd been a bit more careful. And these weigh on you, emotionally, pretty consistently.
And while it's pretty absurd, in the story, such tiny mistakes having such outsized consequences, the story reminds us that such severe consequences are well within the realm of possibility. People do lose limbs off of little, careless mistakes. Doubly so with all the incredibly concentrated sources of energy we have in the modern world - power tools, automobiles, explosives.
Would one really lose ten years trying to pick out a single Netflix show? No. But could one wake up one day and realize that they'd accomplished nothing of note for a decade, that all their free time was dumped into Netflix shows that weren't even that good?
So, what do you do with all that? Memento Mori, I guess.