I've recently been going down the rabbit hole of creating a "fast start dev env" and it's interesting to see how this article differs from other approaches (codesandbox has some fantastic blogs, the fly.io blog on sprites has interesting pointers, e2b and daytona are related open source tools). Everyone has a different solution based on their tradeoffs.
I thought the memory snapshotting part in particular was clever since most container based systems don't bother (VM/firecracker based ones can use UFFD and call it a day), but by having emulated syscalls you can actually do single-process restore pretty well.
I am a bit dubious of the use of fuse (though it clearly works well!), and I wonder if ublk (what I ended up using) might alleviate some of the pain/magic in fuse tuning. I'd personally also be looking at forking gvisor to take a memfd which you enable UFFD on for the page loading (I have some firecracker patches where I do the same). It's nice because you can optimistically push pages, rather than waiting for the requests to come in. The series of three codesandbox blog posts are good background reading.
I'm super interested to hear more on anything you can share about your projects, or the niche of gov projects you're aware of - I've been doing some work with gov and haven't seen this requirement yet, so want to be prepared if it does come up.
One annoying part of using chroot if you're creating them on the fly is teardown - you have to manually invoke umount, and also take care to get this right for partially created chroots (maybe you detected an error after mounting proc, in the process of getting other files in place).
This was my original motivation in creating machroot (mentioned elsewhere in this thread) and having it use namespaces.
As a number of comments have noted, there are a bunch of different axes that chroot could be 'better' on - e.g. security and sandboxing.
I wrote https://github.com/aidanhs/machroot (initially forked from bubble wrap) a while ago to lean into the pure "pretend I see another filesystem" aspect of chroot with additional conveniences (so no security focus). For example, it allows setting up overlay filesystems, allows mounting squashfs filesystems with an overlay on top...and because it uses a mount namespace, means you don't need to tear down the mount points - just exit the command and you're done.
The codebase is pretty small so I just tweaked it with whatever features I needed at the time, rather than try and make it a fully fledged tool.
(honestly you can probably replicate most of it with a shell script that invokes unshare and appropriate mount commands)
Out of curiosity, what would you say the current state of the art is for full compilable decompilation? This is something I have a vague interest in but I'm not involved enough in the space to be on top of the latest and greatest tooling.
Interesting, that's helpful, thanks - so with the eventual arrival of memory64 and assuming I only wanted to target desktop systems and assuming browser implementations permit large allocations (e.g. 8GB) - large 64bit apps could work fine. I have a use case for this I've been poking at for a bit, but implementing my own version of cheerpx would be a lot of work, maybe I'll just wait!
On open source - I can only give you feedback as an outside fresh pair of eyes :) I incorrectly interpreted that it was full stack OSS based on the overall blog post 'vibe' and had to deliberately double check because I was aware of cheerpx beforehand. Perhaps it's just me. I look forward to the cheerpx blog post!
All the emulation of desktop machines in WASM I've seen so far have been for x86 - do you think there are significant additional hurdles for x86_64? Or is it just a matter of time?
Separately, one bit of feedback - it's cool that webvm is open source, but I think it's fair to ask you to be upfront that cheerpx itself is not (which is fine!) in the blog post itself where you talk about webvm licensing. If I wasn't already familiar with the wasm emulation space I would have felt rather misled.