FYI, a microVM isn't a small VM, nor necessarily one that has been allocated little RAM.
The term comes from the fact that a microVM is launched with Firecracker (a Virtual Machine Monitor), and Firecracker itself, as far as Virtual Machine Monitors go, doesn't consume many resources per VM and can start them relatively fast.
Given this, an 8GB agent VM started with Firecracker would still be a microVM. Firecracker doesn't place restrictions on how much memory you can assign it.
Current serverless offerings come with a substantial litany of complex, frustrating and expensive issues that together constitute a far cry from the promise of serverless. In this post we look into why that is, what serverless should be, and how to get there.
On kraft.cloud we use Dockeffiles to build into extremely specialized VMs for deployment. With this in place, we can have say an nginx server cold started and ready to serve at a public URL in about 20 millis (not quite the 10ms you mention, but in the right ballpark, and we're constantly shaving that down). Heavier apps can take longer of course, but not too much (e.g., node/next < 100ms). Autoscale and scale to zero also operate in those timescales.
Underneath, we use specialized VMs (unikernels), a custom controller and load balancer, as well as a number of perf tweaks to achieve this. But it's (now) certainly possible.
We don't need all of those layers and abstractions of course. But if we do things right we also don't need to go the bare metal server route -- cloud platforms, if done right, can provide both strong, hardware-level (read: vm) isolation plus fast starts.
On kraft.cloud (shameless plug) we build extremely specialized VMs (aka unikernels) where most of the code in them is the application code, and pair this with a fast, custom controller and other perf tweaks. We use Dockerfiles to build from, but when deploying we eliminate all of those layers you mention. Cold boot times are in milliseconds (e.g., nginx 20ms, a basic node app ~50ms), as are scale to zero and autoscale.
Actually it means both, in an unfortunate case of term overload. Though I can understand the embedded/IoT world being frustrated by this, as the term existed first within that context.
The edge within this context means running a server close, in terms of Internet latency, to users. For example, if a user if sending a request from Germany, then the response should come from a server running in say Frankfurt, not the US. There are now many providers that allow devels to deploy services at many different locations at once, and to ensure that client requests are routed to the closest available location. An understandable source of confusion is that wasm comes from the browser world, but it's also possible to run it as standalone (no browser) server code.
Also not to be confused with the term edge within the context of IoT/embedded, where the edge is devices running at the very edge of the Internet, e.g., factory floors, trucks, etc.
Yes, though I'd like to point out that "scale to zero" is a loose definition to mean anything that can be transparently scaled to 0 whenever an app/service is idle, and then wake up when traffic to the service arrives once again.
The problem in practice with Cloud Run (and similar products from other providers) is that it can take seconds or minutes for the platform to detect idleness, during which you're still paying, and then seconds to wake up -- during which users/clients have to wait for a response or possibly leave the service/site.
For my taste, real scale to 0 would be: detection and scale to 0 within < 1 second of a idleness, and wakeup within an RTT, such that the mechanism is transparent to end users.
As a shameless plug, this is what we do at kraft.cloud (based on years or research, LF OSS work, unikernels, a custom controller and overall non-negligible engineering effort).
In almost all cloud deployment, whether transparently or not, you'll have a hypervisor/VM underneath for hardware-level/strong isolation reasons. Using wasm on top of that stack only for isolation purposes might not be the best use of it. Having said that, if wasm is useful for other reasons (e.g., you need to run wasm blobs on behalf of your users/customers), then my (admittedly biased) view is that you should run these in an extremely specialized VM, that has the ability to run the blob and little else.
If you do this, it is entirely possible to have a VM that can run wasm and still only consume a few MBs and cold start/scale to 0 in milliseconds. On kraft.cloud we do this (eg, https://docs.kraft.cloud/guides/wazero/ , wazero, 20ms cold start).
On kraft.cloud we can (done internal stress tests for this) run thousands of specialized VMs (aka unikernels) scaled to zero, meaning that when a request for one of them arrives we can wake it up and respond within the timescales of an RTT. You can take it out for a spin, just use the -0 flag when deploying to do scale to 0 (https://docs.kraft.cloud/guides/features/scaletozero/).
Hi, on kraft.cloud we use FC, along with a custom controller and very specialized VMs (unikernels) to have extremely efficient deployments (eg, millisecond cold starts). For a PHP web server, for instance, we can cold start things in about 30ms (https://docs.kraft.cloud/guides/php/). It's also possible to run wasm workloads/blobs (e.g., https://docs.kraft.cloud/guides/wazero/).
The builds are based on Dockerfiles, but for deployment we transparently convert that to unikernels.
Fully agree, doing reactive autoscaling when the actual boot time is slow is an inherently hard problem. We've done years of research into building specialized VMs (unikernels) and fast controllers to be able to provide infra that allows VMs/containers to cold start, and thus autoscale/scale to zero in milliseconds (eg, a simple Node app cold starts in ~50 ms). If interested, you can try it out at kraft.cloud, or check out info about the tech in our blogs (https://unikraft.io/blog/) or the corresponding LF OSS project (www.unikraft.org).
[Disclaimer: I'm with KraftCloud]
For what it's worth, Firecracker/the VMM is only one part of the boot process. Among others, there's also the controller and the VM/OS itself that typically slow things down. In other words, it's not enough to just switch in Firecracker and expect cold starts to immediately drop to sub-second levels.
On kraft.cloud we've fundamentally redesigned the cloud stack to be able to cold start containers/VMs in milliseconds (eg, about 20 millis for nginx, about 50 millis for a basic Node app), and also scale them to zero and autoscale them in milliseconds. If interested there's more info about the tech in our blog posts https://unikraft.io/blog/ .
Author here, we did this, first by continuing the research alongside the creation of the Unikraft LF OSS project -- the result of which was the Eurosys 2021 best paper award (https://dl.acm.org/doi/10.1145/3447786.3456248).
Commercially, we leverage Unikraft on kraft.cloud to provide a cloud platform with millisecond semantics.
Hey, author/founder here, thanks for providing that answer, all correct there :) . I would also add that KraftCloud unikernels are built using Unikraft, and that its modularity allow us to tailor/specialize those images to obtain great perf.
Finally, we also had to design and implement a controller from scratch -- nothing out there provided the millisecond semantics and scalability we needed (plus we also did tweaks to network interface creation and a few other things to get the end to end experience to be fast).
The 125ms is using Linux. Using a unikernel and tweaking Firecracker a bit (on KraftCloud) we can get, for example, 20 millis cold starts for NGINX, and have features on the way to reduce this further.
Agree with epr's definition of a unikernel (and no, no mental decline on your part, this isn't always well defined).
First off, a unikernel is a virtual machine, albeit a pretty specialized one. They're are often based on modular operating systems (e.g., Unikraft), in order to be able to easily pick the OS modules needed for each application, at compile time. You can think of it as a VM that has a say NGINX-specific distro, all the way down to the OS kernel modules.
VMs provide what's called hardware-level isolation, running on top of a hypervisor like KVM, Xen or Hyper-V. Wasm runs higher up the stack, in user-space, and provides what's called language-level isolation (i.e., you could even create a wasm unikernel, that is, a specialized VM that inside runs wasm (eg, see https://docs.kraft.cloud/guides/wazero/). Generally speaking, the higher you go up the stack, the more code you're running and the higher the chances of a vulnerability.
The model of a single container within a VM just adds overhead. The ideal case would be to remove the container layer and have the application(s) within the container run directly in the VM (which hopefully only includes the libs and OS modules needed for the app to run, and nothing more).
This is the approach we take at kraft.cloud (based on the LF Unikraft project): use Dockerfiles to specify app/filesystem, and at deploy automatically convert to a lightweight VM (unikernel) without the container runtime/layer.
[One of the authors of the paper] I wouldn't recommend writing a network stack from scratch, that is a lot of effort. Instead, with the Unikraft LF project (www.unikraft.org) we took the lwip network stack and turned it into a Unikraft lib/module. At KraftCloud we also have a port of the FreeBSD stack.
[Author of the paper here] You hit the nail on the head, this is precisely what we do (kernel API compatibility) with the LF Unikraft project (the evolution of the 2017 paper) at www.unikraft.org, and kraft.cloud, a cloud platform that leverages Unikraft.
Correct, we never did release Tinyx, mostly because it was in a very unclean/researchy state = not ready for public consumption. In retrospect, we probably should have either (a) made it available in whatever state it was in or (b) put more cycles into it.
The term comes from the fact that a microVM is launched with Firecracker (a Virtual Machine Monitor), and Firecracker itself, as far as Virtual Machine Monitors go, doesn't consume many resources per VM and can start them relatively fast.
Given this, an 8GB agent VM started with Firecracker would still be a microVM. Firecracker doesn't place restrictions on how much memory you can assign it.