OS Technologies to Watch(gfxmonk.net)
gfxmonk.net
OS Technologies to Watch
http://gfxmonk.net/2015/01/04/os-technologies-to-watch.html
45 comments
This article put into words the way I feel about Docker in the presence of Nix and GNU Guix. Docker is really a weak tool by comparison.
While Docker specifically is bit lackluster, I still feel like the underlying container concept/tech and nix would complement each other nicely. Nix for building the rootfs and lxc (or whatever suitable runtime) to run it.
That's exactly the kind of tool I'd love to see in 2015. Not only but in particular for Haskell development. nix seems to gain traction to set up your dev environment, but it's still pretty clunky to use at this point.
Do you mind elaborate more on what's wrong with self-contained VM?
I'm confused by your question. I don't have any problem with VMs. Perhaps you thought I was pro-container, anti-VM?
Oh, let me be a bit more clear. To quote the article
> The main push of docker seems to be for completely self-contained applications, basically a super cheap and consistent VM. But VMs are a hack, and docker in many ways is just as hacky.
So I just thought that the problem you have with Docker is similar to the author of the article (which is anti Docker/VM, since he dislike both).
> The main push of docker seems to be for completely self-contained applications, basically a super cheap and consistent VM. But VMs are a hack, and docker in many ways is just as hacky.
So I just thought that the problem you have with Docker is similar to the author of the article (which is anti Docker/VM, since he dislike both).
Oh, I see. I think VMs and containers are great for different reasons. I just don't like Docker. One of the cool features of Nix is that it can manage systems on bare metal, a VM, or a container.
Docker was written off with the caveat of, "but I still use it for my local development environment". I feel like that's a huge advantage for Docker. Maybe I missed it, but all the other mentioned technologies didn't try to solve the problem of providing a consistent development to production environment. That said, Docker still has a lot of problems around making the dev-to-production experience smooth, especially for teams that use OS X for development and run Linux in production.
That's always been the least interesting thing about Docker for me, since Vagrant solves that particular problem much more completely, and has been around a lot longer than Docker has.
Using Docker for this forces you to do things the Docker way, whereas Vagrant is much more OS and CM agnostic. (CM is configuration manager: you can use puppet, chef, ansible, shell, salt or even Docker: whatever you're already using in production).
Using Docker for this forces you to do things the Docker way, whereas Vagrant is much more OS and CM agnostic. (CM is configuration manager: you can use puppet, chef, ansible, shell, salt or even Docker: whatever you're already using in production).
My current web app is composed of 10 database hosts and 11 REST APIs. 21 VMs would blow my laptop's resources away while 21 containers run just fine. For me, that makes Docker a lot more attractive for local development. Also, Docker removes the need for a CM if you are into stateless deployments.
If you're running a large number of VM's, vagrant-lxc is awesome, so are offsite targets. (digital ocean, AWS, et cetera)
Vagrant also has more options for stateless deployments than Docker: docker, packer, nixos, et cetera.
Vagrant also has more options for stateless deployments than Docker: docker, packer, nixos, et cetera.
That's probably the most important thing to me. Also, docker will probably succeed in the enterprise world you, being a Linux based solution.
What a great list; I used this holiday season to build a similar list actually. Genode's potential is really understated in the post - it can apparently run a number of different microkernels including the formally verified seL4 which provides secure isolation of each individual driver, VM, or process. They seem to have some Linux interoperability going too. I really want to find the time to dig into this one some more.
Qubes is great, however the project really is a bit absolutist in my (amateur) opinion in placing all the trust in isolation: I still care about integrity of the isolated environments too, but Qubes seems to somewhat throw that baby out with the bathwater by, for example, insisting on passwordless sudo in all AppVMs.
I get where they are coming from - local privescs are a dime a dozen and they have a small team. Linux process isolation sucks but is it that worthless that we should deliberately introduce new vulnerabilities by leaving the doors completely wide open? When the skids find a new RCE in my crappy mail client or attachment viewer, the effort required to engineer exploits is massively reduced.
Qubes is great, however the project really is a bit absolutist in my (amateur) opinion in placing all the trust in isolation: I still care about integrity of the isolated environments too, but Qubes seems to somewhat throw that baby out with the bathwater by, for example, insisting on passwordless sudo in all AppVMs.
I get where they are coming from - local privescs are a dime a dozen and they have a small team. Linux process isolation sucks but is it that worthless that we should deliberately introduce new vulnerabilities by leaving the doors completely wide open? When the skids find a new RCE in my crappy mail client or attachment viewer, the effort required to engineer exploits is massively reduced.
> which provides secure isolation of each individual driver
seL4 is great and all, but this is simply not possible as long as hardware devices can access memory. For example, a graphics driver can program the GPU to overwrite arbitrary system memory, regardless of how drivers are "isolated" by the kernel.
seL4 is great and all, but this is simply not possible as long as hardware devices can access memory. For example, a graphics driver can program the GPU to overwrite arbitrary system memory, regardless of how drivers are "isolated" by the kernel.
> a graphics driver can program the GPU to overwrite arbitrary system memory
Unless there's a hardware IOMMU configured by Genode, Xen, etc, http://transpute.github.io/test1/related/
Unless there's a hardware IOMMU configured by Genode, Xen, etc, http://transpute.github.io/test1/related/
I'm surprised BOSH isn't brought up more when I see discussions on stateless deployment. Maybe I'm biased as I spent some months as an intern on that team but I definitely think it has a place in a thread like this. Check out http://github.com/cloudfoundry/bosh or the #bosh channel on freenode
I agree, BOSH is pretty darn cool. Once you have learned the quirks of it.
Currently I think a couple of things that stands in the way of it being more widespread.
* Its tedious to create a new release.
* The documentation is scattered.
* Distributing a release can be a bit tricky. Spiff + deployment templates doesn't really scale that well. For CF we have to pipe to a mod script just because the original templates are not flexible enough, i.e "./generate-manifest infra stub.yml | ./add-extra-releases-and-change-static-logging-levels-and-other-random-stuff > manifest.yml"
* Its sufficiently different even for a seasoned admin that it will take some time to get up and running and understanding how stuff fits together end to end. Currently it feels like the only way to understand BOSH is to understand BOSH.
Currently I think a couple of things that stands in the way of it being more widespread.
* Its tedious to create a new release.
* The documentation is scattered.
* Distributing a release can be a bit tricky. Spiff + deployment templates doesn't really scale that well. For CF we have to pipe to a mod script just because the original templates are not flexible enough, i.e "./generate-manifest infra stub.yml | ./add-extra-releases-and-change-static-logging-levels-and-other-random-stuff > manifest.yml"
* Its sufficiently different even for a seasoned admin that it will take some time to get up and running and understanding how stuff fits together end to end. Currently it feels like the only way to understand BOSH is to understand BOSH.
Those are really good points and they're actively working on improving all of the points you've mentioned. I can't encourage you enough to join the IRC channel or create Github issues, especially with regards to the documentation. I personally spent a significant portion of time working directly with the project manager on improving the documentation, but your feedback too is invaluable. Please speak up about these issues to the team or submit pull requests. They love to see this kind of feedback and these kinds of contributions.
For what it's worth, I tried to run the CloudFoundry release on my local BOSH installation and it froze my Macbook. I'm sure it's very useful in cloud environments or on beefier machines, but it's not very useable for local development vs something like Docker which doesn't require a bunch of VMs.
To be fair, deploying the CF release to your local machine with something like BOSH lite is not the most resource conservative. I think that you hit the nail on the head when you say it's intended for beefier machines or smaller cloud deployments. That being said CF release is huge. If you want to deploy your own software packages in a custom release against bosh lite for development purposes I think you might find that to be less temperamental.
I thought creating a BOSH release would be too much work but I will try that. What would be nice is if BOSH could abstract a cluster of VMs as a single Docker host and let me operate on containers instead of VMs (I'm not sure if that's possible at the moment without significant work).
Excellent article. I like the sound of OSv, which attempts to make appliances that run on hypervisors: https://github.com/cloudius-systems/osv
Sandstorm looks interesting. I'm slightly skeptical as the homepage has a link to their enterprise sales, yet it's hyped as a project of good will. I'd like to hear more about their business model. From a technical side, the project makes a lot of sense. But I'd rather see something like OSv be made to run on users' machines and phones and not just cloud systems, which would let app authors distribute directly to users with little hassle.
Sandstorm looks interesting. I'm slightly skeptical as the homepage has a link to their enterprise sales, yet it's hyped as a project of good will. I'd like to hear more about their business model. From a technical side, the project makes a lot of sense. But I'd rather see something like OSv be made to run on users' machines and phones and not just cloud systems, which would let app authors distribute directly to users with little hassle.
I really like the concept of MirageOS. I think its the only way to scale up above c10m [1].
In Mirage single application servers are able to git rid of a ton of the multiprocess kernel complexity. One really cool improvement/simplification I'd like to see: I'd like to remove all CPU level security (ie. context switches). For now I don't see the benefit of removing memory safety/abstraction but that could be discussed.
Something like a SQL DB is the perfect example. In production its the only application on the OS. It already has its own set of security concerns (the majority of which are not solved by context switching). Yes, it increases the scope of a DBs security concerns, but it allows the application to choose its own security model. The DB has as much power as the kernel now, so maybe the DB would reimplement context switches for all network activity (be as safe as the Kernel lets it be today)? Maybe instead it would optimize context switches away for a majority of calls except writes? The possibilities/optimizations only grow, at the cost of only a 5-10% code size increase (ie to implement the subset of kernel security features that it actually needs).
The other really nice thing about this could be: Imagine if Mirage allowed you to run a process with kernel level permissions and (as I understand it) it already allows that one process to run arbitrary OCaml on a selected CPU core. Imagine then the one process built into Mirage was the code for multi-process/multi-thread management with scheduling, message passing, security, signals and all the bells and whistles of a monolithic kernel.
You could essentially construct monolithic kernels on the same framework that allowed us to build optimized databases, optimized servers.
I think this set of lower level libraries, would be a game changer. If these libraries (framework) for building monolithic applications could be extracted out of the Linux kernel that would be huge since you could run it on any device Linux currently has drivers for (think embedded). However, even as it stands if you could create a framework for building Kernels with MirageOS on Xen, that would be really impressive.
[1] http://c10m.robertgraham.com/p/manifesto.html
In Mirage single application servers are able to git rid of a ton of the multiprocess kernel complexity. One really cool improvement/simplification I'd like to see: I'd like to remove all CPU level security (ie. context switches). For now I don't see the benefit of removing memory safety/abstraction but that could be discussed.
Something like a SQL DB is the perfect example. In production its the only application on the OS. It already has its own set of security concerns (the majority of which are not solved by context switching). Yes, it increases the scope of a DBs security concerns, but it allows the application to choose its own security model. The DB has as much power as the kernel now, so maybe the DB would reimplement context switches for all network activity (be as safe as the Kernel lets it be today)? Maybe instead it would optimize context switches away for a majority of calls except writes? The possibilities/optimizations only grow, at the cost of only a 5-10% code size increase (ie to implement the subset of kernel security features that it actually needs).
The other really nice thing about this could be: Imagine if Mirage allowed you to run a process with kernel level permissions and (as I understand it) it already allows that one process to run arbitrary OCaml on a selected CPU core. Imagine then the one process built into Mirage was the code for multi-process/multi-thread management with scheduling, message passing, security, signals and all the bells and whistles of a monolithic kernel.
You could essentially construct monolithic kernels on the same framework that allowed us to build optimized databases, optimized servers.
I think this set of lower level libraries, would be a game changer. If these libraries (framework) for building monolithic applications could be extracted out of the Linux kernel that would be huge since you could run it on any device Linux currently has drivers for (think embedded). However, even as it stands if you could create a framework for building Kernels with MirageOS on Xen, that would be really impressive.
[1] http://c10m.robertgraham.com/p/manifesto.html
That's actually an interesting point wiht regard to SQL DB: deploying PostgreSQL and juggling around with all the different permission/ login setting have always been one of my most dislike aspect of it. Considering that it's true most DB server would only run the DB, is there any unikernel for such task?
There's also bromium, which is very similar to qubes-os but backed by a big startup ,so it's polished for enterprise usage. The drawback is probably the price.
Ling, the Erlang VM directly on Xen seems neat.[0]
Folks have talked about composable OS frameworks / domain-specific OSes in the usual suspects of Erlang, OCaml, perhaps Haskell... especially useful for soft-realtime / embedded.
Irdis is neat.[1]
0: http://build.erlangonxen.org/
1: http://www.idris-lang.org/
Folks have talked about composable OS frameworks / domain-specific OSes in the usual suspects of Erlang, OCaml, perhaps Haskell... especially useful for soft-realtime / embedded.
Irdis is neat.[1]
0: http://build.erlangonxen.org/
1: http://www.idris-lang.org/
HaLVM is similar sort of project which uses the Haskell VM.
https://github.com/GaloisInc/HaLVM
https://github.com/GaloisInc/HaLVM
Also perhaps worth mentioning Baserock (http://wiki.baserock.org) which can reproducibly build an OS from sources in git and deploy the resulting artifact to various popular targets, kvm, virtualbox, openstack etc.
This seems to be associated with GENIVI, Yocto and OpenEmbedded for automotive. If this can reproducibly build any Yocto-based OS, are there plans to upstream this capability to Yocto?
There are videos of talks on both Genode and OpenMirage from the OS conference I organized back in November [1] - I tried to get a Qubes talk too, hopefully this year.
[1] https://operatingsystems.io/
[1] https://operatingsystems.io/
consider inviting the muen developers (http://muen.sk). They work on a verified separation kernel, which is not exactly a "new way" (SKs have literature going back to the 60s, but so does nearly everything else on that list, too), but it's certainly not common knowledge these days.
Yesterday, I've attended a container presentation held by Microsoft. They mentioned CoreOS (https://coreos.com/) for a potential alternative to Docker.
CoreOS is a Linux distribution which was originally designed to _run_ Docker. So it's not really an alternative to Docker.
I guess they meant Rocket (https://github.com/coreos/rocket). Rocket is an app container runtime similar to Docker, which is developed by the CoreOS group.
I guess they meant Rocket (https://github.com/coreos/rocket). Rocket is an app container runtime similar to Docker, which is developed by the CoreOS group.
You must be right, as a non-engineer I probably caught the wrong name during the conversation.
For windows users, a docker alternative is sandboxie. WinNT series has almost all required features ever since the early nineties - it's just not exposed in the Win32 UI. The Windows backwards compatibility layer and IE 10+ also use the sandbox functionality. Maybe Win10 comes with a easy to use sandbox UI?
An open WinNT alternative that is still in alpha is ReactOS.
An open WinNT alternative that is still in alpha is ReactOS.
Maybe not easy to use, but Win8+ includes Client Hyper-V to run Linux, FreeBSD and Windows virtual machines, http://www.howtogeek.com/196158/how-to-create-and-run-virtua...
I really like the idea of Qubes OS. I just wish a "lighter weight" method existed to sandbox apps or groups of apps just as well, and adopted by more mainstream operating systems, too.
Actually, Clive seems to be dead. There's been no activity on the Google Group since May 2014.
The other “alternative” to Docker is SmartOS, which combines KVM with Solaris zones and ZFS.
Not really.
SmartOS is an "operating system". The way it designed to decoupled may give the illusion of Docker alternative, but it certainly doesn't serve the main purpose of docker which is to only prepackaged dependencies/environment, not the whole machine.
Its the same on how LXC/KVM/Jails is not docker alternative.
SmartOS is an "operating system". The way it designed to decoupled may give the illusion of Docker alternative, but it certainly doesn't serve the main purpose of docker which is to only prepackaged dependencies/environment, not the whole machine.
Its the same on how LXC/KVM/Jails is not docker alternative.
[1] http://nymote.org/blog/2013/introducing-nymote/