Look Ma, no OS(slides.com)
slides.com
Look Ma, no OS
http://slides.com/technolo-g/intro-to-unikernels-and-erlang-on-xen-ling-demo#/
90 comments
I hated Reveal.js until I realized that I could press Space to advance, and Shift-Space to move backwards, and it will just Do The Right Thing.
Can you go down? Space will go down. Otherwise, Space will go to the right. It re-linearizes the presentation, while still preserving the 2D layout for quick navigation between major topics.
Can you go down? Space will go down. Otherwise, Space will go to the right. It re-linearizes the presentation, while still preserving the 2D layout for quick navigation between major topics.
I would have never said that, but now that I've come across it a few times, I think it's great – easy to get an overview of high level topics and dig into those that are interesting. Definitely not very discoverable, but pretty useful.
For those who are confused:
Hit the down arrow to go thru the slides, when you get to the end of one, hit the right arrow to go to the next section, then rinse and repeat.
Just hitting right will show the you the headers for each section. Right jumps from section to section
Hitting down goes slide to slide.
Just hitting right will show the you the headers for each section. Right jumps from section to section
Hitting down goes slide to slide.
just use space for next slide, why all down, right arrows..??
I've used this aspect of Reveal to present an overview of my talk at the beginning: "We're going to talk about X, Y, and Z. Okay, so X..."
Believe it or not, on a phone I found it to be a completely reasonable interface.
It's actually all three. I like the format.
Unikernels are cool to play with. Erlang is cool too. But...
- [name] is a kernel thread not a user space process.
- its libssl, not "libopenss"
- installing rpc, some obscure 55k listening port, and calling it bloated.. heh so cheap.
Once past all this FUD - what does a unikernel (basically the same as running linux and your app as /sbin/init except it's in Erlang and not C here)?
- its obscure. you get security mainly because nobody knows what you're running.
- its library based. like regular OSes.
- it runs only one program. or a million of them, like a regular OS, but with zero separation between them so compromise one means compromise all, actually. Else you have to add the separation layer yourself by writing more apps per "system" and make them communicate.
- its undebuggable without a lot of knowledge
- 'script kiddies' will run ./erlang-hack-for-x instead of ./rootshell.sh, this changes nothing to that. Remember the XEN exploit a few month ago? That'd still work and give access to all the Erlang instances. Its not that hard.
- its not faster, its actually slower, when compared to C kernel+code.
- it doesnt compile itself, you need a "regular OS" to deploy it
- it uses the same memory space for mostly everything (regular OSes use protected memory per process, its hardware enforced by the CPU)
- namespaced linux boots in < 10ms already...
There are much better implementations of safer, faster, cleaner OSes with better paradigms that also do away with backward compat and using modern, memory safe languages (http://en.wikipedia.org/wiki/Singularity_%28operating_system... for a well-known one). For the record MS also developed Drawbridge (process+lib, so not exactly a unikernel) because they could not do away with windows compatibility. So yeah.
If you have doubts, look at the example code at https://github.com/technolo-g/unikernel-demo
- [name] is a kernel thread not a user space process.
- its libssl, not "libopenss"
- installing rpc, some obscure 55k listening port, and calling it bloated.. heh so cheap.
Once past all this FUD - what does a unikernel (basically the same as running linux and your app as /sbin/init except it's in Erlang and not C here)?
- its obscure. you get security mainly because nobody knows what you're running.
- its library based. like regular OSes.
- it runs only one program. or a million of them, like a regular OS, but with zero separation between them so compromise one means compromise all, actually. Else you have to add the separation layer yourself by writing more apps per "system" and make them communicate.
- its undebuggable without a lot of knowledge
- 'script kiddies' will run ./erlang-hack-for-x instead of ./rootshell.sh, this changes nothing to that. Remember the XEN exploit a few month ago? That'd still work and give access to all the Erlang instances. Its not that hard.
- its not faster, its actually slower, when compared to C kernel+code.
- it doesnt compile itself, you need a "regular OS" to deploy it
- it uses the same memory space for mostly everything (regular OSes use protected memory per process, its hardware enforced by the CPU)
- namespaced linux boots in < 10ms already...
There are much better implementations of safer, faster, cleaner OSes with better paradigms that also do away with backward compat and using modern, memory safe languages (http://en.wikipedia.org/wiki/Singularity_%28operating_system... for a well-known one). For the record MS also developed Drawbridge (process+lib, so not exactly a unikernel) because they could not do away with windows compatibility. So yeah.
If you have doubts, look at the example code at https://github.com/technolo-g/unikernel-demo
What FUD? There appears to be more FUD in your comment than I've seen from any of the unikernel projects.
Most of the claims you've made seem to be coming from the existing paradigm of how people write code for the cloud. If you're determined to stay in that mindset, then unikernels will probably not make much sense to you. The whole point is to reevaluate our assumptions about developing for cloud-native environments, which leads to different approaches (many cloud-services are already single-purpose). No-one is claiming that unikernels are some kind of panacea but they are a useful addition to the toolbox.
Also, if you really want to compare security and vulnerabilities, then you should consider things like TCB and look at CVEs for both Xen and Linux. The way you describe it is almost disingenuous. The following talks have useful discussion points.
https://fosdem.org/2015/schedule/event/zombieapocalypse/
https://media.ccc.de/browse/congress/2014/31c3_-_6443_-_en_-...
Most of the claims you've made seem to be coming from the existing paradigm of how people write code for the cloud. If you're determined to stay in that mindset, then unikernels will probably not make much sense to you. The whole point is to reevaluate our assumptions about developing for cloud-native environments, which leads to different approaches (many cloud-services are already single-purpose). No-one is claiming that unikernels are some kind of panacea but they are a useful addition to the toolbox.
Also, if you really want to compare security and vulnerabilities, then you should consider things like TCB and look at CVEs for both Xen and Linux. The way you describe it is almost disingenuous. The following talks have useful discussion points.
https://fosdem.org/2015/schedule/event/zombieapocalypse/
https://media.ccc.de/browse/congress/2014/31c3_-_6443_-_en_-...
If you read the first 3 sentences, 2 are technically completely wrong and the third is a deliberately fake data. That's called FUD.
I'm all for the positive attitude stuff - unless the presentations start like that, I then have no sympathy.
The linked presentation actually claims unikernels are the panacea, by the way. Have you read it?
I'm all for the positive attitude stuff - unless the presentations start like that, I then have no sympathy.
The linked presentation actually claims unikernels are the panacea, by the way. Have you read it?
The problem with Mesa/Cedar, Modula-3/Spin, Modula-2/Lilith, Oberon/Native Oberon, Oberon-2/AOS, Sing#/Singularity and many others is compatibility and lack of buy-in from OS vendors.
Personally I see application compartments, wider adoption of Swift, Java, .NET Native, OCaml, Haskell, Go, D, Rust, Erlang, ..., alongside unikernels as the way forward to mainstream adoption.
One day we will get C free OS stacks.
Personally I see application compartments, wider adoption of Swift, Java, .NET Native, OCaml, Haskell, Go, D, Rust, Erlang, ..., alongside unikernels as the way forward to mainstream adoption.
One day we will get C free OS stacks.
C/C++ is the language you write the library in, so that it's callable from every other language. Hopefully Rust, Nim (and Golang one day) will be able to produce C-API compatible replacements. API can also be something better than what .so and .dlls provide, as long as everyone is on board.
I guess Rust and Nim are the real contenders, as only a language with "zero runtime" is going to cut it.
I guess Rust and Nim are the real contenders, as only a language with "zero runtime" is going to cut it.
It is an historical accident that C == OS ABI, as mainstream OSes copied UNIX model.
This wasn't true in other OSes.
Windows is currently moving to COM as the main OS ABI, specially with the WinRT focus.
Android also has little support for C ABI, beyond wrapping .so in JNI wrappers.
Another example are mainframe systems like OS/400, where the ABI is bytecode based (TIMI).
This wasn't true in other OSes.
Windows is currently moving to COM as the main OS ABI, specially with the WinRT focus.
Android also has little support for C ABI, beyond wrapping .so in JNI wrappers.
Another example are mainframe systems like OS/400, where the ABI is bytecode based (TIMI).
Yeah exactly Microsoft's point I think.
I'm not sure if its the best trade off right now, but it certainly makes sense.
It might help bridging the current solutions, hopefully (even thus that will perhaps take 10 years)
It might help bridging the current solutions, hopefully (even thus that will perhaps take 10 years)
> with zero separation between them
the hypervisor separates unikernels from each-other.
the hypervisor separates unikernels from each-other.
I think he means the 'programs' running under same unikernel
The security he means is not because obscurity but because you have less stuff you don't need. At least that's what he means I think.
namespaced linux boots in < 10ms already
Can someone show me this?
Can someone show me this?
The OP means the creation time of a Linux "container" is less than 10ms, ie the time for the system calls you need to create the namespaces and cgroups. While you can boot a unikernel from nothing in that time.
Actually, I meant booting!
Had to look for it in the HN history, look at this guy's implementation:
https://www.youtube.com/watch?v=6SxLvz6pcGs ~30ms here, the post indicates some kind of intel NUC (so not exactly the fastest thing)
The syscalls themselves are probably sub-millisecond
Had to look for it in the HN history, look at this guy's implementation:
https://www.youtube.com/watch?v=6SxLvz6pcGs ~30ms here, the post indicates some kind of intel NUC (so not exactly the fastest thing)
The syscalls themselves are probably sub-millisecond
I have to say, this is one of the most exciting technologies I've seen in awhile. It predates docker by a fair bit, and I would have hoped it would have gotten more traction. So I'm very happy to see slides from a user group.
Alas, the brilliance of Erlang has not been sufficiently appreciated, so hopefully Elixir -- aka: Erlang Returns -- might catch on and become a popular hipster language.
With Docker you still have all the overhead of Linux, and personally, I'm finding it a bit overwrought, and then on top of that more and more infrastructure is being built. All well intentioned and I'm not saying it's wrong-- but it's starting to feel like open stack.
One of the earlier demos of Erlang on Xen was a system where a VM was spawned to handle each requests. EG: An HTTP request would come in, an entire VM would be spawned, handle the request and go away. It was very fast.
I find that kinda astounding!
http://erlangonxen.org/zerg
PS- Not intended to start a flame war, the dogmatic sounding parts of this post are all tongue in cheek.
Alas, the brilliance of Erlang has not been sufficiently appreciated, so hopefully Elixir -- aka: Erlang Returns -- might catch on and become a popular hipster language.
With Docker you still have all the overhead of Linux, and personally, I'm finding it a bit overwrought, and then on top of that more and more infrastructure is being built. All well intentioned and I'm not saying it's wrong-- but it's starting to feel like open stack.
One of the earlier demos of Erlang on Xen was a system where a VM was spawned to handle each requests. EG: An HTTP request would come in, an entire VM would be spawned, handle the request and go away. It was very fast.
I find that kinda astounding!
http://erlangonxen.org/zerg
PS- Not intended to start a flame war, the dogmatic sounding parts of this post are all tongue in cheek.
> and then on top of that more and more infrastructure is being built.
I don't see how unikernels solve orchestration. You still need databases, load balancing, service discovery and so on. Kubernetes or something like it would still be a necessity.
I don't see how unikernels solve orchestration. You still need databases, load balancing, service discovery and so on. Kubernetes or something like it would still be a necessity.
Good point. My feeling isn't so much that it exists, but that it's all new and dockery-- and doesn't feel like it's gelled into something really useable yet. And by usable, I mean "usable for non-ops people". I'm hoping managing hypervisors will be more of a solved problem given more time in the market... but I may well be wrong.
So.. CGI?
The next logical step is to elide the virtualized network and disk interfaces, and then we're back to shared hosting! See you again in twenty years.
Yep. I don't get the value in this, pushing the problem somewhere else just creates new problems. Instead of being able to rely on distributions and gargantuan open software, now we go off and roll it all ourselves?
This is a good path to job security, but I don't see what else.
This is a good path to job security, but I don't see what else.
What's to roll yourselves? (Serious question, I might have missed it.) The erlang platform provides a great deal of everything you need. And if it doesn't, there's a library out there to do it.
Elixir is extremely active, growing and moving fast and even if you do have to roll it yourself, it can still be a net win.
For instance, I found it was easier to integrate a mail sending library or interface with mailgun than it was to get an STMP service working on linux.
But then, I'm a developer, not ops.
Elixir is extremely active, growing and moving fast and even if you do have to roll it yourself, it can still be a net win.
For instance, I found it was easier to integrate a mail sending library or interface with mailgun than it was to get an STMP service working on linux.
But then, I'm a developer, not ops.
Disclaimer: I have no idea what I'm talking about and should not be taken seriously.
Eliding the virtualized network and disk interfaces is not what they are talking about though. They are sticking with those. So, given virtualized CPU, RAM, network and disk interfaces; a runtime that talks directly to those interfaces; and the intent to exclusively run a single process per VM; what do they really need Linux for?
Eliding the virtualized network and disk interfaces is not what they are talking about though. They are sticking with those. So, given virtualized CPU, RAM, network and disk interfaces; a runtime that talks directly to those interfaces; and the intent to exclusively run a single process per VM; what do they really need Linux for?
You don;t roll all a Linux distro yourself; aside from the fact you use open source libraries, you need much less as it is only for running networked applications, which needs less stuff.
Arguably we only got this way because, although UNIX neatly abstracted the file system with user access control, it predated networking which was slapped on the side with no way of allocating network resources between users.
If SysV had had filesystem-like means of allocating the right to listen on a particular port to particular user accounts, the world would have been very different, and we'd have been spared entire categories of security hole and workaround.
If SysV had had filesystem-like means of allocating the right to listen on a particular port to particular user accounts, the world would have been very different, and we'd have been spared entire categories of security hole and workaround.
Research 9th and 10th had a /cs pretty much identical to Plan 9's /net. I doubt with any of that user-level control, though. IIRC Research's fmount(2) would've allowed people to run their own (unrestricted) copy of tcpmgr and mount it on by themselves.
While true, it would only help if C would be out of the picture as well.
Plan 9 and later Inferno (with Alef and Limbo) would have been better, but they went nowhere.
Plan 9 and later Inferno (with Alef and Limbo) would have been better, but they went nowhere.
I can't seem to shake the first thought i had when reading about unikernels. That they are DOS in a VM.
Maybe it is overly simplified, but as i understand it we are looking at a "kernel" stripped down to handling the disk and network interfaces provided by the VM. No user separation (there is only a single "user"), no memory protection (it is only running a single process as best i can tell), and resource provisioning is done by the VM.
Maybe it is overly simplified, but as i understand it we are looking at a "kernel" stripped down to handling the disk and network interfaces provided by the VM. No user separation (there is only a single "user"), no memory protection (it is only running a single process as best i can tell), and resource provisioning is done by the VM.
The point is that you only run a single application in the VM anyway. And if you do that, you should make your VM as slick as possible, because anything in there that doesn't serve your application is just increasing the attack surface and the probability of errors through side effects. That's what is already done today, and unikernels are just the natural progression towards that goal.
So you essentially don't separate your applications via users within your OS, but via VMs in your hypervisor.
Not sure if this turns out to be a good idea or not, but it's an interesting approach that shouldn't be put down easily.
So you essentially don't separate your applications via users within your OS, but via VMs in your hypervisor.
Not sure if this turns out to be a good idea or not, but it's an interesting approach that shouldn't be put down easily.
Yeah i may be coming across as overly dismissive.
But hopefully with a smaller stack?? :)
Also relevant: http://openmirage.org
and https://www.youtube.com/watch?v=bYQ_lq5dcvM
and https://www.youtube.com/watch?v=bYQ_lq5dcvM
Mirage supports Ocaml
the video shows people making Python Unikernels.
This slide deck supports Elixir and Erlang languages.
(just adding details for people)
This slide deck supports Elixir and Erlang languages.
(just adding details for people)
This has to be one of the worst UIs for a slide deck. "Why aren't these questions being answered?" ... "Oh, the slide advance UI changes. Too inconspicuous."
To be honest I kind of like it. I was confused at first as well but this does allow you to more quickly go between sections of a powerpoint which just isn't possible with the traditional "you can only swipe left or right" presentation.
Not only that but it also makes it much easier get an overview about what things are discussed in the presentation.
Not only that but it also makes it much easier get an overview about what things are discussed in the presentation.
I'm shocked that the top two HN comments as of writing are bikeshedding the submission's web design.
How's it trivial that the terrible design is a barrier to the content?
This is "Why is the bike shed missing three walls and half the roof?"ing.
Chapters bother you that much huh?
Not when they're arranged as a book, rather than a quilt disguised as a book.
A presentation is designed to be mainly useful for the presenter. In an actual presentation I could see it useful to navigate between main chapters when I get questions at the end. Now that it is shared and some people have minor problems navigating it at first is much lower priority. Maybe the shared slides could be linearized.
To jump on the bandwagon here, I recently set up a gopher site for a friend running for local office (sort of as a geeky in-joke). I came away from the experience actually wanting more gopher. Yes, the web can be a platform for things that should be simple documents, but gopher all but guarantees a uniform UI and nothing but content.
I didn't bother to figure out the multi-dimensional navigation of these slides. Paraphrasing Tufte, Richard Feynman managed to write about much of physics without resorting to more than two levels of headings. I'm similarly unconvinced that multi-dimensional, nested documents are really that helpful here.
I didn't bother to figure out the multi-dimensional navigation of these slides. Paraphrasing Tufte, Richard Feynman managed to write about much of physics without resorting to more than two levels of headings. I'm similarly unconvinced that multi-dimensional, nested documents are really that helpful here.
Oh! You go down until you run out of slides then you go right.
I didn't spend enough time to figure that out. I just assumed that the audio piece that went along with the slide deck wasn't uploaded and closed my browser.
I don't think this is intuitive enough.
If your slide deck needs a secondary nav, it probably shouldn't be a slide deck.
If your slide deck needs a secondary nav, it probably shouldn't be a slide deck.
similarly at first it confused me, but now I understand it is like a table of contents which is useful and easy to use.
Or you could put a TOC at the beginning (you know, like in a book) and do no sliding at all.
-> No confusion, great overview, direct jumps with hyperlinks, easy to realize with pure HTML.
-> No confusion, great overview, direct jumps with hyperlinks, easy to realize with pure HTML.
I'm not sure Erlang with a immutable infrastructure concept is good idea. Dynamic code reloading and upgrading without downtime is a key feature of Erlang, and immutable infrastructure do not play well with these features. If you make Erlang immutable, you should add another layer to ensure availability like HAProxy, which adds management burden. If you use dynamic code reloading, it is no longer 'immutable'.
My thoughts as well. And if you one does accept the restrictions I'd hope they remove the performance penalties associated with binding everything so late so it can be swapped.
In theory it should be possible to run Erlang applications on OSv as well [0]. I saw some discussions around this earlier, but I am not sure what the current status is.
[0] https://github.com/cloudius-systems/osv-apps/tree/master/erl...
[0] https://github.com/cloudius-systems/osv-apps/tree/master/erl...
Russell Pavlicek had a pretty good high level talk on Unikernels at Linuxfest Northwest this year -http://linuxfestnorthwest.org/2015/sessions/next-generation-...
He gave the same (or at least substantially similar) talk at SCaLE this year: https://www.youtube.com/watch?v=8RH7m5LYv5s
Thanks for all the comments! Looking forward to reading them in depth and tuning up the ole' preso :)
-- Matt
Twitter: @mattbajor
Twitter: @mattbajor
Here is another one for your list: https://boxfuse.com
We launched Boxfuse last month with support for running JVM apps as unikernels on VirtualBox and AWS.
Here is a blog article from last week about deploying Dropwizard unikernels to EC2: https://boxfuse.com/blog/dropwizard-aws.html
Disclaimer: I'm the founder.
We launched Boxfuse last month with support for running JVM apps as unikernels on VirtualBox and AWS.
Here is a blog article from last week about deploying Dropwizard unikernels to EC2: https://boxfuse.com/blog/dropwizard-aws.html
Disclaimer: I'm the founder.
It is not a unikernel though, just a minimal Linux.
Boxfuse falls squarely into the "generalized" unikernel type mentioned in the presentation. You are only thinking of the "specialized" ones.
In a sense it follows the same principle as OSv, just with a proven Linux kernel instead of a custom one.
Update: Why the downvote? If you disagree, feel free to say why.
In a sense it follows the same principle as OSv, just with a proven Linux kernel instead of a custom one.
Update: Why the downvote? If you disagree, feel free to say why.
No it doesnt. And the presentation is not very accurate. Rump kernel is a proper unikernel (based on the proven NetBSD kernel) but actually running as a library operating system, in the same address space as the single application it is linked to. You compile and link the kernel library and application into a single elf file. OSv is a single address space too, it just tries to pretend to look more like a normal OS. I think Drawbridge is a proper unikernel too. So the examples in the slides are simply wrong.
PS no idea who downvoted you, there is really no point in mentioning it.
PS no idea who downvoted you, there is really no point in mentioning it.
[deleted]
I thought Erlang relied heavily on threads...how does that even work without as OS?
I was curious as well and Googled it. Apparently, the Erlang Runtime runs within a single OS thread and implements its own thread scheduler in user space. http://stackoverflow.com/questions/605183/how-if-at-all-do-e...
Unikernels are very able to schedule threads to CPUs. Often, that scheduling is static. But, when your intent is to run a single process exclusively on the virtual machine, static scheduling is actually preferable.
Is it me or is the wrong word being used (0.o)?
See -- http://en.wikipedia.org/wiki/Microkernel
It is you. See https://queue.acm.org/detail.cfm?id=2566628
This is pretty amazing. I really like to idea of using more Erlang for my projects and this would make the deployment to EC2 easier.
Possible next step in this direction is to get rid of both Xen and unikernel by making a specified Java/Erlang VM that only has syscall-calling op-codes for network traffic and virtual disk I/O, while uses hardware virt goodies to secure execution of JITed code. (;
> Linux is a black box to a lot of devs.
That's not a problem you should get rid of, devs need to understand the underlying system.
> no shell == no shellshock
Only servers that needed the shell for some service exposed the issue, so this point is irrelevant.
> no libopenss = no heartbleed
no libopenssl = no ssl either.
That's not a problem you should get rid of, devs need to understand the underlying system.
> no shell == no shellshock
Only servers that needed the shell for some service exposed the issue, so this point is irrelevant.
> no libopenss = no heartbleed
no libopenssl = no ssl either.
Sounds like unikernels are giving the traditional OS a lobotomy and leaving only the brain stem to handle basic functions.
Can the same be done for Node.js? What about PHP?
No one has tried Node.js yet as far as I know (possibly under OSv not sure). PHP, yes, is running as a unikernel, using a rump kernel.
Can you post some links?
https://github.com/mato/rump-php
http://rumpkernel.org/
https://github.com/rumpkernel/wiki/wiki/Info:-Community
Ask on the mailing list/irc if you need help... its still under development so not completetly obvious yet ;)
http://rumpkernel.org/
https://github.com/rumpkernel/wiki/wiki/Info:-Community
Ask on the mailing list/irc if you need help... its still under development so not completetly obvious yet ;)
I wonder if this is how AppEngine was built.
http://wiki.xenproject.org/wiki/Unikernels
'nuff said.
'nuff said.
OK, maybe not 'nuff said; this is a great paper on unikernels from last year:
Unikernels: Rise of the Virtual Library Operating System https://queue.acm.org/detail.cfm?id=2566628
Unikernels: Rise of the Virtual Library Operating System https://queue.acm.org/detail.cfm?id=2566628
Off-topic: Anyone else found it difficult to navigate through the presentation?
This website must be broken. It broke my browser's back button.
I went in expecting to read a bit about unikernels. I got there and felt like I was playing The Legend of Zelda for NES. You know, the one with the golden cartridge.
- no one ever