Xhyve – Lightweight Virtualization on OS X Based on Bhyve(pagetable.com)
pagetable.com
Xhyve – Lightweight Virtualization on OS X Based on Bhyve
http://www.pagetable.com/?p=831
74 comments
thanks!
what about volume mounting and port forwarding,
are supported already?
Your script is pretty much the same as mine, good to see :)
For everyone who wants to give it a try and wants some instructions http://simonvanderveldt.nl/boot2docker-on-xhyve/
I've also done some quick benchmarks, they are at the bottom of the post. Virtualbox had about 50% higher disk IO performance in my measurements.
Regarding the root permissions for networking: As far as I know it shouldn't be necessary if you sign the xhyve binary, haven't tried that though.
I've also done some quick benchmarks, they are at the bottom of the post. Virtualbox had about 50% higher disk IO performance in my measurements.
Regarding the root permissions for networking: As far as I know it shouldn't be necessary if you sign the xhyve binary, haven't tried that though.
Really glad to see someone using Hypervisor.framework, looks like it has a lot of potential. Fusion/Parallels always seem far too heavyweight to me.
Not to mention they're paid solutions, closed source and are horrible for battery life. Their only advantage is nice set of GUI tools and tight OS X integration.
VMWare Fusion is, in my experience, everything but horrible for battery life. I frequently spin up VMs with Vagrant, do something with them and then switch to other tasks. More often than not, I forget to shut them off, mostly because their impact on the battery life is so minimal that I don't notice them until I run `vagrant global-status`.
This probably depends greatly upon whether your machine has discrete graphics or not. On Macbook Pros with nVidia graphics like my 2014 higher-tier Macbook Pro, there is a dramatic loss in battery when the discrete graphics kicks in. However, most Macs sold now don't have discrete graphics because the Intel integrated graphics is honestly pretty decent nowadays for most tasks. The difference is 3 hours of battery and 8+ for me.
Google Chrome, similarly, tends to drain my battery a lot, and that alone has encouraged me to go to Safari on Macs.
Google Chrome, similarly, tends to drain my battery a lot, and that alone has encouraged me to go to Safari on Macs.
Or just use gfxCardStatus to lock your machine to the integrated GPU all the time
Can anyone tell me why this isn't an inbuilt feature of OS X? Preferably in the 'Energy Saver' pane of System Preferences, e.g. while on battery power, only use integrated graphics.
There was a discussion that sometimes it is better to run a task quick on GPU than to run it for longer time on CPU. Would be cool if someone could try to prove this or inverse. :)
Maybe it'll be different in Metal-backed El Capitan, but my experience so far has been that the discrete GPU universally degrades my battery life. I'm glad I have it though; sometimes I need it, but it'd be nice if I could just disable it completely most of the time.
Ehm when we talk about Vmware Fusion and discrete Graphics you are obviously wrong. You could change the graphic mode in Vmware Fusion, so that you save lots of power..
I get increased battery drain with VMWare Fusion without even turning a VM on for similar reasons as just starting Google Chrome - it causes a switch to discrete graphics merely by starting the program up with no clear pattern of releasing it. It has rather little to do with a VM workload in my observations. If I completely disable discrete graphics, that stops the switch and the subsequent battery drain. However, I lose my external graphics output when I do that evidently, so if I need to use my external monitor, I will lose battery anyway.
I can see the graphics switch happen with gfxCardStatus notifying me. The process occupying the GPU is from Fusion plain as day (I think it's vpx agent?). This happens with nothing else running. This switching is one of the known quirks with OS X after years and years now.
Also, older versions of Fusion may have been less careful about avoiding graphics modes that potentially cause graphics acceleration to kick in and maybe more recent versions than mine do better (still on 6 because I'm a cheapskate).
I can see the graphics switch happen with gfxCardStatus notifying me. The process occupying the GPU is from Fusion plain as day (I think it's vpx agent?). This happens with nothing else running. This switching is one of the known quirks with OS X after years and years now.
Also, older versions of Fusion may have been less careful about avoiding graphics modes that potentially cause graphics acceleration to kick in and maybe more recent versions than mine do better (still on 6 because I'm a cheapskate).
I'm thinking you mean "tight UI integration."
This is great, is it possible to make it work with Docker?
Edit: Yes, it is.
Edit2: It is possible, but whether is it worth it or not is still a question. xhyve is a supervisor after all, you still need to boot a vm to use Docker. So if you like me, thought for a second that this can make containers in OSX a reality, don't get excited too early.
Edit: Yes, it is.
Edit2: It is possible, but whether is it worth it or not is still a question. xhyve is a supervisor after all, you still need to boot a vm to use Docker. So if you like me, thought for a second that this can make containers in OSX a reality, don't get excited too early.
It explicitly says: xhyve may make a good solution for running Docker on your Mac, for instance.
Well it might be an alternative to virtualbox for boot2docker.
If you want a better solution on OSX than you already have, try https://github.com/codekitchen/dinghy - it does a really great job of wrapping Vagrant and taking care of setting up NFS instead of the incredibly slow default vboxfs.
Alternatively I've used Parallels + NFS with Vagrant for boot2docker but it's a real pain to get set up correctly. Dinghy just did it automatically, and also set up DNS and NTP for me. Can't recommend it enough.
Alternatively I've used Parallels + NFS with Vagrant for boot2docker but it's a real pain to get set up correctly. Dinghy just did it automatically, and also set up DNS and NTP for me. Can't recommend it enough.
Not sure how useful OS X containers would be. Are you deploying on OS X?
The benefit for me would be quick, isolated development environments. I want to boot a contained swarm of services connected to each other in a single command, and shut them down in a single command as well
And installing docker / docker-compose / machine, etc. using one brew command would be a great addition in order to ease introduction of the tool to the whole team.
brew install docker docker-compose docker-machine
open https://kitematic.com/download/
Are you aware that your OS X machine is, itself, a deployment target for developers somewhere? All environments benefit from enforceable loose coupling.
I develop containers on OS X. I just spin up a linux box in vmware running a docker server listening on TCP.
From my development experience, there's no difference from working directly on a linux box.
From my development experience, there's no difference from working directly on a linux box.
Definitely time to add Hypervisor.framework to the list of interesting things on OS X to check out. Related: I'd love to see a Vagrant provider for Xhyve, may have to hack on that a bit later.
OK. That's freaking awesome. I like seeing FreeBSD technology being brought into the mainstream in OS X. (I don't even use OS X, but this is cool.)
I guess this is pronounced "X-hive" and "B-hive"?
Initially I thought it was "zive" and "bive", like "bive" was a foreign word from somewhere. Bhyve, the muse of unit tests!
Initially I thought it was "zive" and "bive", like "bive" was a foreign word from somewhere. Bhyve, the muse of unit tests!
Fair warning, this may cause a Kernel panic. I've contributed to the bug with my trace, but it's definitely something to keep in mind when checking it out.
I really want this to work, though. It would be a great replacement for VirtualBox.
I really want this to work, though. It would be a great replacement for VirtualBox.
I honestly wouldn't notice. 10.10 kernel panics on me about once a day. It's the WORST EVER release of OSX by an amazing margin.
Did you have a VirtualBox VM running before xhyve?
I wrote a quick set of scripts to get CoreOS booting under xhyve as a container host. The defaults will expose SSH and a docker client port and drop you directly into a shell. Some quick getting started info here:
https://github.com/coreos/coreos-xhyve#coreos--xhyve
Lots of GitHub issues filed for next steps in exploring this technology. Really exciting to see a simple CLI driven virtualization option on OSX. Patches accepted. :)
https://github.com/coreos/coreos-xhyve#coreos--xhyve
Lots of GitHub issues filed for next steps in exploring this technology. Really exciting to see a simple CLI driven virtualization option on OSX. Patches accepted. :)
This project is awesome. No more vmware fusion for me. Props to netapp for funding and driving the bhyve project. FreeBSD was desperately in need of native virtualization capabilities.
Note that apart from bhyve (available today), FreeBSD 11 will also come with Xen Dom0 support.
Great news. That will allow ZFS in a FreeBSD driver VM, providing storage for other guests, even with a Linux dom0. Is FreeBSD 11 expected this year?
Next year in july: https://www.freebsd.org/releases/11.0R/schedule.html
I hadn't heard about that - that's great news! I may finally be able to start stealthing it into test environments.
It would be great if Docker integrated / embedded that instead of forcing users to download / configure VirtualBox + boot2docker.
I'm not sure docker would want to do this or if that's even an appropriate feature to integrate.
I'd rather see it supported by vagrant, which would seem an appropriate tool to manage them with.
I'd rather see it supported by vagrant, which would seem an appropriate tool to manage them with.
Docker has a huge interest in making it easy for the legions of Mac-toting developers to use it easily without first telling you to get complex, fragile external tooling like vagrant or virtualbox working.
I doubt it'd go in the mainline product but I'd be shocked if in a year or two there isn't a Docker.app one-stop option which makes it trivial to run popular Linux apps.
I doubt it'd go in the mainline product but I'd be shocked if in a year or two there isn't a Docker.app one-stop option which makes it trivial to run popular Linux apps.
> I'd be shocked if in a year or two there isn't a Docker.app one-stop option which makes it trivial to run popular Linux apps.
That sounds really compelling, and something along the lines of what Kitematic is doing, but even simpler.
That sounds really compelling, and something along the lines of what Kitematic is doing, but even simpler.
Thanks for mentioning Kitematic – I'd forgotten the name but this was my first reaction when I saw their product announcement. It's a nice way to make their product even smoother.
> use it easily without first telling you to get complex, fragile external tooling like vagrant or virtualbox working.
I won't get into virtualbox. But I haven't found vagrant very fragile at all since Mitchell moved to omnibus (quite a while back). Most of the issues I had with vagrant were directly related to virtualbox, not vagrant itself.
> Docker has a huge interest in making it easy for the legions of Mac-toting developers
Well, sort of. They have a product that is inherently linux based. Getting a nice setup working on OS X is pretty trivial, takes a developer maybe a couple of hours one time to have something working.If they use someone else's solution, it takes minutes.
Here's an ansible task I use to configure an ubuntu machine[1]. It's all of 24 lines. If you're running it against a VM, you just forward the 2375 port to your host. Then the workflow looks like:
[1] https://gist.github.com/jameskyle/b8e573d63da4dbaaf31a
I won't get into virtualbox. But I haven't found vagrant very fragile at all since Mitchell moved to omnibus (quite a while back). Most of the issues I had with vagrant were directly related to virtualbox, not vagrant itself.
> Docker has a huge interest in making it easy for the legions of Mac-toting developers
Well, sort of. They have a product that is inherently linux based. Getting a nice setup working on OS X is pretty trivial, takes a developer maybe a couple of hours one time to have something working.If they use someone else's solution, it takes minutes.
Here's an ansible task I use to configure an ubuntu machine[1]. It's all of 24 lines. If you're running it against a VM, you just forward the 2375 port to your host. Then the workflow looks like:
vagrant up
docker build -t jkyle/awesomeness .
[1] https://gist.github.com/jameskyle/b8e573d63da4dbaaf31a
Yeah, I know many people who use and like vagrant but I've heard a too many people mention it as a major impediment when they were just getting starting. Sure, someone with a bit of sysadmin experience can usually get it working fairly quickly but I've seen entire meetups where 90% of the people used a Linux instance on EC2 rather than try to debug it.
Anything which makes that process easier for newcomers is welcome, particularly for non-specialists like front-end specialists, scientists, data-analysts and other technical professionals for whom figuring out a toolchain is a road-block to their job rather than the job itself.
Anything which makes that process easier for newcomers is welcome, particularly for non-specialists like front-end specialists, scientists, data-analysts and other technical professionals for whom figuring out a toolchain is a road-block to their job rather than the job itself.
Hopefully Apple will add Hypervisor.framework support for Intel's hardware-virtualized graphics, https://01.org/zh/blogs/skjain/2014/intel®-graphics-virtuali...?
I'm curious why there is a restriction of 2010 or newer mac. Is it something in the CPU that's needed? I've 10.10.3 installed on a non supported mac and wanted to use this, but getting errors when I try the test Linux
You need a CPU that supports Intel's virtualization instruction set. Specifically VT-x. You can check if your CPU supports it with:
sysctl machdep.cpu.features
You're looking for VMX. Here's an example from mine (highlighted the flag you're looking for):
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL * VMX * SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
You're looking for VMX. Here's an example from mine (highlighted the flag you're looking for):
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL * VMX * SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
Most Macs have had VT-x enabled since.... 2007?
The critical hardware feature they didn't all have is EPT[1], which may be required here in addition to VT-x.
[1] https://en.wikipedia.org/wiki/Second_Level_Address_Translati...
The critical hardware feature they didn't all have is EPT[1], which may be required here in addition to VT-x.
[1] https://en.wikipedia.org/wiki/Second_Level_Address_Translati...
This (missing) EPT must be the reason my MacBook Pro from 2009 is crashing when running it. Thanks for pointing that
Thanks, was running to a client and didn't have time to check it out right away.
Hmm, turns out my Mac Pro does have VMX : machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 DTES64 MON DSCPL VMX EST TM2 SSSE3 CX16 TPR PDCM Maybe it's the EPT mentioned by @mrsteveman1
Hmm, turns out my Mac Pro does have VMX : machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 DTES64 MON DSCPL VMX EST TM2 SSSE3 CX16 TPR PDCM Maybe it's the EPT mentioned by @mrsteveman1
I would imagine it needs Intel vt-x.
That's really great! I had to rebuy VMware at work and Parallels at home when Yosemite came out because of the new hypervisor framework modifications that these two needed under Yosemite (I presume).
I don't really use the OSX integration of both systems, other than file mapping/sharing (I run Linux and Windows VMs).
Glad to see this, many thanks!
I don't really use the OSX integration of both systems, other than file mapping/sharing (I run Linux and Windows VMs).
Glad to see this, many thanks!
I don't know about Parallels but VMWare Fusion 6 runs just fine on Yosemite – there's almost no reason to upgrade unless you want new icons.
That said, I'm completely with you on the value of Hypervisor.framework for people who are e.g. running Linux test instances and not really seeing much benefit from the UI integration but who still want to avoid the performance / stability problems caused by using virtualbox.
That said, I'm completely with you on the value of Hypervisor.framework for people who are e.g. running Linux test instances and not really seeing much benefit from the UI integration but who still want to avoid the performance / stability problems caused by using virtualbox.
I had been making do with VMWare Fusion 5, so it truly refused to work on Yosemite. (I also had Parallels 8 at home, which refused to work).
Hopefully the upgrades will be only necessary every two years.
Stability problems with VirtualBox are very real. I haven't touched that for years.
Hopefully the upgrades will be only necessary every two years.
Stability problems with VirtualBox are very real. I haven't touched that for years.
> Stability problems with VirtualBox are very real. I haven't touched that for years.
What issues do you have? I've been running VirtualBox on OS X for 6+ years and have never had stability issues. I usually have a few VMs running at a time via vagrant.
What issues do you have? I've been running VirtualBox on OS X for 6+ years and have never had stability issues. I usually have a few VMs running at a time via vagrant.
Under Snow Leopard (yes it was that long ago) it would frequently kernel panic so I stopped using it. I have never had VMware kernel panic on me; Parallels has crashed once or twice but never kernel panicked.
I did use VirtualBox for running Windows 95 the other month though. I am not complaining about VirtualBox or advising against its use, just I had some issues on OSX.
Under Windows and Linux it ran fine, just OSX was dodgy for me.
I did use VirtualBox for running Windows 95 the other month though. I am not complaining about VirtualBox or advising against its use, just I had some issues on OSX.
Under Windows and Linux it ran fine, just OSX was dodgy for me.
Until at least a couple of years ago, it was common for it to cause kernel panics if running during a sleep/wake cycle or when exercising the GPU. I used to work with a couple of people who used it and they'd have an unplanned coffee break at least once a week.
Integration with VSphere was our reason to upgrade to 7.0. Its not a bad upgrade.
The title is a bit misleading, it could say 'Xhyve – Lightweight Virtualization for OS X based on FreeBSD's Bhyve'
Bhyve is FreeBSDs...? I don't follow how it is misleading.
It is in fact - it is called The BSD Hypervisor [1] and so far it exists only on FreeBSD, and this is where it was born. There are some forks to port it to other platforms (like Joyent's SmartOS) but they're closed and not out yet.
I consider technology awareness, competition and diversity a very good thing, therefore I think its important to mention FreeBSD in this case (and other BSD's too in other cases).
By the way, have you donated to FreeBSD [2] and OpenSSH [3] yet? ;)
[1] http://bhyve.org/
[2] https://www.freebsdfoundation.org/donate/
[3] http://www.openbsd.org/donations.html
I consider technology awareness, competition and diversity a very good thing, therefore I think its important to mention FreeBSD in this case (and other BSD's too in other cases).
By the way, have you donated to FreeBSD [2] and OpenSSH [3] yet? ;)
[1] http://bhyve.org/
[2] https://www.freebsdfoundation.org/donate/
[3] http://www.openbsd.org/donations.html
I've been buying OpenBSD CD's since 3.4 and a long time friend has commit bits with the FreeBSD project. I am a big BSD supporter :)
It requires that you copy the vmlinuz64 and initrd.img files from the B2D ISO to a local filesystem location. As of right now enabling networking requires root privileges. Not sure if the Hypervisor framework has user-level accommodations for that.