Using bubblewrap to add sandboxing to NetBSD(blog.netbsd.org)
blog.netbsd.org
Using bubblewrap to add sandboxing to NetBSD
https://blog.netbsd.org/tnf/entry/gsoc2025_bubblewrap_sandboxing
36 comments
I documented what I came up with for Claude Code: https://blog.gpkb.org/posts/ai-agent-sandbox/ However, I couldn't get this to work for Codex, it kept failing at the auth bit and I couldn't figure out how to fix it. Anyone got a working solution for Codex?
A question.
How do you know what permissions are required by an application, to write a bubblewrap script?
In AppArmor, you exercise the application and aa-logprof suggests permissions requested by application. If you know AppArmor, usually you can refine those suggestions and write a profile. It may not be ideal, as aa-logprof’s permissions are multiple choice suggestions, require user knowledge and may be too broad or specific, but it could work. You will see that there are many and all kinds of permissions, and there is no way that you will be able to guess them without aa-logprof.
What is the equivalent of aa-logprof in bubblewrap and how do you find the required permissions?
How do you know what permissions are required by an application, to write a bubblewrap script?
In AppArmor, you exercise the application and aa-logprof suggests permissions requested by application. If you know AppArmor, usually you can refine those suggestions and write a profile. It may not be ideal, as aa-logprof’s permissions are multiple choice suggestions, require user knowledge and may be too broad or specific, but it could work. You will see that there are many and all kinds of permissions, and there is no way that you will be able to guess them without aa-logprof.
What is the equivalent of aa-logprof in bubblewrap and how do you find the required permissions?
it’s a bit baroque but I use strace personally
Have you seen Leash?
https://github.com/strongdm/leash
It even has a --darwin macOS-native mode which goes beyond the capabilities and guarantees of sandbox-exec and bubblewrap.
Full-disclosure: I am one of the authors.
https://github.com/strongdm/leash
It even has a --darwin macOS-native mode which goes beyond the capabilities and guarantees of sandbox-exec and bubblewrap.
Full-disclosure: I am one of the authors.
bubblewrap escapes are not unheard of. Infact, it's a common theme that the general linux landscape lacks strong sandboxing, even if you use bwrap, firejail, etc. Especially linux desktop, a security firehazard to say the least unless you are using QubesOS
I am curious but what are your thoughts on
https://hanako.codeberg.page/ (Flatpak is not a sandbox)
https://hanako.codeberg.page/ (Flatpak is not a sandbox)
As a side note NetBSD is THE ONLY operating system of which binaries are fully bit by bit reproducible.
Moreover it vendores patched gcc compiler, so the entire toolchain is reproducible too as well.
How cool is that?! For the record golang brings it further - its entire package registry containing +40mln packages is bit by bit reproducible.
Moreover it vendores patched gcc compiler, so the entire toolchain is reproducible too as well.
How cool is that?! For the record golang brings it further - its entire package registry containing +40mln packages is bit by bit reproducible.
Are you sure GNU Guix System doesn't share this attribute?
Unfortunately it doesn't - even if our package definition doesn’t change output may, since software referenced by the package’s build function is not bit by bit reproducible.
The biggest issue is with Linux Kernel itself. Once kernel reproducibility will be handled, we can practically claim Guix, NixOS, Debian or any other Linux distro is reproducible as well, since with little effort we can avoid installing packages which will colour our reproducibility.
For desktop environments it’s a bit worse, but there are huge efforts to make this happen for all packages in package registries of aforementioned distros.
The biggest issue is with Linux Kernel itself. Once kernel reproducibility will be handled, we can practically claim Guix, NixOS, Debian or any other Linux distro is reproducible as well, since with little effort we can avoid installing packages which will colour our reproducibility.
For desktop environments it’s a bit worse, but there are huge efforts to make this happen for all packages in package registries of aforementioned distros.
Debian has an ongoing project (https://wiki.debian.org/ReproducibleBuilds) to have 100% reproducible builds and are getting pretty close (numbers wise). Other Linux distros are working toward it as well. Many people care about this ... https://reproducible-builds.org/.
Is NetBSD also reproducible from source without using NetBSD binaries?
https://bootstrappable.org/ https://stagex.tools/
https://bootstrappable.org/ https://stagex.tools/
I thought FreeBSD has it as well with v15?
I think I have read something like that similar as well.
Freebsd seems like an amazing project too.
I am always confused tho by what is the actual difference b/w freebsd,netbsd,openbsd as I feel like freebsd is taking a lot of positive ideas from both netbsd and openbsd or independently building them.
I might want to play with freebsd one day but I feel like it might have limited software or I might need to compile a lot of things as compared to archlinux or even debian etc.
Freebsd seems like an amazing project too.
I am always confused tho by what is the actual difference b/w freebsd,netbsd,openbsd as I feel like freebsd is taking a lot of positive ideas from both netbsd and openbsd or independently building them.
I might want to play with freebsd one day but I feel like it might have limited software or I might need to compile a lot of things as compared to archlinux or even debian etc.
They're quite different, but in subtle ways, and sometimes it's not about what they offer but rather what they don't offer.
What they all have in common is that they have great documentation, and they're all very unix-y and all have some virtualization support for linux.
Some things FreeBSD offer are good performance, IME the largest package repository, zfs integration, capsicum and jails. The experience IMO is a bit like Arch, except you'll rely less on a wiki and more on their official documentation.
OpenBSD and NetBSD offer a more "complete" experience like you'd expect from a desktop OS, while at the same time being quite minimal and simple.
OpenBSD focuses a lot on security, and is probably more likely to work with your hardware with fast wifi. It doesn't offer the same virtualization nor security the same type of security as Linux, but instead encourages you to pledge and unveil your applications, which means you lock down each application by saying what it can and can't do. It notoriously removes features from the kernel that are unmaintained and also doesn't include things it considers insecure like bluetooth. It's probably the OS that gives you the most "works out-of-the-box" experience.
NetBSD gives you a full desktop experience as well, but also has some unique security features like kauth, veriexec, extra-hardened chroot and security.curtain. It also offers some cool features like rumpkernels, smolbsd, and is probably the easiest to hack and compile yourself (even cross-compiling is very simple). It has a package manager, pkgsrc, that runs on many other OSes and it's also quite easy to port to new platforms. It's a fun OS to tinker with.
What they all have in common is that they have great documentation, and they're all very unix-y and all have some virtualization support for linux.
Some things FreeBSD offer are good performance, IME the largest package repository, zfs integration, capsicum and jails. The experience IMO is a bit like Arch, except you'll rely less on a wiki and more on their official documentation.
OpenBSD and NetBSD offer a more "complete" experience like you'd expect from a desktop OS, while at the same time being quite minimal and simple.
OpenBSD focuses a lot on security, and is probably more likely to work with your hardware with fast wifi. It doesn't offer the same virtualization nor security the same type of security as Linux, but instead encourages you to pledge and unveil your applications, which means you lock down each application by saying what it can and can't do. It notoriously removes features from the kernel that are unmaintained and also doesn't include things it considers insecure like bluetooth. It's probably the OS that gives you the most "works out-of-the-box" experience.
NetBSD gives you a full desktop experience as well, but also has some unique security features like kauth, veriexec, extra-hardened chroot and security.curtain. It also offers some cool features like rumpkernels, smolbsd, and is probably the easiest to hack and compile yourself (even cross-compiling is very simple). It has a package manager, pkgsrc, that runs on many other OSes and it's also quite easy to port to new platforms. It's a fun OS to tinker with.
Wow really appreciate your message
I actually didn't know that openbsd and netbsd offer a more complete experience than as compared to freebsd
I had heard about openbsd a lot as well but freebsd I think is a lot more known in the nas community partially/wholely because of the zfs integration
Capsicum is definitely really interesting as well, I had heard of pledge partially because of justine tunney's cosmopolitan-esque port of pledge to linux and I was always fascinated by it
I wonder what is the state of software of openbsd /bsd's in general. I think another interesting thing that I found was that SSPL license could probably be bypassed if one can run freebsd etc., I am pretty sure that I had read one such comment regarding something similar on HN
Also, if I may ask, what is the difference with the linux kernel itself, I feel like bsd's have some security advantages which might make it a little bit more suitable for multi-tenancy. But the same things can be done in linux as well with things like podman / docker etc. or with firejail/bubblewrap/flatpak but with a higher cost like ram etc. or they can be considered bloated solutions as compared to bsd's but still.
What are some operating openbsd or some other servers which are used and where are they used, I used to see some apache servers in openbsd in some websites but what are your thoughts on the whole situation, I would love to know more about and thanks once again for your insightful comment!
I actually didn't know that openbsd and netbsd offer a more complete experience than as compared to freebsd
I had heard about openbsd a lot as well but freebsd I think is a lot more known in the nas community partially/wholely because of the zfs integration
Capsicum is definitely really interesting as well, I had heard of pledge partially because of justine tunney's cosmopolitan-esque port of pledge to linux and I was always fascinated by it
I wonder what is the state of software of openbsd /bsd's in general. I think another interesting thing that I found was that SSPL license could probably be bypassed if one can run freebsd etc., I am pretty sure that I had read one such comment regarding something similar on HN
Also, if I may ask, what is the difference with the linux kernel itself, I feel like bsd's have some security advantages which might make it a little bit more suitable for multi-tenancy. But the same things can be done in linux as well with things like podman / docker etc. or with firejail/bubblewrap/flatpak but with a higher cost like ram etc. or they can be considered bloated solutions as compared to bsd's but still.
What are some operating openbsd or some other servers which are used and where are they used, I used to see some apache servers in openbsd in some websites but what are your thoughts on the whole situation, I would love to know more about and thanks once again for your insightful comment!
I think much of what the BSDs offers can be achieved with Linux (and hardened Linux is probably just as secure), but it can be a lot more work and there is less guidance.
Locking down an Openbsd system is quite easy for example as it tries to have sane defaults. I find myself more in the dark with Linux.
Installing zfs on Linux is possible, but again can be a lot of work and it doesn't cleanly integrate with containers the same way zfs integrates with jails on Freebsd.
Locking down an Openbsd system is quite easy for example as it tries to have sane defaults. I find myself more in the dark with Linux.
Installing zfs on Linux is possible, but again can be a lot of work and it doesn't cleanly integrate with containers the same way zfs integrates with jails on Freebsd.
Can you explain what reproducible mean in this context? Does that mean that you can recompile everything from "scratch" or does it have a deeper meaning?
Reproducible builds ensure that you can build the same binaries with the same source code. Nothing like the current date for instance gets in the way of getting a different build.
This allows independent people to check that provided binaries don't contain malicious stuff for instance. Ultimately, it lets you download binaries instead of rebuilding everything yourself locally if the binaries have been independently reproduced.
You might be interested in checking:
https://reproducible-builds.org/
https://en.wikipedia.org/wiki/Reproducible_builds
This allows independent people to check that provided binaries don't contain malicious stuff for instance. Ultimately, it lets you download binaries instead of rebuilding everything yourself locally if the binaries have been independently reproduced.
You might be interested in checking:
https://reproducible-builds.org/
https://en.wikipedia.org/wiki/Reproducible_builds
The provided binaries may still contain malicious code but it guarantees that no malicious code has been inserted in between the build process of the published code. So if your binaries contain malicious code, you can be sure that all other users of the software version are affected, too.
does anyone practice dual build pipeline? eg: 1 by your devops team and another one by your security team and compare binaries hash later. To verify everything is reproducible.
is it a common practice?
is it a common practice?
It is not common outside of security inclined communities like cryptocurrencies. It should be and we are slowly moving there.
Indeed, thanks for the precision!
It's possible to run a full DE bwrapped.
That's obviously super permissive, but from there can take things away to expose as much or little of the host system as needed.
For my system I'm working on making a few bwrapped "zones". E.g. start a terminal that can see a certain subset of files, and can configure whether it should use gpu or not, have internet or not, have access to local network or not, etc. A bit more project/environment focused than bwrapping programs one by one.
bwrap --dev-bind /dev /dev --tmpfs /tmp -- labwc
bwrap --dev-bind /dev /dev --tmpfs /tmp -- kwin_wayland konsole
bwrap --dev-bind /dev /dev --tmpfs /tmp -- startplasma-wayland
That's obviously super permissive, but from there can take things away to expose as much or little of the host system as needed.
For my system I'm working on making a few bwrapped "zones". E.g. start a terminal that can see a certain subset of files, and can configure whether it should use gpu or not, have internet or not, have access to local network or not, etc. A bit more project/environment focused than bwrapping programs one by one.
Always nice to see NetBSD posts here, that nice OS gets hardly any press.
I experimented bubblewrap as a better alternative to firejail. Unfortunately there are scripts for few applications. It’s tool for developers. Users are better off flatpaks powered by bubblewrap .
I have been using firejail for a long time now. It seem to work well for me. At times I have to write my own profiles, yeah.
The situation with firejail is different. There are a lot of profiles, and they are easier to customize and write.
But firejail needs root privilege, so there are arguments that it may actually reduce the security, because even though it sandboxes some applications, non-privilege processes may gain privilege if there is a flaw in the fire jail itself.
Firejail is also criticized for complexity.
But firejail needs root privilege, so there are arguments that it may actually reduce the security, because even though it sandboxes some applications, non-privilege processes may gain privilege if there is a flaw in the fire jail itself.
Firejail is also criticized for complexity.
Firejail does not need root privileges at all. You are probably referring to SUID. It is a "setuid ELF 64-bit" executable. I run it with my own user only.
Complexity? It just supports a lot of stuff. See the manual page. It seems simple enough to me.
Complexity? It just supports a lot of stuff. See the manual page. It seems simple enough to me.
Yeah, I meant the same thing! If you run it under your non-root user, and there is a vulnerability in firejail, the jailed process could run as root, right?
Complexity referred to large code base and attack surface.
I haven’t tried it myself. I would be interested knowing, whether it might lower the security in some cases. Maybe it’s useful to sandbox apps that don’t have flatpaks.
Complexity referred to large code base and attack surface.
I haven’t tried it myself. I would be interested knowing, whether it might lower the security in some cases. Maybe it’s useful to sandbox apps that don’t have flatpaks.
SUID by itself is not a security liability. Typically, programs using SUID quickly drop their additional privileges during startup to avoid the scenario you're describing. So the vulnerability in firejail would have to be early during startup before any of that.
Still, it can happen of course, particularly with memory-unsafe languages. Conversely without root privileges you have to sacrifice strength of your sandbox, widening the vulnerability window in the app you were trying to constrain to begin with.
Meanwhile bubblewrap relies on user namespaces, which come with their own set of security problems. They are disabled by default in many distributions for that reason.
Still, it can happen of course, particularly with memory-unsafe languages. Conversely without root privileges you have to sacrifice strength of your sandbox, widening the vulnerability window in the app you were trying to constrain to begin with.
Meanwhile bubblewrap relies on user namespaces, which come with their own set of security problems. They are disabled by default in many distributions for that reason.
Yeah, it runs with escalated privs. People wager that SUID binaries are extremely risky and this seems totally reasonable to me! I don't know where to find a really good security analysis of the risks, however. Firejail is most likely not bullet-proof, although not soft by any stretch of the imagination and I don't know how often and how many security researchers are profiling this tool.
My gut sense is that flatpak gets much more scrutiny since it ships, and firejail is typically not shipped, but another package as far as I can tell (maybe in some specialized distros?).
My gut sense is that flatpak gets much more scrutiny since it ships, and firejail is typically not shipped, but another package as far as I can tell (maybe in some specialized distros?).
I wanted to build flatpaks cli applications and I still don't understand how to build a flatpak cli in the first place and I have asked in some communities and they mention things like vim being in flatpak but I more so need a definitive guide on how to publish a cli application to flatpak
A lot of focus in flatpak sadly gets focused on guis and not clis
Like how much efforts would it take to convert a golang static binary/project cli to flatpak?
A lot of focus in flatpak sadly gets focused on guis and not clis
Like how much efforts would it take to convert a golang static binary/project cli to flatpak?
I would have loved to see something like OCI containers.
This would underpin that, right? First you build the namespace primitives, then you put the handy wrapper on top that composes a root fs and spawns a container from it using those namespace primitives
bubblewrap popping )) https://brainteaser.top/bubblespop.html
https://github.com/containers/bubblewrap
It's the base for Flatpack, the thing that makes Flatpack be sandboxed.
I use it to run Claude Code / Codex / Gemini CLI, to make sure that they have a limited / fake view of my system.
You can bind directories into it, or overlay them into it, restrict other kinds of access.
If Docker is a thing in a scale between a VM and your OS, Bubblewrap is a thing a scale between Docker and your OS. You use your OS, instead of installing and managing an OS like you do with Docker, but you get filesystem and process isolation like with Docker.
Though I had an issue where I cannot use `--new-session`, which is kind of dangerous to not use, but you can get around it if you use seccomp to block ioctl calls, and ptrace.