OpenBSD's Pledge and Unveil from Python(nullprogram.com)
nullprogram.com
OpenBSD's Pledge and Unveil from Python
https://nullprogram.com/blog/2021/09/15/
40 comments
I was not familiar with these syscalls. Are they primarily in a “if you build it they will come” stage right now? Or is there an upswing of usage in the BSD world? Does the Linux kernel have any analog to pledge and unveil?
> BSD world
Pledge/unveil are openbsd-specific. Freebsd has capsicum which, like seccomp (mentioned else-thread) is much more complex and flexible.
Pledge/unveil are manifestly more successful, being used pervasively (though not really by programs which do not primarily target openbsd). Seccomp and capsicum are barely used at all (likely due to their higher complexity), and capsicum usage was even removed from a few freebsd utilities. At the same time, there have been some valid criticisms of pledge/unveil by capsicum people; IIRC something to do with its restrictions not persisting following an exec?
Pledge/unveil are openbsd-specific. Freebsd has capsicum which, like seccomp (mentioned else-thread) is much more complex and flexible.
Pledge/unveil are manifestly more successful, being used pervasively (though not really by programs which do not primarily target openbsd). Seccomp and capsicum are barely used at all (likely due to their higher complexity), and capsicum usage was even removed from a few freebsd utilities. At the same time, there have been some valid criticisms of pledge/unveil by capsicum people; IIRC something to do with its restrictions not persisting following an exec?
> At the same time, there have been some valid criticisms of pledge/unveil by capsicum people; IIRC something to do with its restrictions not persisting following an exec?
The validity of that criticism is so-so. It's a common complaint from people who are trying to build an externally imposed sandbox that dictates what a program can do.
But that's not what pledge and unveil are. They're more of an internally imposed set of constraints: the program just announces what it's going to do. After that, if it breaks the contract (due to a bug or malicious intervention), the system has license to kill it.
The program knows what it is going to do, so it can write the contract for itself. But it doesn't know what some other program is going to do, so it doesn't make sense for these restrictions to persist after exec. The program-to-be-exec'd should have its own pledges.
In reality it's even more complicated than that: programs often need to perform some "privileged" operations before they are ready to put on their straight jacket. It'd be very hard for program A to say that program B starts with privileges X, Y, Z, and then after instruction Q drops Y and Z. And program B might require more privileges than what A had when it called exec, so again it's just not going to work for this at all. If A's privileges were to persist, then it would have to have some way to elevate them or it'd never drop them in the first place. Both seem like a bad deal.
It's just a completely different mechanism, but people think sandbox sandbox sandbox and if that's all one can think of, pledge and unveil might seem like a terrible tool for that. They're not a tool for sandboxing untrusted programs.
The validity of that criticism is so-so. It's a common complaint from people who are trying to build an externally imposed sandbox that dictates what a program can do.
But that's not what pledge and unveil are. They're more of an internally imposed set of constraints: the program just announces what it's going to do. After that, if it breaks the contract (due to a bug or malicious intervention), the system has license to kill it.
The program knows what it is going to do, so it can write the contract for itself. But it doesn't know what some other program is going to do, so it doesn't make sense for these restrictions to persist after exec. The program-to-be-exec'd should have its own pledges.
In reality it's even more complicated than that: programs often need to perform some "privileged" operations before they are ready to put on their straight jacket. It'd be very hard for program A to say that program B starts with privileges X, Y, Z, and then after instruction Q drops Y and Z. And program B might require more privileges than what A had when it called exec, so again it's just not going to work for this at all. If A's privileges were to persist, then it would have to have some way to elevate them or it'd never drop them in the first place. Both seem like a bad deal.
It's just a completely different mechanism, but people think sandbox sandbox sandbox and if that's all one can think of, pledge and unveil might seem like a terrible tool for that. They're not a tool for sandboxing untrusted programs.
It's also worth noting that you can pledge for an execed process using the second argument to pledge. This is for when you know what the child is expected to do on your behalf.
> Pledge/unveil are manifestly more successful, being used pervasively (though not really by programs which do not primarily target openbsd).
Because of their simplicity, they are easily added as a patch to a ported application. There are probably more ports using them than programs that target openbsd.
Because of their simplicity, they are easily added as a patch to a ported application. There are probably more ports using them than programs that target openbsd.
It really doesn't make much sense to compare pledge/unveil to seccomp and capsicum or any other sandboxing solution.
Pledge and Unveil are really part of the program's specification. They are much closer in practice to asserts, pre-conditions, or contracts depending on the programming language you have used. They basically make sure that the program you are writing doesn't do anything stupid with bad inputs from the outside world. Its part of the development process, and not very hard to add to an existing program.
Pledge and Unveil are really part of the program's specification. They are much closer in practice to asserts, pre-conditions, or contracts depending on the programming language you have used. They basically make sure that the program you are writing doesn't do anything stupid with bad inputs from the outside world. Its part of the development process, and not very hard to add to an existing program.
A major limitation of seccomp is that you can't turn off inheritance. With pledge inheritance is opt-in.
This has major consequences because it makes it possible for a process to secure itself without affecting its ability to spawn child processes. You could apply seccomp in-process or use a spawn-helper before securing it. But more realistically it means that one applies an external seccomp profile to a group of processes (e.g. a container) that contains the union of all needed syscalls.
So while seccomp is more flexible in some sense (being able to run BPF filters on every syscall) its architecture leads to it being applied in a less flexible way.
This has major consequences because it makes it possible for a process to secure itself without affecting its ability to spawn child processes. You could apply seccomp in-process or use a spawn-helper before securing it. But more realistically it means that one applies an external seccomp profile to a group of processes (e.g. a container) that contains the union of all needed syscalls.
So while seccomp is more flexible in some sense (being able to run BPF filters on every syscall) its architecture leads to it being applied in a less flexible way.
> Freebsd has capsicum which, like seccomp (mentioned else-thread) is much more complex and flexible.
Capsicum may be more flexible in some ways, but it's also less flexible in others.
After you a process entera capsicum mode, it can't open new sockets, except by accepting on an existing listen socket or by receiving them on a unix socket, sent by a cooperating non-capsicum process. This means you can't capsicum a TLS proxy like hitch, which would be a great thing to capsicum since the operation is pretty simple and OpenSSL is scary.
Capsicum may be more flexible in some ways, but it's also less flexible in others.
After you a process entera capsicum mode, it can't open new sockets, except by accepting on an existing listen socket or by receiving them on a unix socket, sent by a cooperating non-capsicum process. This means you can't capsicum a TLS proxy like hitch, which would be a great thing to capsicum since the operation is pretty simple and OpenSSL is scary.
No true, of course you can do that. Not being able to open files or sockets only means you need a small separate process to do that for you and then send you the file descriptors.
I said you could get them from a cooperating uncapsicumed process. But, it's not simple, and what are you going to write that loophole process in, and why does it get access to the filesystem if it only needed sockets, etc.
Capsicum is simply not flexible in this way. Maybe if there was a way to open a new socket with a capability you setup earlier, that would be flexible enough.
Capsicum is simply not flexible in this way. Maybe if there was a way to open a new socket with a capability you setup earlier, that would be flexible enough.
It is pretty simple - implementing that for irssi(1) took a dozen or two lines of C, IIRC. Sure, it could be simpler, and hopefully libcasper(3) will make it happen, but it's not much of a difference.
They are also available (and used) in SerenityOS:
https://awesomekling.github.io/pledge-and-unveil-in-Serenity...
https://awesomekling.github.io/pledge-and-unveil-in-Serenity...
they only exist on OpenBSD. They're used a lot for default OpenBSD binaries, but they are mostly hopeless for openbsd developers/porters to try to apply to portable software after the fact if the upstream devs aren't supporting that work.
(There are other mechanisms like containers or freebsd' jail that try to accomplish the same thing, but those tend to be "lots of functionality inside the sandbox" solutions, whereas openbsd is mostly aiming for "allow nothing besides the minimum in the sandbox".
(edited for clarity, thanks for the nudge ghoward).
(There are other mechanisms like containers or freebsd' jail that try to accomplish the same thing, but those tend to be "lots of functionality inside the sandbox" solutions, whereas openbsd is mostly aiming for "allow nothing besides the minimum in the sandbox".
(edited for clarity, thanks for the nudge ghoward).
I upvoted, but I wanted to correct a small thing.
> ...they are mostly hopeless to try to apply to portable software after the fact if the devs aren't constantly testing with it.
They don't have to be hopeless. They were pretty easy ([1], [2]) to add to my bc.
[1]: https://git.yzena.com/gavin/bc/commit/3e8cd345de9d5b65ac7c33...
[2]: https://git.yzena.com/gavin/bc/commit/b6c65bb44c910c054bedfa...
> ...they are mostly hopeless to try to apply to portable software after the fact if the devs aren't constantly testing with it.
They don't have to be hopeless. They were pretty easy ([1], [2]) to add to my bc.
[1]: https://git.yzena.com/gavin/bc/commit/3e8cd345de9d5b65ac7c33...
[2]: https://git.yzena.com/gavin/bc/commit/b6c65bb44c910c054bedfa...
Yes. That's awesome!
I was referring to the OpenBSD devs trying to hack pledges into other people's software by trying to guess what capabilities it needs while porting. That's not a good thing to try to do -- see also all the various attempts at guessing seccomp profiles in the Linux world, which has the same problem.
I was referring to the OpenBSD devs trying to hack pledges into other people's software by trying to guess what capabilities it needs while porting. That's not a good thing to try to do -- see also all the various attempts at guessing seccomp profiles in the Linux world, which has the same problem.
looks like unveil() has an rough analog in Linux 5.13: https://landlock.io/, there was even a HN discussion which I missed: https://news.ycombinator.com/item?id=27215563
Would be useful to have a unified API…
Indeed, but on the other hand, almost all of the projects mentioned in the comments here are of the 'for this project, we do it this way' type of implementation meaning that in their local use case their way is the unified API.
That is both the benefit and the downside of non-commercial projects: you get to do whatever you want.
That is both the benefit and the downside of non-commercial projects: you get to do whatever you want.
True. It may be worth postponing a unified API when the semantics solidify. I really like the pledge/unveil semantics, but it's not a consensus.
To my understanding, you can use existing Linux kernel facilities like seccomp to build equivalents for pledge and unveil. The latter are less flexible but also easier to use, which is a point in their favour; ease of use matters for security.
> I was not familiar with these syscalls. Are they primarily in a “if you build it they will come” stage right now?
It’s an openbsd feature they built for themselves and deployed throughout the base system.
It’s an openbsd feature they built for themselves and deployed throughout the base system.
seccomp in linux is _roughly_ similar to pledge. they aim to accomplish the same goals but pledge is more straightforward. think of it like sudo vs doas.
Seccomp profiles are typically applied externally to a process in my experience though. I think this is more analogous to using capset() to remove capabilities in highly permissioned processes. For instance dnsmasq makes use of this ability to be minimally permissioned.
> Python functions that accept paths, such as open, generally accept either strings or bytes.
Or a pathlib.Path, hence os.fspath.
In fact for this specific use case there’s even better:
> os.fsencode(filename)
> Encode path-like filename to the filesystem encoding with 'surrogateescape' error handler, or 'strict' on Windows; return bytes unchanged.
Or a pathlib.Path, hence os.fspath.
In fact for this specific use case there’s even better:
> os.fsencode(filename)
> Encode path-like filename to the filesystem encoding with 'surrogateescape' error handler, or 'strict' on Windows; return bytes unchanged.
> This means no checking for OpenBSD specifically but instead feature sniffing for their presence.
Indeed it sniffs for any functions named pledge() and unveil() that exist in any library loaded into the process… and then assumes that, if they exist, they have not only the same purpose but also the exact same signatures as the corresponding functions from OpenBSD. ctypes cannot validate function signatures, so if they have different signatures, you get undefined behavior. I wouldn’t recommend this approach.
Indeed it sniffs for any functions named pledge() and unveil() that exist in any library loaded into the process… and then assumes that, if they exist, they have not only the same purpose but also the exact same signatures as the corresponding functions from OpenBSD. ctypes cannot validate function signatures, so if they have different signatures, you get undefined behavior. I wouldn’t recommend this approach.
Just ask OpenBSD to provide a pledge_sig and unveil_sig that simply return a C string where each character is the type of the corresponding argument.
E.g., if pledge_sig returns "ss" you know you're dealing either with the bona fide OpenBSD pledge or an evil demon.
E.g., if pledge_sig returns "ss" you know you're dealing either with the bona fide OpenBSD pledge or an evil demon.
What would be the right way to do it? A plain old Python C extension?
Or using API-mode cffi which basically does that for you, though it’s still not quite safe you can combine `cdef` and `set_source` to re-export exactly what you’re looking for. `set_source` will basically create an intermediate module under your control.
Sadly AFAIK you always need a `cdef` which defines the binding between Python and C, I don’t think you can tell cffi to get this information from a real header file. But by providing a custom source you can more easily ensure the `cdef` and the function for it match correctly, with `set_source` bridging to the real underlying functions.
One drawback of using API-level CFFI is it requires a C compiler (and probably all sorts of dev packages / headers), whereas ABI-level use doesn’t.
Sadly AFAIK you always need a `cdef` which defines the binding between Python and C, I don’t think you can tell cffi to get this information from a real header file. But by providing a custom source you can more easily ensure the `cdef` and the function for it match correctly, with `set_source` bridging to the real underlying functions.
One drawback of using API-level CFFI is it requires a C compiler (and probably all sorts of dev packages / headers), whereas ABI-level use doesn’t.
I would check to make sure I was running on an OpenBSD system first before anything else.
You could use sys.platform()
https://docs.python.org/3/library/sys.html#sys.platform
You could use sys.platform()
https://docs.python.org/3/library/sys.html#sys.platform
> I would check to make sure I was running on an OpenBSD system first before anything else.
But SerenityOS has pledge and unveil too.
https://awesomekling.github.io/pledge-and-unveil-in-Serenity...
Dunno if Python runs on SerenityOS or not yet tho
But SerenityOS has pledge and unveil too.
https://awesomekling.github.io/pledge-and-unveil-in-Serenity...
Dunno if Python runs on SerenityOS or not yet tho
Not wanting to tie it to OpenBSD only was the reason he chose feature sniffing. From the article:
> "Systems other than OpenBSD may support these functions, now or in the future, and it would be nice to automatically make use of them when available. This means no checking for OpenBSD specifically but instead feature sniffing for their presence."
> "Systems other than OpenBSD may support these functions, now or in the future, and it would be nice to automatically make use of them when available. This means no checking for OpenBSD specifically but instead feature sniffing for their presence."
That's a good point. But I would still include a check for OS's that I know support it. This means it wouldn't work on OS's without first explicitly allowing it.
I view this as a feature and not a bug. There's a good chance there are other things to consider when a new OS adds pledge or unveil, and this gives the developer a chance to test support on the new OS before anyone uses it.
Basically, I disagree with the article that you want to implement this in Python in a completely OS agnostic manner.
I view this as a feature and not a bug. There's a good chance there are other things to consider when a new OS adds pledge or unveil, and this gives the developer a chance to test support on the new OS before anyone uses it.
Basically, I disagree with the article that you want to implement this in Python in a completely OS agnostic manner.
Make "failed to find `pledge` function' a hard error on OSes known to have it.
so at what level does this happen? The forked python interpreter and its children (but not any parent process)?
I love Python and OpenBSD. I've been using `pyopenbsd` [1] to access pledge/unveil in some chat bots. They may not be the most well-coded, secure, or stable bots, but having a one- [2] to three-line [3] change to harden them so much is great for me.
1. https://github.com/yuce/pyopenbsd
2. https://github.com/rendello/TTD2_Bot/blob/dev/src/main.py#L1...
3. https://github.com/rendello/pipe_bot/blob/develop/src/main.p...
1. https://github.com/yuce/pyopenbsd
2. https://github.com/rendello/TTD2_Bot/blob/dev/src/main.py#L1...
3. https://github.com/rendello/pipe_bot/blob/develop/src/main.p...
This was written by Chris "skeeto" Wellons, a computer scientist for which I've personally grown a lot of admiration. Check out his Github if you're interested, he does a lot of really interesting things and is worth following.
Personal favorite is the branchless utf-8 decoder which I've used quite a few times in lexers and other projects.
https://github.com/skeeto?tab=repositories&sort=stargazers
Personal favorite is the branchless utf-8 decoder which I've used quite a few times in lexers and other projects.
https://github.com/skeeto?tab=repositories&sort=stargazers
We would be better off if every program on every operating system was required to call these functions on startup. Even better if the data was encoded into the binary, to be read by the kernel before starting the process.
Other discussions: https://discussions.xojoc.pw/?url=https%3A%2F%2Fnullprogram....