Lua in the Kernel?(lwn.net)
lwn.net
Lua in the Kernel?
https://lwn.net/Articles/830154/
48 comments
What happened to the implementation of a Scheme interpreter in Linux from quite a while back? I can't immediately find a reference, and don't remember details. I vaguely remember there was something similar for SunOS too (in the '90s?).
NetBSD supports Lua in the kernel since 2012.
https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2...
https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2...
It's in fact exactly the same thing, since NetBSD Lua started as a port of Lunatik, which is the topic of this article.
Freebsd has used lua in its bootloader for a few years now.
https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
https://lists.freebsd.org/pipermail/freebsd-current/2018-Feb...
https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
https://lists.freebsd.org/pipermail/freebsd-current/2018-Feb...
Wow, it actually sounds like a very elegant idea. I always viewed GRUB as overly complex and rather unpleasant to configure. I wonder why they hadn't gone this way.
Note that we haven't yet introduced .lua configuration capabilities (that's a project branch off to the side), but it does certainly make modifying menu entries and other fun stuff a lot more palatable in my experience.
Grub also has an optional module run to Lua code https://git.savannah.nongnu.org/cgit/grub-extras.git/tree/lu...
> Scripts should not be able to crash the system, run indefinitely, or corrupt other parts of the system. To ensure that, Lunatik uses the Lua VM facilities for sandboxing the scripts so that they run in a safe execution environment [...] the number of instructions can be limited, so that infinite loops are not possible.
Lua's sandbox should be able to prevent scripts from crashing or corrupting the system, but I'd be surprised if it could prevent a script from running indefinitely. In particular, limiting the number of Lua bytecode instructions is insufficient - even a single call to `string.find` can lock up the CPU [0].
> They are using the standard Lua, rather than the LuaJIT fork, Neto said, in answer to another question.
Using LuaJIT would make sandboxing significantly more difficult, because it's much more complex than regular Lua and hence more likely to have exploitable bugs. This is the reason that game consoles disallow JIT-compilation - and hence games tend to use regular Lua rather than LuaJIT.
[0] http://lua-users.org/lists/lua-l/2011-02/msg01595.html
Lua's sandbox should be able to prevent scripts from crashing or corrupting the system, but I'd be surprised if it could prevent a script from running indefinitely. In particular, limiting the number of Lua bytecode instructions is insufficient - even a single call to `string.find` can lock up the CPU [0].
> They are using the standard Lua, rather than the LuaJIT fork, Neto said, in answer to another question.
Using LuaJIT would make sandboxing significantly more difficult, because it's much more complex than regular Lua and hence more likely to have exploitable bugs. This is the reason that game consoles disallow JIT-compilation - and hence games tend to use regular Lua rather than LuaJIT.
[0] http://lua-users.org/lists/lua-l/2011-02/msg01595.html
> This is the reason that game consoles disallow JIT-compilation - and hence games tend to use regular Lua rather than LuaJIT.
You can disable the JIT in LuaJIT and still get better performance because it's a different implementation of the VM.
> but I'd be surprised if it could prevent a script from running indefinitely.
You'd have to patch the standard library a bit to avid things like `string.find` causing trouble, but as far as core Lua features go, yes, you can do that and all you need is to install a debug hook (Preferably from C, because performance, but you can even do this 100% from within Lua)
-----
EDIT: Here's some benchmarks I did for this a while ago https://gist.github.com/DarkWiiPlayer/f639a38df53e39df63497c...
You can disable the JIT in LuaJIT and still get better performance because it's a different implementation of the VM.
> but I'd be surprised if it could prevent a script from running indefinitely.
You'd have to patch the standard library a bit to avid things like `string.find` causing trouble, but as far as core Lua features go, yes, you can do that and all you need is to install a debug hook (Preferably from C, because performance, but you can even do this 100% from within Lua)
-----
EDIT: Here's some benchmarks I did for this a while ago https://gist.github.com/DarkWiiPlayer/f639a38df53e39df63497c...
I'd be surprised if it could prevent a script from running indefinitely. In particular, limiting the number of Lua bytecode instructions is insufficient
FTA: “Lua has a facility to interrupt a script after it has run a certain number of instructions.”
FTA: “Lua has a facility to interrupt a script after it has run a certain number of instructions.”
string.find is implemented in C, not Lua. As far as Lua is concerned, that call is a single instruction.
You could have a string.find C implementation which gets an additional budget parameter and depending on it might limit search or similar (and reports back how much "budged" was used)
Through always under the assumption that the C function is implemented correctly and adding need to do some explicit budged keeping makes it more complex and in turn more prone to errors.
Through always under the assumption that the C function is implemented correctly and adding need to do some explicit budged keeping makes it more complex and in turn more prone to errors.
A single instruction that can run for a long time. That's the point.
ditto for eBPF when acessing the C helpers
Just add a time limit to the sandbox as well
Apparently, the sandbox cannot enforce such a time limit.
From a similar thread to the one linked from the root comment:
> ..Attempts to control untrusted scripts may be in vain. E.g. string.find, string.rep and other C functions can be abused and will either run forever or allocate unlimited amounts of memory. This is true for both Lua and LuaJIT.
> The only reasonably safe way to handle untrusted Lua scripts is to isolate them in a process context and to make use of per-process quotas/limits provided by the operating system.
Improved functions sandboxing in Lua and LuaJIT - http://lua-users.org/lists/lua-l/2011-02/msg01106.html
From a similar thread to the one linked from the root comment:
> ..Attempts to control untrusted scripts may be in vain. E.g. string.find, string.rep and other C functions can be abused and will either run forever or allocate unlimited amounts of memory. This is true for both Lua and LuaJIT.
> The only reasonably safe way to handle untrusted Lua scripts is to isolate them in a process context and to make use of per-process quotas/limits provided by the operating system.
Improved functions sandboxing in Lua and LuaJIT - http://lua-users.org/lists/lua-l/2011-02/msg01106.html
This solution is problematic because there's no one time limit that makes sense across all past and present CPUs that span orders of magnitude of performance.. your lua script would always fail on some CPU and there wouldn't be a way to fix it.
> In particular, limiting the number of Lua bytecode instructions is insufficient - even a single call to `string.find` can lock up the CPU [0].
At first glance, the second part of this sentence doesn't seem to support the initial premise. I am not too familiar with Lua bytecode but wouldn't "string.find" actually be compiled to multiple bytecode instructions ?
Unless a single bytecode instruction can lock up the cpu, doing so would most likely require either looping or recursive calls. Thus, by puting a hard limit on the number of instructions the sandbox can execute, it would end up aborting.
At first glance, the second part of this sentence doesn't seem to support the initial premise. I am not too familiar with Lua bytecode but wouldn't "string.find" actually be compiled to multiple bytecode instructions ?
Unless a single bytecode instruction can lock up the cpu, doing so would most likely require either looping or recursive calls. Thus, by puting a hard limit on the number of instructions the sandbox can execute, it would end up aborting.
> I am not too familiar with Lua bytecode but wouldn't "string.find" actually be compiled to multiple bytecode instructions ?
It's probably a single opcode of "invoke native-code standard-library function/intrinsic".
It's probably a single opcode of "invoke native-code standard-library function/intrinsic".
Yes. `string.find` is implemented in C. Invoking it (no matter how complex its arguments) requires just a few bytecode instructions.
Well, then that's library dependent and not an intrinsic part of the language. So shipping Lua safely would mean having to swap or drop parts of the standard library for something a bit more robust.
Apparently, that's what Lunatik does so the work seems to already have been done.
Apparently, that's what Lunatik does so the work seems to already have been done.
Appending strings together is a single bytecode instruction and this cannot really be fixed.
You can't prevent infinite loops unless you've solved the halting problem right?
They could of course just limit the run time if they really wanted to make sure scripts don't run indefinitely.
They could of course just limit the run time if they really wanted to make sure scripts don't run indefinitely.
>You can't prevent infinite loops unless you've solved the halting problem right?
Only in general arbitrary turing complete runtimes. For restricted subsets of instructions and specific implementations of runtimes you can. E.g. one could trivially disallow "jump" control flow, and limit loops to N iterations.
Only in general arbitrary turing complete runtimes. For restricted subsets of instructions and specific implementations of runtimes you can. E.g. one could trivially disallow "jump" control flow, and limit loops to N iterations.
It's more simple than that: pull the plug on the entire turing machine. Lua runs in a VM, so you can stop it from the outside, which wouldn't be possible if the code was running directly in the CPU (Without an OS underneath it, that is).
You could argue this doesn't really count as preventing though. It's preventing a symptom instead of the underlying problem.
The halting problem only says that you cannot tell if a program ever stops, i.e. takes a finite number of steps but without any a priori bounds. If you limit the number of steps to N, you can just run N steps of the program on your given input and look if it has stopped already.
The problem only says you can’t tell if the programs stops without actually running it. But here you’re running it anyway.
If you run it and it stops, then it stops. If you run it and it runs forever, then it runs forever. But the point is that you want to know if it's going to run forever before you hit "forever".
> They could of course just limit the run time
It seems as though this would be more robust than limiting the number of bytecode instructions. I wonder if that’s true and if so, why it’s not more commonly recommended.
It seems as though this would be more robust than limiting the number of bytecode instructions. I wonder if that’s true and if so, why it’s not more commonly recommended.
[deleted]
ePBF prevents infinite loops by restricting the language:
https://lwn.net/Articles/794934/
https://lwn.net/Articles/794934/
> Investigation of a "typed Lua" for compilation is something on the roadmap. That is the approach that the main Lua project is taking to compete with LuaJIT on performance.
I haven't heard anything about an official "typed Lua". Is there information about it somewhere?
I haven't heard anything about an official "typed Lua". Is there information about it somewhere?
* Typed Lua: https://github.com/andremm/typedlua (not actively maintained)
* tl: https://github.com/teal-language/tl a compiler for Teal a typed dialect of lua. (actively maintained)
* tl: https://github.com/teal-language/tl a compiler for Teal a typed dialect of lua. (actively maintained)
There is also Luau and Ravi but I don't think any of them is official which was the original question
Indeed, Teal looks like it has the right brains behind it (author was involved with several earlier attempts at typed Lua) and is actively maintained. I'm personally watching it for future game development usage.
I believe he was talking about Pallene, which is being currently developed in LabLua
Paper: http://www.inf.puc-rio.br/~roberto/docs/pallene-sblp.pdf
Git Repo: https://github.com/pallene-lang/pallene
Paper: http://www.inf.puc-rio.br/~roberto/docs/pallene-sblp.pdf
Git Repo: https://github.com/pallene-lang/pallene
Shameless self promotion:
https://github.com/Benjamin-Dobell/IntelliJ-Luanalysis
[deleted]
It's only a matter of time until we have metered WebAssembly in the kernel :)
Actually, why not? The main reason that "in the kernel" is concerning is because untrusted code needs some kind of sandboxing, either by static analysis or by run-time checks. (Most modern systems do a combination of both).
After all, the whole "in the kernel" vs. "in userland" distinction only exists because we get hardware-accelerated sandboxing from the CPU and OS combined.
After all, the whole "in the kernel" vs. "in userland" distinction only exists because we get hardware-accelerated sandboxing from the CPU and OS combined.
Yeah, my comment is actually unironic :)
There are a few ways to limit the execution time of wasm programs:
- inject code after a set of instructions with a certain duration and in each loop that calls an external function which meters and destroys/suspends execution when a limit is reached (e.g. https://github.com/ewasm/wasm-metering)
- use a VM which counts instructions (e.g. https://github.com/perlin-network/life)
- use hardware interrupts
More random thoughts: https://esolangs.org/wiki/RarVM
There are a few ways to limit the execution time of wasm programs:
- inject code after a set of instructions with a certain duration and in each loop that calls an external function which meters and destroys/suspends execution when a limit is reached (e.g. https://github.com/ewasm/wasm-metering)
- use a VM which counts instructions (e.g. https://github.com/perlin-network/life)
- use hardware interrupts
More random thoughts: https://esolangs.org/wiki/RarVM
I guess running an alternative language in kernel is not very challenging. (I tried it two years ago and managed to run i8042 keyboard driver in WebAssembly on Qemu) The hard part is interoperability. Even in languages with automatic C header -> native prototypes conversion capabilities, many kernel features are still not properly handled, especially the macros. Status quo requires the user to manually extract wrappers which could be super tedious.
Smells like an attack vector waiting to happen to me. Malformed network request, sandbox escape, front page of Hacker News like 8-10 years from now?
I am often irrationally paranoid about this sort of thing however.
I am often irrationally paranoid about this sort of thing however.
The neat thing about Lua is: It's simple. It's reasonable to expect that a sandbox can be proven to be inescapable considering there's only a very limited set of features that would allow this sort of thing in the first place (mainly the `debug` library, which I doubt will be exposed)
ZFS compiles a modified version Lua 5.2 into zfs.ko and loaded into Linux kernel.
To expand slightly, it's used for "channel programs", scripts that can interact with ZFS to create or modify datasets and similar tasks. There's a write-up here[1] about the feature.
Main advantage is that it allows for multiple operations to be applied atomically, as well as better error handling.
[1]: https://www.delphix.com/blog/delphix-engineering/zfs-channel...
Main advantage is that it allows for multiple operations to be applied atomically, as well as better error handling.
[1]: https://www.delphix.com/blog/delphix-engineering/zfs-channel...