When to use memory safe languages(verdagon.dev)
verdagon.dev
When to use memory safe languages
https://verdagon.dev/blog/when-to-use-memory-safe-part-1
59 comments
This seems like a long axiomatic rationalization for using memory-unsafe languages. "[W]ith the right tooling, practices, and discipline, one can reduce the risk of memory safety bugs to an acceptable level for their situation". Well, empirically, no, that's not the case: the largest, most sophisticated teams in the world have tried, and it hasn't worked out.
>> Well, empirically, no, that's not the case
Rust itself has had more CVEs in the past 18 months than Postfix (written in C) has had in the past 18 years [1].
So it must be more nuanced than you're claiming.
EDIT: Postfix is somewhere in the region of 1,000x to 10,000x more popular than rust - it's hard to compare a compiler to a mail transfer agent but the MTA is in the more risky use case and yet fares better despite having more sysadmins deploying it than there are rust developers.
[1] https://www.cvedetails.com/vulnerability-list/vendor_id-1902...
https://www.cvedetails.com/vulnerability-list/vendor_id-8450...
Rust itself has had more CVEs in the past 18 months than Postfix (written in C) has had in the past 18 years [1].
So it must be more nuanced than you're claiming.
EDIT: Postfix is somewhere in the region of 1,000x to 10,000x more popular than rust - it's hard to compare a compiler to a mail transfer agent but the MTA is in the more risky use case and yet fares better despite having more sysadmins deploying it than there are rust developers.
[1] https://www.cvedetails.com/vulnerability-list/vendor_id-1902...
https://www.cvedetails.com/vulnerability-list/vendor_id-8450...
There are standard criteria for what can become a CVE, and I guess some Rust users are very zealous with bug reporting. At first glance, it looks like a lot of these issues are minor, but not normal, bugs to see in a programming language. I guess abnormality rises to that level.
A lot of Rust CVEs are filed for things that C/C++ software would just consider normal bugs, or not bugs at all.
For example: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html
The C++ stdlib is affected by the same issue, but they don't consider it a security bug, because the API doesn't claim to be safe of race conditions. Rust doesn't explicitly claim that either, but the Rust community has a higher bar for correctness around these things, so it gets a CVE.
Prior discussion here:
https://news.ycombinator.com/item?id=30461036
https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou...
For example: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html
The C++ stdlib is affected by the same issue, but they don't consider it a security bug, because the API doesn't claim to be safe of race conditions. Rust doesn't explicitly claim that either, but the Rust community has a higher bar for correctness around these things, so it gets a CVE.
Prior discussion here:
https://news.ycombinator.com/item?id=30461036
https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou...
You're comparing a programming language's standard library with an application. In the former case, most of the CVEs are pointing out bugs in 'unsafe' implementations of std APIs that result in UB where they should not. In the latter case, the CVEs are mostly pointing out actual exploits that permit privilege escalation. If you applied the standard by which CVEs are created for Rust to C---an apples to apples comparison---then most APIs in libc would have CVEs filed for them and they would be marked as 'wontfix'.
It's not at all a direct refutation of the GP's claim.
It's not at all a direct refutation of the GP's claim.
I'm sorry, but you don't understand how Rust's safety guarantees work. There would be multiple CVEs filed against almost every single library in libc and C++'s standard library if we used the same standards as Rust for those two. We don't, because it would obviously be pointless to do so, because C and C++ libraries generally don't have explicit safety guarantees, and those that do usually put an extremely high correctness burden on the caller. AFAIK, virtually all of Rust's CVEs found to date would go away (assuming the rest of the code remained the same) if Rust libraries just declared all the APIs in question "unsafe," which is basically what C and C++ do. The Rust standard library, your "random" example here, would have 0 CVEs by this metric, and so would pretty much every Rust library (the vast majority of Rust CVEs are filed against libraries).
In other words--a library can't be blamed for being insecure if it never makes any security promises in the first place. Rust libraries make stronger and more explicit guarantees, and the code is encapsulated in a much smaller region of the program, so it is easier to prove that they are in violation of their contract; but it emphatically does not mean that it results in more end to end vulnerabilities. Claiming that Rust simply isn't used in enough, or critical enough, places yet to have real exploits like C programs do is pretty naive, as it's started to be used in places like Android's bluetooth stack.
In fact, in practice there's even more of a gap than this--most C and C++ programs don't report CVEs even for reliably reproducible instance of UB unless you can demonstrate a full end to end attack chain, and many won't even fix or prioritize them if they consider it "benign" UB (what this means is anyone's guess!). Even projects like Chrome that actually care about security often won't upgrade UB bugs to CVEs if they believe (erroneously or not) that other hardening mechanism in the program will prevent it from being used as part of an RCE chain. Applying the same standards people typically do to C and C++ projects (an end to end exploit chain), I'm not sure whether there would be a single known Rust CVE in any widely used program.
Basically: the standard of evidence for a CVE in your average C or C++ program is astronomically high, while in Rust it is extremely low. Any attempt to decide which language is practically safer by counting CVEs (which is in any case bad methodology) needs to have awareness of this fact if it's trying to actually measure anything real.
In other words--a library can't be blamed for being insecure if it never makes any security promises in the first place. Rust libraries make stronger and more explicit guarantees, and the code is encapsulated in a much smaller region of the program, so it is easier to prove that they are in violation of their contract; but it emphatically does not mean that it results in more end to end vulnerabilities. Claiming that Rust simply isn't used in enough, or critical enough, places yet to have real exploits like C programs do is pretty naive, as it's started to be used in places like Android's bluetooth stack.
In fact, in practice there's even more of a gap than this--most C and C++ programs don't report CVEs even for reliably reproducible instance of UB unless you can demonstrate a full end to end attack chain, and many won't even fix or prioritize them if they consider it "benign" UB (what this means is anyone's guess!). Even projects like Chrome that actually care about security often won't upgrade UB bugs to CVEs if they believe (erroneously or not) that other hardening mechanism in the program will prevent it from being used as part of an RCE chain. Applying the same standards people typically do to C and C++ projects (an end to end exploit chain), I'm not sure whether there would be a single known Rust CVE in any widely used program.
Basically: the standard of evidence for a CVE in your average C or C++ program is astronomically high, while in Rust it is extremely low. Any attempt to decide which language is practically safer by counting CVEs (which is in any case bad methodology) needs to have awareness of this fact if it's trying to actually measure anything real.
I mean, you're not wrong, but cherry-picking two examples that aren't really comparable isn't a good argument
Agreed. It's hard to find examples of successful large Rust projects to compare against. The compiler and maybe deno are about it i think?
This is not what you'd expect for an alleged developer rocket ship after 10+ years. It doesn't compare favourably with C++ at 10 years old, or Java, or Go, or...
This is not what you'd expect for an alleged developer rocket ship after 10+ years. It doesn't compare favourably with C++ at 10 years old, or Java, or Go, or...
Largest Rust systems aren't public, like both Dropbox server and client.
Millions of lines of Rust are shipping in Firefox, so we could count CVEs in Firefox attributable to Rust. I agree this is annoying, but someone should do this.
I point out Rust in Firefox is an order of magnitude (10x) larger than rustc or deno.
Millions of lines of Rust are shipping in Firefox, so we could count CVEs in Firefox attributable to Rust. I agree this is annoying, but someone should do this.
I point out Rust in Firefox is an order of magnitude (10x) larger than rustc or deno.
It sounds like you just have a bone to pick here, but I would like to point out that C++ came out of Bell Labs, Java had massive corporate backing including a huge marketing campaign and Go has Google's backing. Each of your examples had huge advantages unrelated to any technical benefit.
A closer comparison might be Python, which took a long time to reach its potential, and I see Rust on the same trajectory, with high-profile teams now adopting it, like AWS.
A closer comparison might be Python, which took a long time to reach its potential, and I see Rust on the same trajectory, with high-profile teams now adopting it, like AWS.
To be clear, if you're using number of CVEs to determine whether some software is more secure than other software then you are very directly incentivizing people to not file CVEs. Your actions are directly harming the ecosystem.
It's definitely more nuanced. That said, I'd rather only have to fix memory unsafety in the language than potentially every program written with it. It reduces the audit surface area to just "unsafe" and Rust itself.
That data point doesn't say much before accounting for some confounding variables:
- the complexity of the code base
- the attack surface
- the amount of effort to find the CVEs
- the complexity of the code base
- the attack surface
- the amount of effort to find the CVEs
Sure - and i wouldn't put too much weight on my examples, i picked them as a direct refutation to the GP's claim "Well, empirically, no, that's not the case".
That said - 2 & 3 in your list seem like they'd lend further weight to Postifx rather than Rust in this scenario. Postfix is a network service forced to accept any and all input from remote attackers. Postfix has been around longer and has marketed itself as secure, that's meant it's attracted the attention of capable attackers keen to demonstrate their prowess.
I think the rust code base is more complex than postfix though. Postfix's architecture borders on a work of art.
That said - 2 & 3 in your list seem like they'd lend further weight to Postifx rather than Rust in this scenario. Postfix is a network service forced to accept any and all input from remote attackers. Postfix has been around longer and has marketed itself as secure, that's meant it's attracted the attention of capable attackers keen to demonstrate their prowess.
I think the rust code base is more complex than postfix though. Postfix's architecture borders on a work of art.
Completely uncomparable. This is a great example of why CVE counts are a useless metric.
I’ve worked on C++ code bases that were stabilized to the point where crashes were usually hardware bugs.
I strongly prefer Rust these days, though it has poor support for certain advanced use cases.
I strongly prefer Rust these days, though it has poor support for certain advanced use cases.
Another example is Redis. Redis crashes are usually hardware bugs. This is why Redis includes a memory tester, which is run on crashes, so that it does not receive spurious bug reports.
https://github.com/redis/redis/blob/unstable/src/memtest.c
https://github.com/redis/redis/blob/unstable/src/memtest.c
But as the article points out, it depends on the situation. The article is right that memory safety matters a lot more in certain areas than others.
The teams you refer to are working in areas where every exploit is a huge risk, places like a browser or an OS kernel. There is a reason Rust originated among people working on a browser. What you describe is correct for them. But the article has several examples of other applications where the tradeoffs are different.
Another example: LLVM is written in C++, but that's basically fine. A memory safety exploit in clang might be annoying, but it's not a vector for taking over people's machines like a browser is. A safety exploit in clang's output is where the real risks lie, but memory safety of clang itself doesn't help there.
The teams you refer to are working in areas where every exploit is a huge risk, places like a browser or an OS kernel. There is a reason Rust originated among people working on a browser. What you describe is correct for them. But the article has several examples of other applications where the tradeoffs are different.
Another example: LLVM is written in C++, but that's basically fine. A memory safety exploit in clang might be annoying, but it's not a vector for taking over people's machines like a browser is. A safety exploit in clang's output is where the real risks lie, but memory safety of clang itself doesn't help there.
Well, LLVM being written in C++ is fine for memory safety, but let's not kid ourselves. If LLVM were written in Rust it would have function level parallel compilation already. LLVM compilation is not parallelized yet because it is written in C++.
I don't know enough about why LLVM doesn't use parallel compilation, you might be right there.
But there are many C++ compiler codebases that do have it, such as: every major JS/Wasm VM (V8, SpiderMonkey, JSC), Binaryen, etc. Those do run in environments where parallel compilation matters a lot, unlike LLVM (which assumes parallelism in the build system, as you compile to object files).
But there are many C++ compiler codebases that do have it, such as: every major JS/Wasm VM (V8, SpiderMonkey, JSC), Binaryen, etc. Those do run in environments where parallel compilation matters a lot, unlike LLVM (which assumes parallelism in the build system, as you compile to object files).
Rust wants parallel LLVM, because Rust has larger compilation unit.
Interesting. That might be very specific to Rust, though. In that case, LLVM might not be the optimal backend for rustc.
But this isn't an issue for most LLVM languages, which either assume parallelism in the build system, or they get enough benefit from ThinLTO (which is the parallelism LLVM does support at the link stage).
But this isn't an issue for most LLVM languages, which either assume parallelism in the build system, or they get enough benefit from ThinLTO (which is the parallelism LLVM does support at the link stage).
> LLVM is written in C++, but that's basically fine. A memory safety exploit in [LLVM] might be annoying, but it's not a vector for taking over people's machines like a browser is.
Hrm, is LLVM no longer used for shader compilation? I think most people wouldn't like it if compiling a shader could delete your home directory.
Hrm, is LLVM no longer used for shader compilation? I think most people wouldn't like it if compiling a shader could delete your home directory.
In such cases one can sandbox LLVM, either in another process or using wasm etc., and that has been common for many years. It's practical to do so because it's a pure computation with a clear interface (compile source code to machine code).
Yup, Mesa is no longer using LLVM except for CPU fallback.
I think this argument gets sillier every year, and passed the silliness event horizon probably at least a decade ago. Taking what the article says for convenience, memory safety prevents "buffer overflows, use after free, and use after return". My question is, what are you doing that actually depends on those things? Why would you consider them features?
There is of course a small handful of answers like accessing memory-mapped buffers for hardware access, but these are fractional percentage exceptions, not things to bend your language choice around. They're fractional percentage exceptions even in programs where that's the primary use case; you're still better off using an "unsafe" function to do your little bit of magic, then using the memory safety everywhere else.
And... it's not like C is a nice language. By modern standards, C is just terrible. Unpleasant. Poor affordances that lead to bad designs, where you end up giving up on the right answer because it's just so darned hard so let's just take some shortcuts. Spikes everywhere you walk, knives everywhere you touch. Missing basic features to any modern language like closures, and if you do bash them in they come with so many caveats that they're hardly recognizable.
If you are still someone who learned C 30 years ago and still think it's just somehow incomparably amazing and nothing can compare, please, spend some time with a couple of other system languages long enough to come to fluency, then re-evaluate. All the other languages have been polishing for decades now, and they've straight-up passed C for all language-related purposes. Yea verily these ~25 years ago when I started in this industry there were still cases to be made that C or the C++ of the time was still necessary to get the performance you needed, but those days have passed. The only remaining place I can understand using it are those fringes where it's just the only choice, and even then, the newer languages are marching along and if I were stuck in those industries I'd be trying to get on board as quickly as possible.
There is of course a small handful of answers like accessing memory-mapped buffers for hardware access, but these are fractional percentage exceptions, not things to bend your language choice around. They're fractional percentage exceptions even in programs where that's the primary use case; you're still better off using an "unsafe" function to do your little bit of magic, then using the memory safety everywhere else.
And... it's not like C is a nice language. By modern standards, C is just terrible. Unpleasant. Poor affordances that lead to bad designs, where you end up giving up on the right answer because it's just so darned hard so let's just take some shortcuts. Spikes everywhere you walk, knives everywhere you touch. Missing basic features to any modern language like closures, and if you do bash them in they come with so many caveats that they're hardly recognizable.
If you are still someone who learned C 30 years ago and still think it's just somehow incomparably amazing and nothing can compare, please, spend some time with a couple of other system languages long enough to come to fluency, then re-evaluate. All the other languages have been polishing for decades now, and they've straight-up passed C for all language-related purposes. Yea verily these ~25 years ago when I started in this industry there were still cases to be made that C or the C++ of the time was still necessary to get the performance you needed, but those days have passed. The only remaining place I can understand using it are those fringes where it's just the only choice, and even then, the newer languages are marching along and if I were stuck in those industries I'd be trying to get on board as quickly as possible.
[deleted]
In my experience, having a large team makes it harder to achieve memory safety in 'MMM' languages. The ideal situation is a tractable codebase written by a small team of competent developers who can understand the entire scope. As you introduce more people, there are more potential points of failure
The author gave the following suggestions for how to use manual memory management (MMM) languages more safely:
- Don't use malloc.
- For long-lived allocations, use per-type arrays.
- In a game, we might have an array for all Ships, an array for all Missiles, and an array for all Bases.
- For temporary allocations, one can also use an arena allocator.
- For temporary allocations whose pointers don't escape, one can also use the stack.
- All unions must be tagged 11 and treated as values.
- Always use bounds checking.
The author then gives a list of tools like ASAN to perform various analyses (static and dynamic) to help further increase safety.
What I did not see, however, was one of the simplest and most effective suggestions for MMM programs:
- Don't use malloc.
- For long-lived allocations, use per-type arrays.
- In a game, we might have an array for all Ships, an array for all Missiles, and an array for all Bases.
- For temporary allocations, one can also use an arena allocator.
- For temporary allocations whose pointers don't escape, one can also use the stack.
- All unions must be tagged 11 and treated as values.
- Always use bounds checking.
The author then gives a list of tools like ASAN to perform various analyses (static and dynamic) to help further increase safety.
What I did not see, however, was one of the simplest and most effective suggestions for MMM programs:
Never free.
Along these lines, there is also (from 1988!) the Boehm Garbage Collector just an -lboehm away on your link line which in your suggested mode can reclaim (most) of the memory you might think you would lose. [1] (C does have many gotchas, though.)
[1] https://en.wikipedia.org/wiki/Boehm_garbage_collector
[1] https://en.wikipedia.org/wiki/Boehm_garbage_collector
The best way to do “never free” is suggestion #4, an arena allocator.
This is such a broad topic with so much to cover, I admire the author's willingness to tackle it. There's a ton of information one could say belongs in the article, but I don't think it would ever be written if they had tried to cover everything. All in all, I think this is a pretty good introduction, with a good trade-off between completeness and digestibility
This. Let’s not jump to the conclusion that making this point today is a silly argument or even an evil deed.
For someone who writes mostly in GC languages this was a great read and can recommend.
Even with switching to <your_safe_language> let’s not loose awareness of what is possible to strengthen current approaches.
For someone who writes mostly in GC languages this was a great read and can recommend.
Even with switching to <your_safe_language> let’s not loose awareness of what is possible to strengthen current approaches.
Given that many 'memory safe' languages offer escape hatches for non-memory-safe code, shouldn't the answer rather obviously be "always"? Assuming no extra constraints like compatibility with a large amount of non-memory-safe code, and that the language is actually fit for purpose (don't use js for kernel modules, nim as a procedural language in postgres)
Sadly, no, there isn’t always a memory safe language that is fit for purpose. For one thing, the article lists 5 memory management techniques. It is reasonably common for sufficiently large, high performance C++ code bases to rely on three or four of the things on that list, all in the same process.
The compiler can only partially verify that the application is not crossing the streams by passing memory from one allocator subsystem into another.
Also, memory unsafe languages are sometimes safer on some other dimension than the most obvious memory safe alternative. For instance, it is trivial to write C++ code that provides constant, non-null references (in the absence of UB), but completely impossible to express those invariants in the Java type system.
The compiler can only partially verify that the application is not crossing the streams by passing memory from one allocator subsystem into another.
Also, memory unsafe languages are sometimes safer on some other dimension than the most obvious memory safe alternative. For instance, it is trivial to write C++ code that provides constant, non-null references (in the absence of UB), but completely impossible to express those invariants in the Java type system.
RC alone can never be memory safe, only GC can.
First: cyclic structures (just leaks).
Second: wrong refcounts leading to UAF (use-after-free), in minor cases leaks.
First: cyclic structures (just leaks).
Second: wrong refcounts leading to UAF (use-after-free), in minor cases leaks.
Nice article. It would be nice if it touched on thread safety, since that is inseparable from memory safety these days. Maybe in part 2?
There are generally four approaches to memory safety:
[...]
Reference counting (RC), like in Swift, Nim, Lobster, etc.
You have a point, but memory leaks generally aren't considered as problematic as other memory issues, such as buffer overruns and use after free. While they can cause programs to crash and degrade, they generally don't corrupt program data or lead to security vulnerabilities.
As such, people often don't include protection from memory leaks as a criterion for 'memory safety'. It's possible to explicitly leak memory in the safe subset of Rust (e.g. Box::leak). It's also possible to 'leak' memory in GC'd languages too by keeping old references around you'll never use, although this example is just arguing semantics even more.
As such, people often don't include protection from memory leaks as a criterion for 'memory safety'. It's possible to explicitly leak memory in the safe subset of Rust (e.g. Box::leak). It's also possible to 'leak' memory in GC'd languages too by keeping old references around you'll never use, although this example is just arguing semantics even more.
> memory leaks generally aren't considered as problematic as other memory issues, such as buffer overruns and use after free. [...] they generally don't corrupt program data or lead to security vulnerabilities.
I see.
Although that definition of safety feels a little wrong to me, I can't think of any security issue that could be caused by a memory leak. Perhaps OOM killer butchering random processes on the system, bringing it to some vulnerable state? But then again - that's a system failure, as any well-behaved process might as well use more memory than is available.
I see.
Although that definition of safety feels a little wrong to me, I can't think of any security issue that could be caused by a memory leak. Perhaps OOM killer butchering random processes on the system, bringing it to some vulnerable state? But then again - that's a system failure, as any well-behaved process might as well use more memory than is available.
Safety usually means the program isn’t doing something wrong. You are thinking of liveness, which means the program continues to do something good.
If killing a process leaves the system in an unsafe state, that’s probably not the fault of the OOM killer. (What if the killed process was slow to start, or failed an assert and exited instead?)
If killing a process leaves the system in an unsafe state, that’s probably not the fault of the OOM killer. (What if the killed process was slow to start, or failed an assert and exited instead?)
It can depend on your precise definition of "safe". If you define it in terms of accessing correct memory, leaked memory may be inconvenient, it may be "incorrect", but it is safe. It is not, on its own, a security issue (other than eventually leading to possible program termination due to resource exhaustion, which is considered a security issue but I've always thought of this as generally "just barely" in the security class of issue). It may be chained into one if some other bug allowed access to it, but that "some other bug" would itself be the memory unsafe bug.
It may not even be incorrect. For instance if the only thing you care about is raw speed, an allocator which just grabs pages of memory and never deallocates or overwrites might be the optimal solution, especially if it's provable allocations will never grow beyond a certain upper limit.
> which would lead to memory leaks in any sufficiently complex program.
Memory leaks (and more generally resource leaks) are not generally considered memory safety issues.
After all, that's basically what an advanced GC manages for you: you leak memory, and it tries to clean up after you. But whether it does is entirely up to it and its configuration (nothing precludes disabling it and leaking all the memory, there's nothing actionable at the language level).
Memory leaks (and more generally resource leaks) are not generally considered memory safety issues.
After all, that's basically what an advanced GC manages for you: you leak memory, and it tries to clean up after you. But whether it does is entirely up to it and its configuration (nothing precludes disabling it and leaking all the memory, there's nothing actionable at the language level).
[deleted]
I think weak references solves that issue.
> A weak reference is a reference that doesn’t keep a strong hold on the instance it refers to, and so doesn’t stop ARC from disposing of the referenced instance. [0]
[0] https://docs.swift.org/swift-book/LanguageGuide/AutomaticRef...
> A weak reference is a reference that doesn’t keep a strong hold on the instance it refers to, and so doesn’t stop ARC from disposing of the referenced instance. [0]
[0] https://docs.swift.org/swift-book/LanguageGuide/AutomaticRef...
They would solve the issue only if the language itself forbid the programmer from creating cyclic strong references. If that's the case, then I was wrong in my original comment.
Although the thought of forbidding arbitrary references feels a little restrictive to me - it would be placing the burden of keeping a tree of references onto the programmer. I don't know how well that works in practice. Perhaps trying out some of the mentioned languages would be insightful.
Although the thought of forbidding arbitrary references feels a little restrictive to me - it would be placing the burden of keeping a tree of references onto the programmer. I don't know how well that works in practice. Perhaps trying out some of the mentioned languages would be insightful.
Rust provides explicit mechanisms to leak memory and its notion of memory safety does not include “never leaks” . A rust program that leaks memory like a sieve is considered safe.
Nim is choice, including choice among a variety of memory management options including manual, markAndSweep style collection, reference counting variants, etc.