Rust in Linux Revisited(drewdevault.com)
drewdevault.com
Rust in Linux Revisited
https://drewdevault.com/2024/08/30/2024-08-30-Rust-in-Linux-revisited.html
180 comments
> I don't see why it's so important to rewrite parts of the kernel in rust, or why Drew is so sympathetic to the cause. The linux project uses static analysis tools which provide some of what rust guarantees (outside of unsafe).
It’s important because static analysis tools get nowhere near the coverage of a memory safe language.
Static analysis can find bugs but it cannot prove their absence. A memory safe type system proves absence. Rust’s type system proves absence module uses of unsafe, which is still much better than what any C static analysis could do.
(I am saying this as a Rust hater who’s trying to obviate the need for it.)
> The language isn't the problem, it's the architecture. In Linux, all of the drivers run with the rest of the kernel. A bad driver can take down the whole thing, or perform arbitrary badness.
This is a case of “why not both”.
Sure, you can harden an OS by compartmentalizing. And you can also harden using memory safety. As time goes on, folks will do both. The fact that you could (and should) do one of these things is not an argument to not do the other.
It’s important because static analysis tools get nowhere near the coverage of a memory safe language.
Static analysis can find bugs but it cannot prove their absence. A memory safe type system proves absence. Rust’s type system proves absence module uses of unsafe, which is still much better than what any C static analysis could do.
(I am saying this as a Rust hater who’s trying to obviate the need for it.)
> The language isn't the problem, it's the architecture. In Linux, all of the drivers run with the rest of the kernel. A bad driver can take down the whole thing, or perform arbitrary badness.
This is a case of “why not both”.
Sure, you can harden an OS by compartmentalizing. And you can also harden using memory safety. As time goes on, folks will do both. The fact that you could (and should) do one of these things is not an argument to not do the other.
> It’s important because static analysis tools get nowhere near the coverage of a memory safe language
It’s hard for me to believe a HN reader would not know this, especially given there are regular exploits revealed in the kernel.
It’s hard for me to believe a HN reader would not know this, especially given there are regular exploits revealed in the kernel.
> I don't see why it's so important to rewrite parts of the kernel in rust,
It'd be nice to be able to say, "We're pretty sure these source code files don't contain any null pointer dereferences, array out-of-bounds errors, use-after-free errors, double frees, or (if the kernel rust project fully embraces the Rust way of doing concurrency, I'm not familiar with the technical details) race conditions, because otherwise it wouldn't have compiled." When absolutely necessary, unsafe code blocks can provide a back door around some of these checks, but as long as the unsafe code blocks are relatively small, it's a much smaller surface area for these kinds of bugs to show up.
Not to mention the type system is in some ways much more user friendly. Proper algebraic types are easier than trying to cobble something ad-hoc together with tagged unions.
> or why Drew is so sympathetic to the cause.
He's not, he's suggesting in the most polite and diplomatic way possible that he thinks they should stop what they're doing, and do something else instead that kernel maintainers don't have to deal with.
It'd be nice to be able to say, "We're pretty sure these source code files don't contain any null pointer dereferences, array out-of-bounds errors, use-after-free errors, double frees, or (if the kernel rust project fully embraces the Rust way of doing concurrency, I'm not familiar with the technical details) race conditions, because otherwise it wouldn't have compiled." When absolutely necessary, unsafe code blocks can provide a back door around some of these checks, but as long as the unsafe code blocks are relatively small, it's a much smaller surface area for these kinds of bugs to show up.
Not to mention the type system is in some ways much more user friendly. Proper algebraic types are easier than trying to cobble something ad-hoc together with tagged unions.
> or why Drew is so sympathetic to the cause.
He's not, he's suggesting in the most polite and diplomatic way possible that he thinks they should stop what they're doing, and do something else instead that kernel maintainers don't have to deal with.
> why Drew is so sympathetic to the cause
I think this is a mischaracterization. If you read the original post[1] it's clear he's not at all sympathetic to the cause, though given recent news he is more sympathetic to the people who have the cause (even if he thinks it's misguided).
[1] https://drewdevault.com/2022/10/03/Does-Rust-belong-in-Linux...
I think this is a mischaracterization. If you read the original post[1] it's clear he's not at all sympathetic to the cause, though given recent news he is more sympathetic to the people who have the cause (even if he thinks it's misguided).
[1] https://drewdevault.com/2022/10/03/Does-Rust-belong-in-Linux...
How is he sympathetic to the cause? He isn’t. The original article gave the project a thumbs down and so does this one… only this time he is here to admit defeat on behalf of the project because of “burnout”.
All the praise is just nerdstroking to soften the kernel of the argument that they are working on the wrong thing.
All the praise is just nerdstroking to soften the kernel of the argument that they are working on the wrong thing.
I am sympathetic to the cause. I respect the Linux project as a whole forming a consensus on the right direction for the kernel and I respect the right of the Rust developers to participate in the consensus-making process and advocate for what they believe is the right direction -- and put in the work to get there. I can hold this view and disagree with their opinion at the same time. I celebrate the difficult technical and political work they're doing to advance their cause, and I respect the hell out of that -- the fact that I would make different choices doesn't contradict this in any way.
The fact of the matter is that politics was always going to be difficult, and for all of the respect and admiration I have for the Rust-for-Linux team -- which I do have, thank you -- I am equally sympathetic to the kernel hackers who didn't ask for this project on their doorstep, and for their own needs to be accommodated. I condemn the toxicity that has bubbled up in this process, from C hackers and Rust hackers alike, but even absent that toxicity I think that the political challenges of Rust-for-Linux are enormous and distract from the fundamental work of the project.
People have burnt out and quit the project, and you cannot erase their experience when it questions the viability of the project. I'm offering them compassion and a different path that might celebrate their work without leading to burnout. People having burnt out and quitting the project is a historical fact, and not my fault, even if I'm the easy polemic for you to pin the blame on. I think it'd be a fucking shame if they quit pursing their passions for OS development in Rust over it and I've said as much and that's more than I can say for you and everyone else filling my messages with personal attacks and bad faith reading of everything I have to say.
The fact of the matter is that politics was always going to be difficult, and for all of the respect and admiration I have for the Rust-for-Linux team -- which I do have, thank you -- I am equally sympathetic to the kernel hackers who didn't ask for this project on their doorstep, and for their own needs to be accommodated. I condemn the toxicity that has bubbled up in this process, from C hackers and Rust hackers alike, but even absent that toxicity I think that the political challenges of Rust-for-Linux are enormous and distract from the fundamental work of the project.
People have burnt out and quit the project, and you cannot erase their experience when it questions the viability of the project. I'm offering them compassion and a different path that might celebrate their work without leading to burnout. People having burnt out and quitting the project is a historical fact, and not my fault, even if I'm the easy polemic for you to pin the blame on. I think it'd be a fucking shame if they quit pursing their passions for OS development in Rust over it and I've said as much and that's more than I can say for you and everyone else filling my messages with personal attacks and bad faith reading of everything I have to say.
This just gives all the power to abusers.
I'd rather fix the abuse (the cause of the particular burnout), because it impacts not only the Rust developers, but all current and future contributors, and indirectly us users as well.
I'd rather fix the abuse (the cause of the particular burnout), because it impacts not only the Rust developers, but all current and future contributors, and indirectly us users as well.
> and that's more than I can say for you and everyone else filling my messages with personal attacks and bad faith reading of everything I have to say.
In a metaphorical sense… since HN is not your inbox and this is not your submission.
In a metaphorical sense… since HN is not your inbox and this is not your submission.
gtm strategy trumps nerdy perfectionism. Linux is successful because of its timing and probably accidental social engineering - regularly breaking out-of-tree drivers incentivized upstreaming hardware support. Doesn't matter how good an alternative would be on technical merits, there's too much momentum behind Linux.
Except maybe a fork where oxidation proceeds much faster, sponsored by a large corporation with bespoke hardware they can develop for and interest in security. Cloud OS-es or Android maybe.
Except maybe a fork where oxidation proceeds much faster, sponsored by a large corporation with bespoke hardware they can develop for and interest in security. Cloud OS-es or Android maybe.
Do we get also realtime (10ths us latency) out-of-the-box, where badly written drivers cannot negatively interfere good ones?
To get to that point (mixed criticality, some processes and drivers can be critical and others not be), you need to be able to reason about time.
That's exclusively seL4 right now, and the very state of the art.
(incidentally, we really should be putting our weight behind seL4, it really is good)
That's exclusively seL4 right now, and the very state of the art.
(incidentally, we really should be putting our weight behind seL4, it really is good)
> Here’s the pitch: a motivated group of talented Rust OS developers could build a Linux-compatible kernel, from scratch, very quickly, with no need to engage in LKML politics.
The BSDs, Solaris/Illumos, and Windows all have tried, some more than once even. I think there have been at least 4 Linux kernel ABI compatibility layers for those OSes. They've all failed. And all future attempts will fail too for the same reason:
To top it all off that undefined (but stable) ABI is a moving target: it's always evolving / getting extended.
The only thing you have to help you is that whatever the ABI, Linux [mostly] commits to maintaining backwards compatibility with it. If your compatibility ever gets good enough, that will help you, but in the meantime you'll be chasing a never ending and ever growing ill-defined ABI.
Good luck with that!
The BSDs, Solaris/Illumos, and Windows all have tried, some more than once even. I think there have been at least 4 Linux kernel ABI compatibility layers for those OSes. They've all failed. And all future attempts will fail too for the same reason:
The Linux kernel ABI is not
specified anywhere, and it's
insanely large (and growing).
To top it all off that undefined (but stable) ABI is a moving target: it's always evolving / getting extended.
The only thing you have to help you is that whatever the ABI, Linux [mostly] commits to maintaining backwards compatibility with it. If your compatibility ever gets good enough, that will help you, but in the meantime you'll be chasing a never ending and ever growing ill-defined ABI.
Good luck with that!
There's at least one more to add to the pile, Google's Fuchsia is primarily written in Rust and aims to support the Linux ABI through "starnix".
See https://fuchsia.dev/fuchsia-src/concepts/components/v2/starn... and https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/s...
See https://fuchsia.dev/fuchsia-src/concepts/components/v2/starn... and https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/s...
No, just download the source and check. It's 14M C/C++ and 4M in Rust. There is another 3.3M in Go and 1.1 in Dart. This is a usual trope that Rust is just about to replace C++ but in fact more and more gets written in C++.
curl -s "https://fuchsia.googlesource.com/fuchsia/+/HEAD/scripts/boot..." | base64 --decode | bash
curl -s "https://fuchsia.googlesource.com/fuchsia/+/HEAD/scripts/boot..." | base64 --decode | bash
Fuchsia is written in C++, not Rust.
It's half-half. The kernel is c++, but it is small relative to the overall OS which is predominantly use space. Growth in rust far outpaces c++ so in a few years c++ will likely be a much smaller fraction. Also, notably, starnix is entirely written in rust.
No, just download the source and check. It's 14M C/C++ and 4M in Rust. There is another 3.3M in Go and 1.1 in Dart. This is a usual trope that Rust is just about to replace C++ but in fact more and more gets written in C++.
curl -s "https://fuchsia.googlesource.com/fuchsia/+/HEAD/scripts/boot..." | base64 --decode | bash
curl -s "https://fuchsia.googlesource.com/fuchsia/+/HEAD/scripts/boot..." | base64 --decode | bash
A lot of what you're seeing is third party dependencies, not code fuchsia developers have written. A lot of that is third party code is also dead and isn't compiled into anything (like mesa). If you do a more detailed analysis of what ends up in an actual fuchsia image, it'll look a bit more like 50/50 (and that's close to what you get if you just remove the third_party/ directory). It would be strange on fuchsia to start on a new project and choose c++ over rust these days. Most critical components of the os including filesystems, network stack, linux emulation layer, init system, etc are all rust. Things which are not are likely to be rewritten in rust eventually.
Go and dart are basically gone from the system as well. Go is only used in the legacy netstack which has been displaced by one written in rust, and is otherwise only used in host tools for building fuchsia. I believe dart is almost gone, with a few remnants left in the form of build tools. Flutter does support fuchsia (and that is dart), but that support is not maintained in a fuchsia repo.
Source: I work on fuchsia.
Go and dart are basically gone from the system as well. Go is only used in the legacy netstack which has been displaced by one written in rust, and is otherwise only used in host tools for building fuchsia. I believe dart is almost gone, with a few remnants left in the form of build tools. Flutter does support fuchsia (and that is dart), but that support is not maintained in a fuchsia repo.
Source: I work on fuchsia.
Thanks for the clarification, I was definitely under the misapprehension that Fuchsia was basically 100% C++.
I think the real reason is very few developers are interested in that and because virtualisation solve most users issues related to not being able to run software X on BSD and most open source software are relatively easy to port to these OS if there is interest.
You do not become a BSD or illumos developer because you are interested in replicating Linux.
You do not become a BSD or illumos developer because you are interested in replicating Linux.
Sun actually paid for the development of some of this. Sun definitely had a business interest in having a Linux compatible product that wasn't Linux.
I am split on this. Maybe I am idealistic and naive, but I will always wish people can just stop fighting, and work together. Fragmenting the community and starting a project out of grudge, is always the last resort, IMO.
OTOH, I also recognize sometimes it is really that bad and going your own way is the best thing to do. But I think Linux isn't at that position yet.
OTOH, I also recognize sometimes it is really that bad and going your own way is the best thing to do. But I think Linux isn't at that position yet.
Replacing C with Rust within the kernel is not going to be peaceful. It's been made clear by now that most Linux developers, including those most prominent, prefer C and do not wish to invest any time or effort accommodating Rust developers.
>but I will always wish people can just stop fighting, and work together.
Two separate ideas are conflated here.
It would make me happy if they stopped fighting, and instead worked separately on their C and Rust kernels respectively.
>but I will always wish people can just stop fighting, and work together.
Two separate ideas are conflated here.
It would make me happy if they stopped fighting, and instead worked separately on their C and Rust kernels respectively.
> including those most prominent,
The most prominent one is supportive of Rust. And traditionally when it comes to technical decisions, it is his opinion that matters most.
The most prominent one is supportive of Rust. And traditionally when it comes to technical decisions, it is his opinion that matters most.
>The most prominent one is supportive of Rust
It is the premise. Without his acceptance of Rust, we wouldn't be at this point.
But he can't realistically do the relevant rust-accommodation work all by himself.
Not does he seem supportive of Rust enough as to replace the top maintainers he trusts and has worked with for a very long time with those that will extend the red carpet for Rust developers.
Perhaps Linus was trying to be kind. But sometimes, just saying "No." is most kind.
He didn't have the luxury of hindsight. He tried to be accommodating. Now, we are in a situation that is not pleasant for anybody involved.
It is the premise. Without his acceptance of Rust, we wouldn't be at this point.
But he can't realistically do the relevant rust-accommodation work all by himself.
Not does he seem supportive of Rust enough as to replace the top maintainers he trusts and has worked with for a very long time with those that will extend the red carpet for Rust developers.
Perhaps Linus was trying to be kind. But sometimes, just saying "No." is most kind.
He didn't have the luxury of hindsight. He tried to be accommodating. Now, we are in a situation that is not pleasant for anybody involved.
The problem is that neither Linus nor the other prominent maintainers will live forever. C was the right choice in 1991 but today the landscape is different enough that its shortcomings, for the younger generations, are painful to ignore.
So saying yes to Rust, or to some other language that is not filled with foot guns and could also work in the kernel space, is not only a matter of kindness but a matter of long-term strategy for the kernel.
So saying yes to Rust, or to some other language that is not filled with foot guns and could also work in the kernel space, is not only a matter of kindness but a matter of long-term strategy for the kernel.
>The problem is that neither Linus nor the other prominent maintainers will live forever.
I agree. But likewise, Linux's design is far from the state of the art. Maybe there's value in letting it be what it always has been. It is not realistic to try to migrate millions of LoCs written with poor structure, to properly structured Rust, with barely any support from the pre-existing developers.
At some time, it makes sense to move on to something better. This hypothetical system be written in a different language. It could be Rust, it could be C23, or something else entirely. But what it will definitely be is better structured. It would have clean, versioned APIs. And drivers will no doubt run in user space.
I agree. But likewise, Linux's design is far from the state of the art. Maybe there's value in letting it be what it always has been. It is not realistic to try to migrate millions of LoCs written with poor structure, to properly structured Rust, with barely any support from the pre-existing developers.
At some time, it makes sense to move on to something better. This hypothetical system be written in a different language. It could be Rust, it could be C23, or something else entirely. But what it will definitely be is better structured. It would have clean, versioned APIs. And drivers will no doubt run in user space.
I’ve become convinced that complex things in tech either ossify and get stuff built on top of them, or get Ship of Theseus’d. It’s extremely rare for an outright replacement to replace something overnight. Linux is not going to be suddenly replaced by something incompatible, but I’m concinved it will become more oxidized over time. If not under Linus, then under an initially fully compatible corporate fork which would ultimately become more popular. So there’s some pressure on Linus to move it along .
This is typically the case.
But the argument being made is that Linux is particularly ill suited for an in-place conversion to a better design.
But the argument being made is that Linux is particularly ill suited for an in-place conversion to a better design.
> The most prominent one is supportive of Rust.
"supportive" is not a binary.
Interpreting "no immediate objections" as "this is what I want" is specious and intellectually dishonest.
"supportive" is not a binary.
Interpreting "no immediate objections" as "this is what I want" is specious and intellectually dishonest.
Prefer C? Or haven’t used Rust?
All the Rust kernel developers have to know C. Is that true of the bigger maintainers who are arguing against it?
It’s fine to prefer one over the other.
If you haven’t learned one and refuse to try it and your argument boils down to “it’s not what we’ve done and I don’t want to change” that’s not good technical decision making.
All the Rust kernel developers have to know C. Is that true of the bigger maintainers who are arguing against it?
It’s fine to prefer one over the other.
If you haven’t learned one and refuse to try it and your argument boils down to “it’s not what we’ve done and I don’t want to change” that’s not good technical decision making.
> If you haven’t learned one and refuse to try it and your argument boils down to “it’s not what we’ve done and I don’t want to change” that’s not good technical decision making.
It sounds reasonable - you don't waltz into a large existing project $FOO and tell the existing maintainers "Here's our rules going forward - you shall learn $BAR!"
It's both rude and arrogant.
I have not seen any good arguments for why there should be an exception when $BAR == Linux and $FOO == Rust.
It sounds reasonable - you don't waltz into a large existing project $FOO and tell the existing maintainers "Here's our rules going forward - you shall learn $BAR!"
It's both rude and arrogant.
I have not seen any good arguments for why there should be an exception when $BAR == Linux and $FOO == Rust.
> starting a project out of grudge
How many times has this succeeded for fundamental work? I can think of a couple, including Xorg and GCC. I don't know if those cases can be characterized as "grudges" exactly but, in each instance, someone was unsatisfied with the prevailing conditions, forked the work, and then subsumed the legacy.
Doubtless there have been far more failures than successes. I also know that computing probabilities based on such ratios is simple minded: other factors exist and the equation is non-linear.
> But I think Linux isn't at that position yet.
I was disturbed by the irrational behavior I saw in this talk[1]. What the presenter sought was reasonable: communicate the intended semantics so Rust kernel developers can track the evolution. This was somehow understood as: forego C and become a Rust programmer, take responsibility for our work and maintain it going forward.
No, I do not believe the former implies the latter. There are endless silos, both formal and informal, for both good reasons and bad reasons, permeating all intellectual work, whether "open" or not. It is entirely reasonable to imagine that this pattern can be employed among reasonable people for the matter at hand, especially for something like file systems, where even parallel implementations can and do exist simultaneously.
If this behavior is prevalent enough then drastic steps are justified. Whether they're feasible or not is another matter, and also a many factored, non-linear calculation. One could, for instance, reimplement only some fraction of the kernel and be entirely sufficient for most of the backend cloud server Linux use case. Given the enthusiasm for Rust among major cloud operators, I suspect this might be attempted at some point.
[1] https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=1566s
How many times has this succeeded for fundamental work? I can think of a couple, including Xorg and GCC. I don't know if those cases can be characterized as "grudges" exactly but, in each instance, someone was unsatisfied with the prevailing conditions, forked the work, and then subsumed the legacy.
Doubtless there have been far more failures than successes. I also know that computing probabilities based on such ratios is simple minded: other factors exist and the equation is non-linear.
> But I think Linux isn't at that position yet.
I was disturbed by the irrational behavior I saw in this talk[1]. What the presenter sought was reasonable: communicate the intended semantics so Rust kernel developers can track the evolution. This was somehow understood as: forego C and become a Rust programmer, take responsibility for our work and maintain it going forward.
No, I do not believe the former implies the latter. There are endless silos, both formal and informal, for both good reasons and bad reasons, permeating all intellectual work, whether "open" or not. It is entirely reasonable to imagine that this pattern can be employed among reasonable people for the matter at hand, especially for something like file systems, where even parallel implementations can and do exist simultaneously.
If this behavior is prevalent enough then drastic steps are justified. Whether they're feasible or not is another matter, and also a many factored, non-linear calculation. One could, for instance, reimplement only some fraction of the kernel and be entirely sufficient for most of the backend cloud server Linux use case. Given the enthusiasm for Rust among major cloud operators, I suspect this might be attempted at some point.
[1] https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=1566s
> This was somehow understood as: forego C and become a Rust programmer, take responsibility for our work and maintain it going forward.
That's how I understand it as well: the proposal is
1. Put our Rust code in.
2. If you make C-code changes that break Rust-code, then your change can't be merged until you either learn Rust or wait on us to fix things.
This is an unreasonable expectation, IMO, for any large project written in any language; this is not specific to Linux and C and Rust.
Making unreasonable proposals in civil language does not magically turn that proposal into a reasonable one.
Pressing forward in spite of feedback that the proposal is unreasonable is uncivil, even if you spread a thin veneer of civility over it.
That's how I understand it as well: the proposal is
1. Put our Rust code in.
2. If you make C-code changes that break Rust-code, then your change can't be merged until you either learn Rust or wait on us to fix things.
This is an unreasonable expectation, IMO, for any large project written in any language; this is not specific to Linux and C and Rust.
Making unreasonable proposals in civil language does not magically turn that proposal into a reasonable one.
Pressing forward in spite of feedback that the proposal is unreasonable is uncivil, even if you spread a thin veneer of civility over it.
I'll stipulate your view of the proposal. Does an uncivil, almost hysterical reaction that is perceived (wrongly or otherwise) as elitist and irrationally hostile an effective approach?
I question the basis of the reaction and find it wanting. Paraphrasing now; "50 filesystems won't be instantaneously converted to Rust." Has anyone, anywhere suggested such a thing? I don't think anyone credible has done so. A strawman argument. The accusation of some forced conversion to a new "religion" is also baseless, not to mention insulting.
An aside: the get_or_create_inode method example that was left on the display for nearly half the presentation is very compelling to me. It is explicit and comprehensive: one can trivially comprehend both the likely implementation of the method and the obligations of the caller without reading a line of code beyond that declaration. Practically self documenting and vastly superior to conventional systems programming C. At one point a speaker likened this to Java, I suppose because of the type composition. That's ignorant and false: that signature conveys so much more value than what one sees in typical Java code that it's not in the same ballpark at all. The verbosity has actual value.
Rust is great; a legitimate advance in systems languages. People are compelled by it. That has produced some conflict. Not all conflict is avoidable or inherently evil. The C side of this conflict will need better tactics if they're not going to just devolve into irrationality with false claims, baseless accusations and hysteria.
I question the basis of the reaction and find it wanting. Paraphrasing now; "50 filesystems won't be instantaneously converted to Rust." Has anyone, anywhere suggested such a thing? I don't think anyone credible has done so. A strawman argument. The accusation of some forced conversion to a new "religion" is also baseless, not to mention insulting.
An aside: the get_or_create_inode method example that was left on the display for nearly half the presentation is very compelling to me. It is explicit and comprehensive: one can trivially comprehend both the likely implementation of the method and the obligations of the caller without reading a line of code beyond that declaration. Practically self documenting and vastly superior to conventional systems programming C. At one point a speaker likened this to Java, I suppose because of the type composition. That's ignorant and false: that signature conveys so much more value than what one sees in typical Java code that it's not in the same ballpark at all. The verbosity has actual value.
Rust is great; a legitimate advance in systems languages. People are compelled by it. That has produced some conflict. Not all conflict is avoidable or inherently evil. The C side of this conflict will need better tactics if they're not going to just devolve into irrationality with false claims, baseless accusations and hysteria.
> The C side of this conflict will need better tactics if they're not going to just devolve into irrationality with false claims, baseless accusations and hysteria.
The "C side" don't need any tactics; it's their project and they are free to refuse entry of some other group of people.
They can make false claims, they can make baseless accusations and they can throw hysterics all they want to, because they don't have to convince anyone of anything.
The Rust side has to provide an argument more compelling than "you are using inferior tools", because even if true, it's irrelevant!
The Rust side has to do the convincing here, not the C side.
The "C side" don't need any tactics; it's their project and they are free to refuse entry of some other group of people.
They can make false claims, they can make baseless accusations and they can throw hysterics all they want to, because they don't have to convince anyone of anything.
The Rust side has to provide an argument more compelling than "you are using inferior tools", because even if true, it's irrelevant!
The Rust side has to do the convincing here, not the C side.
This response goes back to my first post on the topic: such attitudes and behavior have led to successful forks of major components that ultimately subsumed the legacy work. The Rust side doesn't even have to go so far as to reimplement the entire kernel, as Drew proposes. They can simply fork, govern the evolution of the fork as they see fit, including whatever adaptations they prefer to accommodate Rust and, should their efforts yield a compelling product (to one or all of Google, AWS, Microsoft, et al.,) they can win.
Also, I don't believe the "C side" has the degree of control over all of this that you hypothesize. The Linux BDFL isn't anti-Rust, and he is endowed with an inhuman degree of wisdom that will likely prevent him from supporting irrational C dogmatists. The acceptance by Linus of a possible future that included Rust in the kernel is the start of all of this. He knew then the road would not be trouble free; he's been calling these kinds of shots for too long to imagine otherwise.
Also, I don't believe the "C side" has the degree of control over all of this that you hypothesize. The Linux BDFL isn't anti-Rust, and he is endowed with an inhuman degree of wisdom that will likely prevent him from supporting irrational C dogmatists. The acceptance by Linus of a possible future that included Rust in the kernel is the start of all of this. He knew then the road would not be trouble free; he's been calling these kinds of shots for too long to imagine otherwise.
> Here’s the pitch: a motivated group of talented Rust OS developers could build a Linux-compatible kernel, from scratch, very quickly, with no need to engage in LKML politics.
Rust is many things, but devoid of political drama it is not.
Rust is many things, but devoid of political drama it is not.
I also disagree with his take that the Kernel could be replicated in Rust by 6 motivated volunteers in 4 or 5 years. That could be said of many projects, you could probably reproduce AAA entertainment software with such a team in such a span of time, but the trick is getting these people to stay on track, fed, and satisfied for that time. Who's going to pay for their rent/mortgages? Are they just not going to work for the duration?
It's naive grandstanding in the best of cases, and malicious proselytizing in the worst.
It's naive grandstanding in the best of cases, and malicious proselytizing in the worst.
You are misrepresenting the original article. Drew did not say six volunteers or 4 or 5 years. Those are your numbers, so feel free to agree to disagree with yourself.
A Linux kernel clone is the epitome of a large Rust community project (for many reasons, some noble, some not so noble). It would likely pull in hundreds if not thousands of developers in an arms race assuming the end goal is well defined.
Nobody claims it's a small task, but I believe it can probably be accomplished. Particularily in the scope of the original claim "applied to a new Linux-compatible OS we could have something production ready for some use-cases within a few years."
A Linux kernel clone is the epitome of a large Rust community project (for many reasons, some noble, some not so noble). It would likely pull in hundreds if not thousands of developers in an arms race assuming the end goal is well defined.
Nobody claims it's a small task, but I believe it can probably be accomplished. Particularily in the scope of the original claim "applied to a new Linux-compatible OS we could have something production ready for some use-cases within a few years."
I disagree. There is already a project for a Rust OS, and its been around for years and I don't even know if it'll actually function on real hardware or just in VMs.[1] Or even enough to matter. There is already a project, where are the hundreds, if not thousands, of developers?
[1] https://www.redox-os.org/
[1] https://www.redox-os.org/
But that's not the project discussed in the article.
Now I'm not saying I agree, but his premise was a Linux-compatible kernel, which Redox most definitely is not. Redox explicitly does not intend to be POSIX, has its own custom (in-house) filesystem, desktop environment, and is a microkernel as well. A better comparison for Redox would be Hurd, not Linux.
Now I'm not saying I agree, but his premise was a Linux-compatible kernel, which Redox most definitely is not. Redox explicitly does not intend to be POSIX, has its own custom (in-house) filesystem, desktop environment, and is a microkernel as well. A better comparison for Redox would be Hurd, not Linux.
This isn't a Linux kernel clone. The whole argument is that producing a "bug-by-bug compatible" Linux kernel clone should be much easier to pull off than a "research kernel" where you may get lost in exploring design dead ends.
Much easier? How much easier?
Or, the original claim: How small a team, and how quickly?
Sure, the Linux development process was inefficient. There were a lot of false directions and dead ends and things that were OK ideas but were superseded by better ideas. If you know the destination, you can drive straight there without all the wandering around.
But, fine, how inefficient was the development process? 90% wandering around? It probably wasn't 99% wandering around. So you're going to need something like 10% of the man-hours that went into Linux.
You say Rust is more productive? How much more? Maybe a factor of two? OK, you need 5% of the man-hours that went into Linux, over the last 20+ years.
That's still not a small team and quickly, no matter how you slice it.
Or, the original claim: How small a team, and how quickly?
Sure, the Linux development process was inefficient. There were a lot of false directions and dead ends and things that were OK ideas but were superseded by better ideas. If you know the destination, you can drive straight there without all the wandering around.
But, fine, how inefficient was the development process? 90% wandering around? It probably wasn't 99% wandering around. So you're going to need something like 10% of the man-hours that went into Linux.
You say Rust is more productive? How much more? Maybe a factor of two? OK, you need 5% of the man-hours that went into Linux, over the last 20+ years.
That's still not a small team and quickly, no matter how you slice it.
[deleted]
https://imgur.com/a/FcmNLbx
I’m extrapolating but this would hardly be “misrepresenting” his position.
I’m extrapolating but this would hardly be “misrepresenting” his position.
I don't know where that image is from, so you are working on information I do not have, but still, the first "paragraph" sets up a possibility reinforced by an anecdote in the second. It's not asserting that only 5 contributors could re-create the entirety of the Linux kernel in 5 years.
I very much took his argument as an open-source project with a small number of leaders, with the help of the larger OSS community, could make a mature subset of a Linux compatible kernel in a small amount of time (years)
I very much took his argument as an open-source project with a small number of leaders, with the help of the larger OSS community, could make a mature subset of a Linux compatible kernel in a small amount of time (years)
nanos wrote a quite reasonable subset of the linux kernel in C using less than 5 people at a time on average and somewhat less than 5 years. this can totally be done.
https://github.com/nanovms/nanos
> A kernel designed to run one and only one application in a virtualized environment
> 1. Security:
> Nanos aims to be a much more secure system than Linux. It does this through several thrusts. Not having the notion of users, running a single process per vm, and limiting the amount of code that is incorporated into each vm.
> 2. Minimalist:
> KISS. As Nanos is not intended to be ran on bare metal we strive to keep the core as simple as possible.
They patched erlang and gpu-nvidia to work with Nanos
A product based on Nanos
> A kernel designed to run one and only one application in a virtualized environment
---
> 1. Security:
> Nanos aims to be a much more secure system than Linux. It does this through several thrusts. Not having the notion of users, running a single process per vm, and limiting the amount of code that is incorporated into each vm.
> 2. Minimalist:
> KISS. As Nanos is not intended to be ran on bare metal we strive to keep the core as simple as possible.
---
They patched erlang and gpu-nvidia to work with Nanos
---
A product based on Nanos
How do the current Rust/Linux maintainers pay their rent?
Also your estimate for AAA game development is massively optimistic, AAA team sizes are in the hundreds today (not a good thing for quality and innovation, but that's how it is).
Also your estimate for AAA game development is massively optimistic, AAA team sizes are in the hundreds today (not a good thing for quality and innovation, but that's how it is).
>Who's going to pay for their rent/mortgages?
The very loud Rust (and toxic) community could maybe spare in average 1$ a month each.
The very loud Rust (and toxic) community could maybe spare in average 1$ a month each.
Out of curiosity since I don't follow Rust.
Why/how are they toxic?
Not trying to stoke arguments, just a curious person thinking of learning Rust for fun with less drama.
I find Golang community to be less opinionated and just chill + pumping out libs, modules, projects, products. I like that. Less drama more output.
But I'm curious to pick on Rust as well.
Why/how are they toxic?
Not trying to stoke arguments, just a curious person thinking of learning Rust for fun with less drama.
I find Golang community to be less opinionated and just chill + pumping out libs, modules, projects, products. I like that. Less drama more output.
But I'm curious to pick on Rust as well.
I can't give you a complete answer, just partial stuff I am aware of
1. Rust fanboys poping up on other language topic and shitting on them and promoting Rust
2. Ton of low quality promotion, like you would get on HN crap like TODO application in Rust, and the Rust community would just upvote and promote this low level 1 weekend project just because is Rust
3. RIR rewrite it in rust, toxic Rust fans would popup on projects communities and demand to rewrite it in rust, or why is this not in Rust, or I would contribute if you rewrite it in rust.
I am not sure why this community is so toxic, I do not remember Java,Python to have this problem, I suspect that the toxic part is just a portion of the developers but the community of Rust devs did not manage to keep the toxic people in check.
But honestly use a search engine and find more details, I am avoiding this kind of drama so I do not have latest gossip (just seen a link here that the ladybird devs also called the Rust people toxic , probably because they are butt hurt that people are making a more popular browser then their Rust alternatives)
1. Rust fanboys poping up on other language topic and shitting on them and promoting Rust
2. Ton of low quality promotion, like you would get on HN crap like TODO application in Rust, and the Rust community would just upvote and promote this low level 1 weekend project just because is Rust
3. RIR rewrite it in rust, toxic Rust fans would popup on projects communities and demand to rewrite it in rust, or why is this not in Rust, or I would contribute if you rewrite it in rust.
I am not sure why this community is so toxic, I do not remember Java,Python to have this problem, I suspect that the toxic part is just a portion of the developers but the community of Rust devs did not manage to keep the toxic people in check.
But honestly use a search engine and find more details, I am avoiding this kind of drama so I do not have latest gossip (just seen a link here that the ladybird devs also called the Rust people toxic , probably because they are butt hurt that people are making a more popular browser then their Rust alternatives)
I can’t understand how you missed the modifier “LKML” for Christ’s sake.
It does not make sense to conflate LKMl politics with politics.
Does your comment contains anything more than drama?
Someone in a sub-thread accused Drew of being incurious, and reading the article, I kind of agree.
It's a very polite, high-effort, superficially humble piece of writing, that nevertheless boils down to "You guys should probably leave us alone and work on stuff we don't need to worry about".
Now, working on a Linux fork as a "proof of value" thing could be interesting, but it also means that this hypothetical Linust project would be stuck forever chasing Linux's API decisions without any power to influence them (and, if recent history is any indication, quite a lot of hostility from OG Linux maintainers).
I can't help but notice that Drew's plan doesn't include any exit strategy, any point where the projects merge or Linux starts taking components from Linust or something.
Maybe Drew thinks that forever being stuck between shadowing a concurrent project's API and trying to convince its billion users to switch to yours instead is an attractive prospect. If I was a Rust-on-Linux developer, I'd find that patronizing.
It's a very polite, high-effort, superficially humble piece of writing, that nevertheless boils down to "You guys should probably leave us alone and work on stuff we don't need to worry about".
Now, working on a Linux fork as a "proof of value" thing could be interesting, but it also means that this hypothetical Linust project would be stuck forever chasing Linux's API decisions without any power to influence them (and, if recent history is any indication, quite a lot of hostility from OG Linux maintainers).
I can't help but notice that Drew's plan doesn't include any exit strategy, any point where the projects merge or Linux starts taking components from Linust or something.
Maybe Drew thinks that forever being stuck between shadowing a concurrent project's API and trying to convince its billion users to switch to yours instead is an attractive prospect. If I was a Rust-on-Linux developer, I'd find that patronizing.
This is all predicated on the assumption that forks and clones are a bad thing, when they're actually a good thing? Why does a clone require an 'exit strategy'? Does Linux have an exit strategy for merging back into Unix? When is Unix getting merged back into Multics, for that matter?
If Rustnux is safer and more performant in the ways that Rust advocates believe it will be, Rustnux will gain traction and demonstrate the potential benefits of Rust in the kernel itself. This is intrinsically beneficial, and I'm unsure how convincing a billion people to switch has anything to do with it. (Also, bizarre metric, does every purcharser-of-an-IoT-thermostat get their choice of OS kernels?)
For those who don't want to work on "shadowing" an API (quick, someone tell the Wine crowd they're 'stuck' and should stop), Redox already exists. He's not saying to shut it down. For a bunch of people accusing Drew of being incurious, these some some aggressively 'in the box' takes.
If Rustnux is safer and more performant in the ways that Rust advocates believe it will be, Rustnux will gain traction and demonstrate the potential benefits of Rust in the kernel itself. This is intrinsically beneficial, and I'm unsure how convincing a billion people to switch has anything to do with it. (Also, bizarre metric, does every purcharser-of-an-IoT-thermostat get their choice of OS kernels?)
For those who don't want to work on "shadowing" an API (quick, someone tell the Wine crowd they're 'stuck' and should stop), Redox already exists. He's not saying to shut it down. For a bunch of people accusing Drew of being incurious, these some some aggressively 'in the box' takes.
> This is all predicated on the assumption that forks and clones are a bad thing
No, they are just different things. Linux is what people use. It would be nice to be able to write drivers in Rust for the system people use. It's a very practical desire to want to use Rust and Linux to build real systems, right now, instead of having to choose one or the other.
No, they are just different things. Linux is what people use. It would be nice to be able to write drivers in Rust for the system people use. It's a very practical desire to want to use Rust and Linux to build real systems, right now, instead of having to choose one or the other.
You can write out of tree Rust drivers for Linux right now, no one is stopping you. Your 'very practical desire' to write Rust drivers doesn't extend into some sort of corresponding obligation for anyone else to accept that code into their projects. Your desires do not take priority over the desires and needs of others.
I feel like people have some kind of misconception about what OSS requires on this point. The only meaningful right you have with the Linux kernel, that you don't have with - say - NT, is the right to fork it. Nothing about the idea of OSS extends you a right to merge whatever code you want into someone else's project to satisfy your 'very practical desires'.
You could have the best idea of all time, and you still wouldn't have any rights to anyone else's OSS project other than forking it.
Drew is offering a practical solution here to all sides, and all that the critics can come up with is 'yeah but wouldn't it be nicer if they got along?' (it would! they don't) or 'I hate Drew', which is an irrelevant and cruel thing to say about an actual human being.
I feel like people have some kind of misconception about what OSS requires on this point. The only meaningful right you have with the Linux kernel, that you don't have with - say - NT, is the right to fork it. Nothing about the idea of OSS extends you a right to merge whatever code you want into someone else's project to satisfy your 'very practical desires'.
You could have the best idea of all time, and you still wouldn't have any rights to anyone else's OSS project other than forking it.
Drew is offering a practical solution here to all sides, and all that the critics can come up with is 'yeah but wouldn't it be nicer if they got along?' (it would! they don't) or 'I hate Drew', which is an irrelevant and cruel thing to say about an actual human being.
> Your 'very practical desire' to write Rust drivers doesn't extend into some sort of corresponding obligation for anyone else to accept that code into their projects.
Never said it did?
> Nothing about the idea of OSS extends you a right to merge whatever code you want into someone else's project to satisfy your 'very practical desires'.
And that's why I called them practical desires, not rights?
> Drew is offering a practical solution here to all sides, and all that the critics can come up with is 'yeah but wouldn't it be nicer if they got along?' (it would! they don't)
My solution is to first deal with the bad behavior or it will fester. Then I'd ask if the technical reasons for including Rust in the Linux kernel still hold. I think they do. So I'm not sure we've reached the point where a fork makes sense yet. If we forked every time a kernel dev acted nasty to others, we'd have far too many forks.
> or 'I hate Drew', which is an irrelevant and cruel thing to say about an actual human being.
That's not what I said. If anyone is curious, I address this elsewhere: https://news.ycombinator.com/item?id=41407312
As to why I really can't stand Drew's writing, which is at the core of this complaint, one might see: https://news.ycombinator.com/item?id=41404644
Never said it did?
> Nothing about the idea of OSS extends you a right to merge whatever code you want into someone else's project to satisfy your 'very practical desires'.
And that's why I called them practical desires, not rights?
> Drew is offering a practical solution here to all sides, and all that the critics can come up with is 'yeah but wouldn't it be nicer if they got along?' (it would! they don't)
My solution is to first deal with the bad behavior or it will fester. Then I'd ask if the technical reasons for including Rust in the Linux kernel still hold. I think they do. So I'm not sure we've reached the point where a fork makes sense yet. If we forked every time a kernel dev acted nasty to others, we'd have far too many forks.
> or 'I hate Drew', which is an irrelevant and cruel thing to say about an actual human being.
That's not what I said. If anyone is curious, I address this elsewhere: https://news.ycombinator.com/item?id=41407312
As to why I really can't stand Drew's writing, which is at the core of this complaint, one might see: https://news.ycombinator.com/item?id=41404644
Just seems pragmatic to me. The people contributing rust to Linux are having a bad time so the advice is to do something else. It sucks I guess but it's no different than any other open source project.
"Fine I'll do it myself" is a story basically as old as time.
"Fine I'll do it myself" is a story basically as old as time.
- Walt Disney
- Ferruccio Lamborghini
- King Henry VIII
- Juan Pujol Garcia
The elephant in the room, which is Linux's untenable complexity and lack of internal APIs (even for drivers), is not addressed.
Doing anything on Linux is many times as hard as doing it on systems that put a strong emphasis on structure. This is true not just for microkernel, multiserver systems, but for other unices/unix-like such as the BSDs (which put far more emphasis on structure relative to Linux) as well.
Thus, I agree entirely with Drew, with extra reasons, that we (developers in general, not specific to rust) should try and put some effort elsewhere than Linux.
It doesn't have to be Linux. It isn't the end-all in operating systems. It's just what happens to be most popular -right now-.
Doing anything on Linux is many times as hard as doing it on systems that put a strong emphasis on structure. This is true not just for microkernel, multiserver systems, but for other unices/unix-like such as the BSDs (which put far more emphasis on structure relative to Linux) as well.
Thus, I agree entirely with Drew, with extra reasons, that we (developers in general, not specific to rust) should try and put some effort elsewhere than Linux.
It doesn't have to be Linux. It isn't the end-all in operating systems. It's just what happens to be most popular -right now-.
> It doesn't have to be Linux. It isn't the end-all in operating systems. It's just what happens to be most popular -right now-.
That's part of the problem, I think: The Rust group's primary goal appears to be increasing the usage and/or popularity of Rust, not merely hardening the Linux kernel.
After all, both the kernel devs as well as the Rust group knows very well that the Rust group can maintain out-of-tree patches indefinitely. By using a out-of-tree patch, they can slowly and effectively replace almost all of the drivers. Many people actually would download and use their `rustix` kernel.
But that doesn't allow them to achieve their goal of spreading the movement, hence their resistance to prove the concept out-of-tree and the pushback from kernel devs who see this purely as a popularity-booster.
There is no good argument against the Rust group proceeding with an out-of-tree development effort; all the ones around "it's too much work" reflects the fear that the kernel devs have that the Rust group would push Rush code into the kernel, and then not be around to maintain it when some C code breaks it.
That's part of the problem, I think: The Rust group's primary goal appears to be increasing the usage and/or popularity of Rust, not merely hardening the Linux kernel.
After all, both the kernel devs as well as the Rust group knows very well that the Rust group can maintain out-of-tree patches indefinitely. By using a out-of-tree patch, they can slowly and effectively replace almost all of the drivers. Many people actually would download and use their `rustix` kernel.
But that doesn't allow them to achieve their goal of spreading the movement, hence their resistance to prove the concept out-of-tree and the pushback from kernel devs who see this purely as a popularity-booster.
There is no good argument against the Rust group proceeding with an out-of-tree development effort; all the ones around "it's too much work" reflects the fear that the kernel devs have that the Rust group would push Rush code into the kernel, and then not be around to maintain it when some C code breaks it.
Lack of internal APIs is not the problem if the putative Rust-based clone doesn't aim to be compatible with Linux drivers written in C, which I think it wouldn't. The real problem is that the Linux kernel ABis to user-land are merely stable, myriad, and poorly documented. Keeping up is not possible. The BSDs, Solaris/Illumos, and Windows all had at least one (sometimes two) attempts at being compatible with Linux kernel user-land ABIs, and all failed due to being woefully incomplete in spite of being huge.
> "Fine I'll do it myself" is a story basically as old as time.
Heavy survivorship bias in your list of examples, though.
Heavy survivorship bias in your list of examples, though.
But that's the point. With Rust's absolute superiority in all dimensions and highly motivated developers why wouldn't Rust OS project succeed?
That kind of sarcasm and putting words in other people's mouths is why people are burning out of the project.
And maybe you're right and some Rust-based UNIX will eventually outcompete Linux.
But it'll be a shame if it gets that far because one community distrusted the other so much it would rather sneer at them like you're doing and tell them to piss off than put any investment whatsoever into sharing work.
And maybe you're right and some Rust-based UNIX will eventually outcompete Linux.
But it'll be a shame if it gets that far because one community distrusted the other so much it would rather sneer at them like you're doing and tell them to piss off than put any investment whatsoever into sharing work.
[deleted]
There's an attempt at heading off the driver commentary with a note at the bottom. The note makes some hand-wavy thing which is essentially "assuming you have an army of people with specifications, this won't be a problem", but that assumption is crazy. Fuchsia is a real OS effort funded by Google (at varying levels of funding through it's lifecycle, which is still ongoing AIUI) - I worked on Fuchsia for a number of years, from approximately the time the kernel had become usable, through to launching on nest hubs where it replaced the existing OS from bootloader to GUI. Drivers remain a huge problem for Fuchsia despite funding. SOC vendors don't provide docs or assistance, even for serious efforts, they demand money first - they're literally holding the software world hostage with this activity.
After leaving Fuchsia and reflecting on the whole thing, I really believe that despite the many problems Linux may have over the long term, the only viable strategies for a broadly and practically usable safer OSS system are either: someone donates at least $2bn to get it done in a PBC well managed and focused over a decade, or you slowly mutate Linux into that thing. Anything drastically different is, for now, dependent on the world being a different shape / people & companies having different attitudes than they do. I'd love to see something like Redox prove me wrong, but real world experience suggests that's extremely unlikely. RISCv is the one thing in flight that might "change the shape of the world" enough to alter this, but RISCv comes with it's own stack of long challenges to overcome to meet the originally stated goal of broadly and practically usable.
Writing a core kernel and usable userspace isn't something to be trivialized, but by cost and work volume, getting a broad set of drivers is far far more effort by an astronomical margin. The OSDev community has thousands of OSes to explore, and barely any drivers that do things a user actually cares about.
After leaving Fuchsia and reflecting on the whole thing, I really believe that despite the many problems Linux may have over the long term, the only viable strategies for a broadly and practically usable safer OSS system are either: someone donates at least $2bn to get it done in a PBC well managed and focused over a decade, or you slowly mutate Linux into that thing. Anything drastically different is, for now, dependent on the world being a different shape / people & companies having different attitudes than they do. I'd love to see something like Redox prove me wrong, but real world experience suggests that's extremely unlikely. RISCv is the one thing in flight that might "change the shape of the world" enough to alter this, but RISCv comes with it's own stack of long challenges to overcome to meet the originally stated goal of broadly and practically usable.
Writing a core kernel and usable userspace isn't something to be trivialized, but by cost and work volume, getting a broad set of drivers is far far more effort by an astronomical margin. The OSDev community has thousands of OSes to explore, and barely any drivers that do things a user actually cares about.
> SOC vendors don't provide docs or assistance, even for serious efforts, they demand money first - they're literally holding the software world hostage with this activity.
This 1000%
Working with 9front has showed me that open source hardware mostly isn't. You're at the mercy of the hardware manufacturer who's only commitment to open source is a Linux kernel full of drivers you can't understand because there's no hardware manual. That or you have to be a corporate partner with licensees and NDA's which is a massive wall for little projects and individuals. This effects EVERY alternative OS to Linux.
It's disheartening as it makes Linux into an unwitting tyrant as you have no choice but to use Linux or find another SoC that provides docs.
This 1000%
Working with 9front has showed me that open source hardware mostly isn't. You're at the mercy of the hardware manufacturer who's only commitment to open source is a Linux kernel full of drivers you can't understand because there's no hardware manual. That or you have to be a corporate partner with licensees and NDA's which is a massive wall for little projects and individuals. This effects EVERY alternative OS to Linux.
It's disheartening as it makes Linux into an unwitting tyrant as you have no choice but to use Linux or find another SoC that provides docs.
> RISCv is the one thing in flight that might "change the shape of the world" enough to alter this
I don't think RISC-V will have any effect on the availability of drivers.
I don't think RISC-V will have any effect on the availability of drivers.
If its compatible with linux then the linux drivers would work right?
In short: no. Longer: there is no such thing as "compatible with linux drivers", linux has a stable(ish, very ish) userspace, but it does not have a stable HAL or driver interface. All of the drivers in Linux are GPLv2, so there are constraints to "borrowing code" from them, if you'd even be able to or want to. On the "ish" about userspace, while the POSIX layer is pretty stable, the deeper you go beyond that, extended ioctls for devices and drivers, some stuff on netlink, etc are not nearly as stable as the "never break userspace" mantra would imply too. Running a subset of userspace software is pretty easy (https://fuchsia.dev/fuchsia-src/concepts/components/v2/starn..., https://docs.freebsd.org/en/books/handbook/linuxemu/) but getting much further is extremely hard, and probably practically untenable as Linux is a moving target, even in userspace.
In such a scenario, it makes sense to design a sane, stable driver api and move drivers to it.
Such drivers would require little maintenance and would thus be much more valuable than Linux drivers.
They would be free from the insanity that is Linux's lack of internal structure.
Such drivers would require little maintenance and would thus be much more valuable than Linux drivers.
They would be free from the insanity that is Linux's lack of internal structure.
androids gki is an effort in this direction which has been helping android vendors. it has a number of limitations which would need to be overcome for broader use, but definitely is a good direction https://source.android.com/docs/core/architecture/kernel/gen...
It would indeed make a lot of sense for Google to untie drivers from Linux, as it could mean drivers that work on both Linux and Fuchsia.
I don't know if KMI is all that stable (I've not looked), and some of the modules may be constrained to that symbol space (though not boot drivers), and they'll still likely encode assumptions about being in a single kernel address space. KMI alone is likely not sufficient for driver portability. The original goal of this Android project was to reduce the cost and complexity of vendor forks, which as explained in the docs, sometimes ran up to 50% patched code prior to GKI.
The Linux kernel is stable for userspace only. Drivers are expected to be merged up stream and the internal APIs change often with no regard for out of tree drivers. So there's no interface that an alternative kernel can target that doesn't constantly change.
Pretty sure you could just choose an LTS abi and use that.
> Two years ago, seeing the Rust-for-Linux project starting to get the ball rolling, I wrote “Does Rust belong in the Linux kernel?”, penning a conclusion consistent with Betteridge’s law of headlines. Two years on we have a lot of experience to draw on to see how Rust-for-Linux is actually playing out, and I’d like to renew my thoughts with some hindsight – and more compassion. If you’re one of the Rust-for-Linux participants burned out or burning out on this project, I want to help. Burnout sucks – I’ve been there.
Now this compassionate revisit offers the same conclusion: don’t do Rust in Linux.
This person read that email about one Rust Kernel developer resigning because of burnout. Now he goes into how the Linux project is a “burnout machine” and how his heart goes out to the “developers who have been burned” (how did we get to plural?).[1]
The “so where do we go now?” almost gets ahead of itself before it says in the next paragraph that “the path is theirs to choose”. Well yeah because the only person who implied there was a crossroads is the author here.
The predictable conclusion is to abandon the project and do something adjacent to the Linux Kernel. But what if you cared about working on the Linux Kernel specifically? What if you cared about the code in the Linux Kernel itself, its long term health… hush, hush now. You are burned out and don’t know what you are saying.
The penultimate paragraph then declares that the Rust-for-Linux project itself is “burned out” (“and that’s awful”).
Who needs enemies with compassionate friends like this.
[1] How often do we read about maintainers suffering burnout? Every day? Do we then declare that the project is a failure, even when there are other maintainers left on the project?
Now this compassionate revisit offers the same conclusion: don’t do Rust in Linux.
This person read that email about one Rust Kernel developer resigning because of burnout. Now he goes into how the Linux project is a “burnout machine” and how his heart goes out to the “developers who have been burned” (how did we get to plural?).[1]
The “so where do we go now?” almost gets ahead of itself before it says in the next paragraph that “the path is theirs to choose”. Well yeah because the only person who implied there was a crossroads is the author here.
The predictable conclusion is to abandon the project and do something adjacent to the Linux Kernel. But what if you cared about working on the Linux Kernel specifically? What if you cared about the code in the Linux Kernel itself, its long term health… hush, hush now. You are burned out and don’t know what you are saying.
The penultimate paragraph then declares that the Rust-for-Linux project itself is “burned out” (“and that’s awful”).
Who needs enemies with compassionate friends like this.
[1] How often do we read about maintainers suffering burnout? Every day? Do we then declare that the project is a failure, even when there are other maintainers left on the project?
>This person read that email about one Rust Kernel developer resigning because of burnout. Now he goes into how the Linux project is a “burnout machine” and how his heart goes out to the “developers who have been burned” (how did we get to plural?).[1]
There are several Rust-for-Linux folks who have complained about the same things and been at various levels of burned out over the course of the project. Ignoring them because it raises uncomfortable questions regarding the viability of the project doesn't make it go away, it just erases their experiences.
>The “so where do we go now?” almost gets ahead of itself before it says in the next paragraph that “the path is theirs to choose”. Well yeah because the only person who implied there was a crossroads is the author here.
>The predictable conclusion is to abandon the project and do something adjacent to the Linux Kernel.
This article is in response to someone who already decided to abandon the project, and to suggest what's next. I didn't impose the conclusion to abandon it on anyone, and in fact I explicitly supported it if burnout victims choose to return to the fold.
Yes, I stand by the conclusion that Rust-for-Linux is probably not a great idea, and I'm allowed to say that without being anyone's "enemy". I also believe people when they say they're burned out and quitting the project and take their needs seriously, something I think is missing from your comment. All of this is compatible with compassion. I'm not and have never been your enemy: I can say that I think it's not a good idea and wish you well in your efforts nevertheless, and I have.
There are several Rust-for-Linux folks who have complained about the same things and been at various levels of burned out over the course of the project. Ignoring them because it raises uncomfortable questions regarding the viability of the project doesn't make it go away, it just erases their experiences.
>The “so where do we go now?” almost gets ahead of itself before it says in the next paragraph that “the path is theirs to choose”. Well yeah because the only person who implied there was a crossroads is the author here.
>The predictable conclusion is to abandon the project and do something adjacent to the Linux Kernel.
This article is in response to someone who already decided to abandon the project, and to suggest what's next. I didn't impose the conclusion to abandon it on anyone, and in fact I explicitly supported it if burnout victims choose to return to the fold.
Yes, I stand by the conclusion that Rust-for-Linux is probably not a great idea, and I'm allowed to say that without being anyone's "enemy". I also believe people when they say they're burned out and quitting the project and take their needs seriously, something I think is missing from your comment. All of this is compatible with compassion. I'm not and have never been your enemy: I can say that I think it's not a good idea and wish you well in your efforts nevertheless, and I have.
keybored(2)
> resigning because of burnout
It wasn't even because of burnout.
It wasn't even because of burnout.
I must be missing something obvious. I’m not aware of any too-frequent toxicity in the rust lkml.
Just seems like a lot of groundwork is still being laid, without much even having been started.
The article makes it seem like things are actually being rewritten in rust.
Are they?
Just seems like a lot of groundwork is still being laid, without much even having been started.
The article makes it seem like things are actually being rewritten in rust.
Are they?
Oh, I started searching for some hot topics in the rust lkml, ouch, there’s some bad stuff in there.
Linus seems reasonable in comparison…
Linus seems reasonable in comparison…
I’m not sure it’s rewritten so much as new code. But it sounds like it’s been taking a very long time (year+) to merge even smaller things like structure definitions Rust developers need, let alone “real” code.
The reddit threads on this subject were interesting, with hundreds of comments in r/programming and r/linux all bemoaning the lack of Rust adoption in the kernel and the supposed mistreatment of the Rust maintainers.
Are these people just users with a strong opinion about what language their kernel uses? If they're Rust devs and reflective of how popular Rust is in general, where are all of the killer apps written in Rust? Why do Rust programmers seem to mostly just produce partial, feature-incomplete rewrites of Unix utilities?
Are these people just users with a strong opinion about what language their kernel uses? If they're Rust devs and reflective of how popular Rust is in general, where are all of the killer apps written in Rust? Why do Rust programmers seem to mostly just produce partial, feature-incomplete rewrites of Unix utilities?
> where are all of the killer apps written in Rust
What like Ripgrep, uv, Rustdesk, Deno, Tauri, Zed, Meillisearch, Typst, Nushell...?
What like Ripgrep, uv, Rustdesk, Deno, Tauri, Zed, Meillisearch, Typst, Nushell...?
Your parent is also ignoring the tens of millions of lines of Rust that powers products from many massive tech companies, like Amazon, Meta, Cloudflare, and many many more.
with respect to linux kernel rust projects: is there anything new that is complete and ready to use or anything that existed in c that has been rewritten that is stable and has reached feature parity with the existing implementations yet?
The android binder driver is feature complete [1] and is apparently going to ship in android soon (I assume Kees is a reasonably reliable source for this as a google employee... certainly work has been ongoing related to upstreaming this) [2]
[1] https://lore.kernel.org/rust-for-linux/20231101-rust-binder-...
[2] https://lwn.net/Articles/987949/
[1] https://lore.kernel.org/rust-for-linux/20231101-rust-binder-...
[2] https://lwn.net/Articles/987949/
this is pretty cool, the binder always struck me as a potential security risk!
also pretty fun to think that beos will continue to secretly live on deep in the guts of linux powering android, even in rust!
also pretty fun to think that beos will continue to secretly live on deep in the guts of linux powering android, even in rust!
The gpu driver for apple m1 silicon is written is Rust.
And have never had a single crash from their code. And wrote a fully compliant Open GL stack for it in rust.
Shipped by Fedora. Used on all Apple Silicon Macs running Linux.
The value exists and has been proven. The community can say no to it, but dismissing the work of Lina and the Asahi team members who have worked on Rust stuff as a “toy project” is just being rude.
Shipped by Fedora. Used on all Apple Silicon Macs running Linux.
The value exists and has been proven. The community can say no to it, but dismissing the work of Lina and the Asahi team members who have worked on Rust stuff as a “toy project” is just being rude.
I think the new Nvidia driver is also being written in Rust, though it's not ready yet.
Didn't they announce hat they use Ada and SPARK for such things some years ago?
I believe this poster meant the new Linux driver for Nvidia cards called Nova (announcement: https://lore.kernel.org/dri-devel/Zfsj0_tb-0-tNrJy@cassiopei...), not the driver maintained by Nvidia.
this is also very cool! some pretty nice blog posts from the asahi folks about this and the code looks nice...
Not Linux => Windows GDI regions code, Windows CoreWrite, Copilot+ PC EUFI firmware, Azure networking cards firmware.
Requirement that unless required by existing code, C and C++ are deprecated for new projects on Azure Infrastructure team, Rust or compiled managed languages (C#, Java, Go) are the official ones for new projects.
Requirement that unless required by existing code, C and C++ are deprecated for new projects on Azure Infrastructure team, Rust or compiled managed languages (C#, Java, Go) are the official ones for new projects.
I think this is intended as more of a proof of concept than as a useful-to-end-users change, but apparently this driver is going to be enabled on Fedora rawhide soon.
https://github.com/torvalds/linux/blob/master/drivers/net/ph...
MR where they are working though a build system issue: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/3...
https://github.com/torvalds/linux/blob/master/drivers/net/ph...
MR where they are working though a build system issue: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/3...
Fedora is also apparently considering enabling rust to render QR-codes on kernel panics (and has upstreamed code to do the rendering already).
fish shell
Cygwin is a compatibility layer between linux syscalls and windows. And that is all userspace. So if you somehow implement everything in there plus the raw kernel functions, maybe there is a chance I think.
Cygwin requires recompiling.
Care to explain a little bit more? In which cases recompiling is required? Do you agree that Cygwin is a moderately successful reimplementation of a part of Linux? If you agree then I wonder how recompiling affects that Cygwin is a (partial) reimplementation?
Cygwin is a POSIX implementation, not a Linux implementation. Linux is mostly POSIX compatible and so Cygwin is source-compatible, but not binary-compatible, with many programs that work on Linux.
[deleted]
What the other guy said. Cygwin is a set of libraries to compile and run GNU programs on Windows. It does not run existing Linux binaries and definitely does not translate Linux syscalls into Windows (that would be WSL v1). If recompiling is OK, I suggest starting with one of the BSDs, at least they have a fork() that works.
What I meant is that it can be used as a checklist of what to do, since so many linux programs can be built against it and it has been around for more than two decades.
Could not agree more.
I think there’s a parable in the Bible about this one:
No one puts new wine into old wineskins; or else the new wine will burst the wineskins and be spilled, and the wineskins will be ruined. But new wine must be put into new wineskins, and both are preserved.
He was specifically talking about bringing new paradigms to an old system. Sometimes, it’s just easier to start over. It has been my experience in programming that starting over almost always yields better results, but it does take time.
That’s why apples m1 was so successful. They took the time to rebuild just for apple. But it was 100% worth it.
I think there’s a parable in the Bible about this one:
No one puts new wine into old wineskins; or else the new wine will burst the wineskins and be spilled, and the wineskins will be ruined. But new wine must be put into new wineskins, and both are preserved.
He was specifically talking about bringing new paradigms to an old system. Sometimes, it’s just easier to start over. It has been my experience in programming that starting over almost always yields better results, but it does take time.
That’s why apples m1 was so successful. They took the time to rebuild just for apple. But it was 100% worth it.
The counter has started. I am looking forward that drop-in kernel, that could replace Linux on all my production systems "has-is".
This sounds very easy. Very.
This sounds very easy. Very.
This is an interesting take. Has DeVault done a lot of Linux kernel development?
No. He himself said it here.
https://fosstodon.org/@drewdevault/113052127771211132
What he did though, he developed a Unix clone using hare in roughly 30 days.
https://fosstodon.org/@drewdevault/112319697309218275
https://fosstodon.org/@drewdevault/113052127771211132
What he did though, he developed a Unix clone using hare in roughly 30 days.
https://fosstodon.org/@drewdevault/112319697309218275
Probably a significant number of HN readers have done something like this at one point. IIRC there is a MIT course where you do a modern Unix clone in one semester.
There is an ocean between that and being an expert in Linux internals.
Oh, and the original UNIX was done in one week by one guy.
There is an ocean between that and being an expert in Linux internals.
Oh, and the original UNIX was done in one week by one guy.
I have not contributed much to Linux -- my claim to fame is submitting a one-line patch which generated 50+ emails of arguments from LKML before Linus merged it -- but I have read a ton of the Linux source code and familiarized myself with many of its internals many times, as well as doing extensive low-level systems work in userspace against the Linux API/ABI. I often use it as a reference in my own osdev work, or working on Hare, etc. Have read a lot of the syscall API surface, DRM internals in depth, dcache and several filesystem implementations, io_uring, etc. Not ignorant to what would be involved in making a Linux-compatible kernel.
Nobody said he is an expert in Linux internals.
What this brings to light is the ability to compare Helios (microkernel) and Unix clone development, or how it was said in the post: reimplementing already existing design vs designing something new.
Because of this, I think his statements carry some weight.
What this brings to light is the ability to compare Helios (microkernel) and Unix clone development, or how it was said in the post: reimplementing already existing design vs designing something new.
Because of this, I think his statements carry some weight.
Thanks! I don't follow him on Mastodon, wouldn't have known. I wasn't so much curious about whether he was capable of writing OS kernel code, since that's not at all what his take is about.
TLDR: In response to the recent resignation of Rust-for-Linux's Wedson Almeida Filho [0] and consequent debate over the politics of Rust in the kernel, Drew DeVault suggests that the Rust for Linux crowd build their own ABI-compatible Linux clone, reasoning that a clone is easier than a greenfield research OS (like the existing Redux).
It's not a bad idea, imho. Rust folks love rewriting things, and are often quite good at it. It would keep both the curmudgeonly kernel Never-Rusters and the Rust evangelism brigades happy (by keeping them apart). And it's a great proving ground for Rust's claims - assuming licence compatibility, successful rewrites of individual components could be lifted by Linux down the line.
[0] https://lwn.net/Articles/987635/
It's not a bad idea, imho. Rust folks love rewriting things, and are often quite good at it. It would keep both the curmudgeonly kernel Never-Rusters and the Rust evangelism brigades happy (by keeping them apart). And it's a great proving ground for Rust's claims - assuming licence compatibility, successful rewrites of individual components could be lifted by Linux down the line.
[0] https://lwn.net/Articles/987635/
I expect the result of Filho's tantrum will be the ouster of Ted Tso from the Linux kernel.
I think he is wrong on two accounts:
1. While the Linux guy mentioned here is not a great politician and his rant was pathetic, his technical argument was sane. You can put months into creating the perfect data model for your Rust implementation but Linux have a habit of completely changing things every year or so. That would make Rust the bottleneck of Linux development
2. These "hugely talanted" Rust devs are competing with ten times more talented Linux devs on their home turf.
My point is, stop creating drama. This task requires hard work and compromises.
1. While the Linux guy mentioned here is not a great politician and his rant was pathetic, his technical argument was sane. You can put months into creating the perfect data model for your Rust implementation but Linux have a habit of completely changing things every year or so. That would make Rust the bottleneck of Linux development
2. These "hugely talanted" Rust devs are competing with ten times more talented Linux devs on their home turf.
My point is, stop creating drama. This task requires hard work and compromises.
Stop contributing to the drama with the “ten times more talented” comment. You have no way of measuring such a thing and are just fanning the flames.
Linux devs and rust devs aren't two opposing groups. There exists major pockets within Linux community who embrace rust. That doesn't mean they turned in their Linux dev cards.
These arguments are pointless because in a few years you can have your AI of choice translate Linux to your language of choice.
The real question is will you even bother to do it? Will it matter at that point?
The real question is will you even bother to do it? Will it matter at that point?
From the Linux conference video I watched I think they have the wrong approach. They’re trying to improve a bad api by adding types do describe it’s awkward behavior. The C programmers who love the Linux style want to be able to change semantics or api without modifying rust code. The solution is straight forward, build an adapter later in C that has well describable and clean semantics. Since it is C and in the kernel and has clear semantics the C programmers can maintain it without having to look at Rust. Perhaps the general lack of encapsulation and layers in the kernel will defeat them as they will need a lot of adapter layers. But for file systems it might be achievable.
It's more or less working for ladybird browser, could work for a kernel.
Ladybird is getting migrated to Swift, not Rust. Andreas Kling goes over why he chose Swift here: https://x.com/awesomekling/status/1822236888188498031 and his views on Rust here: https://x.com/awesomekling/status/1822241531501162806
The point is not the specific language, but the approach. The split from Serenity OS (and its "C++ only, no external dependencies" approach) made Ladybird a lot more attractive for both funding and potential contributors.
It is intended to be migrated. Nothing has been done in Swift yet.
Ladybird is all C++ though a somewhat unique dialect of it.
Servo is a browser engine written in Rust.
Ladybird is all C++ though a somewhat unique dialect of it.
Servo is a browser engine written in Rust.
I am well aware, the point is that ladybird is a new project following some standards and they are free to use a memory safe language now.
The language isn't the problem, it's the architecture. In Linux, all of the drivers run with the rest of the kernel. A bad driver can take down the whole thing, or perform arbitrary badness.
In Drew's own Helios microkernel, the architecture is right, and it's ready to have folks start piling on drivers in user-space. Those drivers can be written in pretty much any low level language that can produce ELF binaries.
It's surely a better path forward to slap a Linux syscall API on top of Helios + some new drivers than to clone the whole linux system architecture in another programming language. At least then, we would have gained something tangible (drivers can't ruin the kernel). Instead of having another Linux written in a different language, with all the same design flaws.