Linux kernel in-tree Rust support(lore.kernel.org)
lore.kernel.org
Linux kernel in-tree Rust support
https://lore.kernel.org/lkml/CAKwvOdmuYc8rW_H4aQG4DsJzho=F+djd68fp7mzmBp3-wY--Uw@mail.gmail.com/T/#u
506 comments
GCC 10 supports Dlang directly. (The support is still immature and it's being updated to the newer version of Dlang.)
Dlang compiles quickly not because the language is simple, but because the compiler authors care about compilation speed:
Lexer skips four spaces at once https://github.com/dlang/dmd/pull/11095
Optimize core logic to 7x compilation speed of language feature https://github.com/dlang/dmd/pull/11303
Cache rarely used types to improve memory locality https://github.com/dlang/dmd/pull/11363
Dlang compiles quickly not because the language is simple, but because the compiler authors care about compilation speed:
Lexer skips four spaces at once https://github.com/dlang/dmd/pull/11095
Optimize core logic to 7x compilation speed of language feature https://github.com/dlang/dmd/pull/11303
Cache rarely used types to improve memory locality https://github.com/dlang/dmd/pull/11363
It's important to note that run-time tradeoffs were made in the interest of fast compilation speed. When it comes to optimizing code emitted from D source, it's important to leverage the alternate implementations from GCC or the LLVM-based LDC...which don't sport nearly as fast compiles as DMD, the reference D compiler.
For DMD's backend, that is true. It's like Rust's MIR project, but finished.
However, the commits I linked to describe front-end optimizations. LLVM-based D Compiler and GCC share DMD's front-end.
I've seen some generated code optimizations for DMD's back-end as well, which could get one lightning fast compiles for decent code. (Maybe in the future one could write a JIT that first compiles with DMD then tiers up to LLVM D Compiler or GCC.)
However, the commits I linked to describe front-end optimizations. LLVM-based D Compiler and GCC share DMD's front-end.
I've seen some generated code optimizations for DMD's back-end as well, which could get one lightning fast compiles for decent code. (Maybe in the future one could write a JIT that first compiles with DMD then tiers up to LLVM D Compiler or GCC.)
You're right, I didn't distinguish between overall compile times and the front-end time taken, which is what you were focusing on. For the Rust toolchain, the (vast IIRC) majority of compilation time is in LLVM's backend. So, to me, front-end optimizations are interesting, but not the highest priority for Rust specifically. LMD seems to be a great thing to compare against, so here's my follow-up question: What's the proportion of front-end to back-end time in LMD?
The D language uses GC, so it's unlikely to be acceptable for the Linux kernel. (That's not to say that you can't write a kernel in D, just that the Linux kernel maintainers don't want to write a kernel that requires GC.)
Rust's ownership model, which allows you to write straightforward, non-leaky, non-use-after-free-filled code without a GC, is quite complex at compile time. It's definitely possible to make it faster, but it's definitely a thing Rust does that D doesn't do.
Rust's ownership model, which allows you to write straightforward, non-leaky, non-use-after-free-filled code without a GC, is quite complex at compile time. It's definitely possible to make it faster, but it's definitely a thing Rust does that D doesn't do.
D has a nogc flag that can be used on methods and taints methods down the line. It's pretty neat. I'm a big fan of D, but I still think I wouldn't write a Kernel in D. I find it much more suited to quicker smaller programs, kind of like Python but with better typing. The script support for D is also really neat, and I've used that instead of Perl a few times.
When you turn off D's GC then its no different from C or C++ . So yes, it can be done. At last D conference, someone presented on building kernel drivers in D.
D is definitely trying to support it as well, how far they will get remains to be seen.
Nick Nethercote and others have done a lot of work with traditional profiling and optimization like that. They've done a great job, but for my project the fundamental problem seems to be limited parallelism, which I think requires more fundamental design changes (perhaps getting away from the crate-at-a-time approach of rustc).
Though TBH I'm not an expert on the compiler and I'm not confident I know the root problems or the solutions.
Though TBH I'm not an expert on the compiler and I'm not confident I know the root problems or the solutions.
Performance of the generated code is always the #1 priority in both Rust and llvm, performance improvements are rarely declined, and this prioritization needs to change radically, to get a much faster compiler.
Prioritization is not that straightforward. Every open source project has different sub-groups with different goals. While we do care a ton about generated code, there is also movement by other folks to work on the compiler speed problem. It has been a lot of work to come this far, and it will be even more work to get to a better place, but that work is underway. That's not even to mention that since rustc is written in Rust, work to make generated code faster can also make the compiler faster.
I'm referring to the rust-analyzer work, which is effectively an outside-in re-write of the compiler, if you squint at it right.
At the end of the day, rustc is a very large codebase, and has already undergone major architectural changes, and will be undergoing even more. It is not fast to steer a large ship in a new direction, but that is what is happening.
See https://pbs.twimg.com/media/ENC1ls6XYAEdKkv?format=jpg&name=... for a time I ran https://github.com/erikbern/git-of-theseus on the rustc codebase. Thank goodness Rust is such a good refactoring language.
I'm referring to the rust-analyzer work, which is effectively an outside-in re-write of the compiler, if you squint at it right.
At the end of the day, rustc is a very large codebase, and has already undergone major architectural changes, and will be undergoing even more. It is not fast to steer a large ship in a new direction, but that is what is happening.
See https://pbs.twimg.com/media/ENC1ls6XYAEdKkv?format=jpg&name=... for a time I ran https://github.com/erikbern/git-of-theseus on the rustc codebase. Thank goodness Rust is such a good refactoring language.
dYou are talking incremental improvements. Radical is needed not just using cranelift for debug builds, but something like that for all builds. Current llvm model does not work well for delivering a fast compiler (opt and non-debug)
You can deliver radical improvements in an incremental fashion. The move towards a query-based architecture, and the rust-analyzer work, are both fundamental, radical improvements.
If anything, the cranelift stuff is less radical. Doesn't require any architecture changes, "only" involves implementing the backend API.
If anything, the cranelift stuff is less radical. Doesn't require any architecture changes, "only" involves implementing the backend API.
then there's also ongoing work in rustc_codegen_cranelift, which [as of 2 weeks ago] can now compile rustc:
https://github.com/bjorn3/rustc_codegen_cranelift/issues/743...
https://github.com/bjorn3/rustc_codegen_cranelift/issues/743...
Yeah to be clear, I am very excited about cranelift too!
Sure, that's seriously cool but I don't see how it changes the material balance so to speak... If you still want llvm to codegen the same items, we don't have any radical departure at all, just a different way to do the same thing, when we are release-compiling any particular crate, from scratch.
Oh sure, if you're talking purely about a final release build, then yes. But most of the time, compiles are not final release build compiles. People don't generally complain about final release build times. They complain about the build time as they're developing.
I regularly need to run release builds while developing. Not as commonly as I need to do a debug build or a cargo check, but still often enough that the slowness is annoying.
Final release build times are up there as build automation systems are often at capacity
For sure, and I don't mean to say that they don't matter. I have worked on codebases that only use release mode, for reasons, for example. I just hear less complaints about it, that's all I'm saying :)
Usually I've observed release is used by automated systems and debug locally. If you're not managing the automation or working closely with those people, you're less likely to see those complaints but they definitely exist.
What is wrong having 2 versions: slower at runtime but super fast compile for debug and the other way around for production?
Of course, that is the status quo for rustc (as it is for all production compilers I know of). The issues are that (1) debug builds are still slow enough to be annoying, and (2) they are sometimes _so much slower_ to execute that a debug build + test run is slower than a release build + run.
A lot of work is being done to address these issues. The big projects I'm aware of are a move to a query-based compiler (which should improve incremental build times) and the new cranelift backend, which should dramatically speed up debug build times.
A lot of work is being done to address these issues. The big projects I'm aware of are a move to a query-based compiler (which should improve incremental build times) and the new cranelift backend, which should dramatically speed up debug build times.
The slowness of building crates makes an immense impact on ordinary rust project build times, but as the Linux kernel would just be using rustc directly, and Kbuild already parallelizes CC invocations, the relevant gains here would be single-TU compilation speed.
But the translation unit in Rust is the crate. Running `rustc` directly won't change that fact. Unlike in C, where you can compile TUs in any order since you've got interfaces in headers, in Rust you can't compile a crate until you've compiled all its dependencies--and if it's not Cargo that's going to sort that out for you, Kbuild will have to.
> in Rust you can't compile a crate until you've compiled all its dependencies
Since Rust 1.38 (https://blog.rust-lang.org/2019/09/26/Rust-1.38.0.html#pipel...) that's no longer the case: "To compile a crate, the compiler doesn't need the dependencies to be fully built. Instead, it just needs their "metadata" (i.e. the list of types, dependencies, exports...). This metadata is produced early in the compilation process. Starting with Rust 1.38.0, Cargo will take advantage of this by automatically starting to build dependent crates as soon as metadata is ready."
Since Rust 1.38 (https://blog.rust-lang.org/2019/09/26/Rust-1.38.0.html#pipel...) that's no longer the case: "To compile a crate, the compiler doesn't need the dependencies to be fully built. Instead, it just needs their "metadata" (i.e. the list of types, dependencies, exports...). This metadata is produced early in the compilation process. Starting with Rust 1.38.0, Cargo will take advantage of this by automatically starting to build dependent crates as soon as metadata is ready."
Every time a new Rust version comes out I compile my toy Gtk-rs application ported from a C/C++ User Journal article about Gtkmm.
The C++ version still wins by several minutes on a Asus 1215B.
The C++ version still wins by several minutes on a Asus 1215B.
Has the gap at least narrowed over time?
Definitely, the improvements on rustc are visible, so eventually it will reach there.
And I test it on that hardware on purpose, because I should not have to buy a top class desktop to achieve usable compile times similar to other AOT compiled languages.
However, I also admit that what is missing is having a backend that favours debug workflows, which do exist just aren't yet there.
And I test it on that hardware on purpose, because I should not have to buy a top class desktop to achieve usable compile times similar to other AOT compiled languages.
However, I also admit that what is missing is having a backend that favours debug workflows, which do exist just aren't yet there.
Is this a fresh compile each time?
Yes, make clean all, cargo clean build.
If it does compile, could you try reporting results for multiple Rust compiles (e.g. the current one, and the ones from 6,12,18,24 months ago) ? (If you can, maybe do the same for C, but I don't know how easy that is).
This is good but it doesn't work well enough in my experience. I still spend a lot of time waiting for dependencies to build before the crates that depend on them.
Given the modular nature of the kernel, how big will each "crate" be? Will they mostly end up interfacing with the rest of the kernel using the C ABI and not need to be compiled as a single unit?
The Rust compiler folks care immensely about compile speed, and the language is a factor in both cases.
I don't know about D, but a difference between Rust and Go, for instance, in this area might be that the Go language designers care about compilation speed, to the extent that it has influenced the language design, while in Rust that concern may be important in terms of implementing the compiler, but not enough to influence the language design.
It's not that Rust doesn't care about compilation speed, is that final binary speed, zero cost abstractions and ergonomics lie higher in the priority list. If we didn't care about ergonomics you wouldn't be able to have diamond dependencies inside of a crate (like defining a struct in a file and it's impl in a different one, or splitting impl items between two files, this is very handy but requires that the compilation unit be a crate). If we didn't care about zero cost abstractions and final binary speed we wouldn't always monomorphism generics and have a threshold where we automatically use dyn traits instead (vtables), like Swift does, but we want people to have full control over the behavior of their code so you have to change your code in order for that behavior to change. This means that your code's performance characteristics won't change because of unrelated changes crossing some threshold, but it does mean that unless you opt into using a vtable your code will spend a lot of time expanding code for every type used in generics. This doesn't mean that we don't care about performance or that there aren't things we can do to improve the current situation or that we don't actively pursue those actions (take a look at https://perf.rust-lang.org/ over larger periods of time and you'll see that for the most part things are getting significantly better).
[deleted]
Yes, agreed fully. This is a good example of what I meant by the language being a factor :)
Walter Bright founded Dlang and continues to be the primary author of its compiler frontend. He is the sole author of dmc++/Zortech C++, the world's fastest C++03 compiler.
Absolutely true. I'm not sure what you're implying though, none of that contradicts what I've said in any way.
Are you suggesting to hire Walter to work on Rust compiler/language itself?
Such things are giving me hope that eventually all compilation to native CPU code will converge on a singular project. There's so much fragmented effort!
Slightly off topic, I see D brought up incredibly consistently when Rust is mentioned on HN.. Is it just me?? (it's usually by walter themself but I digress..)
Some of us believe in systems programming languages with some form of GC, so it usually pops up as alternative.
For me D is what .NET (thus C#) should have been if the Ext-VOS project (COM Runtime) decided to go native, instead of embracing J++ and then turning into .NET due to the lawsuit.
But unfortunately D is a tiny community that gets by with small contributions and herding cats is hard.
For me D is what .NET (thus C#) should have been if the Ext-VOS project (COM Runtime) decided to go native, instead of embracing J++ and then turning into .NET due to the lawsuit.
But unfortunately D is a tiny community that gets by with small contributions and herding cats is hard.
> but because the compiler authors care about compilation speed
Is this meant to imply that the Rust compiler authors don't care about compilation speed?
Is this meant to imply that the Rust compiler authors don't care about compilation speed?
Everybody cares about compilation speed. What would be interesting is how much time in percentage is spent optimizing it though. Does any one have any hard daya here?
We don't track time of who works on what, so there's no way to answer this specific question with hard data.
They haven't made it a priority until the last year or two, no efforts were made to improve it before then.
This is simply not true.
For example, here's the 2017 roadmap, which was authored in 2016 https://github.com/rust-lang/rfcs/blob/master/text/1774-road...
> The edit-compile-debug cycle in Rust takes too long, and it's one of the complaints we hear most often from production users. We've laid down a good foundation with MIR (now turned on by default) and incremental compilation (which recently hit alpha). But we need to continue pushing hard to actually deliver the improvements. And to fully address the problem, the improvement needs to apply to large Rust projects, not just small or mid-sized benchmarks.
We didn't do these sorts of yearly plans before that, but note that it isn't just talking about what should be done then, but the work that was being done on this previously.
For example, here's the 2017 roadmap, which was authored in 2016 https://github.com/rust-lang/rfcs/blob/master/text/1774-road...
> The edit-compile-debug cycle in Rust takes too long, and it's one of the complaints we hear most often from production users. We've laid down a good foundation with MIR (now turned on by default) and incremental compilation (which recently hit alpha). But we need to continue pushing hard to actually deliver the improvements. And to fully address the problem, the improvement needs to apply to large Rust projects, not just small or mid-sized benchmarks.
We didn't do these sorts of yearly plans before that, but note that it isn't just talking about what should be done then, but the work that was being done on this previously.
The push for Rust is coming from above, top-down. Watch what programmers do, not say. They prefer Go to Rust because it compiles fast.
That depends on the programmer. Personally, I will choose rust ovet go almost every time, because I think it has a better type system, and is in many (but not all) ways more ergonomic. And that is more important to me than compile time, although I concede that other programmers (especially those with more experience with interpreted languages) prioritize compile time I over other language aspects. Besides which, my experience has been that when incrementally compiling during development, go compilation isn't actually that much faster.
Every language that goes all-in on type safety (Haskell, OCaml, F#, Scala) ends up being too hard to actually use in day-to-day programming. But people publicly say that they prefer it so as to not signal that they themselves find it too difficult. In an ideal world, I'd be writing everything in Coq, but I still live on earth and have deadlines, so I'll "do that some other day". All the code ends up being in some practical language, but when I get a developer survey I'll say I love the type safe hippie language.
I think the difficulty with those languagrs comes more from the pure functional paradigm than type safety. And I do use scala as a day-to-day language. Also, I'm not saying I prefer rust over go after reading some documentation and playing around with toy examples. I've written real code in both, and prefer the rust experience. I think go's main advantage is it is easy to learn. That is a real advantage in some cases (like onboarding new employees/contributors), but for me and many programmers I know it is not a sufficiently compelling reason to give up a more powerful type system.
Exactly this!
I use Go because it's easy when I sometimes need to provide small binaries to colleagues to help with their day-to-day work. And those are colleagues who don't have a Python environment installed or anything, which I'd be opting for in small "workflow improvement" cases like this. So whip up a small Go program, compile to Windows, and make a colleague happy.
But when I build larger codebases with my developer colleagues (or private projects at home) I really prefer Rust as it gives me so much more trust in the code that I might have to maintain for years and where the linecount is not just <200, but actually thousands and thousands.
> Every language that goes all-in on type safety (Haskell, OCaml, F#, Scala) ends up being too hard to actually use in day-to-day programming.
F# is the most pleasant, practical day-to-day programming language that I have personally used. Scala is also quite nice.
F# is the most pleasant, practical day-to-day programming language that I have personally used. Scala is also quite nice.
D was dead by all practical measures since its inception. It was never revolutionary enough to warrant the huge costs of migrating there. This is where Rust comes in...
Typescript allows to compile without type checking at the sacrifice of correctness. If you would have a slow computer or huge code-base, you could consider using the type checker only for tests, pre-checkin check, or CI. (Not that Typescript is slow - but as an conceptual example)
Would something similar be possible for Rust, so you still have the ability for correctness for CI and release builds, but allow for fast compilation when necessary or desired?
Would something similar be possible for Rust, so you still have the ability for correctness for CI and release builds, but allow for fast compilation when necessary or desired?
Type checking doesn't seem to be major bottleneck at all. The major issues seem to be
1) Technical debt in rustc producing large amounts of LLVM IR and expecting LLVM to optimize it away
2) generic monomorphization producing vast amount of IR
IIRC.
1) Technical debt in rustc producing large amounts of LLVM IR and expecting LLVM to optimize it away
2) generic monomorphization producing vast amount of IR
IIRC.
Typescript would have a far easier time in that scenario because much of it's syntax is simply Javascript with extra types attached. So, it could (I don't know the implementation details at all) 'just' strip away all the typescript-syntax, and give you your Javascript files.
Rust has the issue that there's a lot of type-checking done, even in basic scenarios: such as setting a variable `let alpha = some_function(abc);`, you have to do some form of basic typechecking here to get the type of `alpha` (since that affects how it's stored on the stack). The simple case of `fn some_function(abc: u32) -> i16;` would be 'easy', but more complicated scenarios like generics would make your life harder.
Though, I'm sure there's parts of the checking that could be eventually written so that they can be turned off, but I don't think it would provide nearly as much benefit as Typescript's non-correct compilation.
Personally, I think it would be better to spend that time on just making the compiler faster. This is perhaps easier than I made it sound, but this is my assumptions as to why it wouldn't work that well.
> Would something similar be possible for Rust
Yes. The `mrustc` alternative implementation doesn't do things like borrow checking, it assumes the source is "correct".
> [skipping typechecking would] allow for fast compilation when necessary or desired?
The real expenses in a "release" compilation pipeline are in the optimisation passes and codegen, so the gains are mostly in avoiding optimisations (debug build) or avoiding codegen entirely (`cargo check`).
For instance on the latest ripgrep head on my machine:
* cargo check takes ~200s user
* cargo build (debug) takes ~300s user
* carbo build --release takes ~900s user
Yes. The `mrustc` alternative implementation doesn't do things like borrow checking, it assumes the source is "correct".
> [skipping typechecking would] allow for fast compilation when necessary or desired?
The real expenses in a "release" compilation pipeline are in the optimisation passes and codegen, so the gains are mostly in avoiding optimisations (debug build) or avoiding codegen entirely (`cargo check`).
For instance on the latest ripgrep head on my machine:
* cargo check takes ~200s user
* cargo build (debug) takes ~300s user
* carbo build --release takes ~900s user
I’ve heard someone say about the rust compiler that it will never allow compilation without its safety checks in place because it would decrease faith in any rust binary.
You could always replace the fn body of large parts of your codebase with unimplemented!() (or use the nightly only everybody-loops flag), but I don't think this will have as big a benefit as people would expect. cargo check though is probably something more people should know about.
[deleted]
I assume dependency trees in the kernel would be much more shallow.
That might be true for drivers.
Outside the kernel, I don't think it's going to be true for Android or Windows.
Outside the kernel, I don't think it's going to be true for Android or Windows.
But this discussion is mostly about the kernel, isn't it? I'm not sure why throwing out rust weaknesses in general is appropriate when we're talking about a specific use case such as kernel drivers and other modules which do tend to be quite a bit more shallow than most large c/c++ projects I've been on.
That's a fair point. I do think it's appropriate to point out that build times are a problem for a lot of projects and may affect the kernel so careful attention needs to be paid to that issue.
I know a lot of people will disagree with me, and I know that the rust compiler has room for improvement, but compile time isn't really that bad for rust when you consider what it actually does for you.
Assuming no unsafe blocks, rust's type system eliminates entire classes of memory safety errors. It fundamentally eliminates data races. It eliminates large swaths of logical errors, due to its use of algebraic data types, and strong type enforcement. No null pointers, explicit error handling, and no undefined behavior.
It boggles my mind that people will balk at compile times for rust, but then not even bat an eye at their C++ testkit that runs for 8+ hours, primarily checking for errors that rust would never let you make in the first place.
Assuming no unsafe blocks, rust's type system eliminates entire classes of memory safety errors. It fundamentally eliminates data races. It eliminates large swaths of logical errors, due to its use of algebraic data types, and strong type enforcement. No null pointers, explicit error handling, and no undefined behavior.
It boggles my mind that people will balk at compile times for rust, but then not even bat an eye at their C++ testkit that runs for 8+ hours, primarily checking for errors that rust would never let you make in the first place.
Part of what makes rustc's slow compile times so frustrating is that the slow compile times are mostly unrelated to all of the fancy compile-time analysis you listed. You can see this in action if you run `cargo check`. That will run all the special static analysis (typeck, borrowck, dropck, etc.) and complete in a very reasonable amount of time.
To a first approximation, rustc is slow because it generates terrible LLVM IR, and relies on a number of slow LLVM optimization passes to smooth that IR into something reasonable. As far as I can tell this is entirely a result of not having the necessary engineering resources to throw at the problem; Mozilla has a limited budget for Rust, and needs to spread that budget across not just the compiler toolchain, but the language and ecosystem.
Brian Anderson's been working on a blog post series that covers Rust's slow compile times in more detail, if you're curious: https://pingcap.com/blog/tag/Rust
To a first approximation, rustc is slow because it generates terrible LLVM IR, and relies on a number of slow LLVM optimization passes to smooth that IR into something reasonable. As far as I can tell this is entirely a result of not having the necessary engineering resources to throw at the problem; Mozilla has a limited budget for Rust, and needs to spread that budget across not just the compiler toolchain, but the language and ecosystem.
Brian Anderson's been working on a blog post series that covers Rust's slow compile times in more detail, if you're curious: https://pingcap.com/blog/tag/Rust
Yeah, I realize that Rust has a lot of room for improvement. I'm just saying that compared to the alternative, you get a relatively inefficient compiler that, despite its inefficiency, still gives you a more efficient development process. You get instant feedback, which you can't get from a testkit, and you can eliminate massive amounts of testing.
If the focus is entirely on reducing compile times, it can very quickly lead down a path that limits runtime performance. There is so much potential for increased runtime performance in strongly typed languages, due to the guarantees made by the type system. It allows for analysis and reasoning that not just enables micro-optimizations but also macro-optimizations across entire codebases. Entire theses have been written describing optimizations that rely on immutability alone, and we haven't even touched so many other possibilities that rust's design enables (such as the ownership system).
I'm fairly familiar with the project to work on the LLVM-IR code emitter. If the focus is on reducing the complexity and cruft of the LLVM-IR emitted, that's a win all around (because you get faster code and faster compile times). I just hope that reducing compile times doesn't become the core objective, because that eliminates a lot of future potential.
If the focus is entirely on reducing compile times, it can very quickly lead down a path that limits runtime performance. There is so much potential for increased runtime performance in strongly typed languages, due to the guarantees made by the type system. It allows for analysis and reasoning that not just enables micro-optimizations but also macro-optimizations across entire codebases. Entire theses have been written describing optimizations that rely on immutability alone, and we haven't even touched so many other possibilities that rust's design enables (such as the ownership system).
I'm fairly familiar with the project to work on the LLVM-IR code emitter. If the focus is on reducing the complexity and cruft of the LLVM-IR emitted, that's a win all around (because you get faster code and faster compile times). I just hope that reducing compile times doesn't become the core objective, because that eliminates a lot of future potential.
Note that Rust doesn't really generate worse LLVM IR than clang does, given the same input. It's just that as a language, Rust needs MIR optimizations more than C and C++ do, given the increased abstraction (largely necessary for safety).
I think your and OP's perspectives are not exclusive:
A. Yes, as you say, taken from a distance and considering the whole development lifecycle from idea to shipping, "compile time isn't really that bad for rust when you consider what it actually does for you".
B. However, -during- development, you want compile speed, regardless of all the benefits. I don't know about you, but I have a hard time rationalizing "it's okay, rustc is doing many things" at that moment. I just want it to be faster, and as OP mentioned, I hope there's a future where companies / sponsors get to "hiring about 25 people" to crunch and improve things noticeably (and I will do more than hope, I already donate money to Ferrous Systems to improve rust-analyzer, and will donate if a "make rust faster" fund asks for it). Your argument sounds to me a bit like "who cares about making Python faster? It's just glue". To which I answer: maybe (though not always), but even then but I'll take faster glue anytime :) .
To sum up: rust being already awesome and being "fast given all the thing it does" doesn't mean it shouldn't be faster, and doesn't mean speaking about it is futile.
A. Yes, as you say, taken from a distance and considering the whole development lifecycle from idea to shipping, "compile time isn't really that bad for rust when you consider what it actually does for you".
B. However, -during- development, you want compile speed, regardless of all the benefits. I don't know about you, but I have a hard time rationalizing "it's okay, rustc is doing many things" at that moment. I just want it to be faster, and as OP mentioned, I hope there's a future where companies / sponsors get to "hiring about 25 people" to crunch and improve things noticeably (and I will do more than hope, I already donate money to Ferrous Systems to improve rust-analyzer, and will donate if a "make rust faster" fund asks for it). Your argument sounds to me a bit like "who cares about making Python faster? It's just glue". To which I answer: maybe (though not always), but even then but I'll take faster glue anytime :) .
To sum up: rust being already awesome and being "fast given all the thing it does" doesn't mean it shouldn't be faster, and doesn't mean speaking about it is futile.
I'm not opposed to faster compile times, and there is plenty of room for improvement, as I have already stated.
Generally speaking, if you are using incremental compilation, Rust is fast enough for development. It could be better, especially on large projects, but I haven't found it to be an issue generally. There are a bunch of improvements that could still be made. The LLVMIR issue mentioned above is one of them. But also, why do we have to rely on a code emitter at all? In development mode, the type checker should be incremental, synchronous, and fast...but the code emitter could be relatively slow as long as it is asynchronously coupled to the type checker.
But relentlessly pursuing the path of compiler performance can be a bad route to take, because it invariably leads to stagnation in compiler capabilities. I'd gladly accept a 10x increase in `opt-level=3` compile times if it meant 10% faster runtime performance. I'd gladly accept a 10x increase in `opt-level='z'` compile times if it meant a 50% decrease in binary sizes. We can't get to those capabilities if those possibilities are squashed by concerns about compile times.
Generally speaking, if you are using incremental compilation, Rust is fast enough for development. It could be better, especially on large projects, but I haven't found it to be an issue generally. There are a bunch of improvements that could still be made. The LLVMIR issue mentioned above is one of them. But also, why do we have to rely on a code emitter at all? In development mode, the type checker should be incremental, synchronous, and fast...but the code emitter could be relatively slow as long as it is asynchronously coupled to the type checker.
But relentlessly pursuing the path of compiler performance can be a bad route to take, because it invariably leads to stagnation in compiler capabilities. I'd gladly accept a 10x increase in `opt-level=3` compile times if it meant 10% faster runtime performance. I'd gladly accept a 10x increase in `opt-level='z'` compile times if it meant a 50% decrease in binary sizes. We can't get to those capabilities if those possibilities are squashed by concerns about compile times.
> Rust in Linux will be fantastic except for compile time. Rust (and the world) needs a Manhattan Project to build a fast Rust compiler (where by "fast" I mean both efficient and scalably parallel when compiling a workspace with many crates and long dependency chains).
A basic technique to parallelize sections is to split a project into modules. Is this not an option for the linux kernel?
A basic technique to parallelize sections is to split a project into modules. Is this not an option for the linux kernel?
Rust has great support for modules (crates in Rust lingo). It can compile modules in parallel when they don't depend on each other.
The problem is that when you have module A depending on module B, rustc doesn't do a good job of compiling A and B in parallel. In contrast, in C/C++ you can handcraft your header files to ensure that A and B compile in parallel.
The problem is that when you have module A depending on module B, rustc doesn't do a good job of compiling A and B in parallel. In contrast, in C/C++ you can handcraft your header files to ensure that A and B compile in parallel.
In C and C++ it is also quite common to use binary dependencies, something that cargo still doesn't do.
However C++ is also now looking at parallelization opportunities for modules, which is a similar situation as crates.
However C++ is also now looking at parallelization opportunities for modules, which is a similar situation as crates.
Every time you compile a rust project after an initial compile, cargo does use "binary" dependencies (bitcode dependencies).
It is much better at this than C++.
For example, change a trait in your Rust library, and recompile the tests, and compare that with changing a type in, say, range-v3 (or any other C++ header-only library).
In C++, recompiles take pretty much the same time as clean builds (for range-v3, exactly the same time). In Rust, they are infinitely faster (100000x faster, depending on the project, but essentially almost instant, in contrast with initial compiles).
So for development at least I'll pick Rust over C++ every day. I only do an initial compile once, but I edit and recompile a lot, and that's the bottleneck that I really care about.
(Same with C btw, add a new API to a commonly use C header and re-compile, and you end up re-compiling the world, while from Rust, you just re-compile the part of the crate where the new API was added, and since no downstream crates use it, that's pretty much it - instant change).
It is much better at this than C++.
For example, change a trait in your Rust library, and recompile the tests, and compare that with changing a type in, say, range-v3 (or any other C++ header-only library).
In C++, recompiles take pretty much the same time as clean builds (for range-v3, exactly the same time). In Rust, they are infinitely faster (100000x faster, depending on the project, but essentially almost instant, in contrast with initial compiles).
So for development at least I'll pick Rust over C++ every day. I only do an initial compile once, but I edit and recompile a lot, and that's the bottleneck that I really care about.
(Same with C btw, add a new API to a commonly use C header and re-compile, and you end up re-compiling the world, while from Rust, you just re-compile the part of the crate where the new API was added, and since no downstream crates use it, that's pretty much it - instant change).
https://github.com/StanfordSNR/gg - should be almost the same as the GCC/LLVM thunk extractor. You have to pay for the borrow checker, but LLVM IR optimization passes should be the same complexity.
> increasing the number of people paid to work on the Rust compiler by 10x would only mean hiring about 25 people.
Citation needed ?
I know at least 20 people being paid to work on the Rust compiler, which crank's your 10x number from 25 to 200. And this is only the people I can come up with from the top of my head.
Citation needed ?
I know at least 20 people being paid to work on the Rust compiler, which crank's your 10x number from 25 to 200. And this is only the people I can come up with from the top of my head.
How much of the needed speedup is engineering and how much is research?
Depends on what you mean by "research". If you mean "problems with a high degree of novelty" then quite a lot of research, I think. Rust is unusual: it requires a lot of tricky type inference and static analysis from the compiler, and doesn't make developers structure their code with header files etc to make separate compilation easier; BUT unlike most languages with those properties, people are writing large real-world projects in it.
My understanding is that Rust's slowness is mainly from code generation and optimization. `cargo check` performs full parsing and type-checking, and runs reasonably quickly.
The problem starts when Rust throws a ton of IR at LLVM. The current focus is on adding optimizations in Rust at higher level (MIR) so that it would cause less work for LLVM.
The problem starts when Rust throws a ton of IR at LLVM. The current focus is on adding optimizations in Rust at higher level (MIR) so that it would cause less work for LLVM.
That is a problem.
There is another major problem, which is that rustc/cargo do not do a good job of compiling crates in parallel when one depends on the other.
There may well be other problems too.
There is another major problem, which is that rustc/cargo do not do a good job of compiling crates in parallel when one depends on the other.
There may well be other problems too.
Imagine you have 16 cores available. Allowing parallel compilation where it is sequential at the moment, you get 16 times speedup on such a machine. 32 cores machine -- 32 times speedup etc. Or even 10 such machines -- 320 times speedup.
On the other side, imagine you can even reduce the "ton of IR" tenfold but if the compilation is sequential, the tenfold speedup is fixed even if you have 32 or 320 cores.
In general, allowing for parallel compilation is much more important topic, and it's worth even redesigning language to make that more achievable.
Which also doesn't mean that producing "ton of IR" is a good thing. But working on allowing maximum parallelism is more fundamental. Also, discovering algorithms to allow all compiler passes as effective as possible is important for something to reach wider usability. But Rust must also get to achieve its promises of being "safer" in the scenarios where C is otherwise used, and I time and again see a lot of "unsafe" keywords in the sources wherever I see some implementation of anything fundamental.
On the other side, imagine you can even reduce the "ton of IR" tenfold but if the compilation is sequential, the tenfold speedup is fixed even if you have 32 or 320 cores.
In general, allowing for parallel compilation is much more important topic, and it's worth even redesigning language to make that more achievable.
Which also doesn't mean that producing "ton of IR" is a good thing. But working on allowing maximum parallelism is more fundamental. Also, discovering algorithms to allow all compiler passes as effective as possible is important for something to reach wider usability. But Rust must also get to achieve its promises of being "safer" in the scenarios where C is otherwise used, and I time and again see a lot of "unsafe" keywords in the sources wherever I see some implementation of anything fundamental.
Modulo the necessarily sequential (as mentioned by rayiner), you are correct in terms of absolute speed achievable. But you achieve that speedup at a roughly linear increase in $$$ cost. When you have idle cores sitting around, maybe it was opportunity cost you were already paying, but that's bounded.
On the other hand, reductions in the amount of computation needed mean you get a faster build for the same resources, or a similar speed build for cheaper.
Depending on your circumstances, one of these things (speed vs. cost) might be much more important than the other. We cannot forget about either.
On the other hand, reductions in the amount of computation needed mean you get a faster build for the same resources, or a similar speed build for cheaper.
Depending on your circumstances, one of these things (speed vs. cost) might be much more important than the other. We cannot forget about either.
Except Amdahl’s law.
Replace "faster" with "more incremental".
Trying to get a program to quickly do the exact same thing over and over again is a colossal waste of resource.
Trying to get a program to quickly do the exact same thing over and over again is a colossal waste of resource.
When you’re trying to balance a bad budget, you can’t dismiss many of the cost centers and still succeed.
You can prioritize, but that makes the rest a matter of “when” not if.
It can be hard to tell from the outside if someone is avoiding a problem because they don’t want to solve it, or because they are hoping for a more inspired solution. Tackling a problem when all the options are bad may block another alternative when it does finally surface. I’d lump this in with irreversible decisions and the advice to delay them to the last responsible moment.
Who know what internal plumbing has to be moved around to make the compiler more incremental, parallel, or both.
You can prioritize, but that makes the rest a matter of “when” not if.
It can be hard to tell from the outside if someone is avoiding a problem because they don’t want to solve it, or because they are hoping for a more inspired solution. Tackling a problem when all the options are bad may block another alternative when it does finally surface. I’d lump this in with irreversible decisions and the advice to delay them to the last responsible moment.
Who know what internal plumbing has to be moved around to make the compiler more incremental, parallel, or both.
Yes, let's keep on all options on the table.
But trying to constant-factor optomize O(code I didn't change) is a ridiculous approach that needs justification, and the burden of proof is on those that advocate it. Yet, this seem to be the default thing people beg for.
Switching to incremental is hard programming, and that's why it hasn't happened overnight, but at least it is an algorithmically sane approach.
But trying to constant-factor optomize O(code I didn't change) is a ridiculous approach that needs justification, and the burden of proof is on those that advocate it. Yet, this seem to be the default thing people beg for.
Switching to incremental is hard programming, and that's why it hasn't happened overnight, but at least it is an algorithmically sane approach.
Yes, we should have a national politburo assign funds to Rust development. Although most of private industry (proles) disagrees, us in the Administration believe that Rust is superior. The proles--ahem I mean programmers with jobs--keep causing problems by using Unsafe technologies and being too dumb to use them. Rust should be funded by The People. It is a Public Good after all because using anything else would be an act of defiance against the will of The People.
> increasing the number of people paid to work on the Rust compiler by 10x would only mean hiring about 25 people. Compared to the size of the projects that are starting to depend on Rust, that's a rounding error.
And are you going to pay for the "rounding error" or just expecting someone to pay for millions a year for your idea?
And are you going to pay for the "rounding error" or just expecting someone to pay for millions a year for your idea?
It's not my idea. This is well-known.
As for who pays for it: that's a tricky issue! But if we get into a situation where (for example) 10,000 developers spend an hour a day waiting for builds, we know 50 developers could fix the issue in a year, but that doesn't happen because we can't figure out how to structure it economically --- that would be a failure of organizational imagination.
As for who pays for it: that's a tricky issue! But if we get into a situation where (for example) 10,000 developers spend an hour a day waiting for builds, we know 50 developers could fix the issue in a year, but that doesn't happen because we can't figure out how to structure it economically --- that would be a failure of organizational imagination.
This is such a low effort sneer-comment, it's a dismissive way to attack and putdown any idea that includes any amount of payment. It's the discussion equivalent of "whoever smelt it, dealt it" - if someone mentions an idea which costs money, demand as a first response if they are going to pay for it (assuming they aren't, instant dismissal), or accuse that the alternative is a kind of entitled and unfair expectation held of others, when neither need be the case.
"That volcano looks like it's becoming active, maybe some sensors could give us an early warning of problems" - Are YOU going to pay for your little "idea"??? Then WHO IS?
"That tree is getting dangeoursly high, it's at risk of falling down in a big storm this winter, maybe we could get it cut down before that" - Are YOU going to pay for it?!
"Dumping raw sewage in the river is making people sick, treating it first would take a small amount of space and a small fraction of the council's existing budget" - and you want to TAX ME for YOUR clean river, I suppose?!
As if OP is the only person who would benefit, as if automatically assuming the worst possible intent for who would pay for it, and as if "who would pay" is the first and only thing worth demanding an answer to, before even having a discussion about whether it's worth doing at all - and which of many ways it might be done.
"That volcano looks like it's becoming active, maybe some sensors could give us an early warning of problems" - Are YOU going to pay for your little "idea"??? Then WHO IS?
"That tree is getting dangeoursly high, it's at risk of falling down in a big storm this winter, maybe we could get it cut down before that" - Are YOU going to pay for it?!
"Dumping raw sewage in the river is making people sick, treating it first would take a small amount of space and a small fraction of the council's existing budget" - and you want to TAX ME for YOUR clean river, I suppose?!
As if OP is the only person who would benefit, as if automatically assuming the worst possible intent for who would pay for it, and as if "who would pay" is the first and only thing worth demanding an answer to, before even having a discussion about whether it's worth doing at all - and which of many ways it might be done.
If the idea involved more than "hire 25 people" then, sure, I might've said something better but just saying more people would solve the problem, is that even an idea?
[deleted]
I’m not sure why everyone cares so much about compile times compared to the only thing that matters - runtime performance.
For the lifetime of a binary, compilation time is like 0.000001% of its existence.
I’m more than happy to wait longer compared to other languages (even zero time at all with scripting languages) while cargo does it’s thing.
For the lifetime of a binary, compilation time is like 0.000001% of its existence.
I’m more than happy to wait longer compared to other languages (even zero time at all with scripting languages) while cargo does it’s thing.
Business reason: When you're in the business of building software, people are your biggest expense. You want to waste their time as little as possible.
Dev reason: tight feedback loops are encouraging, and make it easier to build momentum. I hate being deep in a problem then having to "wait" for a compile.
Dev reason: tight feedback loops are encouraging, and make it easier to build momentum. I hate being deep in a problem then having to "wait" for a compile.
But compile times have a massive effect on developer time, which has a massive effect on the productivity and the amount of work that can actually be done.
Comparing compile time to run time is rather strange, but compared as a percentage of a developer’s time, it’s very obvious why people are worried about it.
Comparing compile time to run time is rather strange, but compared as a percentage of a developer’s time, it’s very obvious why people are worried about it.
I think it's a bit of a shadow argument.
So for business, the logic goes:
Goal := save(Money).
time(Developer) := costs(Money).
Therefore, to achieve our Goal, we optimise developer time.
But more often than not, businesses don't know how their developers spend their time. So as such, they optimise for results. The business doesn't care how you spend your time, as long as by the end of Sprint/Deadline/Whatever you have a result and/or a reason why not. And I'd bet some money that the reason will very rarely be "my compiler took to long".
From a developer standpoint, I understand better (mainly because I'm a developer). But it still lacks a bit: So I started using Rust and I have a long history with C. The first compile with Rust takes very long (because of loading all dependencies), but afterwards it is slower than C, but it's okay. In return you get a lot of extra safety for your program, it's basically time you spent on your C program to fix all those nitty gritty memory safe optimisations and making sure there's no dangling pointers and free(NULL) errors. Of course with experience, these get less and less, but there's always risk. Nobody is perfect.
For me as a developer, it's more of a shift in my thinking - which is, to be fair, hard to enforce on people and can only come from oneself. But now that I have a compiler which supports me handling memory and pointers, my mind is more at ease. It does these things I spend time debugging on for me (most of them, not all) and in return for longer compile time, I save development/debugging time. At first it feels a bit less productive, since you are just sitting around watching your compilation (there's an XKCD for that https://xkcd.com/303/), but you could also spend this time writing documentation, reading up on something or just grabbing your coffee (or leave the office and check in tomorrow). Things you would do anyways, it's just a matter of how you mentally approach it.
I think the downside only comes when you want it to be negative, or when you're still transitioning.
But more often than not, businesses don't know how their developers spend their time. So as such, they optimise for results. The business doesn't care how you spend your time, as long as by the end of Sprint/Deadline/Whatever you have a result and/or a reason why not. And I'd bet some money that the reason will very rarely be "my compiler took to long".
From a developer standpoint, I understand better (mainly because I'm a developer). But it still lacks a bit: So I started using Rust and I have a long history with C. The first compile with Rust takes very long (because of loading all dependencies), but afterwards it is slower than C, but it's okay. In return you get a lot of extra safety for your program, it's basically time you spent on your C program to fix all those nitty gritty memory safe optimisations and making sure there's no dangling pointers and free(NULL) errors. Of course with experience, these get less and less, but there's always risk. Nobody is perfect.
For me as a developer, it's more of a shift in my thinking - which is, to be fair, hard to enforce on people and can only come from oneself. But now that I have a compiler which supports me handling memory and pointers, my mind is more at ease. It does these things I spend time debugging on for me (most of them, not all) and in return for longer compile time, I save development/debugging time. At first it feels a bit less productive, since you are just sitting around watching your compilation (there's an XKCD for that https://xkcd.com/303/), but you could also spend this time writing documentation, reading up on something or just grabbing your coffee (or leave the office and check in tomorrow). Things you would do anyways, it's just a matter of how you mentally approach it.
I think the downside only comes when you want it to be negative, or when you're still transitioning.
> increasing the number of people paid to work on the Rust compiler by 10x would only mean hiring about 25 people
I don't think there are 25 people qualified to do that work looking for jobs. It's not something you can just throw money at, you need really qualified people to do that kind of work. Those people are in very high demand.
I don't think there are 25 people qualified to do that work looking for jobs. It's not something you can just throw money at, you need really qualified people to do that kind of work. Those people are in very high demand.
Note that that would be increasing the number for pay, you already have many people who have the relevant qualifications because they’re already doing the work, just only in their spare time.
Second, the compiler team has put a ton of effort into mentorship infrastructure, and you can go from not knowing a ton to being a productive member of the team with their help. It "just" takes time and desire to do it.
Second, the compiler team has put a ton of effort into mentorship infrastructure, and you can go from not knowing a ton to being a productive member of the team with their help. It "just" takes time and desire to do it.
Interesting. What sort of mentorship infrastructure? What sort of people are they targeting for mentorship?
I'd love to learn about compilers by working on an actual real world compiler, but I really only know high level details about how different components fit together. I also don't have a formal CS education, although I am currently working as a senior level data engineer. Not sure if that is the sort of person they are looking to mentor.
I'd love to learn about compilers by working on an actual real world compiler, but I really only know high level details about how different components fit together. I also don't have a formal CS education, although I am currently working as a senior level data engineer. Not sure if that is the sort of person they are looking to mentor.
rust-lang/rust on github, go to issues, and click on the "easy" and/or "mentor" labels. That will show you all the easy tasks with a mentor available.
Just find one that you'd like to work on, write that you are going to work on it, and start doing the work, and asking lots of questions.
Just find one that you'd like to work on, write that you are going to work on it, and start doing the work, and asking lots of questions.
Or on some other project.
Sure, there are probably some projects for which if the primary author leaves, their coworkers will drop it or rewrite it in another language, but for others the management will realize what a liability it was to have all of their eggs in that one basket and they’ll have to hire 2 to replace them.
If you steal 25 to work on the kernel, you will likely see 25 open reqs to replace them. And more people will work with languages that they have seen reqs for.
Sure, there are probably some projects for which if the primary author leaves, their coworkers will drop it or rewrite it in another language, but for others the management will realize what a liability it was to have all of their eggs in that one basket and they’ll have to hire 2 to replace them.
If you steal 25 to work on the kernel, you will likely see 25 open reqs to replace them. And more people will work with languages that they have seen reqs for.
There are a lot of people out there who can do this kind of work. I know a lot of such people. Many of them don't have traditional CVs.
Yes, most of them are employed. Some are under-employed. Some are well employed but might be keen to work on a project this important.
Yes, most of them are employed. Some are under-employed. Some are well employed but might be keen to work on a project this important.
There are certainly way more than 25 skilled programmers who are experts in compilers that could be hired easily by offering enough money (on the order of several hundred thousand dollars per year).
> a Manhattan Project to build a fast Rust compiler
Or work on getting what Rust gives you in a different language that isn't Rust (maybe entirely new, maybe not[1]).
One underdiscussed thing about Rust is that it's so clearly a byproduct of the C++ world. And the most disappointing thing about Rust is that it probably won't ever be seen as a prototype that should be thrown away.
1. http://mbeddr.com/
Or work on getting what Rust gives you in a different language that isn't Rust (maybe entirely new, maybe not[1]).
One underdiscussed thing about Rust is that it's so clearly a byproduct of the C++ world. And the most disappointing thing about Rust is that it probably won't ever be seen as a prototype that should be thrown away.
1. http://mbeddr.com/
If you want to convince people that Rust should be thrown away in favour of mbeddr or something else, you need to make an argument based on specific design flaws of Rust, not just tar it by association with C++.
That argument would have to not just explain why an improved language is needed, but also why Rust can't evolve into that language via the edition system. It would also have to convince people that whatever benefits the improved language brings justify moving away from the existing crate ecosystem, tool ecosystem, and developer pool, and building new ones (or how you would reuse/extend/coopt them).
(In reality I think the influence of Haskell and even node.js were as important to Rust as C++.)
That argument would have to not just explain why an improved language is needed, but also why Rust can't evolve into that language via the edition system. It would also have to convince people that whatever benefits the improved language brings justify moving away from the existing crate ecosystem, tool ecosystem, and developer pool, and building new ones (or how you would reuse/extend/coopt them).
(In reality I think the influence of Haskell and even node.js were as important to Rust as C++.)
>not just tar it by association with C++.
- A syntax that's highly familiar to the legions of C++ programmers already out there
- A bunch of features C++ has had in the works but that aren't quite there yet (specifically, a proper module system comes to mind)
- Features from ML/OCaml/Haskell, like a first-class option type, first-class (powerful) pattern matching, and an emphasis on immutable-by-default variables
Having this without the cruft of the decades of backwards-compatibility that C++ has had (not without good reason, granted) makes Rust the hard refresh of C++ that I and hopefully others have been waiting for for a while.
.NET has these features for decades. It compiles really fast, much easier to use than C++, and has awesome IDEs.
C interop is pretty much same as in Rust.
GC is not an issue in modern .NET even for resource-demanding apps running on slow ARM CPUs. Here's an example: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo BTW it consumes non-trivial amount of C APIs like V4L2, ALSA, and audio decoders.
C interop is pretty much same as in Rust.
GC is not an issue in modern .NET even for resource-demanding apps running on slow ARM CPUs. Here's an example: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo BTW it consumes non-trivial amount of C APIs like V4L2, ALSA, and audio decoders.
The reasons .NET never really competed with C# for the longest time were a handful:
- First-party support for .NET only existed on Windows systems until 2015ish with the release of Rosetta
- Third party support across platforms via Mono never really picked up steam for a number of reasons
- Early on, .NET code, much like JVM code, wasn't in the same league as C++ in terms of performance
- I'd like to be proven wrong on this one, but .NET remains a poor framework to do systems programming in. GC is IMO still a factor because even though modern M&S GCs are impressively performant they still add a degree of unpredictability to your runtime, and sometimes that's not acceptable — so reference counting or something even more basic is preferred, if only for the consistency.
- First-party support for .NET only existed on Windows systems until 2015ish with the release of Rosetta
- Third party support across platforms via Mono never really picked up steam for a number of reasons
- Early on, .NET code, much like JVM code, wasn't in the same league as C++ in terms of performance
- I'd like to be proven wrong on this one, but .NET remains a poor framework to do systems programming in. GC is IMO still a factor because even though modern M&S GCs are impressively performant they still add a degree of unpredictability to your runtime, and sometimes that's not acceptable — so reference counting or something even more basic is preferred, if only for the consistency.
> .NET remains a poor framework to do systems programming in
Have you clicked that link? Essentially, I’ve re-implemented parts of ffmpeg’s libavformat and libavcodec in C#. On Raspberry Pi, the software uses 15% of CPU time while playing 12 mbit/sec 1080p video, on par with VLC, while using 20% less RAM than VLC. If that’s not a system programming, then what is?
BTW, I don’t think I allocate any objects on heap while playing the video, therefore no GC is involved, and no degree of unpredictability is introduced. Wasn’t hard at all, just recycling buffers between frames.
Have you clicked that link? Essentially, I’ve re-implemented parts of ffmpeg’s libavformat and libavcodec in C#. On Raspberry Pi, the software uses 15% of CPU time while playing 12 mbit/sec 1080p video, on par with VLC, while using 20% less RAM than VLC. If that’s not a system programming, then what is?
BTW, I don’t think I allocate any objects on heap while playing the video, therefore no GC is involved, and no degree of unpredictability is introduced. Wasn’t hard at all, just recycling buffers between frames.
Object recycling often works fine to avoid GC issues for simple stuff, but in more complicated applications you really do need to dynamically allocate and free memory and GC is going to get involved.
> for simple stuff, but in more complicated applications
Media engine part of the library has 45k lines of code, this is mostly due to the complexity of V4L2 API, and of the MKV and Mpeg4 containers. I wouldn’t call that “simple stuff”.
> you really do need to dynamically allocate and free memory
Memory allocation in .NET ain’t limited to GC.
First of all there’s stack. I use Spans + stackalloc in quite a few places there. Modern .NET even has these `ref structs` where the compiler guarantees they are never accidentally boxed into the heap.
Another thing, no one stops you from using malloc/free, like I did in that class: https://github.com/Const-me/Vrmac/blob/master/VrmacVideo/Aud...
Finally, the mere presence of GC doesn’t affect performance in any way, only usage of GC does. Even for latency-sensitive applications like that media engine it’s OK to allocate stuff on managed heap, as long as it only happens on startup, or otherwise rarely. I allocate quite a lot of stuff on the heap, megabytes, when opening a media file.
Media engine part of the library has 45k lines of code, this is mostly due to the complexity of V4L2 API, and of the MKV and Mpeg4 containers. I wouldn’t call that “simple stuff”.
> you really do need to dynamically allocate and free memory
Memory allocation in .NET ain’t limited to GC.
First of all there’s stack. I use Spans + stackalloc in quite a few places there. Modern .NET even has these `ref structs` where the compiler guarantees they are never accidentally boxed into the heap.
Another thing, no one stops you from using malloc/free, like I did in that class: https://github.com/Const-me/Vrmac/blob/master/VrmacVideo/Aud...
Finally, the mere presence of GC doesn’t affect performance in any way, only usage of GC does. Even for latency-sensitive applications like that media engine it’s OK to allocate stuff on managed heap, as long as it only happens on startup, or otherwise rarely. I allocate quite a lot of stuff on the heap, megabytes, when opening a media file.
Funny, before your comment showed up, I was actually going to edit in an addendum to link to
https://robert.ocallahan.org/2018/10/the-costs-of-programmin...
... to pair with the mbeddr link.
You seem to be at a place where you think of Rust as already entrenched, but I'm at a place where I still see Rust as a proposition—one that could lead to that place, or not if we think better and nix it (following the reasoning above).
https://robert.ocallahan.org/2018/10/the-costs-of-programmin...
... to pair with the mbeddr link.
You seem to be at a place where you think of Rust as already entrenched, but I'm at a place where I still see Rust as a proposition—one that could lead to that place, or not if we think better and nix it (following the reasoning above).
I don't think Rust is going away. You may not like it; it may not be adopted by the Linux kernel. But it is widely used in real codebases (including, say, Firefox). It's not brand new. And the article linked above discourages creating new languages; valid or not, that horse is well out of the barn.
The post as written, and whether intended or not (admittedly by current signs, I guess it'd be foolish to wager anything other than "not"), ends with an observation that's still highly relevant here but that you're neglecting to account for.
> I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible.
And being widely used is not the same thing as being entrenched, not the same thing as being unable to say, "nice prototype; let's wrap this up now and do it for real this time".
> I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible.
And being widely used is not the same thing as being entrenched, not the same thing as being unable to say, "nice prototype; let's wrap this up now and do it for real this time".
> ends with an observation that's still highly relevant here but that you're neglecting to account for.
No, that's kind of the sentence I was responding to. Rust is already popular; that horse is out of the barn.
> And being widely used is not the same thing as being entrenched, not the same thing as being unable to say, "nice prototype; let's wrap this up now and do it for real this time".
I don't care to argue about the first clause, but I don't understand how you can argue the latter. Who is going to say "nice prototype; let's discard this?" Do you expect the Rust-using projects to just fold? Do you expect Rust developers to just disappear? Again, it sounds like you don't like Rust, which is fine, but as someone external to the community, why do you think anyone would listen to you about spinning down Rust?
No, that's kind of the sentence I was responding to. Rust is already popular; that horse is out of the barn.
> And being widely used is not the same thing as being entrenched, not the same thing as being unable to say, "nice prototype; let's wrap this up now and do it for real this time".
I don't care to argue about the first clause, but I don't understand how you can argue the latter. Who is going to say "nice prototype; let's discard this?" Do you expect the Rust-using projects to just fold? Do you expect Rust developers to just disappear? Again, it sounds like you don't like Rust, which is fine, but as someone external to the community, why do you think anyone would listen to you about spinning down Rust?
As someone who's spent the last few years writing a 170K line Rust project ... I consider it entrenched!
I stand by that blog post, though I recognise it's controversial. I think Rust clearly qualifies as "worth the cost of a new language". There simply wasn't, and still isn't, any other serious contender for safe GC-free systems programming. New contenders will arise but in the meantime lots of projects are doing safe systems programming that without Rust would have been done in C or C++ and would have had to be unsafe.
I stand by that blog post, though I recognise it's controversial. I think Rust clearly qualifies as "worth the cost of a new language". There simply wasn't, and still isn't, any other serious contender for safe GC-free systems programming. New contenders will arise but in the meantime lots of projects are doing safe systems programming that without Rust would have been done in C or C++ and would have had to be unsafe.
Even if the benefits are "worth the cost of a new language", is using the new language worth the cost of it being Rust?
You have not made your argument clear.
Are you questioning whether Rust is simply not good enough to be worth using by anyone? Lots of people, including me, have judged that is worth using for us, both before we tried it and after we've put Rust projects into production.
(BTW once again you have implied there is something deeply wrong with Rust that cancels out its benefits, without saying what that might be. That is annoying.)
Are you questioning whether Rust is simply not good enough to be worth using by anyone? Lots of people, including me, have judged that is worth using for us, both before we tried it and after we've put Rust projects into production.
(BTW once again you have implied there is something deeply wrong with Rust that cancels out its benefits, without saying what that might be. That is annoying.)
You're not seeking answers; in place of anything that would lead to a better understanding, you're asking the questions that make it easier to defend the position you've staked out. (Great job, Brendan!)
If someone released a memory-safe, highly concurrent, systems-oriented version of INTERCAL, you recognize that someone might look at that and say, "Gee, I don't think that's really worth using", and the comment's lack of detail doesn't nullify the substance of what makes the comment true or not? And an opponent could approach in the same way you are now, with the primary goal of "beating" the other participant—by making it burdensome to respond—and by all appearances do so?
It's not my job to painstakingly prepare a perfect, incontrovertible response for a hostile adversary, particularly since absolutely nothing would come of that time investment. If your idea of success is the sound of all your colleagues telling yourselves that you won the conversation and you're on the right track and nothing needs to change because there really aren't any issues... then, hey, that's just fine.
If someone released a memory-safe, highly concurrent, systems-oriented version of INTERCAL, you recognize that someone might look at that and say, "Gee, I don't think that's really worth using", and the comment's lack of detail doesn't nullify the substance of what makes the comment true or not? And an opponent could approach in the same way you are now, with the primary goal of "beating" the other participant—by making it burdensome to respond—and by all appearances do so?
It's not my job to painstakingly prepare a perfect, incontrovertible response for a hostile adversary, particularly since absolutely nothing would come of that time investment. If your idea of success is the sound of all your colleagues telling yourselves that you won the conversation and you're on the right track and nothing needs to change because there really aren't any issues... then, hey, that's just fine.
You asserted that Rust should be thrown away, but decline to provide any reasons why you think that. So yeah, in the absence of any supporting argument, I reject your claim, and so would any other reasonable person who doesn't already believe it. You have added nothing to the discussion.
> You asserted that Rust should be thrown away, but decline to provide any reasons why you think that.
That simply isn't true.
> I reject your claim
That's fine.
That simply isn't true.
> I reject your claim
That's fine.
• Rust is an ML-family language dressed in a C-family syntax to look palatable to existing systems programmers. It's a byproduct of the C++ world only to the extent that it has to work on the CPU architectures and operating systems influenced by C and C++.
• Rust is mainly based on established research and existing languages from 80s and 90s. It's a distilled version of these, not a prototype.
I recommend checking out the first pitch deck for Rust: http://venge.net/graydon/talks/intro-talk-2.pdf
• Rust is mainly based on established research and existing languages from 80s and 90s. It's a distilled version of these, not a prototype.
I recommend checking out the first pitch deck for Rust: http://venge.net/graydon/talks/intro-talk-2.pdf
RAII-based memory management without GC, and the concept of ownership vs. borrowing are very C++-inspired concepts. The borrow checker, which is the main innovation of Rust, is automated checking of things C++ programmers spend a lot of effort thinking about.
I think the influence of C++ is much more than just the syntax.
I think the influence of C++ is much more than just the syntax.
Isn't this more of an artifact of our memory architecture and the way we (de-)allocate memory? In that sense, the C++ flying-by-the-seat-of-your-pants approach and Rust's borrow checker solve the same problem, and I'm sure a lot of C++ insight has made it into the ideas behind the borrow checker, but I'm not sure that makes Rust in a way derivative of ideas extant in C++-land.
To me, the borrow checker just feels like an additional type layer I have to reason and think about. In that sense, my thinking when reasoning about lifetimes in Rust is a lot more along the same lines I do in Haskell than something I would be doing in C++ (which I refuse to code in, I'm not capable of writing secure C/++ code.)
To me, the borrow checker just feels like an additional type layer I have to reason and think about. In that sense, my thinking when reasoning about lifetimes in Rust is a lot more along the same lines I do in Haskell than something I would be doing in C++ (which I refuse to code in, I'm not capable of writing secure C/++ code.)
I think the programs that fit neatly into Rust's borrow checking patterns are the same ones that fit neatly into C++(11)'s approach to managing object lifetimes (move/value semantics).
The question is what happens when programs don't fit those patterns. Inevitably in any large real world application, there are some objects with lifetimes that need to be managed carefully and don't match conveniently with some sort of scope in a program.
With C++, this is where one would use std::shared_ptr, or something else. In rust, there's Rc. But either way, you've now stepped into a territory where object lifetimes and bugs can be messy.
The question is what happens when programs don't fit those patterns. Inevitably in any large real world application, there are some objects with lifetimes that need to be managed carefully and don't match conveniently with some sort of scope in a program.
With C++, this is where one would use std::shared_ptr, or something else. In rust, there's Rc. But either way, you've now stepped into a territory where object lifetimes and bugs can be messy.
I do not find reference counting problematic in practice. In Rust you have to go out of your way to introduce interior mutability in refcounted values, which discourages problematic patterns.
I find that in most cases where ownership patterns don't fit the Rust model, you can abstract the problematic parts into some kind of collection type and hide them in its own crate --- or better still, find an existing crate that does what you need.
The only remotely common case that still doesn't get handled well, in my experience, is the self-referential struct pattern --- when a value needs to contain references to other parts of the same value.
I find that in most cases where ownership patterns don't fit the Rust model, you can abstract the problematic parts into some kind of collection type and hide them in its own crate --- or better still, find an existing crate that does what you need.
The only remotely common case that still doesn't get handled well, in my experience, is the self-referential struct pattern --- when a value needs to contain references to other parts of the same value.
> The only remotely common case that still doesn't get handled well, in my experience, is the self-referential struct pattern --- when a value needs to contain references to other parts of the same value.
There's a basic reason for that, namely you can't memcpy a self-referential structure while keeping desired semantics. Rust now has a Pin<> feature to mark data that should not be moved in memory, but its use is still unintuitive and IIRC requires unsafe. In many cases one should perhaps avoid references to self-addresses entirely; often they can be replaced by indexes and/or offsets.
There's a basic reason for that, namely you can't memcpy a self-referential structure while keeping desired semantics. Rust now has a Pin<> feature to mark data that should not be moved in memory, but its use is still unintuitive and IIRC requires unsafe. In many cases one should perhaps avoid references to self-addresses entirely; often they can be replaced by indexes and/or offsets.
> There's a basic reason for that, namely you can't memcpy a self-referential structure while keeping desired semantics.
You actually can! For instance, consider a struct with two fields: a Vec<T>, and a &T which references one of the elements of the Vec. Moving the struct (which does the memcpy) does not reallocate the Vec, so the reference in the &T would still be valid. But as far as I know there's no way to represent that in safe Rust; the &T in the struct requires a lifetime, and there's no way to represent "the lifetime of the other field" or even "the lifetime of the struct itself" (something like &'self T).
Yeah, you can use an index in my example (replacing the &T by an usize, and looking up on the Vec every time), but it's not a perfect replacement. A reference would always be valid (and always pointing to the same element), while an index can become invalid if the Vec is truncated, or point to a wrong element if an element is inserted or removed before it in the Vec.
You actually can! For instance, consider a struct with two fields: a Vec<T>, and a &T which references one of the elements of the Vec. Moving the struct (which does the memcpy) does not reallocate the Vec, so the reference in the &T would still be valid. But as far as I know there's no way to represent that in safe Rust; the &T in the struct requires a lifetime, and there's no way to represent "the lifetime of the other field" or even "the lifetime of the struct itself" (something like &'self T).
Yeah, you can use an index in my example (replacing the &T by an usize, and looking up on the Vec every time), but it's not a perfect replacement. A reference would always be valid (and always pointing to the same element), while an index can become invalid if the Vec is truncated, or point to a wrong element if an element is inserted or removed before it in the Vec.
Right, I know what the reason is, but that doesn't stop it from being a problem.
Maybe that has more to do with almost everything new have been garbage collected for the last decades (and that is such a tragedy) and less to do with C++.
It is a low-level programming language after all. Not being explicit about memory would be a bad thing (it almost always is).
It is a low-level programming language after all. Not being explicit about memory would be a bad thing (it almost always is).
I was under the impression that RAII specifically (as opposed to manually calling malloc and free by hand) was an innovation of C++, but I could be wrong.
Ada and Object Pascal also have it.
Don't know, but the concept is very useful and has found its way into many languages, such as thr using statement in C#.
It is just that it is typically more useful/powerful in languages that lack a GC (doesn't necessarily need to be that way though, but my impression).
It is just that it is typically more useful/powerful in languages that lack a GC (doesn't necessarily need to be that way though, but my impression).
C# using is Common Lisp's with-....
[deleted]
> RAII-based memory management without GC, and the concept of ownership vs. borrowing are very C++-inspired concepts.
Why are you crediting C++ for an idea that is probably much older than it?
Why are you crediting C++ for an idea that is probably much older than it?
I think that's how it started out (and the compiler was first written in Ocaml), but the language has largely abandoned that heritage: the C++ camp is squarely in charge.
In the beginning, I feel like Rust aimed to be a general purpose language, but not anymore. Really the only reason why anyone would consider Rust for a project is if GC was unacceptable. And the amount of complexity in the language just to avoid that is truly staggering.
There aren't that many situations where GC can't be tolerated (and I think some would argue there are zero situations), so I don't actually see Rust as a particularly useful language, and certainly not one in the ML tradition.
Moreover, most zero allocation C code is a total lie. These code bases are littered with arena based allocator that burden the project with all the problems of GC and none of the benefits.
GC works for operating systems, they work for hard real time systems, they work for mostly everything except severely constrained embedded systems.
In the beginning, I feel like Rust aimed to be a general purpose language, but not anymore. Really the only reason why anyone would consider Rust for a project is if GC was unacceptable. And the amount of complexity in the language just to avoid that is truly staggering.
There aren't that many situations where GC can't be tolerated (and I think some would argue there are zero situations), so I don't actually see Rust as a particularly useful language, and certainly not one in the ML tradition.
Moreover, most zero allocation C code is a total lie. These code bases are littered with arena based allocator that burden the project with all the problems of GC and none of the benefits.
GC works for operating systems, they work for hard real time systems, they work for mostly everything except severely constrained embedded systems.
We have a 170K line project written in Rust (https://pernos.co). I've written a lot of C++ in the past, but also Java and other things (I've been programming for nearly 40 years now), so I'm familiar in developing with and without GC. Pernosco is mostly a server app that could use GC. I'm happy with the choice of Rust.
> And the amount of complexity in the language just to avoid that is truly staggering.
In terms of day-to-day coding this isn't my experience. Once in a while I have tricky ownership problems to resolve, but most of the time I get by just fine without thinking much about the ownership model.
In exchange we don't have to think at all about GC tuning or object recycling, we get RAII, we get to use affine types to encode typestates, and we get easy parallelism. I'm happy about this trade.
As you can see when I started this thread I am far more concerned about build times than I am about the cognitive overhead of ownership and lifetimes.
> And the amount of complexity in the language just to avoid that is truly staggering.
In terms of day-to-day coding this isn't my experience. Once in a while I have tricky ownership problems to resolve, but most of the time I get by just fine without thinking much about the ownership model.
In exchange we don't have to think at all about GC tuning or object recycling, we get RAII, we get to use affine types to encode typestates, and we get easy parallelism. I'm happy about this trade.
As you can see when I started this thread I am far more concerned about build times than I am about the cognitive overhead of ownership and lifetimes.
I have to say my experience with the language has just been very different from this. It absolutely feels like a general-purpose language to me. I actually prefer Rust over Python, as I find the language simpler to use and anywhere from 10-100x faster (if not much more than that in some cases). cargo is much better than pip, and the Rust compiler is much smarter about guiding me than something like mypy.
Maybe I'm unusual in some way, but I do not find Rust hard to use once you've learned the language. You say the complexity to avoid a GC is staggering, but it seems straightforward to me. You have to appease the borrow checker, and you need to learn about some tools like Rc, Arc, Cell, etc., that provide interior mutability. Other than that, the language has been an absolute pleasure and dream to use.
I tend to write Rust the way I write Haskell, not the way I write C++. I start by sketching out all the types that I need to model the problem space at hand. This can be done almost 1:1 with Haskell in many cases. Then I start filling in the implementation, getting me from "input" types to "output" types and only very rarely running into any kind of issue with the borrow checker. Of course, instead of using immutable data structures (as is common in Haskell), I use mutable ones in Rust. The compiler guides me the entire way, and often it feels like an entire program writes itself.
As for GCs, I almost never want one. Performance matters in almost everything I write these days, and faster is always better. Rust is so easy to write once you learn it that I see no reason to accept a GC in 2020 and beyond for any software where performance matters. I do not write software for severely constrained embedded systems. I mostly write real-time backend services that process data or handle large numbers of clients. I could tolerate a GC in much of this, but there's no reason to in a world where Rust exists, and everything is faster, more responsive, and considerably easier to reason about as a result. A few years ago, I would've used a mixture of C++ and Go. Now, I'd just use Rust for all of it.
Maybe I'm unusual in some way, but I do not find Rust hard to use once you've learned the language. You say the complexity to avoid a GC is staggering, but it seems straightforward to me. You have to appease the borrow checker, and you need to learn about some tools like Rc, Arc, Cell, etc., that provide interior mutability. Other than that, the language has been an absolute pleasure and dream to use.
I tend to write Rust the way I write Haskell, not the way I write C++. I start by sketching out all the types that I need to model the problem space at hand. This can be done almost 1:1 with Haskell in many cases. Then I start filling in the implementation, getting me from "input" types to "output" types and only very rarely running into any kind of issue with the borrow checker. Of course, instead of using immutable data structures (as is common in Haskell), I use mutable ones in Rust. The compiler guides me the entire way, and often it feels like an entire program writes itself.
As for GCs, I almost never want one. Performance matters in almost everything I write these days, and faster is always better. Rust is so easy to write once you learn it that I see no reason to accept a GC in 2020 and beyond for any software where performance matters. I do not write software for severely constrained embedded systems. I mostly write real-time backend services that process data or handle large numbers of clients. I could tolerate a GC in much of this, but there's no reason to in a world where Rust exists, and everything is faster, more responsive, and considerably easier to reason about as a result. A few years ago, I would've used a mixture of C++ and Go. Now, I'd just use Rust for all of it.
> There aren't that many situations where GC can't be tolerated (and I think some would argue there are zero situations)
Those people would be wrong. As an extreme example: GPU programming, where GC is essentially impossible due to the architecture.
Those people would be wrong. As an extreme example: GPU programming, where GC is essentially impossible due to the architecture.
Can you elaborate on the underdiscussion of Rust as a byproduct of the C++ world?
I'm into Rust, so I guess I assumed it was common knowledge that it was originally written to be like OCaml with better concurrency, and that one of the early goals was to replace C++ in Mozilla code.
But, language-wise, I don't even see that much similarity with C++ except for the philosophy of zero-cost abstraction and the trivial syntax-style of generics being <>.
I'm into Rust, so I guess I assumed it was common knowledge that it was originally written to be like OCaml with better concurrency, and that one of the early goals was to replace C++ in Mozilla code.
But, language-wise, I don't even see that much similarity with C++ except for the philosophy of zero-cost abstraction and the trivial syntax-style of generics being <>.
Rust certainly has C++ attitude to performance, for example 'zero-cost abstractions'. That precludes GC. But this is more indirect influence: targeting the same niche
> That precludes GC
This isn't the case, there are tracing garbage collectors implemented as libraries [1][2], and there is consideration being made for supporting tracing GC in the language and stdlib [0]. As well as reference counted GC having been available in stdlib for a long time [3] (similar to C++'s std::shared_ptr)
[0] https://github.com/rust-lang/rfcs/blob/master/text/1398-kind...
[1] https://manishearth.github.io/blog/2016/08/18/gc-support-in-...
[2] https://boats.gitlab.io/blog/post/shifgrethor-i/
[3] https://doc.rust-lang.org/std/rc/index.html
This isn't the case, there are tracing garbage collectors implemented as libraries [1][2], and there is consideration being made for supporting tracing GC in the language and stdlib [0]. As well as reference counted GC having been available in stdlib for a long time [3] (similar to C++'s std::shared_ptr)
[0] https://github.com/rust-lang/rfcs/blob/master/text/1398-kind...
[1] https://manishearth.github.io/blog/2016/08/18/gc-support-in-...
[2] https://boats.gitlab.io/blog/post/shifgrethor-i/
[3] https://doc.rust-lang.org/std/rc/index.html
It precludes pervasive or mandatory GC
[deleted]
I can elaborate, but first: are you a C++ programmer?
I guess some other people here are assuming malintent with your question. Assuming that the answer might be condescending or dismissive depending on how I answer. I'll assume that's not the case and that you simply want to know so you can answer with relevant examples, etc.
I did write C++ code for about 7 years. It was mostly C++98 and C++11.
Since then I've also done a fair bit of Rust. I'm also very familiar with Swift, Kotlin, Java, some lisps, JS, Go, and a few others. So I have enough breadth in language experience for any compare/contrasts you'd like to do.
I did write C++ code for about 7 years. It was mostly C++98 and C++11.
Since then I've also done a fair bit of Rust. I'm also very familiar with Swift, Kotlin, Java, some lisps, JS, Go, and a few others. So I have enough breadth in language experience for any compare/contrasts you'd like to do.
> I'll assume that's not the case
Thanks, that was actually the right answer! (And those assuming otherwise are not just getting the wrong answer, but I'll point out are breaking the rules[1] here, too.)
The C++-to-Rust background is what I suspected—but the reason had nothing to do with being dismissive about not having the right buzzwords to prove you're smart and well-rounded. (This isn't a job interview, folks, geez.) My instinct was a presumption of deep familiarity.
When I mention Rust having the marks of a being from the C++ world, I'm referring to a property of the language that it shares with Swift. When you say you don't see much similarity, my suspicion is that your first consideration is language semantics and the new affordances, and so when you think of Rust and C++, you're seeing foremost all the ways that they're different, rather than seeing the things that they share. I saw where Hejlsberg once said that he doesn't get why people draw comparisons between C# and Java, and then gives a list of reasons—where those reasons show that his main focus is similar, i.e. he focuses on the way that they differ (and maybe in his case, all the things that he "fixed" with C# in comparison). One of your sibling commenters writes that "Rust is an ML-family language dressed in a C-family syntax", but that's a little off the mark. Rust certainly has a C++-style syntax, but aside from curly braces, idiomatic Rust looks alien when compared to C.
1. https://news.ycombinator.com/newsguidelines.html
Thanks, that was actually the right answer! (And those assuming otherwise are not just getting the wrong answer, but I'll point out are breaking the rules[1] here, too.)
The C++-to-Rust background is what I suspected—but the reason had nothing to do with being dismissive about not having the right buzzwords to prove you're smart and well-rounded. (This isn't a job interview, folks, geez.) My instinct was a presumption of deep familiarity.
When I mention Rust having the marks of a being from the C++ world, I'm referring to a property of the language that it shares with Swift. When you say you don't see much similarity, my suspicion is that your first consideration is language semantics and the new affordances, and so when you think of Rust and C++, you're seeing foremost all the ways that they're different, rather than seeing the things that they share. I saw where Hejlsberg once said that he doesn't get why people draw comparisons between C# and Java, and then gives a list of reasons—where those reasons show that his main focus is similar, i.e. he focuses on the way that they differ (and maybe in his case, all the things that he "fixed" with C# in comparison). One of your sibling commenters writes that "Rust is an ML-family language dressed in a C-family syntax", but that's a little off the mark. Rust certainly has a C++-style syntax, but aside from curly braces, idiomatic Rust looks alien when compared to C.
1. https://news.ycombinator.com/newsguidelines.html
> When I mention Rust having the marks of a being from the C++ world, I'm referring to a property of the language that it shares with Swift.
But what property is that?
> I saw where Hejlsberg once said that he doesn't get why people draw comparisons between C# and Java [...]
That's very amusing!
> Rust certainly has a C++-style syntax, but aside from curly braces, idiomatic Rust looks alien when compared to C.
For sure. But I'm still not sure why you say it's from C++ land. It's definitely more like C++ than C, but JavaScript is also probably more like C++ than C. That doesn't mean JavaScript is actually anything like C++, really.
One thing that I thought of that Rust definitely does share with C++ (and C) that other languages (Swift) don't is that you don't have to decide at class definition if a type will be heap allocated or used as a reference vs value, etc. That's decided at the use-site.
But what property is that?
> I saw where Hejlsberg once said that he doesn't get why people draw comparisons between C# and Java [...]
That's very amusing!
> Rust certainly has a C++-style syntax, but aside from curly braces, idiomatic Rust looks alien when compared to C.
For sure. But I'm still not sure why you say it's from C++ land. It's definitely more like C++ than C, but JavaScript is also probably more like C++ than C. That doesn't mean JavaScript is actually anything like C++, really.
One thing that I thought of that Rust definitely does share with C++ (and C) that other languages (Swift) don't is that you don't have to decide at class definition if a type will be heap allocated or used as a reference vs value, etc. That's decided at the use-site.
C++ programmer of ~10 years here (Google Chrome, mostly).
Rust feels very very much like C++ to me. All the memory ownership stuff is basically common patterns in modern C++ pulled down into the compiler and enforced.
Also arguing with the compiler about types feels very familiar from C++ :-/.
Rust feels very very much like C++ to me. All the memory ownership stuff is basically common patterns in modern C++ pulled down into the compiler and enforced.
Also arguing with the compiler about types feels very familiar from C++ :-/.
That's a good point. The ownership patterns are basically compiler-enforced best-practices of C++. Fair enough.
Sure, so, what is the actual relationship you see between C++ and Rust?
It sounds like you're arguing that idiomatic Rust looks alien compared to C, which I'd agree with - I'd also argue it's alien compared to C++. But I assume you disagree with that?
It sounds like you're arguing that idiomatic Rust looks alien compared to C, which I'd agree with - I'd also argue it's alien compared to C++. But I assume you disagree with that?
Not to throw a mean comment, but i think maybe it can help you in your life by giving you an external feedback :
i hope you don't have this kind of attitude in real life, because it instantly makes you look like an pretentious prick and loose any credit. Your original comment made me curious, and this one loose any interest in what your point of view actually is.
(fyi : HN is (mostly) a developer community with usually knowledgeable people working in the field. if you don't think explaining yourself here is worth it, i have no idea where you think you're going to have the opportunity to give your opinion)
i hope you don't have this kind of attitude in real life, because it instantly makes you look like an pretentious prick and loose any credit. Your original comment made me curious, and this one loose any interest in what your point of view actually is.
(fyi : HN is (mostly) a developer community with usually knowledgeable people working in the field. if you don't think explaining yourself here is worth it, i have no idea where you think you're going to have the opportunity to give your opinion)
> i hope you don't have this kind of attitude in real life [...] you look like an pretentious prick
Yeesh. I don't even know what attitude you're referring to. You've set out to give some life advice, but do you know how real life conversations work? Can you understand that my response will differ greatly depending on what the answer to that question is? It's pretty normal for dialogues to be... dialogues, which includes checking in with each other instead of monologuing based on a lack of shared agreement.
> if you don't think explaining yourself here is worth it
Once again, what are you even talking about? You're responding to something you've imagined I've said or something you want to attribute to me, not what I've actually said or am thinking.
Can you wait for ragnese's reply and then my response before going off the deep end?
Yeesh. I don't even know what attitude you're referring to. You've set out to give some life advice, but do you know how real life conversations work? Can you understand that my response will differ greatly depending on what the answer to that question is? It's pretty normal for dialogues to be... dialogues, which includes checking in with each other instead of monologuing based on a lack of shared agreement.
> if you don't think explaining yourself here is worth it
Once again, what are you even talking about? You're responding to something you've imagined I've said or something you want to attribute to me, not what I've actually said or am thinking.
Can you wait for ragnese's reply and then my response before going off the deep end?
From the guidelines:
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Ironically, by assuming bad faith and a condescending tone in the comment you responded to, you've made a legitimately condescending comment.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Ironically, by assuming bad faith and a condescending tone in the comment you responded to, you've made a legitimately condescending comment.
Personally, I think the put-down was quite on point. I don't see the point of flagging it, that just leaves the content to imagination.
Anyone can see flagkilled comments, or any [dead] comments, by turning 'showdead' on in their profile.
Just be warned that most of what you'll see is comments by banned users. Occasionally we get emails from people who don't realize that and somehow assume that moderators are condoning the worst dreck that appears there.
Just be warned that most of what you'll see is comments by banned users. Occasionally we get emails from people who don't realize that and somehow assume that moderators are condoning the worst dreck that appears there.
> I don't even see that much similarity with C++ except for the philosophy of zero-cost abstraction
That is not really an explicit goal of C++, which is part of the problem rust is trying solve.
That is not really an explicit goal of C++, which is part of the problem rust is trying solve.
Rust is actually a byproduct of OCaml and to a lesser extent Haskell. This becomes obvious the more you use it, especially if you’ve written substantial code in either of those languages before. To really drive this home, I’d wager that you have a better chance of translating OCaml or non-fancy Haskell straight into Rust than you do C++.
> I’d wager that you have a better chance of translating OCaml or non-fancy Haskell straight into Rust than you do C++.
I found that translating OCaml code to Rust is frustrating, because you cannot easily and cheaply replicate the functional approach. It's much more Rustic to take an object by mutable reference and modify it in-situ than it is to take an immutable reference and return a new object. The reason is that in OCaml, you have structural sharing, i.e., multiple pointers pointing into the same data structure, which is a big no-no in Rust, so if you want to replicate the functional OCaml code, you begin allocating a new, complete data structure every time, and that's costly.
I found that translating OCaml code to Rust is frustrating, because you cannot easily and cheaply replicate the functional approach. It's much more Rustic to take an object by mutable reference and modify it in-situ than it is to take an immutable reference and return a new object. The reason is that in OCaml, you have structural sharing, i.e., multiple pointers pointing into the same data structure, which is a big no-no in Rust, so if you want to replicate the functional OCaml code, you begin allocating a new, complete data structure every time, and that's costly.
Well, of course. Rust is not designed around immutable data structures. There are going to be differences because it's simply not the same language. This [0] example, written by a notable Haskeller, illustrates what I mean pretty well and also touches on what you just pointed out. It's also very consistent with my own experience.
[0]: https://donsbot.wordpress.com/2020/07/04/back-to-old-tricks-...
[0]: https://donsbot.wordpress.com/2020/07/04/back-to-old-tricks-...
> And the most disappointing thing about Rust is that it probably won't ever be seen as a prototype that should be thrown away.
Can that ever happen with a language that lots of people use?
Can that ever happen with a language that lots of people use?
JavaScript has entered the chat...
You said two different things about Rust but you did not substantiate anything. What do you mean by
"Rust is so clearly a byproduct of the C++ world"?
Furthermore, in which way is Rust defective to the point that it should be seen as a prototype that should be thrown away?
mbbeddr seems really nice and interesting but it is not just a language. It is a set of integrated tools around one.
Rust shares goals with C++, i.e. it wants to be a high-performance and systems language like C++, but I find the differences to be quite substantial and worthwhile.
"Rust is so clearly a byproduct of the C++ world"?
Furthermore, in which way is Rust defective to the point that it should be seen as a prototype that should be thrown away?
mbbeddr seems really nice and interesting but it is not just a language. It is a set of integrated tools around one.
Rust shares goals with C++, i.e. it wants to be a high-performance and systems language like C++, but I find the differences to be quite substantial and worthwhile.
These threads always devolve into "rust is too slow" written by developers (or enthusiasts) that have never written no_std code in production. I've written and shipped firmware for embedded devices written in rust, yes, still using cargo and external crates, and had zero issues with compile time because the nature of the dependencies in the deps tree is different and very carefully curated.
Anyway, I really just wanted to point out that from the mailing list we have Linus and Greg endorsing this experiment/effort from the Linux side and a commitment from Josh on behalf of the rust team to grow the language itself with the needs of the kernel in mind. That's quite impressive and more than I could have hoped for.
I've actually played with writing kernel code in rust - for Windows/NT, however - and it's quite weird to be able to use such high-level type constructs in code where you typically manually chases pointers and wouldn't be surprised to see statically allocated global variables used to monitor reference counts.
Anyway, I really just wanted to point out that from the mailing list we have Linus and Greg endorsing this experiment/effort from the Linux side and a commitment from Josh on behalf of the rust team to grow the language itself with the needs of the kernel in mind. That's quite impressive and more than I could have hoped for.
I've actually played with writing kernel code in rust - for Windows/NT, however - and it's quite weird to be able to use such high-level type constructs in code where you typically manually chases pointers and wouldn't be surprised to see statically allocated global variables used to monitor reference counts.
Linus, normally being C has its quirks but is great and Anti C++ in general is endorsing Rust?
Why is that? Did he ever gave out his reasons?
Why is that? Did he ever gave out his reasons?
Linus has commented on Rust twice (that I'm aware of).
First, back in 2016:
Q: What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
A: That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
https://www.infoworld.com/article/3109150/linux-at-25-linus-...
Then second, much more recently:
People have been looking at that for years now. I’m convinced it’s going to happen one day. It might not be Rust, but it’s going to happen that we will have different models for writing these kinds of things.” He acknowledges that right now it’s C or assembly, “but things are afoot.” Though he also adds a word of caution. “These things take a long, long time. The kind of infrastructure you need to start integrating other languages into a kernel, and making people trust these other languages — that’s a big step.
https://thenewstack.io/linus-torvalds-on-diversity-longevity...
I was once in the same room as him at a Linux Foundation event, and really wanted to ask him about it, but also didn't want to be a bother.
Note that the C++ opinion everyone cites is from 2007. I don't know how he feels about C++ today. It seems he's using it for some things at least. I know I've changed a lot since 2007, and so has C++.
First, back in 2016:
Q: What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
A: That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
https://www.infoworld.com/article/3109150/linux-at-25-linus-...
Then second, much more recently:
People have been looking at that for years now. I’m convinced it’s going to happen one day. It might not be Rust, but it’s going to happen that we will have different models for writing these kinds of things.” He acknowledges that right now it’s C or assembly, “but things are afoot.” Though he also adds a word of caution. “These things take a long, long time. The kind of infrastructure you need to start integrating other languages into a kernel, and making people trust these other languages — that’s a big step.
https://thenewstack.io/linus-torvalds-on-diversity-longevity...
I was once in the same room as him at a Linux Foundation event, and really wanted to ask him about it, but also didn't want to be a bother.
Note that the C++ opinion everyone cites is from 2007. I don't know how he feels about C++ today. It seems he's using it for some things at least. I know I've changed a lot since 2007, and so has C++.
I don't know Linus's reasons specifically, but our presentation at Linux Security Summit last year laid out why we think that Linus's past objections to C++ don't apply to Rust. See slides 19-21 of https://ldpreload.com/p/kernel-modules-in-rust-lssna2019.pdf .
His previous objections were:
His previous objections were:
In fact, in Linux we did try C++ once already, back in 1992.
It sucks. Trust me - writing kernel code in C++ is a BLOODY STUPID IDEA.
The fact is, C++ compilers are not trustworthy. They were even worse in
1992, but some fundamental facts haven't changed:
- the whole C++ exception handling thing is fundamentally broken. It's
_especially_ broken for kernels.
- any compiler or language that likes to hide things like memory
allocations behind your back just isn't a good choice for a kernel.
- you can write object-oriented code (useful for filesystems etc) in C,
_without_ the crap that is C++.
Is the following code from page/slide 64 supposed to be a talking point?
Also, notably, the presentation says one of the desirable characteristics for a kernel language is "Don't 'hide things like memory allocations behind your back'". I'm not very well versed in Rust, but when I write a small test program using vec![0u8; 100], the buffer is heap allocated. Which would be a hidden allocation in my book. I realize these are just introductions, but if you're pitching Rust code for the kernel, it seems disingenuous not to show the style of Rust code, presumably much more verbose, which would actually be used in the kernel. Constructs like boxes can't really be used, at least not easily. C++ was disliked because the prospect of fiddling and debugging hidden allocators is nightmarish. And to the extent Rust's type system relies on boxes and similar implicit allocation for ergonomics, some of the type safety dividends might be lost. How much would be lost is hard to judge without showing the kind of actual code that would be required when used in Linux. Toy Rust OS experiments don't count, because they can just panic on OOM and other hard conditions, and don't need to wrestle with difficult constructs like RCU.
FFI: calling C from Rust
extern {
fn readlink(path: *const u8, buf: *const u8, bufsize: usize) -> i64;
}
fn rs_readlink(path: &str) -> Result<String, ...> {
let mut r = vec![0u8; 100];
if unsafe { readlink(path.as_ptr(), r.as_mut_ptr(), 100) }
...
Also, notably, the presentation says one of the desirable characteristics for a kernel language is "Don't 'hide things like memory allocations behind your back'". I'm not very well versed in Rust, but when I write a small test program using vec![0u8; 100], the buffer is heap allocated. Which would be a hidden allocation in my book. I realize these are just introductions, but if you're pitching Rust code for the kernel, it seems disingenuous not to show the style of Rust code, presumably much more verbose, which would actually be used in the kernel. Constructs like boxes can't really be used, at least not easily. C++ was disliked because the prospect of fiddling and debugging hidden allocators is nightmarish. And to the extent Rust's type system relies on boxes and similar implicit allocation for ergonomics, some of the type safety dividends might be lost. How much would be lost is hard to judge without showing the kind of actual code that would be required when used in Linux. Toy Rust OS experiments don't count, because they can just panic on OOM and other hard conditions, and don't need to wrestle with difficult constructs like RCU.
`Vec` is always heap-allocated; it's not hidden if you know that. It's intentional that you have to write `vec![0u8; 100]` to get this, rather than just, say, `[0u8; 100]` (which will give you a stack allocation).
It's true that the example should use capacity() rather than hardcoding 100.
If you want a stack allocation, here's an example:
https://play.rust-lang.org/?version=stable&mode=debug&editio...
It's about the same length as the original, but it's hard to make a direct comparison. On one hand, the original snippet is totally wrong: it doesn't properly handle the nul terminator of either the input or the output. I fixed that, but at the cost of making the code a bit more verbose. On the other hand, the original snippet does try to parse the result as UTF-8, which is not what you want, so I removed it (it would add 2 lines or so).
That said… it's not true that avoiding heap allocation is what would actually be done in the kernel. Not for file paths, which is what readlink typically returns. Since kernel stack sizes are so small, paths have to be allocated on the heap. It's only in userland that you can plop PATH_MAX sized buffers on the stack.
On the other hand, if you were really doing heap allocation in the kernel you would need to handle out-of-memory conditions. Rust's standard library doesn't support recovering from OOM at all, but it's not like you'd be using the standard library in the kernel anyway. In contrast, Rust the language is actually quite good at this. A fallible allocation function would return something like Result<String, OOMError>; the use of Result forces you to handle the error case somehow (unlike C, you can't just assume it succeeded), but all you need to do to handle it is use the `?` operator to propagate errors to the caller.
It's true that the example should use capacity() rather than hardcoding 100.
If you want a stack allocation, here's an example:
https://play.rust-lang.org/?version=stable&mode=debug&editio...
It's about the same length as the original, but it's hard to make a direct comparison. On one hand, the original snippet is totally wrong: it doesn't properly handle the nul terminator of either the input or the output. I fixed that, but at the cost of making the code a bit more verbose. On the other hand, the original snippet does try to parse the result as UTF-8, which is not what you want, so I removed it (it would add 2 lines or so).
That said… it's not true that avoiding heap allocation is what would actually be done in the kernel. Not for file paths, which is what readlink typically returns. Since kernel stack sizes are so small, paths have to be allocated on the heap. It's only in userland that you can plop PATH_MAX sized buffers on the stack.
On the other hand, if you were really doing heap allocation in the kernel you would need to handle out-of-memory conditions. Rust's standard library doesn't support recovering from OOM at all, but it's not like you'd be using the standard library in the kernel anyway. In contrast, Rust the language is actually quite good at this. A fallible allocation function would return something like Result<String, OOMError>; the use of Result forces you to handle the error case somehow (unlike C, you can't just assume it succeeded), but all you need to do to handle it is use the `?` operator to propagate errors to the caller.
I realize Rust-the-language can handle OOM, and that with Result types it could prove cleaner and safer than in C. But it's rare to see examples of this, and the wg-allocator working group still seems to have alot of unfinished business.
Yeah, someone reported a bug on that slide before: https://www.reddit.com/r/rust/comments/dc5kky/writing_linux_... That slide is not intended to be debugged production code, certainly. :)
Rust Vecs are, by definition, heap-allocated (just like C++ ones). See my reply to a sibling comment for what I mean by hiding memory allocations.
Here's my plan for RCU: https://github.com/fishinabarrel/linux-kernel-module-rust/is...
Rust Vecs are, by definition, heap-allocated (just like C++ ones). See my reply to a sibling comment for what I mean by hiding memory allocations.
Here's my plan for RCU: https://github.com/fishinabarrel/linux-kernel-module-rust/is...
> In brief, Rust does not rely on C++-style exception handling/unwinding, it does not do memory allocations behind your back
It kind of does a little bit. Panicking is implemented via the same machinery. But of course code is not expected to panic unless things are terribly wrong (ie. kind of like the existing kernel panic thing). It also does sometimes allocate things, but it is true that Rust is a lot more explicit about this - ie. you have to call clone() or Box::new() etc.
It kind of does a little bit. Panicking is implemented via the same machinery. But of course code is not expected to panic unless things are terribly wrong (ie. kind of like the existing kernel panic thing). It also does sometimes allocate things, but it is true that Rust is a lot more explicit about this - ie. you have to call clone() or Box::new() etc.
`no_std` environments are different. They don't allow allocations unless you explicitly add an allocator. You also have to define the panic handler yourself.
Generally `no_std` libraries will not allocate or panic themselves. There is still the possibility of panicking (e.g. out of bounds array access) but there are alternatives that don't panic if that's a concern even with a custom handler.
Generally `no_std` libraries will not allocate or panic themselves. There is still the possibility of panicking (e.g. out of bounds array access) but there are alternatives that don't panic if that's a concern even with a custom handler.
Yeah, and it shouldn't be too hard to hook up existing nightly Rust to use the Kernel's panic functionality; it already supports user-overridable panics for no_std, iirc
How does C++ do memory allocations behind your back?
Consider this code:
In userspace, that's not a huge problem - even if allocation requires asking for more memory from the kernel, which requires paging out some dirty files to disk, which blocks for a few seconds on NFS, that's fine - the kernel will just suspend the userspace process for a few seconds while all that happens. Inside the pager or NFS implementation, though, an unintentional memory allocation can deadlock the whole system.
In Rust you'd have to write .to_owned() to do the equivalent, and it would be obvious that doing it in code that might be responsible for freeing up memory is a mistake.
#include <iostream>
void p(const std::string &x) {
std::cout << x << std::endl;
}
int main() {
p("hello");
}
In userspace, that's not a huge problem - even if allocation requires asking for more memory from the kernel, which requires paging out some dirty files to disk, which blocks for a few seconds on NFS, that's fine - the kernel will just suspend the userspace process for a few seconds while all that happens. Inside the pager or NFS implementation, though, an unintentional memory allocation can deadlock the whole system.
In Rust you'd have to write .to_owned() to do the equivalent, and it would be obvious that doing it in code that might be responsible for freeing up memory is a mistake.
Constructors and copy constructors, mainly.
Something as innocuous as `T x;` or `T x = y;` will allocate in C++. In Rust, the equivalents are `let x = T::new();` and `let x = y.clone();` which are much more obvious as potential allocations.
Something as innocuous as `T x;` or `T x = y;` will allocate in C++. In Rust, the equivalents are `let x = T::new();` and `let x = y.clone();` which are much more obvious as potential allocations.
> Something as innocuous as `T x;` or `T x = y;` will allocate in C++
I think saying these will allocate is wrong. They can allocate, but only if T allocates and manages memory, which depends on what T does.
I am skeptical of the more general claim. Anything but a truly cursory skim of the code is going to tell you that these lines are constructing a new object, regardless of the syntax, and to actually know whether it allocates memory, you need to know more about T, which is true in either Rust or C++.
I think saying these will allocate is wrong. They can allocate, but only if T allocates and manages memory, which depends on what T does.
I am skeptical of the more general claim. Anything but a truly cursory skim of the code is going to tell you that these lines are constructing a new object, regardless of the syntax, and to actually know whether it allocates memory, you need to know more about T, which is true in either Rust or C++.
Unless someone implements Copy on a type that allocates in Rust (which you shouldn't do; Copy is specifically for types that are cheap to copy), you really won't get implicit copies, though.
That means in any reasonable code, `let x = y` won't allocate in Rust while `T x = y` could in C++. `f(x)` in Rust won't either for the same reason. And that's on top of how C++ will implicitly take a reference, so even if you know x is a heavy object, you don't know if `f(x)` will be expensive (or if `f(move(x))` would be helpful).
It's just not exactly equivalent. C++ is more implicit and has less baked-in good practices like `Copy` vs `Clone`. The indirection is often shallower (less often I have to inspect a function prototype or type I'm using for specifics), and I find that very useful.
That means in any reasonable code, `let x = y` won't allocate in Rust while `T x = y` could in C++. `f(x)` in Rust won't either for the same reason. And that's on top of how C++ will implicitly take a reference, so even if you know x is a heavy object, you don't know if `f(x)` will be expensive (or if `f(move(x))` would be helpful).
It's just not exactly equivalent. C++ is more implicit and has less baked-in good practices like `Copy` vs `Clone`. The indirection is often shallower (less often I have to inspect a function prototype or type I'm using for specifics), and I find that very useful.
Minor clarification: afaik it's impossible to implement Copy such that it would allocate. Unlike Clone, the Copy trait is only a marker trait which doesn't have methods, so you cannot customize its behavior. It will always be a bitwise stack copy.
This is not minor. In Rust "x = y" specifically only copies or moves bytes, no magic involved, no code is run except something like memcpy.
- If the type isn't marked Copy, it's a bytes move (which will very probably be optimized out).
- If the type is marked Copy (which is possible only if all its subtypes are also marked Copy, which allocating types are not) then it's a bytes-for-bytes copy.
As soon as you need something more involved, then you have to implement (or derive for simple types) Clone.
- If the type isn't marked Copy, it's a bytes move (which will very probably be optimized out).
- If the type is marked Copy (which is possible only if all its subtypes are also marked Copy, which allocating types are not) then it's a bytes-for-bytes copy.
As soon as you need something more involved, then you have to implement (or derive for simple types) Clone.
The real key here is that something that allocates will want to Drop said memory, and you can’t implement both Drop and Copy.
How do you write OO code in C?
This is a really in depth read about OO design patterns in the Linux Kernel using C: https://lwn.net/Articles/444910/
The easiest to understand example (for me) is that you can manually implement polymorphism support using function pointers that take in a struct instance as an argument. Polymorphism is fundamentally just a method call that's dependent on which target object it's being called on, so this pattern allows you to pass in the target object (as a struct since it's C) explicitly.
From the article:
> Some simple examples of this in the Linux kernel are the file_lock_operations structure which contains two function pointers each of which take a pointer to a struct file_lock, and the seq_operations vtable which contains four function pointers which each operate on a struct seq_file.
The easiest to understand example (for me) is that you can manually implement polymorphism support using function pointers that take in a struct instance as an argument. Polymorphism is fundamentally just a method call that's dependent on which target object it's being called on, so this pattern allows you to pass in the target object (as a struct since it's C) explicitly.
From the article:
> Some simple examples of this in the Linux kernel are the file_lock_operations structure which contains two function pointers each of which take a pointer to a struct file_lock, and the seq_operations vtable which contains four function pointers which each operate on a struct seq_file.
The huge and important difference is when you use this pattern in C the compiler has no idea what you are doing and can't check anything, nor can it reduce the cost of this dispatch mechanism. A compiler for a language where this is a feature can check what you are doing and de-virtualize virtual dispatch. In C you pay the highest possible cost for polymorphism while enjoying the least benefits.
> The huge and important difference is when you use this pattern in C the compiler has no idea what you are doing and can't check anything
Function pointers exist in C and are type-checked by the compiler just fine.
> A compiler for a language where this is a feature can check what you are doing and de-virtualize virtual dispatch.
Virtual dispatch is pretty much only used in the kernel in places where virtualization is necessary. Devirtualization is pretty rare in C++ in practice (I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase).
Function pointers exist in C and are type-checked by the compiler just fine.
> A compiler for a language where this is a feature can check what you are doing and de-virtualize virtual dispatch.
Virtual dispatch is pretty much only used in the kernel in places where virtualization is necessary. Devirtualization is pretty rare in C++ in practice (I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase).
You don't need "virtual final" for devirtualization, the compiler will speculatively devirtualize in some cases.
> Function pointers exist in C and are type-checked by the compiler just fine.
This amount of type safety is almost useless. Many of the function prototypes, for example in inode and dentry ops, have the exact same signature. If you accidentally swap inode.rmdir with inode.unlink, compiler isn't going to say anything. And it won't be caught in code review either, to the extent that Linux even has code review culture.
> I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase
It's common in my experience, for performance-sensitive code.
This amount of type safety is almost useless. Many of the function prototypes, for example in inode and dentry ops, have the exact same signature. If you accidentally swap inode.rmdir with inode.unlink, compiler isn't going to say anything. And it won't be caught in code review either, to the extent that Linux even has code review culture.
> I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase
It's common in my experience, for performance-sensitive code.
I don't think that kind of optim is usually important in a kernel, especially one architected a lot with dynamical modules. Plus, on the convenience side, you can also use other models (e.g. have some function pointers at instance level) than the one of C++ in ways that do not look like completely different when reading the source.
Devirtualization is mainly useful to cope with some C++ self inflicted wounds. Linux obviously never had them the first place.
Devirtualization is mainly useful to cope with some C++ self inflicted wounds. Linux obviously never had them the first place.
I'm really not a compiler expert, but is this optimization common in C++ for example? I thought all virtual methods incurred the cost of a vtable lookup
Just slapping the virtual keyword on some function doesn't do anything. Overrides may or may not happen via vtable. Often compilers can figure out how to dispatch at compile time and don't need the vtable.
C++ was originally a precompiler that generated C code. So it is definitely possible.
Generally:
- an object is a struct
- a method is a function taking a struct (the object) as an argument
- inheritance is done by having a struct that has its parent as the first element
- polymorphism is donc by implementing the vtable manually, these are function pointers
- constructors are functions returning an object (so, a struct)
It is inconvenient but possible. OO is a concept that is not dependent on the language, though some languages make it easier than others.
Generally:
- an object is a struct
- a method is a function taking a struct (the object) as an argument
- inheritance is done by having a struct that has its parent as the first element
- polymorphism is donc by implementing the vtable manually, these are function pointers
- constructors are functions returning an object (so, a struct)
It is inconvenient but possible. OO is a concept that is not dependent on the language, though some languages make it easier than others.
[deleted]
These seem like the statements of a person who knows next to nothing about C++.
Why it is weird? It has been done before in Ada, Object Pascal, C++, Mesa/Cedar, Modula-2, Lisp, Oberon, Sing#, System C#, Objective-C,....
Right, and it would be "weird" if I were using one of those, too. You don't run across drivers written in any of those languages on a daily basis.
I didn't mean it with any negative connotation, btw.
I didn't mean it with any negative connotation, btw.
In Linux yes, however Arduino, MBed, Android, iOS/macOS and Windows drivers tend to be written in a C++ subset, and Swift, Java and .NET are also an option for userspace drivers.
I got your point, and naturally we are in the context of Linux kernel, however I like to make a point that not every OS constrains themselves to C and having been embracing more productive options for a while now.
I got your point, and naturally we are in the context of Linux kernel, however I like to make a point that not every OS constrains themselves to C and having been embracing more productive options for a while now.
The title is not very accurate, this is a thread about a discussion on this topic which will happen at the upcoming Linux Plumbers Conference in late August.
I see what you mean. I just posted it here with the same title that was used on /r/Linux, and that I found accurate (for the same reasons chrismorgan exposed in a sibling comment) but now I agree with you that it could cause some confusion.
Maybe a moderator could rename the post to “discussion about Linux kernel in-tree support” or something like that?
Maybe a moderator could rename the post to “discussion about Linux kernel in-tree support” or something like that?
The title is perfectly accurate, it’s an email thread about Linux kernel in-tree Rust support. Sure, you could misconstrue such a title to be implying that the Linux kernel supports Rust in-tree already it if you wanted to, but half the titles on a site like this could be similarly misconstrued.
> The title is perfectly accurate, it’s an email thread about Linux kernel in-tree Rust support.
The title the person you are responding to is complaining about is not the title of a email thread but the title of a hacker news post.
The title the person you are responding to is complaining about is not the title of a email thread but the title of a hacker news post.
Unless there’s a strong reason not to, the Hacker News post title should match the email thread title. It does match, and I see no even slightly compelling reason for it to deviate.
Immunant folks wrote a blog post [1] about automating conversion of Linux kernel drivers from C to Rust by using their tool c2rust[2].
[1] https://immunant.com/blog/2020/06/kernel_modules/
[2] https://github.com/immunant/c2rust
[1] https://immunant.com/blog/2020/06/kernel_modules/
[2] https://github.com/immunant/c2rust
I don't see them discussing what I view as the biggest question of such conversion: is the converted code safer than C, or is it a direct translation with all the issues that we were trying to fix by using Rust? (This came up, IIRC, with automatic C to Go; it worked, but was only useful as a first step because it gave you unsafe unidiomatic Go code)
It is currently a direct translation into unsafe.
They are also interested in "unsafe to safe refactoring tools" in my understanding, but they're not there yet.
They are also interested in "unsafe to safe refactoring tools" in my understanding, but they're not there yet.
There is a good refactoring tool[1][2] to automate the process. It also allows scripting in Lua[3].
[1] https://github.com/immunant/c2rust/tree/master/c2rust-refact...
[2] https://c2rust.com/manual/c2rust-refactor/commands.html
[3] https://c2rust.com/manual/c2rust-refactor/doc/scripting_api....
[1] https://github.com/immunant/c2rust/tree/master/c2rust-refact...
[2] https://c2rust.com/manual/c2rust-refactor/commands.html
[3] https://c2rust.com/manual/c2rust-refactor/doc/scripting_api....
Oh awesome! Thank you!
Pleasantly surprised by Linus response. IIRC his attitude to C++ was that it should be refused if only to keep C++ programmers out.
Rust is well designed modern and concise language with sound type system, while C++ isn't. The difference is huge and obvious.
Except that one of Linus' most vocal offenses on C++ was due to operator overloading and how basic, seemingly native things like + can actually do a lot of hidden stuff unknown to the programmer. He must have softened on this since Rust offers the same facilities for operator overloading.
https://doc.rust-lang.org/stable/rust-by-example/trait/ops.h...
https://doc.rust-lang.org/stable/rust-by-example/trait/ops.h...
Operator overloading in Rust is less flexible than in C++, so there are fewer possibilities to mess it up.
Overloading is done indirectly via specific named traits which enforce method signatures, invariants, immutability rules. Assignments and moves can't be overloaded. Borrow checker limits what indexing and dereferencing overloads can do (hardly anything beyond pointer arithmetic on immutable data). Comparison operators can't be overloaded separately and can't return arbitrary values (like C++ spaceship operator). Generic code has to explicitly list what overloads it accepts (like C++ concepts), so even if you create a bizarre overload, it won't be used in generic code.
Overloading is done indirectly via specific named traits which enforce method signatures, invariants, immutability rules. Assignments and moves can't be overloaded. Borrow checker limits what indexing and dereferencing overloads can do (hardly anything beyond pointer arithmetic on immutable data). Comparison operators can't be overloaded separately and can't return arbitrary values (like C++ spaceship operator). Generic code has to explicitly list what overloads it accepts (like C++ concepts), so even if you create a bizarre overload, it won't be used in generic code.
Linus’s point was also that C++ makes extensive use of function/method name overloading (is polymorphism) and this can make code hard to read. Not only is this not possible in C but the feature tends to be overused in C++.
Rust generally uses function/method polymorphism based on traits (aka type classes), that's far less prone to overuse and abuse than overloading in C++.
Is this form of polymorphism done at runtime and thus incurs a similar overhead to the C++ virtual?
There are two ways of doing it. The way most people use most of the time does not, it's statically dispatched. The other way is dynamically dispatched, though there are some differences from the way that C++ virtual works. They should be roughly the same in that case, though.
It depends, you have access to both capabilities. By default if you're only dealing with an impld trait then it will use static dispatch. If you are passing around that same object as a trait objects, it will use dynamic dispatch and you can also impl Traits on other traits, which are always used as trait objects (vtable) but that have some restrictions on what they can look like. If you rely on the Deref trait to make it look like you have inheritance (but in reality it is composition) it will be the same as the later using dynamic dispatch.
You can pick whether you want to do this at run-time or compile-time. The default is compile-time.
I agree with this critique, but a good IDE helps, as does lint rules preventing egregious misuses (or even outlawing operator overloading entirely). As one example, many Java/C# linters prevent the use of non-curly-braced single line control statements entirely. The language allows them, but many codebases do not.
Could you give me a pointer to a discussion of type soudness in Rust?
I recently watched perhaps 3 years old video about Rust where Simon Peyton-Jones asked about this, and Niko Matsakis answered it was ongoing work back then.
Couldn't find anything proper by googling.
Couldn't find anything proper by googling.
See the work of Iris[1] project. It's open-source[2] and based on Coq, see Rust-related work[3] in particular.
[1] https://iris-project.org/
[2] https://gitlab.mpi-sws.org/iris/iris/
[3] https://gitlab.mpi-sws.org/iris/lambda-rust
[1] https://iris-project.org/
[2] https://gitlab.mpi-sws.org/iris/iris/
[3] https://gitlab.mpi-sws.org/iris/lambda-rust
Linus nowadays also uses Qt, so.
I would assume he sees difference between requirements for kernel code and a dive tracking GUI.
His attitude towards C++ also changed when using it for his side project [1].
Initially he started with C and GTK+ and later migrated to C++ and QT Framework.
[1] https://subsurface-divelog.org/
Initially he started with C and GTK+ and later migrated to C++ and QT Framework.
[1] https://subsurface-divelog.org/
I think that was after others took over the UI development. The back end of that program also was still C as far as I remember from their presentation and the move was mostly motivated by the GTK community, the documentation and different priorities on cross platform support.
Yes, you're correct. Linus still works on C backend of the application and sometimes has to connect his parts with C++ code.
Did he have an "attitude" about C++ in general? I thought he only commented on it with respect to operating system development. He did make much more general statements about Java, though.
I thought the famous comment on C++ was in the context of git, not operation systems?
My other reply: https://news.ycombinator.com/item?id=23801076
Did he talk about his thoughts precisely ?
I think I watched an interview with him where he talked about Subsurface project. In that interview he said that C++ is not all that bad but he still prefers C because he's so used to it and will continue write C code forever. I don't remember title of the video though.
I love how the bugtracker on that is "check out our mailing list" :D
Perhaps the possibility of rust improving kernel security/robustness makes the idea of rust integration seem like it carries its own weight, where C++ has more downsides (perceived or real) and fewer upsides.
Rust's history/origins - loosely, being designed to allow replacing Mozilla's C/C++ with safer Rust that performs well - feel like a good fit for kernel drivers even if the core kernel bits will always be C.
Rust's history/origins - loosely, being designed to allow replacing Mozilla's C/C++ with safer Rust that performs well - feel like a good fit for kernel drivers even if the core kernel bits will always be C.
Yes, very much so. Not even only in the context of Rust but the insight, to fail fast, integrate early and do work in the open, instead of some hidden work, failing after a long time, when revealed.
So this surprises me, obviously this is early discussion about a potential topic, but the general consensus seemed to be more positive than I thought.
I thought I'd remembered reading something (maybe from linus) that seemed very against having rust in the kernel, can anyone find a source for that, I searched a little and can't?
(caveat, I obviously realise that linus isn't supporting rust in the kernel, and is only saying something bounded that, if we have it, it shouldn't be completely hidden behind some config options, but it doesn't match my memory)
I thought I'd remembered reading something (maybe from linus) that seemed very against having rust in the kernel, can anyone find a source for that, I searched a little and can't?
(caveat, I obviously realise that linus isn't supporting rust in the kernel, and is only saying something bounded that, if we have it, it shouldn't be completely hidden behind some config options, but it doesn't match my memory)
Maybe you're remembering Theo's comments about Rust in OpenBSD?
https://marc.info/?l=openbsd-misc&m=151233345723889&w=2
https://marc.info/?l=openbsd-misc&m=151233345723889&w=2
The sad part is that Theo's answer on the thread seems to be one of the most polite and realistic ones
This was interesting to read:
> Such ecosystems [Rust] come with incredible costs. For instance, rust cannot even compile itself on i386 at present time because it exhausts the address space.
> Such ecosystems [Rust] come with incredible costs. For instance, rust cannot even compile itself on i386 at present time because it exhausts the address space.
This may be solved now that rustc can bootstrap itself via cranelift.
But is that a huge issue? One could compile on better machine with i386 target, no?
According to the link above, in openbsd the policy is that 'base builds base'. Rust can't be included in the base on i386 without violation of that policy.
And this is a reasonable and understandable policy. It just means that Rust in it's current form is not a good fit for OpenBSD base system.
Shame I hadn't seen that thread. He's right that most of the rust projects "replacing" standard Unix utils are not feature equivalent and differ in intent, but that doesn't mean there aren't other efforts to do exactly what he is asking.
Eg here's tac (admittedly not cat, but hey, ./goo | tac | tac) published a few months before his email: https://github.com/neosmart/tac
Eg here's tac (admittedly not cat, but hey, ./goo | tac | tac) published a few months before his email: https://github.com/neosmart/tac
I think you're right actually!
Thanks for finding it for me.
Thanks for finding it for me.
Just stating something obvious since I don't see it noted here: Linus is a smart guy with this idea of not wanting it to be some niche feature that nobody enables and hence nobody cares about and sees breakage from.
I think what he did has nothing to do with smart. it's experience that matters here, he learned such stuff, sometimes even on the hard way.
The smarts, I guess, is in recognizing which pieces need this treatment and which do not.
Also, he is suggesting this change to experienced people who did not do it that way. So experience is not 100%.
Also, he is suggesting this change to experienced people who did not do it that way. So experience is not 100%.
It takes smarts to learn / abstract from experience. The smarter you are the better you do.
I always find these email threads very hard to follow, specially on mobile.
Is there anyway to have them shown more friendly?
Is there anyway to have them shown more friendly?
6 months ago, as I hopped on the Rust train, I told a friend of mine that I'm convinced that Rust will "eat the world" like Python did[0]. These moves to see Rust enter the Linux kernel only prove to me that this pace is starting to pick up.
0. https://www.techrepublic.com/article/python-is-eating-the-wo...
0. https://www.techrepublic.com/article/python-is-eating-the-wo...
This is a deeply weird comparison.
Insufferably lazy "eating the world" headline aside, Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of applications, its safety guarantees are leagues better than C/C++, and it's deliberately simple to write (it was designed in part as a pedagogical language).
Rust's performance is leagues better than Python's, it's safety guarantees are better by degrees, and it is substantially harder to write than Python, not just because it requires a more sophisticated understanding how how programs are constructed and verified, but also because it requires developers to think carefully about memory management.
I fully expect Rust to "eat the world" of C/C++ code over the next 10 years, and look forward to a time where we'll look askance at kernel code still written in C. It will no doubt be important in a bunch of performance-critical settings (AAA gaming, browsers, network infrastructure) as well.
But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.
In case this isn't obvious to anyone reading this thread --- though I'd be surprised --- the reason Rust is in the Linux kernel, while Python isn't, is that Rust was designed to be interoperable with kernels and browsers, and Python, like most other languages that have ambitious runtimes, was not. It's not an indication of Rust's popularity, but rather just Rust doing something it was designed to do.
(Rust is plenty popular.)
Insufferably lazy "eating the world" headline aside, Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of applications, its safety guarantees are leagues better than C/C++, and it's deliberately simple to write (it was designed in part as a pedagogical language).
Rust's performance is leagues better than Python's, it's safety guarantees are better by degrees, and it is substantially harder to write than Python, not just because it requires a more sophisticated understanding how how programs are constructed and verified, but also because it requires developers to think carefully about memory management.
I fully expect Rust to "eat the world" of C/C++ code over the next 10 years, and look forward to a time where we'll look askance at kernel code still written in C. It will no doubt be important in a bunch of performance-critical settings (AAA gaming, browsers, network infrastructure) as well.
But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.
In case this isn't obvious to anyone reading this thread --- though I'd be surprised --- the reason Rust is in the Linux kernel, while Python isn't, is that Rust was designed to be interoperable with kernels and browsers, and Python, like most other languages that have ambitious runtimes, was not. It's not an indication of Rust's popularity, but rather just Rust doing something it was designed to do.
(Rust is plenty popular.)
I think you're misunderstanding the comparison to Python. I am saying Rust will eat the world in the space that it fits in, which is still relatively large, in the same way that Python did, in the space that Python fits in. Not that Rust will take over very high level languages too.
The reason I was moved to comment is that I think it's actually not that large a space (the one that Rust plays in). Instead, I think it's just that the applications in that space loom large in our consciousness, because we all use and pay attention to kernels and browsers. And I expect the space of applications "languages like Python" play in to grow over time, and Rust's to shrink, not just because computers will get faster but because the runtimes for those languages will get cleverer (see, for instance, what's happening with Javascript).
I would actually like to make the exact opposite prediction.
A generation of developers grew up on Python, Ruby, etc., rather than C, and they were all brought up on this notion that performance doesn't matter much anymore because computers keep getting faster. But the problems we're solving with computers keep getting harder (in terms of the amount of computation required).
These days, I can comfortably state that I prefer Rust over Python, even for simple programs. I prefer it because the type system is much more useful than Python 3 with mypy, the performance is extraordinary out of the box, the language is often just as expressive, and I rarely have any actual issues with the borrow checker. I also prefer it because I've had far too many experiences with Python where supposedly performance didn't matter... until it did and Python was no longer appropriate.
I think we're entering into a generation of languages where things like Rust are going to start really seriously displacing C++, and as more people learn them they're even going to start displacing supposedly easier languages. Right now, few folks are going to learn Rust as an alternative to Python. But once they're exposed to it for other work and realize how productive they are, there's just going to be less of a need to reach for Python over time.
A generation of developers grew up on Python, Ruby, etc., rather than C, and they were all brought up on this notion that performance doesn't matter much anymore because computers keep getting faster. But the problems we're solving with computers keep getting harder (in terms of the amount of computation required).
These days, I can comfortably state that I prefer Rust over Python, even for simple programs. I prefer it because the type system is much more useful than Python 3 with mypy, the performance is extraordinary out of the box, the language is often just as expressive, and I rarely have any actual issues with the borrow checker. I also prefer it because I've had far too many experiences with Python where supposedly performance didn't matter... until it did and Python was no longer appropriate.
I think we're entering into a generation of languages where things like Rust are going to start really seriously displacing C++, and as more people learn them they're even going to start displacing supposedly easier languages. Right now, few folks are going to learn Rust as an alternative to Python. But once they're exposed to it for other work and realize how productive they are, there's just going to be less of a need to reach for Python over time.
What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not even Java or Go, the managed compiled languages; no, it's Python.†
You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?
It's interesting, because Rust will probably be a part of the story that makes things like Electron tolerable! But the point isn't that Rust is a doomed language (it's not, it's going to be durably important), but rather that it's not going to be the language most programmers work in.
† It's not pure Python, of course, but that's part of my point.
You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?
It's interesting, because Rust will probably be a part of the story that makes things like Electron tolerable! But the point isn't that Rust is a doomed language (it's not, it's going to be durably important), but rather that it's not going to be the language most programmers work in.
† It's not pure Python, of course, but that's part of my point.
> What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not Java or Go, the managed compiled languages; no, it's Python.
While ML might be the most demanding area in terms of computation, it's also a very small part of what developers in general do. My claim/prediction is more based on the notion that everything is becoming more compute-intensive.
Where I work, we've probably incurred many, many millions of dollars of expenses (primarily hardware) due to the choice to write a lot of software in scripting languages over the years. We made that choice because C++ was so unwieldy in comparison that we figured we'd end up spending the same amount on C++ but in different ways. If we were starting from scratch today, I doubt we'd use scripting languages like Python for hardly any of our software, because there are choices available now that just didn't exist in the past.
I'm not saying there's no place for something like Python, nor am I saying existing Python stuff should be rewritten. I think Python is going to continue being an extremely popular and important language. My claim/prediction is more that in the future we'll increasingly see Python chosen less for completely greenfield projects because there are now options that are high-level, well-designed, and extremely fast, and at least in my experience that has a lot of business value that is going to be harder and harder to ignore over time as it becomes more obvious.
While ML might be the most demanding area in terms of computation, it's also a very small part of what developers in general do. My claim/prediction is more based on the notion that everything is becoming more compute-intensive.
Where I work, we've probably incurred many, many millions of dollars of expenses (primarily hardware) due to the choice to write a lot of software in scripting languages over the years. We made that choice because C++ was so unwieldy in comparison that we figured we'd end up spending the same amount on C++ but in different ways. If we were starting from scratch today, I doubt we'd use scripting languages like Python for hardly any of our software, because there are choices available now that just didn't exist in the past.
I'm not saying there's no place for something like Python, nor am I saying existing Python stuff should be rewritten. I think Python is going to continue being an extremely popular and important language. My claim/prediction is more that in the future we'll increasingly see Python chosen less for completely greenfield projects because there are now options that are high-level, well-designed, and extremely fast, and at least in my experience that has a lot of business value that is going to be harder and harder to ignore over time as it becomes more obvious.
Candidly, there once was a vocal subculture of developers that would aver as to how much more comfortable they were in C than in Python (or Tcl or Perl), and, I'm told, in the long long ago there were them that said the same thing about assembly language. As an engineering manager, your job, if done well, would be to mollify those developers without conceding any ground to them, because the higher-level language was practically always the better option.
People will burn money on compute no matter what language things are built in; they'll ship on Fargate instead of EC2, on-demand instead of reserved instances, RDS instead of running their own Postgres, Postgres instead of just using Sleepycat, anything at all instead of running their own iron, and on and on down the list.
(All of this is out the window if you're shipping a browser! Or a hypervisor! Or an OS kernel! Or a new router! Or a AAA game!)
Most of the time, the most expensive thing you can burn is an hour of programmer time. And, with your fluency in Rust and other languages, you must see immediately why someone would believe Rust is more time-consuming than, say, Java, let alone Python.
Paul Graham got this part, at least, right in "Beating The Averages", when he noted how much more worrisome a competitor would be if their job reqs were for Perl programmer than if they were for C developers. The Perl company is actually getting lots of stuff done.
People will burn money on compute no matter what language things are built in; they'll ship on Fargate instead of EC2, on-demand instead of reserved instances, RDS instead of running their own Postgres, Postgres instead of just using Sleepycat, anything at all instead of running their own iron, and on and on down the list.
(All of this is out the window if you're shipping a browser! Or a hypervisor! Or an OS kernel! Or a new router! Or a AAA game!)
Most of the time, the most expensive thing you can burn is an hour of programmer time. And, with your fluency in Rust and other languages, you must see immediately why someone would believe Rust is more time-consuming than, say, Java, let alone Python.
Paul Graham got this part, at least, right in "Beating The Averages", when he noted how much more worrisome a competitor would be if their job reqs were for Perl programmer than if they were for C developers. The Perl company is actually getting lots of stuff done.
You have two premises here, one explicit, one implicit:
(1) [explicit]: a higher level language is practically always the better choice.
(2) [implicit]: Rust is not high level (or at least not compared to Python).
Our biggest disagreement is with (2). Rust is actually higher level and more expressive than Python in many ways. Of course, it also exposes some lower level concepts like lifetimes and ownership that Python hides.
Your comparison doesn’t apply because there isn’t a simple hierarchy into which Rust and Python fall. Each language is both higher- and lower-level than the other in a number of ways.
(1) [explicit]: a higher level language is practically always the better choice.
(2) [implicit]: Rust is not high level (or at least not compared to Python).
Our biggest disagreement is with (2). Rust is actually higher level and more expressive than Python in many ways. Of course, it also exposes some lower level concepts like lifetimes and ownership that Python hides.
Your comparison doesn’t apply because there isn’t a simple hierarchy into which Rust and Python fall. Each language is both higher- and lower-level than the other in a number of ways.
The word "expose" is doing a lot of work there.
I've written a lot of Python (thankfully not so much anymore), and Rust is part of my day job. I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves. Maybe, you can provide examples, and we can see if we're working from the same premises.
I've written a lot of Python (thankfully not so much anymore), and Rust is part of my day job. I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves. Maybe, you can provide examples, and we can see if we're working from the same premises.
> I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves.
Most Python backend projects I've worked on would have been better off written in Rust (had it existed) the same way most of the Javascript projects I've worked on would have been better off written in Typescript.
Maybe it's just the projects I've worked on (or their size) but maintenance and refactoring to deal with growth of or changes in business requirements were by far more time consuming than actually writing code. The amount of time lost on onboarding and writing trivial tests that could have been handled by the Rust compiler dwarfs true R&D time, let alone time spent debugging bugs that could have been prevented by encoding business logic into the type system.
At my current job I often still pull out Python to prototype logic or Typescript to spin up a quick and dirty intranet app but for the stuff that needs to last, just work, or be maintained into the future, I often find myself yearning to use Rust. The last obstacles are organization talent pool and Rust's ecosystem.
Most Python backend projects I've worked on would have been better off written in Rust (had it existed) the same way most of the Javascript projects I've worked on would have been better off written in Typescript.
Maybe it's just the projects I've worked on (or their size) but maintenance and refactoring to deal with growth of or changes in business requirements were by far more time consuming than actually writing code. The amount of time lost on onboarding and writing trivial tests that could have been handled by the Rust compiler dwarfs true R&D time, let alone time spent debugging bugs that could have been prevented by encoding business logic into the type system.
At my current job I often still pull out Python to prototype logic or Typescript to spin up a quick and dirty intranet app but for the stuff that needs to last, just work, or be maintained into the future, I often find myself yearning to use Rust. The last obstacles are organization talent pool and Rust's ecosystem.
As much as I'd love to use more performant APIs and programs that crash less on my daily basis, I don't see Rust even close to overtaking Python.
For starters, memory management in Python is trivial while in Rust the developer has to learn a whole new set of skills in the form of borrowing, lifetimes and ownership.
And for large projects where Python's limitations hamper team coordination, GC'd languages such as C# and Java offer much softer learning curve and a mature ecosystem. Even when performance is critical, projects get by just fine with using Java like in High Frequency Trading shops.
Perhaps we shouldn't settle for Rust but refine the idea in a new, more ergonomic, language with focus on achieving the same but with lower cognitive load. Faster compilation times wouldn't hurt too.
For starters, memory management in Python is trivial while in Rust the developer has to learn a whole new set of skills in the form of borrowing, lifetimes and ownership.
And for large projects where Python's limitations hamper team coordination, GC'd languages such as C# and Java offer much softer learning curve and a mature ecosystem. Even when performance is critical, projects get by just fine with using Java like in High Frequency Trading shops.
Perhaps we shouldn't settle for Rust but refine the idea in a new, more ergonomic, language with focus on achieving the same but with lower cognitive load. Faster compilation times wouldn't hurt too.
> The word "expose" is doing a lot of work there.
I do not believe this is a statement that I can reply to in a meaningful way. You definitely believe Rust is much harder or more complicated than I think it is, and that’s basically where we disagree.
I do not believe this is a statement that I can reply to in a meaningful way. You definitely believe Rust is much harder or more complicated than I think it is, and that’s basically where we disagree.
[deleted]
Claiming rust is as expressive as Python/JS/Kotlin for non-fp-nerds is precisely what leads to rust evangelism strike force memes.
There’s not any particular way I can respond to this, because there really isn’t any substance except to attempt to link me to something you find distasteful/annoying. I use both languages professionally on a daily basis and it is on that basis that I make my claim.
Here’s one way I could respond. There are numerous things in Python that I find more difficult than in Rust. For instance, I think async/.await in Rust is generally easier than asyncio is in Python, and in the former case the compiler is quite good at detecting various concurrency bugs whereas you’re entirely on your own in Python.
It’s far easier to write multicore programs in Rust than Python thanks to the lack of a global interpreter lock.
Python makes extensive use of runtime exceptions for basic error handling, which I find very exhausting to reason about and account for during development; tools like mypy are also unable to help there at all, and I must rely on the questionable quality of documentation or just scavenging through code.
Pythonic things like None-equivalent values often make it impractical to use Optional values as return types in idiomatic code, thus pressuring you to use more exceptions in places where one would just use Option or Result in Rust.
Python has poor support at best for sum types, which I tend to use extensively for almost every problem space.
Python does not support pattern matching, though some support for this has been proposed for a future version of the language.
Handling dependencies generally is much easier in Rust as well thanks to cargo.
Perhaps you think these are all just the concerns of “FP nerds,” but for me these differences often mean I can write correct, high quality backend services in Rust faster than I can in Python — and the result is often 10x-100x faster with no optimization work whatsoever. That seems worth the relatively small amount of time I spend worrying about lifetimes or ownership or the fact that I use map/filter/iterators instead of list or dictionary comprehensions.
Here’s one way I could respond. There are numerous things in Python that I find more difficult than in Rust. For instance, I think async/.await in Rust is generally easier than asyncio is in Python, and in the former case the compiler is quite good at detecting various concurrency bugs whereas you’re entirely on your own in Python.
It’s far easier to write multicore programs in Rust than Python thanks to the lack of a global interpreter lock.
Python makes extensive use of runtime exceptions for basic error handling, which I find very exhausting to reason about and account for during development; tools like mypy are also unable to help there at all, and I must rely on the questionable quality of documentation or just scavenging through code.
Pythonic things like None-equivalent values often make it impractical to use Optional values as return types in idiomatic code, thus pressuring you to use more exceptions in places where one would just use Option or Result in Rust.
Python has poor support at best for sum types, which I tend to use extensively for almost every problem space.
Python does not support pattern matching, though some support for this has been proposed for a future version of the language.
Handling dependencies generally is much easier in Rust as well thanks to cargo.
Perhaps you think these are all just the concerns of “FP nerds,” but for me these differences often mean I can write correct, high quality backend services in Rust faster than I can in Python — and the result is often 10x-100x faster with no optimization work whatsoever. That seems worth the relatively small amount of time I spend worrying about lifetimes or ownership or the fact that I use map/filter/iterators instead of list or dictionary comprehensions.
> You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?
Browsers already have an immense amount of effort poured into making them fast. They will get faster, but not by much.
But then, JavaScript was never the problem with Electron in the first place. Or at least, not the fact that UI code is written in JavaScript. After all, a lot of the people complaining about Electron (me included) are Mac users pining specifically for native Cocoa apps. Which, up until recently, were all written in Objective-C – a language which is much slower than JavaScript! Even Swift is not necessarily faster.
Rather, from a performance perspective, I think the biggest problems are (1) the DOM being a terrible abstraction to design user interfaces on top of, and (2) the web forcing you to use JavaScript for everything rather than letting you drop down to native code for performance-critical bits (other than what is implemented by the browser itself).
Browsers already have an immense amount of effort poured into making them fast. They will get faster, but not by much.
But then, JavaScript was never the problem with Electron in the first place. Or at least, not the fact that UI code is written in JavaScript. After all, a lot of the people complaining about Electron (me included) are Mac users pining specifically for native Cocoa apps. Which, up until recently, were all written in Objective-C – a language which is much slower than JavaScript! Even Swift is not necessarily faster.
Rather, from a performance perspective, I think the biggest problems are (1) the DOM being a terrible abstraction to design user interfaces on top of, and (2) the web forcing you to use JavaScript for everything rather than letting you drop down to native code for performance-critical bits (other than what is implemented by the browser itself).
"I can comfortably state that I prefer Rust over Python, even for simple programs"
That's fine but that will never be generally true because Python is a high level language that will always have 10x more developers using it, including your Data Science prof, the high school kids learning 'basic programming', and all the AI researchers.
And the tons of devs doing stuff for which Python just makes more sense.
That's fine but that will never be generally true because Python is a high level language that will always have 10x more developers using it, including your Data Science prof, the high school kids learning 'basic programming', and all the AI researchers.
And the tons of devs doing stuff for which Python just makes more sense.
I don't think it's appropriate to call a language with an advanced and useful type system, expressive syntax out of box, and reasonable macro support a "low level" language.
Python, in comparison, lacks a lot. But it has a very low barrier of entry: the syntax is English-like, you can start doing something with very little knowledge of advanced features, and everything is easy to inspect and play with in a REPL.
Python, in comparison, lacks a lot. But it has a very low barrier of entry: the syntax is English-like, you can start doing something with very little knowledge of advanced features, and everything is easy to inspect and play with in a REPL.
"advanced and useful type system, expressive syntax out of box, and reasonable macro support"
Those things are nice, but don't have anything to do really with the question here.
Rust is designed to be 'close to the metal' and due to that fact, and a few other things - it's an 'advanced' programming language.
Javascript, Python (and PHP at least today but that's changing) will have 10x a greater user base than Rust ever will.
In the Enterprise, Java and C# are going nowhere.
Rust will be the domain of only a specific set of software developers, and they will share that space with C++.
Those things are nice, but don't have anything to do really with the question here.
Rust is designed to be 'close to the metal' and due to that fact, and a few other things - it's an 'advanced' programming language.
Javascript, Python (and PHP at least today but that's changing) will have 10x a greater user base than Rust ever will.
In the Enterprise, Java and C# are going nowhere.
Rust will be the domain of only a specific set of software developers, and they will share that space with C++.
Respectfully, I feel like you're dancing around what Python has that Rust does not. It's not the "English-like syntax" (with its invisible syntax errors). It's that Python, like most modern languages, manages memory automatically, and Rust does not.
Rust doesn't for good reason! Not just zero-cost abstraction, but also, because for the applications it's designed for, it makes controlling memory much easier. There are ways in which writing a Rust program is probably easier than writing a Java program, in Rust's stomping grounds. It's just that it doesn't make things easier anywhere else.
Rust doesn't for good reason! Not just zero-cost abstraction, but also, because for the applications it's designed for, it makes controlling memory much easier. There are ways in which writing a Rust program is probably easier than writing a Java program, in Rust's stomping grounds. It's just that it doesn't make things easier anywhere else.
I don't disagree with your premise, but I disagree with the claim that Rust doesn't automatically manage memory. Rust absolutely does automatic memory management - that's one of the main motivations behind the ownership/lifetime system. For instance, when you create a vector, the standard library will automatically allocate, re-allocate, clone and/or deallocate the buffer when necessary.
What Rust doesn't do is to decide for you how to pass or share that memory between different parts of your program - it just provides you with options. One could argue that this should also be under the automatic memory management -umbrella, and I wouldn't necessarily disagree. But I don't think it's fair to say that Rust doesn't have automatic memory management.
I would also argue that for many simpler programs you can get away with basically paying no attention to memory management. If your program is a simple pipeline which reads some input, applies some processing and produces an output, your code might not look substantially different from any GC-based language. That doesn't mean Rust is an easy language or even a good idea to use for these kinds of programs, but I would argue that a developer proficient in both Rust and Python could write the same (simple, pipeline-like) program in either language with comparable time and effort.
What Rust doesn't do is to decide for you how to pass or share that memory between different parts of your program - it just provides you with options. One could argue that this should also be under the automatic memory management -umbrella, and I wouldn't necessarily disagree. But I don't think it's fair to say that Rust doesn't have automatic memory management.
I would also argue that for many simpler programs you can get away with basically paying no attention to memory management. If your program is a simple pipeline which reads some input, applies some processing and produces an output, your code might not look substantially different from any GC-based language. That doesn't mean Rust is an easy language or even a good idea to use for these kinds of programs, but I would argue that a developer proficient in both Rust and Python could write the same (simple, pipeline-like) program in either language with comparable time and effort.
Rust is a high enough level programming language to have a GC as a library. You can go with Rc and Arc all over the place, and have a Python-style ref-counting GC.
OTOH Go is a lower-level, less expressive language with a built-in GC memory management.
To me, the level of a language is the level of abstraction one can comfortably reach with the language, and not being close to the metal or not.
OTOH Go is a lower-level, less expressive language with a built-in GC memory management.
To me, the level of a language is the level of abstraction one can comfortably reach with the language, and not being close to the metal or not.
I think nim is the really interesting one to watch here...
Compared to rust it offers, say, 99% of the performance and 70% of the safety while really being almost as easy as python.
Compared to rust it offers, say, 99% of the performance and 70% of the safety while really being almost as easy as python.
But do you have a HackerNews spamming taskforce that mentions nim in every comment?
Please don’t comment like this. Snarky accusations like this are impossible to respond to and are against the HN guidelines.
As a Rust developer, I completely agree. I'd like to see Rust fully supplant C++ and C, but I don't expect it to fully supplant scripting languages, and I definitely wouldn't suggest rewriting working Python in Rust unless you need to improve performance or scalability, or you're having serious maintainability problems. Let many languages bloom.
Shoutout to the lovely pyo3 library for letting you incrementally rewrite small parts of your Python in Rust.
> Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of application
> But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.
I dunno, I sort of hope that there'll be good and resource-light application development frameworks in Rust that do "eat the world." I spent most of a summer two years back using an (at that point) 12-year-old machine without X for both my personal and work computing. I did browsing with w3m+imlib2, images and PDFs with fbi and fbpdf (iirc), and fbterm (and tmux) as a terminal emulator. That was leagues more responsive than the firefox/eog/zathura/konsole setup I reverted to once the school year started (and I needed to run webapps again).
If you're familiar with https://en.wikipedia.org/wiki/Wirth's_law I think it applies really strongly to this case, and it's gotten remarkably worse as programmers have switched to making general applications with Python and Electron at the same time as single-threaded clock speeds have slowed down.
> But Rust will never "eat the world" of general applications (that is, the overwhelming majority of applications), because Python is good enough across all axes for these applications. It would be engineering malpractice for most companies to rewrite working Python applications in Rust. That was not the case for C-to-Python (or even PHP-to-Python) rewrites.
I dunno, I sort of hope that there'll be good and resource-light application development frameworks in Rust that do "eat the world." I spent most of a summer two years back using an (at that point) 12-year-old machine without X for both my personal and work computing. I did browsing with w3m+imlib2, images and PDFs with fbi and fbpdf (iirc), and fbterm (and tmux) as a terminal emulator. That was leagues more responsive than the firefox/eog/zathura/konsole setup I reverted to once the school year started (and I needed to run webapps again).
If you're familiar with https://en.wikipedia.org/wiki/Wirth's_law I think it applies really strongly to this case, and it's gotten remarkably worse as programmers have switched to making general applications with Python and Electron at the same time as single-threaded clock speeds have slowed down.
Python is high level 'tool' - accessible to grad students from various fields looking to do 'some things' with 'data'.
Rust is effectively for 'software developers'.
Also: ", because Python is good enough across all axes for these applications.", no, Python is good for a variety of things, but not remotely for every thing. Javascript/Typescript, C#, Java and its offspring, and probably Golang, are going nowhere. And of course Swift.
Finally: incumbency is more powerful than anyone expects. If C/C++ 'works' in any given scenario, it will be difficult to replace. So many embedded systems, so much hardware out there.
Think of how much C++ Code Adobe has invested in. Think of the massive array of sophisticated Autocad products all written in C++. They are not going to be refactored. The native parts of the JVM, Oracle DB, the list goes on.
I predict in 10 years 'greenfield' projects might favour Rust, but many won't.
Rust is effectively for 'software developers'.
Also: ", because Python is good enough across all axes for these applications.", no, Python is good for a variety of things, but not remotely for every thing. Javascript/Typescript, C#, Java and its offspring, and probably Golang, are going nowhere. And of course Swift.
Finally: incumbency is more powerful than anyone expects. If C/C++ 'works' in any given scenario, it will be difficult to replace. So many embedded systems, so much hardware out there.
Think of how much C++ Code Adobe has invested in. Think of the massive array of sophisticated Autocad products all written in C++. They are not going to be refactored. The native parts of the JVM, Oracle DB, the list goes on.
I predict in 10 years 'greenfield' projects might favour Rust, but many won't.
> incumbency is more powerful than anyone expects
I agree, and would also point out that the competitors aren't static targets. D seems to be well on its way to getting a borrow checker. Well written C++20 looks nothing like C++98. Etc.
I agree, and would also point out that the competitors aren't static targets. D seems to be well on its way to getting a borrow checker. Well written C++20 looks nothing like C++98. Etc.
Yes! The number of settings where $RUST_COMPETITORS are applicable is simply smaller than the number of settings where a high-level, dynamic or at least garbage-collected language is applicable. It's hard for me to imagine having a real justification for building services in Rust given the relative productivity of those high-level languages.
This is basically the reason I haven't picked it up for any personal projects, either. I admire the goals, but my interests don't call for a C++ killer.
This is basically the reason I haven't picked it up for any personal projects, either. I admire the goals, but my interests don't call for a C++ killer.
Ok, you've convinced me. What's the best way to learn Rust in 2020?
The Rust book, which is free online, an idea of a simple program you might have written in some other language, and a text editor.
> Python is incredibly popular because its runtime performance is "good enough" for a huge fraction of applications
Is there anything slower than pure python?
Is there anything slower than pure python?
Interpreted BASIC, of which (in my opinion) Python is the spiritual successor.
I took an online class in college on Python and one of the assignments required the program to scan a plaintext wordlist for a word. I hesitantly made Python load in the entire wordlist and split it by newlines into values in a list, before something like "if word in wordlist". I fully expected my computer to grind to a halt because the interpreted language was loading in a four megabyte textfile, and I expected to have to make a much more complicated function to scan to see whether the word was in the list. I ran it and it ran in less than a second before printing whatever I put to signal that it worked. I didn't believe it so I ran it again with the same result.
Python is like BASIC if it was stable enough to be mission-critical and fast enough to work where it needs to. My previous experience with interpreted languages was with TI-BASIC and it blew my mind that my laptop was powerful enough to do something that fast. Sure, it's slow, but if you really need speed you're better off writing something in something like C or FORTRAN (or Rust now) anyway.
I took an online class in college on Python and one of the assignments required the program to scan a plaintext wordlist for a word. I hesitantly made Python load in the entire wordlist and split it by newlines into values in a list, before something like "if word in wordlist". I fully expected my computer to grind to a halt because the interpreted language was loading in a four megabyte textfile, and I expected to have to make a much more complicated function to scan to see whether the word was in the list. I ran it and it ran in less than a second before printing whatever I put to signal that it worked. I didn't believe it so I ran it again with the same result.
Python is like BASIC if it was stable enough to be mission-critical and fast enough to work where it needs to. My previous experience with interpreted languages was with TI-BASIC and it blew my mind that my laptop was powerful enough to do something that fast. Sure, it's slow, but if you really need speed you're better off writing something in something like C or FORTRAN (or Rust now) anyway.
Ruby.
yes: human beings.
For basically everything we can feasibly ask python to do, it can do that thing faster than we could on our own; few are the things that we need done faster than that.
For basically everything we can feasibly ask python to do, it can do that thing faster than we could on our own; few are the things that we need done faster than that.
I'm pretty sure that's exactly what he meant.
Javascript did this.
Sure, I mean, in my head, Python, Ruby, and Javascript are all lumped together (though Python appears to be by far the most popular of them in commercial settings).
I’ve really only seen python used in academics and data processing.
In the almost 4 years I spent doing security for startups, Python was overwhelmingly the most popular language, not just among clients but among every startup we talked to. That's not counting data science teams.
(I'm not really a Python fan, for what it's worth.)
(I'm not really a Python fan, for what it's worth.)
My recent experiences with startup consulting have involved a lot more new node projects than new python ones. Though I also lump those three languages together as all doing mostly the same thing.
They are close contestant for sure, but don't underestimate the popularity of Javascript among developers: https://madnight.github.io/githut/#/pull_requests/2020/2
There are more niches, industries, and developer cultures than any one of us can fathom.
PHP to Python rewrites? That seems more foolish then Python to Rudt. I have seen a few Python to PHP rewrites.
Python is taught in school instead of java now which took over c/c++ as the default language.
Python is taught in school instead of java now which took over c/c++ as the default language.
I've been keeping an eye on crates.io stats - it's definitely booming. ~50 new crates a day (some of them are total nonsense/tutorial work/etc.).
I keep wondering about whether the hype is justified - I must be slowing down because I find myself hanging for long periods of time on borrow checker errors. One of the errors has stopped my progress dead for a week now, I swear it worked a week ago and then Rust decided that a borrow I was doing was no good.
I've also heard the borrow-checker is a common hold up for new Rust developers, so I keep slogging on.
I keep wondering about whether the hype is justified - I must be slowing down because I find myself hanging for long periods of time on borrow checker errors. One of the errors has stopped my progress dead for a week now, I swear it worked a week ago and then Rust decided that a borrow I was doing was no good.
I've also heard the borrow-checker is a common hold up for new Rust developers, so I keep slogging on.
I'm also relatively new to Rust, just about to ship my first ever project written with it. Borrow checker errors are definitely tricky. In my experience, it usually indicates that you've done something "the wrong way," which means you need to re-think it and architect it somehow else. There's often not a quick fix (unless you just made a typo or something, of course). For example, think about who owns the data in question, why it's trying to be stored in two places at once, and how you can re-arrange your data structures or code flow to avoid that condition. It gets harder to understand once lifetimes get involved, I still have to dig into the book to understand the ins & outs of that, and its syntax. Once you get the hang of writing code in a Rust-friendly manner, borrow checker errors mostly just don't happen, because you wrote your code "the right way" to begin with.
I hope I described that well. It's a fascinating language, and now that I'm on the far side of the learning curve, I find it very pleasant to use. I urge you not to give up.
(Have you read through and written, built, and played with the examples in the book? I found it extremely helpful to follow the exercises, and intentionally break them, to test my understanding. https://doc.rust-lang.org/book/ )
I hope I described that well. It's a fascinating language, and now that I'm on the far side of the learning curve, I find it very pleasant to use. I urge you not to give up.
(Have you read through and written, built, and played with the examples in the book? I found it extremely helpful to follow the exercises, and intentionally break them, to test my understanding. https://doc.rust-lang.org/book/ )
You're absolutely right, I think your explanation is well written. I did get past the error that stopped me for a week, and yeah it is a case of tackling the same workload but in a different way (I guess 'the right way'). What surprised me is that the solution came to me while I wasn't looking at the compiler or source or anything, but was more of an "ah ha!" moment while walking around. This indicates to me that maybe I will eventually come to understand the borrow checker. Also that book is fantastic, I find myself regularly reading/re-reading it as well as "Rust By Example". Also the std docs are great to work through when I am trying to figure out a way to accomplish something.
1) Use rust-analyzer if you aren't already. It decreases the time it takes to see compiler/borrow error results by about 10x. This helps enormously when you're banging your head against something, taking guesses to see what might work.
2) It's often better to slow down and think through what's going on. Make sure you have a good understanding of pointers, allocation, ownership, etc, and if you've been fighting with something for more than 10 minutes see if you can actually understand why it's upset. Rust makes it easier not to accidentally mess these things up, but you will still be chronically frustrated if you don't have a good grasp on them.
3) I've found that the more I use it the more I get an intuition for what fixes certain errors. I don't feel great about fixing something without fully understanding it, but for certain common cases it can really help with the problem of iteration speed.
2) It's often better to slow down and think through what's going on. Make sure you have a good understanding of pointers, allocation, ownership, etc, and if you've been fighting with something for more than 10 minutes see if you can actually understand why it's upset. Rust makes it easier not to accidentally mess these things up, but you will still be chronically frustrated if you don't have a good grasp on them.
3) I've found that the more I use it the more I get an intuition for what fixes certain errors. I don't feel great about fixing something without fully understanding it, but for certain common cases it can really help with the problem of iteration speed.
A week? Might make sense to refactor, or just clone the data? Or ask the subreddit, discord, or whatever place you want for help.
I haven't run into a borrowchecker issue that took me more than a few seconds in a few years. The trick is I'm not crazy about avoiding cloning - Rust is still super fast. The curse of Rust making cloning explicit, and move by default being cheap/ memcopy, is that you often feel like "if only I could express it" and that problem seems to get worse the better you are, and then for me I hit a point where I stopped caring and just got shit done.
It's easy to remove clones later in my experience, once benchmarks show it matters.
I haven't run into a borrowchecker issue that took me more than a few seconds in a few years. The trick is I'm not crazy about avoiding cloning - Rust is still super fast. The curse of Rust making cloning explicit, and move by default being cheap/ memcopy, is that you often feel like "if only I could express it" and that problem seems to get worse the better you are, and then for me I hit a point where I stopped caring and just got shit done.
It's easy to remove clones later in my experience, once benchmarks show it matters.
Rust has one curse (it has many, but this one is critical): inefficient code is generally visible. Experienced developers hate to notice that their code is inefficient. They will balk at seeing `Arc<RefCell<T>>`, but won't bat an eye at using Python. I know because I have the same instinct! This makes it much harder to learn Rust for experienced devs because they go from the "simple code that will work but is slightly inefficient" and in an effort to improve it they land squarely in parts of the language they haven't yet developed a mental model for.
The most important mental model to develop is a model of what can and cannot be feasibly optimized. Sometimes there really is no alternative to stuff like Rc<RefCell<>>, Arc<RwLock<>> or Arc<Mutex<>> - because you'll be working with pieces of data that have to be mutated from multiple places, and don't even have a well-defined lifetime so have to be reference counted. In that case, you should write a comment to that effect and move on. It's still good to see that RefCell<> and the like, because shared mutability is known to be a source of logical bugs. It's a bit like a low-key `unsafe`.
Yes, a week (no shame, damnit). The case was that I couldn't understand what the problem even was. I understood that there can be only one owner, but I didn't see how what I was doing was a problem. I literally looked at every error message it gave me and was like "so what?"
I think Rust initially suggested I implement the Copy trait or something, and so I had to brush up on traits and all that (which I did), only to find that my types can't have traits because vec<T> doesn't implement the Copy trait. So I abandoned that.
The problem turned out to be that I needed a minor refactoring. I think the issue eventually turned out to be that I was mutably iterating through a list reading from TCPStreams, and then trying to write back to those streams. So it was like a read_object_from_stream(stream) on each element in a vector, and then the next line was a broadcast_to_all_streams(message) (I'm psuedocoding).
The fix for me (when I finally came to understand what was happening) was to read_object_from_stream(stream) in to an intermediary vector (I named it cache), and THEN iterate through the intermediary and broadcast each item.
In my notes I paraphrased it as "Instead of doing a readwrite, I separated my algorithm in to a READ and then a separate WRITE", because the way things were written I was double borrowing.
I think Rust initially suggested I implement the Copy trait or something, and so I had to brush up on traits and all that (which I did), only to find that my types can't have traits because vec<T> doesn't implement the Copy trait. So I abandoned that.
The problem turned out to be that I needed a minor refactoring. I think the issue eventually turned out to be that I was mutably iterating through a list reading from TCPStreams, and then trying to write back to those streams. So it was like a read_object_from_stream(stream) on each element in a vector, and then the next line was a broadcast_to_all_streams(message) (I'm psuedocoding).
The fix for me (when I finally came to understand what was happening) was to read_object_from_stream(stream) in to an intermediary vector (I named it cache), and THEN iterate through the intermediary and broadcast each item.
In my notes I paraphrased it as "Instead of doing a readwrite, I separated my algorithm in to a READ and then a separate WRITE", because the way things were written I was double borrowing.
Could you share what the problem you're having is? People in https://users.rust-lang.org/ and https://reddit.com/r/rust are more than happy to help, and if I could peek at a repro case running in https://play.rust-lang.org/ I could try to help you too (and potentially fix the diagnostic in the compiler). Also, depending on what the problem is, you might have an easier to understand error in newer versions of the compiler, which is why I recommend newcomers to use nightly and update often: not because they'll use advanced nightly-only features but because we merge diagnostics improvements aimed at newcomers every week.
Also, don't get too discouraged. If you can avoid doing things that require advanced lifetime checking then you can gain enough experience with the rest of the language letting you focus only on that part of the language at a later time. Also, I can confidently tell you that at some point it clicks and you start anticipating things that can cause issues.
Also, don't get too discouraged. If you can avoid doing things that require advanced lifetime checking then you can gain enough experience with the rest of the language letting you focus only on that part of the language at a later time. Also, I can confidently tell you that at some point it clicks and you start anticipating things that can cause issues.
Thank you for your response. I did figure out the problem after I took the time to slowly and methodically understand what the compiler messages were saying - that being said I was looking for places to ask for help if another week went by and I was still stuck so your resources are in my favorites list now.
As I wrote in another post, I came up with the solution while out walking around and it was like an "Ah Ha!" moment, I think things are finally starting to click a bit more. I purposefully chose to write a chat server/client because it lets me work through language features one at a time. There is some basic printing/reading of console input (though I did go full blown Cursive), some networking, threading, and lots of string manipulations. I am not sure when lifetimes will come in to play, but I have started reading about them and I try to think about the lifetimes of my objects when I write.
As I wrote in another post, I came up with the solution while out walking around and it was like an "Ah Ha!" moment, I think things are finally starting to click a bit more. I purposefully chose to write a chat server/client because it lets me work through language features one at a time. There is some basic printing/reading of console input (though I did go full blown Cursive), some networking, threading, and lots of string manipulations. I am not sure when lifetimes will come in to play, but I have started reading about them and I try to think about the lifetimes of my objects when I write.
Don't get too hung up on fancy structs containing borrowed data with complicated lifetimes. Zero-copy operations via references are an optimization that you can often forego unless you need really high performance code.
I remember it taking me a solid month or two (of side-project experimentation, not daily work) before I felt like I'd really internalized the right mental model for ownership and borrowing.
I'm about a month in. I decided to write a chat server/client to learn Rust by. Multiple side-projects sound like it would also be advantageous. Thanks for your comment.
Please don’t hesitate to ask for help! The forums and discord are there as a resource.
Out of the box, Rust encourages almost everything to be stack allocated, imposes a read-write lock on every value (i.e., you can have multiple immutable readers, but only one mutable writer at a time), and uses move semantics for variable assignments (i.e., values aren't copied but rather ownership of the value changes from one variable to some other variable).
Sometimes you need to drop one or several of these properties. It's very easy to do that -- Rust just makes you be explicit about it. If you want something to be heap allocated, you can use a Box/Rc/etc. If you want a value shared among multiple writers, there are several options; a common one is Arc<Mutex<T>>, which is also thread-safe. If you don't want move semantics, you can just copy the value using .clone().
A few lessons to take away, though:
* Because of that read/write lock I mentioned, it follows that you generally want to avoid taking a mutable reference to an entire data structure, because doing so effectively puts that writer lock on the whole thing. If you can take a mutable reference only to the piece you actually care about, that can help out a lot in some cases.
* Rust makes you feel like you should never clone, but really it's up to you. Exercise basic judgment that you'd use in any other language, and don't let perfect be the enemy of the good. In many cases it's probably not nearly as expensive as you might feel it is, and sometimes it'll just be optimized away in the end anyway.
* Try to stick with structs and data structures that own their data as your default, deviating when it makes sense. I've seen people try to avoid ever cloning values by having many structures only contain references, which leads into complicated lifetime management that they weren't prepared for (and that may not even be possible or make sense).
* Take a moment to understand the various wrapper types (I like [0]). Intuitively, Rust gives you a set of guarantees that you can ask for, and you just pick the guarantees you need and compose them together.
[0] https://manishearth.github.io/blog/2015/05/27/wrapper-types-...
Sometimes you need to drop one or several of these properties. It's very easy to do that -- Rust just makes you be explicit about it. If you want something to be heap allocated, you can use a Box/Rc/etc. If you want a value shared among multiple writers, there are several options; a common one is Arc<Mutex<T>>, which is also thread-safe. If you don't want move semantics, you can just copy the value using .clone().
A few lessons to take away, though:
* Because of that read/write lock I mentioned, it follows that you generally want to avoid taking a mutable reference to an entire data structure, because doing so effectively puts that writer lock on the whole thing. If you can take a mutable reference only to the piece you actually care about, that can help out a lot in some cases.
* Rust makes you feel like you should never clone, but really it's up to you. Exercise basic judgment that you'd use in any other language, and don't let perfect be the enemy of the good. In many cases it's probably not nearly as expensive as you might feel it is, and sometimes it'll just be optimized away in the end anyway.
* Try to stick with structs and data structures that own their data as your default, deviating when it makes sense. I've seen people try to avoid ever cloning values by having many structures only contain references, which leads into complicated lifetime management that they weren't prepared for (and that may not even be possible or make sense).
* Take a moment to understand the various wrapper types (I like [0]). Intuitively, Rust gives you a set of guarantees that you can ask for, and you just pick the guarantees you need and compose them together.
[0] https://manishearth.github.io/blog/2015/05/27/wrapper-types-...
Generally a language with automatic garbage collection is easier to work with. Rust can still be useful in areas like the kernel.
Assuming this happened and some essential drivers were written in Rust, would this effectively exclude GCC from being used to compile the full kernel?
GCC can still compile the components written in C. Only drivers for select hardware might be written in Rust in the future, optional components basically. And GCC won't be more disadvantaged than clang, as clang can't compile Rust code either. The only production compiler that can compile Rust is rustc.
This answer was a very "no"-sounding "yes". :)
rustc uses LLVM like clang. That seems important because features like LTO require the whole kernel to be compiled with the same toolchain.
It seems to me there isn't mainline LTO support yet?
Work in progress. The series was resent two weeks so. We've been shipping it in production in kernels in Android for years.
Is it possible to disable some features in rust to avoid full recompilation of the kernel?
I'm not sure what that means, but I can discuss the issue with LTO. LTO means that the normal compile step turns the source into an intermediate form. The final compilation doesn't happen until link time. The problem is this intermediate form which is a compiler (even compiler version) -specific internal format. GCC and LLVM use different formats so you cannot mix the two compilers on the same LTO program. I'm not certain but it might be possible to use ld -r to obtain a real compiled object from each toolchain and then link those together at the end, but of course that loses some benefits of LTO, especially cross-module inlining.
I wonder if/when GCC will have a Rust frontend
There seems to be some discussions but not sure if it's a goal
There seems to be some discussions but not sure if it's a goal
That would certainly help with some concerns around Rust; compiler diversity and target platform support would both benefit.
So I don't know much about kernel development.
How many other languages apart from C are there in the kernel?
How many other languages apart from C are there in the kernel?
Depends how you define "in the kernel". In terms of what is compiled into the kernel image, it's C plus small pieces of assembler (both inline and in separate files). This proposal (if it ever happens) would be about adding Rust to that. If you mean the tree used to build the kernel, you could say it includes shell script, Makefiles, the Kconfig descriptions, and more. If you mean all the bits you need to have a very minimally working system which boots, you'd want to include all the languages used for the necessary userspace tools like the init system and udev (although the vast majority of that is also C and shell scripts).
I'm a bit uninformed as well. Obviously it must use assembly in some places, it has makefiles for building and some scripts in perl. Here's what a cloc clone has to say.
tokei
===============================================================================
Language Files Lines Code Comments Blanks
===============================================================================
ASN.1 11 484 343 33 108
Assembly 3 2751 2450 0 301
GNU Style Assembly 1296 370153 270691 56418 43044
Autoconf 6 347 303 20 24
Automake 3 31 23 3 5
BASH 54 1991 1438 321 232
C 28971 19846129 14634535 2351170 2860424
C Header 21110 6056898 4465297 1010398 581203
C++ 8 2313 2016 68 229
C++ Header 2 125 59 55 11
CSS 1 137 80 29 28
Device Tree 3351 900862 735380 54238 111244
Gherkin (Cucumber) 1 252 174 50 28
Happy 9 5750 5061 0 689
HEX 1 86 86 0 0
INI 1 7 3 3 1
JSON 328 174872 174872 0 0
LD Script 15 412 317 48 47
Makefile 2587 64666 43896 10957 9813
Module-Definition 2 124 109 0 15
Perl 53 40583 31126 3931 5526
Python 118 36932 29072 2980 4880
ReStructuredText 2621 548840 417105 0 131735
Ruby 1 29 25 0 4
Shell 556 85666 59725 10972 14969
SVG 59 39038 37555 1405 78
Plain Text 3230 239573 0 197373 42200
Unreal Script 5 716 446 162 108
Vim script 1 42 33 6 3
XSL 15 300 183 78 39
YAML 921 91167 72962 3985 14220
===============================================================================
Total 65340 28511276 20985365 3704703 3821208
===============================================================================I was surprised to see C++ in there, but I understand it is simply some of the user-space tooling (in particular, some small parts of the perf tool are written in C++, in order to embed clang/LLVM.)
Likewise, Perl/Python/Ruby are just scripts included for various purposes (such as building documentation).
Interestingly, some of the assembler code included in the kernel is generated by Perl scripts (crypto code for ARM and MIPS). Perl is also used to compile "SCSI SCRIPTS" which are used by the NCR 53c810 SCSI driver.
Likewise, Perl/Python/Ruby are just scripts included for various purposes (such as building documentation).
Interestingly, some of the assembler code included in the kernel is generated by Perl scripts (crypto code for ARM and MIPS). Perl is also used to compile "SCSI SCRIPTS" which are used by the NCR 53c810 SCSI driver.
Assembler. Although there is a movement to port assembler to c.
Which is impossible with ISO C alone, so it will either stay Assembly, just written inline, or replaced by CPU compiler specific intrinsics, yet another GCC C feature.
It's true that some of it will remain assembly forever, but there are large chunks of existing assembly in the kernel that can be ported to regular C and where the current code paths are getting too long to be effectively maintainable in assembly - you can split out the parts that specifically touch magic registers, etc etc and keep those in asm while moving more complex logic into C.
what does linux kernel in-tree support mean? that drivers can be written in rust or something?
Yes. In tree means their sources are in the Linux kernel git repository, as opposed to out of tree drivers. In tree drivers are maintained by the maintainers, so API changes are done together for all in tree drivers. Out of tree drivers don't get this benefit, and are more difficult to build in my experience.
thank you!
Tangential question: what would be a career path for a typical product developer to become one of the programming gods who get invited to such discussions? (Even if it's technically open mailing list, you know what I mean). How many years of experience and how much formal CS education would it take to acsend to that level?
They aren't gods, just normal programmers with specific interests. The knowledge and skill base involves details that are close to the machine. It is less of a "deep CS" thing and more just being patient and interested in details of old things, rather than new things. That is why they self-deprecatingly refer to themselves as plumbers.
The big difference between eg product dev and kernel dev is mindset, not skill. Product dev is about novelty, building new features, inventing new tools. Kernel dev is about making old stuff work better.
Very few kernel devs would succeed in a conventional product dev job, instead of creating value they would be tinkering with some superfluous detail.
If one has the mindset, getting into kernel programming is probably a six month to 2-year initial investment, depending on prior background. Most "operating system" CS grad courses introduce kernel programming as a teaching tool, but a whole CS undergrad isn't required, just an understanding of C and some other basics around low level machine operation.
(Edit: side note- for emphasis- kernel programming is highly specific, extremely empirical and experimental. Not abstract or theoretical. Academic CS tends toward the abstract/theoretical. One can be a poor academic but an excellent kernel programmer.)
The best standalone resource is a book by a fellow named Robert Love called Linux Kernel Development. One of the best technical books I have read and a good indicator of the presence of the mindset.
Cheers!
The big difference between eg product dev and kernel dev is mindset, not skill. Product dev is about novelty, building new features, inventing new tools. Kernel dev is about making old stuff work better.
Very few kernel devs would succeed in a conventional product dev job, instead of creating value they would be tinkering with some superfluous detail.
If one has the mindset, getting into kernel programming is probably a six month to 2-year initial investment, depending on prior background. Most "operating system" CS grad courses introduce kernel programming as a teaching tool, but a whole CS undergrad isn't required, just an understanding of C and some other basics around low level machine operation.
(Edit: side note- for emphasis- kernel programming is highly specific, extremely empirical and experimental. Not abstract or theoretical. Academic CS tends toward the abstract/theoretical. One can be a poor academic but an excellent kernel programmer.)
The best standalone resource is a book by a fellow named Robert Love called Linux Kernel Development. One of the best technical books I have read and a good indicator of the presence of the mindset.
Cheers!
Formal CS education is pretty much useless for systems programming. Systems programming deals with real-world hardware, CS is theorizing about about fantasy machines. "Computers are to Computer Science what telescopes are to Astronomy." They're just different fields. They intersect, of course, but systems programming isn't very near that intersection. Being good at analyzing a DFA on a Turing machine isn't going to be much help when you need to bang bits to a non-compliant USB device.
If you want to get into these discussions, you need to be a kernel dev. That means you need to write kernel stuff. Probably the easiest thing is to write or improve a driver for some device you have (game controllers; steering wheels; audio devices; drawing tablets). It's not rocket science, it's all just bits and bytes and code, just like any other programming.
Edit: Just for an example, here's where the PlayStation controller driver parses the data it gets from the device into something applications can read over the input subsystem. Don't be intimidated, there's nothing magic here, just C code reading data and transforming it. Suppose you maintained this driver and wanted to re-write it in Rust. You could have some input into the discussion! https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
If you want to get into these discussions, you need to be a kernel dev. That means you need to write kernel stuff. Probably the easiest thing is to write or improve a driver for some device you have (game controllers; steering wheels; audio devices; drawing tablets). It's not rocket science, it's all just bits and bytes and code, just like any other programming.
Edit: Just for an example, here's where the PlayStation controller driver parses the data it gets from the device into something applications can read over the input subsystem. Don't be intimidated, there's nothing magic here, just C code reading data and transforming it. Suppose you maintained this driver and wanted to re-write it in Rust. You could have some input into the discussion! https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
Except many universities don't teach CS like that, I guess that might be an US thing.
In the Portuguese university where I took my degree, for example, I had to write an UNIX floppy device driver, code in 80x86 and MIPS Assembly, implement my own userspace threading library, implement a distributed algorithm for maldebrot calculations across a Linux cluster via PWM (MPI predecessor) and report its scalability varying the amount of available machines on the cluster, create a compiler, stuff SUN RPC generated stubs with our own encryption algorithm, implement a B+Tree indexing library using inodes directly.
All look pretty useful to me.
In the Portuguese university where I took my degree, for example, I had to write an UNIX floppy device driver, code in 80x86 and MIPS Assembly, implement my own userspace threading library, implement a distributed algorithm for maldebrot calculations across a Linux cluster via PWM (MPI predecessor) and report its scalability varying the amount of available machines on the cluster, create a compiler, stuff SUN RPC generated stubs with our own encryption algorithm, implement a B+Tree indexing library using inodes directly.
All look pretty useful to me.
And How long ago was that? :)
I would assume most CS in EU dont teach that anymore.
I would assume most CS in EU dont teach that anymore.
Mid-90's, so around 30 years ago.
My former university surely does it, although obviously not with the same exact content.
https://www.di.fct.unl.pt/en/education/integrated-master-com...
My former university surely does it, although obviously not with the same exact content.
https://www.di.fct.unl.pt/en/education/integrated-master-com...
Re: the first paragraph--a lot of "formal CS education" focuses on systems and there are systems tracks. Where BSD came from, for example. So I wouldn't call it useless (while the parts that do not focus on systems do, indeed, not focus on systems). In fact, even the non-systems stuff isn't entirely useless for systems--eg. how ideas from ML and Cyclone influence Rust and its application of those ideas to systems use cases.
That said, all this is independently learnable, as is true for all things software.
That said, all this is independently learnable, as is true for all things software.
CS education typically includes OS theory, computer architecture, and compiler theory, all of which are directly relevant to the kernel.
You may be right that EE is perhaps a better fit for this kind of interest.
You may be right that EE is perhaps a better fit for this kind of interest.
Honest question: why is this downvoted? Is it bad advice? Which of it is?
I upvoted, it's all good advice, but the blanket statement that "formal CS is useless..." is a little strong. CS undergrad is generalist and mostly about types of problems and surveys of solution flavors and POCs of specific solutions in domains that are hot or cool or whatever and geared to students who are just trying to get through and don't have time or patience to work through bespoke granular details. Getting the idea is good enough for a B. Kernel programming is largely the opposite in all respects.
Many topics, problem domains, solution approaches e.g. usually covered in CS undergrad are not relevant for kernel programming. Not useless, but not relevant.
Kernel programming relative to most CS undergrad courses is like taking a semester-long lab, but starting from the best A+ project from the last 5 years, and instead of newly solving the problem, spending time working on understanding why the best A+ solution still sucks, under what edge conditions it fails, and then testing how it can be made better.
Many topics, problem domains, solution approaches e.g. usually covered in CS undergrad are not relevant for kernel programming. Not useless, but not relevant.
Kernel programming relative to most CS undergrad courses is like taking a semester-long lab, but starting from the best A+ project from the last 5 years, and instead of newly solving the problem, spending time working on understanding why the best A+ solution still sucks, under what edge conditions it fails, and then testing how it can be made better.
The problem I perceive with Rust as a language for Linux, is not that Rust is not an excellent language. It is that it evolves rather quickly and code may break when you switch versions.
Does rust/llvm even support all the CPU architectures Linux supports?
If not how will that work? Will some drivers be then limited to only some architectures?
If not how will that work? Will some drivers be then limited to only some architectures?
No, but it's already the case that some drivers are already tightly coupled to an architecture. Does it make sense to build a driver for some internal piece of hardware from an ARM SoC for PPC? Generally, no.
The kernel contains vast number of drivers that are for universal buses like I2C, SPI, USB, PCI(e), SDIO, UART, writing any of these in rust would make these drivers unavailable to architectures unsupported by llvm. I think that would be quite sad.
[deleted]
Right. I don't think anyone is proposing ReWrItE iT iN rUsT. There's way way more drivers that aren't buses (platforms) that don't need the level of portability we're talking about.
I'm not talking about buses, but regular drivers for devices on those buses. The buses are universal, that means that they can be supported on any architecture, so drivers for devices on those busses should not be limited to any single arch just by the choice of the language.
While I like Rust I still picked Modern C++(17) for my next project. All modern languages are converging these days.
Adopted C++ because of its libraries at large. Will still learn Rust on the side. For the moment, boring and verified for product especially when they're updating and remaining up to date.
Adopted C++ because of its libraries at large. Will still learn Rust on the side. For the moment, boring and verified for product especially when they're updating and remaining up to date.
To bootstrap Rustc you have to compile every single version. Say goodbye to reproducible builds. Say goodbye to civilians with cheap laptops being able to compile their kernel.
This is a strawman argument. To be reminded of what it takes to truly "bootstrap" a C compiler, I recommend everyone to read Ken Thompson's Turing Award lecture "Reflections on Trusting Trust".
gcc can still be bootstraped from just an assembler and linker[1] with very few binaries you have to trust and it is possible on comoddity hardware. Bootstrapping Rust is so difficult that adding it to Linux would for all purposes eliminate this feature (being able to be built completely from source) from Linux, probably leading to a fork of Linux without Rust for those that care about it.
[1] https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed...
[1] https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed...
mrustc can shorten that bootstrap, and also, many Linux distros already ship a rustc that they’ve bootstrapped as far back as they thought prudent. So yes, I can see some folks doing something like this, but I doubt that the major distros would.
Happy to see happening!
But to get the most value out of Rust, one needs to use the Cargo ecosystem too (even for kernel code!) and that will be a much bigger political quagmire.
But to get the most value out of Rust, one needs to use the Cargo ecosystem too (even for kernel code!) and that will be a much bigger political quagmire.
We've got Cargo wired up in our prototype, which can let you link modules against no_std crates on crates.io: https://github.com/fishinabarrel/linux-kernel-module-rust
Probably one good end state is if the core kernel exposes appropriate safe abstractions, and vendors with out-of-tree modules (which tend to be worse at security than the main kernel) could use those APIs + things from crates.io to build their modules.
But yes, figuring out how to use useful things that aren't in libcore is one of the open questions.
Probably one good end state is if the core kernel exposes appropriate safe abstractions, and vendors with out-of-tree modules (which tend to be worse at security than the main kernel) could use those APIs + things from crates.io to build their modules.
But yes, figuring out how to use useful things that aren't in libcore is one of the open questions.
Glad to hear it! I just hope Linux doesn't end up with the crates.io vendoring nightmare that the big-corps made for themselves.
My only complaint about Rust is terminology.
"Immutable variable" what an oxymoron!
That's what is known as a constant!
A variable is mutable, a constant is not. Was it that hard to understand?
"Immutable variable" what an oxymoron!
That's what is known as a constant!
A variable is mutable, a constant is not. Was it that hard to understand?
Hmm... Might be time to start looking into Rust. Is it popular in college? I think that's what made Python displace Java.
EDIT: lol. Decided to dive in a hit this almost immediately: https://docs.rust-embedded.org/book/start/io.html
EDIT: lol. Decided to dive in a hit this almost immediately: https://docs.rust-embedded.org/book/start/io.html
That's the guide to embedded programming. If you're just looking to learn Rust, start here: https://doc.rust-lang.org/book/
Rust is probably never going to be the ideal first language for anyone to learn. This is a problem because a lot of people will turn to the language they first learnt, which they know best, when faced with any given problem :-(. But it makes sense that the easiest language to learn might not be the best language to use to get real work done...
No, but C/C++ is widespread and the relative understandability of Rust for those coders and advantages of Rust will certainly aid its adoption.
You can count the college courses on two (or maybe three) hands. Not quite there yet.
Hopefully this will not pass - The requirement of having GCC + LLVM + Rust installed in order to build kernel would be insane.
As someone new to the dev party who has never built anything more lower level than a win32 exe, may I ask why that's insane?
Adding 1000 npm dependencies, I see. Adding 1 rustc one? Doesn't SEEM terrible.
Adding 1000 npm dependencies, I see. Adding 1 rustc one? Doesn't SEEM terrible.
We only just got to a point where 2 C compilers could build Linux; adding rustc brings us back to a monoculture.
Playing devil's advocate, limiting Linux at first to GCC only allowed it to heavily use GCC extensions (inline assembly, statement expressions, and many others), which wouldn't be the case if it had to be portable to different compilers. The other compilers which could build Linux (ICC, sparse, now clang - compiling the Linux kernel with compilers other than GCC is not a new thing) had to implement many of these useful GCC extensions (for instance, clang is compatible with GCC 4.2.1), which made these compilers better.
I suppose there's something to be said for it pushing state-of-the-art and compiler teachers, but it does so in a gcc-centric way and I'm not quite as enthusiastic about that.
It sounds like you're saying that you should never use any programming language until it's supported by at least 2 compilers. Is that accurate?
Without a language specification for Rust, any compiler that isn't the reference implementation, by definition, cannot be correct. Put another way, how is a compiler writer supposed to know what is and is not a bug in the reference implementation?
Most languages do not have a specification, and it works out fine.
Rust's is in-progress. There are some things that are well-specified, and there are some things that are still up in the air.
Rust's is in-progress. There are some things that are well-specified, and there are some things that are still up in the air.
Right, which is why it's a poor choice for the kernel at this time.
Not for anything important, no. For a fun side project, 1 compiler is fine. For a business that depends on a build chain, it's sensible to at least have a plan for switching. For a major OS kernel, even 2 compilers is too low for my comfort.
Should businesses who exclusively build iOS apps still be using Objective-C then, since Swift only has one compiler?
Honestly, that would give me pause if I were targeting iOS. It helps that if you're targeting iOS you already live and die at Apple's whim, so maybe that's a sunk cost; on the other hand, that's the kind of thing that would make me nervous about living in Apple's walled garden in general.
For the Linux kernel? Absolutely. Rust is way too unstable right now for such an important piece of software. Give it 20 years and a standard, then we can talk.
Based on what? The relevant folks and decision makers are already talking, so it doesn’t sound like they think there’s a need to wait 20 years to at least start the conversation.
Writing some optional, non-essential piece != rewriting a piece that will require taking rust as a dependency for the rest of time.
The kernel folks just did a lot of work to remove the strict dependency on GCC. To think any of the big players [Red Hat et al] want to turn around and make their shining star dependent on a language that's been 1.0 for less than 5 years is just not realistic.
The kernel folks just did a lot of work to remove the strict dependency on GCC. To think any of the big players [Red Hat et al] want to turn around and make their shining star dependent on a language that's been 1.0 for less than 5 years is just not realistic.
You could argue not having memory safety when there are tools to do so is insane.
Memory safety is not an absolute good which overrides all other things, like performance, portability, compilation, process, compile time etc.
It is a problem that is well understood and the kernel developers have a lot of experience and tools for dealing with it.
Would it be worth solving well? Definitely, but it has costs and tradeoffs.
It is a problem that is well understood and the kernel developers have a lot of experience and tools for dealing with it.
Would it be worth solving well? Definitely, but it has costs and tradeoffs.
I wouldn't be too worried about it. The kernel people aren't going to take a dependency on something that isn't standardized and doesn't have a spec.
Don't be so sure. Rust lobbying is strong and kernel maintainers don't take a tough stance against things like they used to.
Now there is no legitimate reason to lock out C++.
There are plenty of reasons. As just one of them: C++ is unsafe by default. There's theoretically a safe subset, but there's no precise or consensus definition of that subset. There's no type-system or language support for helping you stay strictly within that subset (and requiring some kind of marker for if you don't). Often, the most obvious way of doing something is unsafe; for instance, in C++, arr[i] is unsafe, and you have to write arr.get(i) to be safe. (In Rust, arr[i] and arr.get(i) are both safe, and you have to write arr.get_unchecked(i) in an unsafe block if you truly want an unsafe indexing operation.)
Can you help me understand why locking C++ out of a kernel written almost entirely in C, and which receives huge volumes of new C code every year, wouldn't be an instance of making the perfect the enemy of the somewhat marginally better?
I would question the "somewhat marginally better" part.
(Please take all of this with a grain of salt, as I do not write "modern C++" in any professional capacity; its complexity has grown far beyond my knowledge.)
I think introducing C++ into the kernel (and getting value from it) would require effort on par with introducing Rust, to provide safer interfaces for things. It would also require a substantial combination of tooling, documentation, and convention, to define a careful subset of C++ for use within the kernel, and to the best of my knowledge that tooling does not already exist. The C++ code would also have to continue interoperating with the existing C code, which means that even if there were a tool to say "warn/error about C++ code that uses one of the unsafe bits that comes from C", the kernel likely couldn't use that without substantial additional integration.
And conversely, C++ introduces new potential errors that don't exist in C. The first example that comes to mind: if you use a C++ smart pointer, and do anything that gives you the raw pointer (e.g. &something->field, which is extremely common in kernel code), and put that in a data structure, and then let the smart pointer go out of scope, you now have a use-after-free vulnerability. In C, that would require explicitly freeing the object; in C++, there's no visible code associated with freeing the memory, and nothing in the type system tells you you've done something wrong there.
A sufficiently well-trained modern C++ developer presumably would not make such an error very often. But what about a large number of expert-level C developers who are attempting to work with modern C++?
Introducing C++ into the kernel would add a substantial amount of complexity, and I'm not at all sure that on balance you'd end up on "marginally better" rather than "worse". I'm not suggesting it couldn't be done, or that it couldn't eventually provide some improvements, but it's not clear to me that it's an obvious win.
(Please take all of this with a grain of salt, as I do not write "modern C++" in any professional capacity; its complexity has grown far beyond my knowledge.)
I think introducing C++ into the kernel (and getting value from it) would require effort on par with introducing Rust, to provide safer interfaces for things. It would also require a substantial combination of tooling, documentation, and convention, to define a careful subset of C++ for use within the kernel, and to the best of my knowledge that tooling does not already exist. The C++ code would also have to continue interoperating with the existing C code, which means that even if there were a tool to say "warn/error about C++ code that uses one of the unsafe bits that comes from C", the kernel likely couldn't use that without substantial additional integration.
And conversely, C++ introduces new potential errors that don't exist in C. The first example that comes to mind: if you use a C++ smart pointer, and do anything that gives you the raw pointer (e.g. &something->field, which is extremely common in kernel code), and put that in a data structure, and then let the smart pointer go out of scope, you now have a use-after-free vulnerability. In C, that would require explicitly freeing the object; in C++, there's no visible code associated with freeing the memory, and nothing in the type system tells you you've done something wrong there.
A sufficiently well-trained modern C++ developer presumably would not make such an error very often. But what about a large number of expert-level C developers who are attempting to work with modern C++?
Introducing C++ into the kernel would add a substantial amount of complexity, and I'm not at all sure that on balance you'd end up on "marginally better" rather than "worse". I'm not suggesting it couldn't be done, or that it couldn't eventually provide some improvements, but it's not clear to me that it's an obvious win.
For what it's worth, I think it's a moot point; it's really hard to imagine major new subsystems being written in C++, and not hard to imagine them being written in Rust. But if you're going to have a polyglot kernel, it seems pretty weird to exclude C++ if you're still accepting the majority of your patches in C.
I think that’s ultimately the question. Given C++ is not technically safer than C (as the GP points out), is it worth the complexity of adding it to the kernel?
I have to say though, most of my hesitation of looking at and trying to contribute to kernel is actually the development model. I think I’ve seen from Linus that it is purposefully obtuse to ensure only people who really want or need to contribute will do so. But now they’re pointing out that they’re having a problem attracting new contributors. Is that due to the language or the development and commit model?
Rust would be a huge improvement, but I don’t see how that alone would fix the contribution model.
I have to say though, most of my hesitation of looking at and trying to contribute to kernel is actually the development model. I think I’ve seen from Linus that it is purposefully obtuse to ensure only people who really want or need to contribute will do so. But now they’re pointing out that they’re having a problem attracting new contributors. Is that due to the language or the development and commit model?
Rust would be a huge improvement, but I don’t see how that alone would fix the contribution model.
I agree that C++ isn't meaningfully safer than C, but the kernel isn't a safe program. If the C kernel was moribund, that'd be one thing, but the opposite thing is true: the standard kernel programming language is C. Meanwhile, it'd be hard (not impossible) to argue that C++ is less safe than C, and there are other considerations besides safety.
(C++ in the kernel has been done before; the Click Modular Router, for instance).
(C++ in the kernel has been done before; the Click Modular Router, for instance).
> the kernel isn't a safe program
But it can be far safer than it is today. I assume you’ve seen Writing an OS in Rust by Philipp Oppermann? The unsafe portions are very small compared to the safe: https://os.phil-opp.com/
I just don’t see why you’d include C++ at this point when there are far better options that would provide meaningful improvements in safety.
But it can be far safer than it is today. I assume you’ve seen Writing an OS in Rust by Philipp Oppermann? The unsafe portions are very small compared to the safe: https://os.phil-opp.com/
I just don’t see why you’d include C++ at this point when there are far better options that would provide meaningful improvements in safety.
Simple: C++ is a much more expressive language than either Rust or C. That is to say, you can write libraries in C++ that are wholly impossible in Rust or C, that encapulate semantics and optimizations that cannot be done in any other language.
There are sound reasons why the highest-paying development in all fields is done exclusively in C++: finance, aerospace, CAE, HPC, telecomms -- all are exclusively C++. Rust is not a blip there, and C is entirely dead (except some aerospace and telecomms).
Safety is not the driving force in systems development: capability is. C++ is simply more capable than C or Rust, and Rust is very far from mature. Given competent modern C++ coding practice, safety is just not a problem, in practice.
There are sound reasons why the highest-paying development in all fields is done exclusively in C++: finance, aerospace, CAE, HPC, telecomms -- all are exclusively C++. Rust is not a blip there, and C is entirely dead (except some aerospace and telecomms).
Safety is not the driving force in systems development: capability is. C++ is simply more capable than C or Rust, and Rust is very far from mature. Given competent modern C++ coding practice, safety is just not a problem, in practice.
C is a far simpler language and C++ is at least an order of magnitude more complex (and getting worse). Complexity is inversely correlated with safety.
Of course additional considerations (understandability, maintainability) are probably even more important in this case.
Of course additional considerations (understandability, maintainability) are probably even more important in this case.
Parts of Darwin too
Having spent some time examining both Linux (written in C) and the Haiku kernel (written in C++) I'm far from clear that C++ is even "marginally better". More complicated is worse right?
A kernel like Linux invariably develops its own idiom that I think probably has more influence than any of the C++ syntactic sugar that you'd be allowed to use in that environment.
Do you have an example in mind of something you wish people did in Linux that they'd likely do with a restricted C++ dialect but not C?
A kernel like Linux invariably develops its own idiom that I think probably has more influence than any of the C++ syntactic sugar that you'd be allowed to use in that environment.
Do you have an example in mind of something you wish people did in Linux that they'd likely do with a restricted C++ dialect but not C?
I think it comes down to your attitude towards complexity, which C++ brings heaps of, and how you rank it among the enemies of software quality.
Here [1] is one of Linus's more well-known rants about C++. It's in reference to the Git source, but I think most of it applies to his attitude towards using C++ in the kernel as well.
[1] http://harmful.cat-v.org/software/c++/linus
[1] http://harmful.cat-v.org/software/c++/linus
Just to be clear, I don't know that anyone in this subthread likes C++. I stopped writing in it in 2003 and I'm glad to be done with it. (I still adore C).
While I don't think C++ belongs in the Linux kernel, Linus rants and diatribes are not something to laud.
Yes, Linus has been wrong on this for a long, long time.
The 'unsafe' part of C++ is not the reason.
C is already unsafe enough.
C++ is a Frankenstein of a language, can't really be parsed, so many idioms, nobody agrees on the common subset.
If there was a C++ totalitarian who ruled with an 'iron set of idioms, conventions and best practices' maybe 20 years ago and forced the world into his benevolent bidding - then C++ might be useful there, even if 'unsafe'.
The memory safety value of Rust is way overblown. That's an important feature but not 'the' feature. Most C++ programs are not dying from memory problems. Rust's value in everything else as well.
C is already unsafe enough.
C++ is a Frankenstein of a language, can't really be parsed, so many idioms, nobody agrees on the common subset.
If there was a C++ totalitarian who ruled with an 'iron set of idioms, conventions and best practices' maybe 20 years ago and forced the world into his benevolent bidding - then C++ might be useful there, even if 'unsafe'.
The memory safety value of Rust is way overblown. That's an important feature but not 'the' feature. Most C++ programs are not dying from memory problems. Rust's value in everything else as well.
Here's our presentation at Linux Security Summit last year outlining why I think you can legitimately object to C++ and not Rust for this particular use case: https://youtu.be/RyY01fRyGhM?t=313 (alternatively, slides 19-21 of https://ldpreload.com/p/kernel-modules-in-rust-lssna2019.pdf)
[deleted]
Those slides are great. I love modern C++, but reading those slides makes me excited about trying out rust.
A lot of the reasons that Linus doesn't want C++ aren't true of Rust.
The biggest issue with C++ is it's an absolute atrocity -- an agglomeration of every feature under the sun, but half-built as many of those features can't possible co-exist.
I mean, for example, std::move doesn't move... anything. std::move "just" static_casts the receiver as a template<class T> typename remove_reference<T>::type&&. Not that any sane human knows what that means. The source object remains alive and potentially invalid. Who even knows? Certainly not the compiler. This is just one example. All of C++'s new features are built like this. And the old ones are just so fragile.
It's an incredibly dangerous, convoluted and absurd language, and every time I turn around it gets worse. It's time to leave C++ in the dustbin. It is an octopus made by stapling extra legs onto a dog.
I mean, for example, std::move doesn't move... anything. std::move "just" static_casts the receiver as a template<class T> typename remove_reference<T>::type&&. Not that any sane human knows what that means. The source object remains alive and potentially invalid. Who even knows? Certainly not the compiler. This is just one example. All of C++'s new features are built like this. And the old ones are just so fragile.
It's an incredibly dangerous, convoluted and absurd language, and every time I turn around it gets worse. It's time to leave C++ in the dustbin. It is an octopus made by stapling extra legs onto a dog.
>> It is an octopus made by stapling extra legs onto a dog.
Never heard that before but it reflects my view of where C++ has been going very well.
Never heard that before but it reflects my view of where C++ has been going very well.
C++: an octopus made by nailing extra legs onto a dog.
— Steve Taylor
http://harmful.cat-v.org/software/c++/
http://harmful.cat-v.org/software/c++/
I used to be a C++ diehard, but modern C++ is the reason I started learning rust.
I figured if I’m going to learn a bunch of new rules and idioms, I might as well do it with a language that has those things built in rather than bolted on as an afterthought.
I figured if I’m going to learn a bunch of new rules and idioms, I might as well do it with a language that has those things built in rather than bolted on as an afterthought.
Same here tbh.
When I tell people their Typescript React projects look like a steaming pile of unrecognizable shit, I get downvoted. My precious simple shitty JavaScript is destined to be a bride dressed up in the shittiest possible gown sewn together in a hodgepodge of programming ideas implemented with the shittiest hacksaw one could afford.
I guess I get you, you complete me.
I guess I get you, you complete me.
I guess Typescript, like C++ is just better than not having it at all.
Plain javascript is a crime against the mind (and QA testers).
Plain javascript is a crime against the mind (and QA testers).
It’s really not, but alas, the great crimes you won’t know about until after the fact. A crime is happening in the JS community, and one day we’ll all be writing the same thing OP wrote about C++, an amalgamation, a monstrosity, but whatever, it’ll be too late.
Just wait until Rust (or lots of new languages) get mature and start getting pushed in weird directions by niche use cases.
In addition, languages always need to be seen improving in order to hold the attention of all the magpie developers. Who wants to use an old, stable language, amiright?
Everyone will want their own small syntactic sugar to make their own ideas more easily expressed in the language they are currently using. It is just part of the lifecycle of programming languages.
I am willing to predict that if Rust is still around in 30 years it will be unrecognizable to present-day developers, and newcomers at that time will decry it as 'an agglomeration of every feature under the sun'.
In addition, languages always need to be seen improving in order to hold the attention of all the magpie developers. Who wants to use an old, stable language, amiright?
Everyone will want their own small syntactic sugar to make their own ideas more easily expressed in the language they are currently using. It is just part of the lifecycle of programming languages.
I am willing to predict that if Rust is still around in 30 years it will be unrecognizable to present-day developers, and newcomers at that time will decry it as 'an agglomeration of every feature under the sun'.
Don't let your feelings get in the way of your valid argument about C++.
The Linux kernel will eventually going use the FFI route from Rust and C someday, but it is no safer in both safe Rust and unsafe Rust according to this memory saftey assessment study that has Rust CVEs [0]
[0] https://arxiv.org/pdf/2003.03296.pdf
The Linux kernel will eventually going use the FFI route from Rust and C someday, but it is no safer in both safe Rust and unsafe Rust according to this memory saftey assessment study that has Rust CVEs [0]
[0] https://arxiv.org/pdf/2003.03296.pdf
Much of C++ is inappropriate for a kernel. And defining which parts of C++ could be safely allowed would take so much time and bikeshedding that it's just probably not worth it.
I have a hard time seeing this happen. Rust has most of the same properties which prevent C++'s inclusion in-tree. (Look up Linus' opinions on C++ if you aren't already familiar.)
Further, as far as I can tell, Rust currently lacks mainline GCC support which I believe will be a blocker.
Further, as far as I can tell, Rust currently lacks mainline GCC support which I believe will be a blocker.
Here's Linus's answer inside the thread, which is quite positive:
From: Linus Torvalds @ 2020-07-10 23:54 UTC (permalink / raw) To: Josh Triplett Cc: Christian Brauner, Nick Desaulniers, alex.gaynor, Greg KH, geofft, jbaublitz, Masahiro Yamada, Miguel Ojeda, Steven Rostedt, LKML, clang-built-linux, Kees Cook
On Fri, Jul 10, 2020 at 3:59 PM Josh Triplett <[email protected]> wrote: > > As I recall, Greg's biggest condition for initial introduction of this > was to do the same kind of "turn this Kconfig option on and turn an > option under it off" trick that LTO uses, so that neither "make > allnoconfig" nor "make allyesconfig" would require Rust until we've had > plenty of time to experiment with it.
No, please make it a "is rust available" automatic config option. The exact same way we already do the compiler versions and check for various availability of compiler flags at config time.
See init/Kconfig for things like
Quite the reverse. I'd want the first rust driver (or whatever) to be introduced in such a simple format that failures will be obvious and simple.
The _worst_ situation to be in is that s (small) group of people start testing their very special situation, and do bad and crazy things because "nobody else cares, it's hidden".
No, thank you.
From: Linus Torvalds @ 2020-07-10 23:54 UTC (permalink / raw) To: Josh Triplett Cc: Christian Brauner, Nick Desaulniers, alex.gaynor, Greg KH, geofft, jbaublitz, Masahiro Yamada, Miguel Ojeda, Steven Rostedt, LKML, clang-built-linux, Kees Cook
On Fri, Jul 10, 2020 at 3:59 PM Josh Triplett <[email protected]> wrote: > > As I recall, Greg's biggest condition for initial introduction of this > was to do the same kind of "turn this Kconfig option on and turn an > option under it off" trick that LTO uses, so that neither "make > allnoconfig" nor "make allyesconfig" would require Rust until we've had > plenty of time to experiment with it.
No, please make it a "is rust available" automatic config option. The exact same way we already do the compiler versions and check for various availability of compiler flags at config time.
See init/Kconfig for things like
config LD_IS_LLD
def_bool $(success,$(LD) -v | head -n 1 | grep -q LLD)
config RUST_IS_AVAILABLE
def_bool $(success,$(RUST) ..sometest..)
Quite the reverse. I'd want the first rust driver (or whatever) to be introduced in such a simple format that failures will be obvious and simple.
The _worst_ situation to be in is that s (small) group of people start testing their very special situation, and do bad and crazy things because "nobody else cares, it's hidden".
No, thank you.
https://www.theregister.com/2020/06/30/hard_to_find_linux_ma...
> "C is still one of the top 10 languages," answered Torvalds. However, he said that for things "not very central to the kernel itself", like drivers, the kernel team is looking at "having interfaces to do those, for example, in Rust... I'm convinced it's going to happen. It might not be Rust. But it is going to happen that we will have different models for writing these kinds of things, and C won't be the only one."
> "C is still one of the top 10 languages," answered Torvalds. However, he said that for things "not very central to the kernel itself", like drivers, the kernel team is looking at "having interfaces to do those, for example, in Rust... I'm convinced it's going to happen. It might not be Rust. But it is going to happen that we will have different models for writing these kinds of things, and C won't be the only one."
The thing is the kernel is kinda hurting for maintainers and opening up the path for the next generation of systems programmers (most of whom don't want to touch C any more than they have to) has to be something Linus is more likely to prioritize than in previous decades.
I for one do not want to touch Rust more than I have to.
But are you the target audience - a C developer contributing, or at least considering contributing, to the Linux kernel?
Why do you have this opinion on Rust compared to C?
The kernel isn't hurting for developers though.
I don't see how a different programming language would help improve the ratio of maintainers vs developers.
I don't see how a different programming language would help improve the ratio of maintainers vs developers.
> The thing is the kernel is kinda hurting for maintainers
Source? AFAIK the kernel is probably one of the most, if not the most, well-funded and maintained open source projects.
Source? AFAIK the kernel is probably one of the most, if not the most, well-funded and maintained open source projects.
From the horse's mouth: https://www.theregister.com/2020/06/30/hard_to_find_linux_ma...
> We do not have enough maintainers. We do have a lot of people who write code, we have a fair number of maintainers, but... it's hard to find people who really look at other people's code and funnel that code upstream all the way, eventually, to my tree... It is one of the main issues we have.
I think that is an issue which would not change with Rust.
Reading other people's code is boring. Linus stated once that he got around that by retyping the code while reviewing, which is a good technique.
One danger is of course the change in culture and politics should Rust actually become prevalent. But there is still BSD.
Reading other people's code is boring. Linus stated once that he got around that by retyping the code while reviewing, which is a good technique.
One danger is of course the change in culture and politics should Rust actually become prevalent. But there is still BSD.
If I read this correctly, they have a lot of contributors, but few people willing to follow through with the code so it gets all the way up to Linus' tree. I don't see how rust would really help there.
It could be. Maybe there will not be more reviewers, but the process of reviewing could be faster. For example, in the kernel there are inline functions, which accept a constant (known at compile time) integer argument, and they make a switch or loop over that integer. The trick is to force compiler to inline function and to throw away most of switch's cases, or unroll loop completely, or make some other optimizations based on knowledge of the passed integer value.
Such a functions are exported to a public kernel API sometimes. So any kernel developer could call them. It make unavoidable a mistake of a programmer, when some of those functions was called with an unknown at compile time integer value. It is not a bug as it is, code would compile and work nevertheless, code would be bigger that it could be, maybe slower, that's all. But reviewer should be finding such a misuse of API.
It is just one example, linux kernel uses a lot of tricks and exposes it via APIs, reviewer must be aware of all of them and to check everything by itself.
Compare it with Rust. In rust one can encode a lot more limits on the right use of an API than in C. Encode and enforce. Even when rust sometimes cannot do it, rust have nice macros, instead of C's macro-horror. So a lot of mistakes that could be made with C, could be avoided with Rust. So it would be easy to review code, it would take less time from a reviewer to review code.
Such a functions are exported to a public kernel API sometimes. So any kernel developer could call them. It make unavoidable a mistake of a programmer, when some of those functions was called with an unknown at compile time integer value. It is not a bug as it is, code would compile and work nevertheless, code would be bigger that it could be, maybe slower, that's all. But reviewer should be finding such a misuse of API.
It is just one example, linux kernel uses a lot of tricks and exposes it via APIs, reviewer must be aware of all of them and to check everything by itself.
Compare it with Rust. In rust one can encode a lot more limits on the right use of an API than in C. Encode and enforce. Even when rust sometimes cannot do it, rust have nice macros, instead of C's macro-horror. So a lot of mistakes that could be made with C, could be avoided with Rust. So it would be easy to review code, it would take less time from a reviewer to review code.
This was linked on reddit and apparently what he meant was other OSS projects lack maintainers but linux is fine.
I would argue that chromium is slightly more funded
How would requiring learning Rust, which is famously difficult, be the solution to that problem?
Driver code has a lot of concurrency and is very unforgiving for any mishandling of memory. If you want to write good kernel code in C you need to have a strong mental model of who owns what, who does what on whom, what can interrupt, what can be interrupted, what shouldn't be interrupted etc...
Having the compiler enforce some of these invariants for you might well end up making writing the code easier, not harder. Rust has a steep learning curve, that's absolutely true, but in my experience I feel a lot more at ease writing complicated parallel code with complicated ownership models in Rust than in C.
C is a trickster: in practice if you want to write correct C code you have to use a memory model very similar to that of Rust, the main difference being that Rust forces you to do it (mostly) right whereas C will gladly let you write completely broken code.
Having the compiler enforce some of these invariants for you might well end up making writing the code easier, not harder. Rust has a steep learning curve, that's absolutely true, but in my experience I feel a lot more at ease writing complicated parallel code with complicated ownership models in Rust than in C.
C is a trickster: in practice if you want to write correct C code you have to use a memory model very similar to that of Rust, the main difference being that Rust forces you to do it (mostly) right whereas C will gladly let you write completely broken code.
Does Rust handle anything with memory barriers? I can handle concurrency, interrupt handlers, etc, but memory barriers always trouble me.
It will force your code to be memory safe, but it can't enforce that you have the correct semantics.
https://doc.rust-lang.org/std/sync/atomic/fn.fence.html exists, but Rust can't force you to say, use the correct ordering.
https://doc.rust-lang.org/std/sync/atomic/fn.fence.html exists, but Rust can't force you to say, use the correct ordering.
Right, so no different than C in this regard.
Really, you should rarely be using raw memory barriers even in C, IMO. The C11/C++11 atomics model (which Rust also uses) is substantially nicer. Though, unfortunately, still extremely tricky to get right.
The only time that I ever had to worry about that was when writing drivers for the Linux kernel.
Writing safe C and C++ is even much more difficult than writing Rust code (especially with many developers). And when you are writing kernel code, both safety and performance are crucial.
The problem seems to be to find people who do not just want to write code, but do the administrative parts:
"We do not have enough maintainers. We do have a lot of people who write code, we have a fair number of maintainers, but... it's hard to find people who really look at other people's code and funnel that code upstream all the way, eventually, to my tree... It is one of the main issues we have."
"We do not have enough maintainers. We do have a lot of people who write code, we have a fair number of maintainers, but... it's hard to find people who really look at other people's code and funnel that code upstream all the way, eventually, to my tree... It is one of the main issues we have."
Finding people willing to review rust must be much easier than finding someone willing to review c.
Actually I think the answer is yes, as Rust is much stricter to the person who's contributing the code. If I was asked to review another person's code, I would much prefer reviewing Rust code, as I would be able to concentrate on the logic and the efficiency of the code instead of potential low level mistakes.
The code smell in Rust is usually marked with ,,unsafe'', Boxing / reference counters, or dynamic behaviour, which is very easy to search for.
The code smell in Rust is usually marked with ,,unsafe'', Boxing / reference counters, or dynamic behaviour, which is very easy to search for.
Is reference counters really a smell? I thought it was the main way to write Rust when the borrow checker is too strict for something you want to do.
Vec + generational indices might be the other way, is that preferred over Rc?
Vec + generational indices might be the other way, is that preferred over Rc?
,,when the borrow checker is too strict for something you want to do''
Most of the times it isn't. Just as an example here's a random part of serde, the high quality serialization library in Rust:
https://github.com/serde-rs/serde/blob/master/serde/src/priv...
The amount of extra automatic memory management is often a good indicator of not well thought out data structures (although of course they are not always needed).
Most of the times it isn't. Just as an example here's a random part of serde, the high quality serialization library in Rust:
https://github.com/serde-rs/serde/blob/master/serde/src/priv...
The amount of extra automatic memory management is often a good indicator of not well thought out data structures (although of course they are not always needed).
I guess your point is that c is harder to review?
But I suspect there are probably more people who are experts in device driver code and know kernel internals than all people who know rust.
But I suspect there are probably more people who are experts in device driver code and know kernel internals than all people who know rust.
I'm sorry but this is being in total denial of the real adoption of rust compared to C.
Indeed. This harks back to old debates:
- do I really need threads when I can share non-mutable messages or copies of same? Maybe (speed, throughput). Maybe not. Depends. What's clear is I need choice.
- how do I get correctness? Rust maybe checks for index bounds. But in C++ I just relegate that to a library and voila std::string/vector.
I am concerned for Rust: placing too much emphasis on correctness through language alone seems to be a weak point. Correctness through libraries and augmenting with formal tools seems like a much stronger solution.
- do I really need threads when I can share non-mutable messages or copies of same? Maybe (speed, throughput). Maybe not. Depends. What's clear is I need choice.
- how do I get correctness? Rust maybe checks for index bounds. But in C++ I just relegate that to a library and voila std::string/vector.
I am concerned for Rust: placing too much emphasis on correctness through language alone seems to be a weak point. Correctness through libraries and augmenting with formal tools seems like a much stronger solution.
One of unsafe's big roles is for people to be able to create new kinds of safety outside of the language proper.
> both safety and performance are crucial
Why not Ada/SPARK though? Was it ever considered, even?
Why not Ada/SPARK though? Was it ever considered, even?
I don't think so.
Despite its ubiquity, the Linux kernel is not actually the golden standard for safety or performance. Not even for code quality. Ada remains the language of choice in areas where safety is actually crucial. Rust isn't going to take it's spot either.
Despite its ubiquity, the Linux kernel is not actually the golden standard for safety or performance. Not even for code quality. Ada remains the language of choice in areas where safety is actually crucial. Rust isn't going to take it's spot either.
I am inclined to agree. By the way: when I first read about Rust and its "safety features" that make it appealing to people, the first programming language that came to mind was Ada. I suppose it lacks the hype (and consequently a vibrant ecosystem, IMO) and people hold absurd misconceptions of the language[1]. :/ I am trying my best to dissipate those misconceptions on Hacker News; hopefully it does not count as shilling. :P
If any of you reading this comment is interested, you may want to check out my previous comments regarding Ada.[2]
By the way, just to be on topic: I found Linus' opinion on Ada vs Rust here: https://news.ycombinator.com/item?id=20838125. He said "We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.". That is all though, he does not get into explaining why he believes Ada to be a disaster. Again, "I do not see any reasons given for why he considers Ada a disaster. Cannot really do much with this opinion as it is."
[1] Some people claimed that it is a legacy language which is just simply not true. I would suggest that they give https://blog.adacore.com/ and https://blog.adacore.com/gnat-community-2020-is-here a read.
[2]
https://news.ycombinator.com/item?id=19122884 (!)
https://news.ycombinator.com/item?id=19245898 (!!)
https://news.ycombinator.com/item?id=19274244 (!!)
https://news.ycombinator.com/item?id=19274412
https://news.ycombinator.com/item?id=19770405
https://news.ycombinator.com/item?id=20776296
https://news.ycombinator.com/item?id=20934511 (!!) [3]
https://news.ycombinator.com/item?id=20939336 (!!)
https://news.ycombinator.com/item?id=21286061
https://news.ycombinator.com/item?id=21286292
https://news.ycombinator.com/item?id=21435869 (!)
https://news.ycombinator.com/item?id=23609499 (!!)
[3] The last link in that comment returns 404, here is a working one: https://www.ei.tum.de/fileadmin/tueifei/rcs/becker/spark2014...
If there are any other broken links, please, do let me know!
If any of you reading this comment is interested, you may want to check out my previous comments regarding Ada.[2]
By the way, just to be on topic: I found Linus' opinion on Ada vs Rust here: https://news.ycombinator.com/item?id=20838125. He said "We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.". That is all though, he does not get into explaining why he believes Ada to be a disaster. Again, "I do not see any reasons given for why he considers Ada a disaster. Cannot really do much with this opinion as it is."
[1] Some people claimed that it is a legacy language which is just simply not true. I would suggest that they give https://blog.adacore.com/ and https://blog.adacore.com/gnat-community-2020-is-here a read.
[2]
https://news.ycombinator.com/item?id=19122884 (!)
https://news.ycombinator.com/item?id=19245898 (!!)
https://news.ycombinator.com/item?id=19274244 (!!)
https://news.ycombinator.com/item?id=19274412
https://news.ycombinator.com/item?id=19770405
https://news.ycombinator.com/item?id=20776296
https://news.ycombinator.com/item?id=20934511 (!!) [3]
https://news.ycombinator.com/item?id=20939336 (!!)
https://news.ycombinator.com/item?id=21286061
https://news.ycombinator.com/item?id=21286292
https://news.ycombinator.com/item?id=21435869 (!)
https://news.ycombinator.com/item?id=23609499 (!!)
[3] The last link in that comment returns 404, here is a working one: https://www.ei.tum.de/fileadmin/tueifei/rcs/becker/spark2014...
If there are any other broken links, please, do let me know!
With Ada you can't use dynamic memory allocation and pointers safely, unless you buy into SPARK, which is heavier-weight than Rust (and also post-dates Rust).
Not a single one of the claims in your post is factually correct.
https://www.adacore.com/uploads/techPapers/Safe-Dynamic-Memo...
> Standard Ada supports safe use of pointers (“access types” in Ada) via strongtype checking, but safety is guaranteed only for programs where there is no explicit deallocation of pointed-to objects
This validates "With Ada you can't use dynamic memory allocation and pointers safely, unless you buy into SPARK". (GC not relevant in this context.)
> In this work, we propose a restricted form of pointers for Ada that is safe enough to be included in the SPARK subset. As our main contribution, we show how to adapt the ideas underlying the safe pointers from permission-based languages like Rust [3] or ParaSail [13]
This validates that the pointer support in SPARK "post-dates Rust".
"heavier-weight" is arguable. I'll withdraw that assertion. SPARK is definitely less expressive than Rust though; it doesn't have lifetime variables, and it doesn't allow borrows of stack values:
https://blog.adacore.com/using-pointers-in-spark
> The most notable one is that SPARK does not allow general access types. The reason is that we did not want to deal with accesses to variables defined on the stack and accessibility levels.
> Standard Ada supports safe use of pointers (“access types” in Ada) via strongtype checking, but safety is guaranteed only for programs where there is no explicit deallocation of pointed-to objects
This validates "With Ada you can't use dynamic memory allocation and pointers safely, unless you buy into SPARK". (GC not relevant in this context.)
> In this work, we propose a restricted form of pointers for Ada that is safe enough to be included in the SPARK subset. As our main contribution, we show how to adapt the ideas underlying the safe pointers from permission-based languages like Rust [3] or ParaSail [13]
This validates that the pointer support in SPARK "post-dates Rust".
"heavier-weight" is arguable. I'll withdraw that assertion. SPARK is definitely less expressive than Rust though; it doesn't have lifetime variables, and it doesn't allow borrows of stack values:
https://blog.adacore.com/using-pointers-in-spark
> The most notable one is that SPARK does not allow general access types. The reason is that we did not want to deal with accesses to variables defined on the stack and accessibility levels.
It's worth noting that Linus Torvalds does not write life-critical software. Take his opinions with a pinch of salt. He's made some of the most incredibly valuable contributions to computing, however this does not make him an authority on all things programming. He's anything but infallible.
Requiring certain classes of drivers, for example, to be written in Rust could be great for maintainers.
The person who submits the code has to do more work up front to satisfy the Rust compiler that there are no ownership issues, data races, etc. Maintainers have less work to do because, assuming the code compiles and does not use unsafe blocks, they can be confident the invariants promised by Rust hold. You have moved work from maintainers to submitters. (But you have also given submitters a way to get quicker, automated feedback on their work.)
The person who submits the code has to do more work up front to satisfy the Rust compiler that there are no ownership issues, data races, etc. Maintainers have less work to do because, assuming the code compiles and does not use unsafe blocks, they can be confident the invariants promised by Rust hold. You have moved work from maintainers to submitters. (But you have also given submitters a way to get quicker, automated feedback on their work.)
I have used a bit of rust and c. I found rust significantly easier since the std lib contains a whole lot more useful stuff. And C is starting to get a bad rep like php and cobal have. The next generation of programmers probably won't bother with it and seek jobs using newer and better languages.
Rust in the kernel is not guaranteed to get stdlib. Running on bare metal is often done with `no_std`. Remembering how memory allocation in the kernel can happen in a multitude of ways, I'm not even sure the usual stdlib could be leveraged. Strings, for example, allocate memory implicitly.
That being said, apart from a good stdlib, I found Rust much easier than C due to a consistent type syntax, sum types, generics, traits, and the compiler hitting me on the head with lifetime errors.
That being said, apart from a good stdlib, I found Rust much easier than C due to a consistent type syntax, sum types, generics, traits, and the compiler hitting me on the head with lifetime errors.
Even no_std Rust has a lot of nice built-ins, for what it is worth.
> Rust has most of the same properties which prevent C++'s inclusion in-tree
It does but it also has some advantages that C++ does not have.
It does but it also has some advantages that C++ does not have.
But you can read his response to Rust in the linked thread.
Correct, he must have changed his attitude on some of those features that apply to C++ as well that he previously railed against.
https://news.ycombinator.com/item?id=23801076
https://news.ycombinator.com/item?id=23801076
Rust integrates easily with C because it's explicitly designed to fall back to the C ABI for external linkage. Whereas writing C++ with any C++-specific features (hence, any "Modern", guidelines-compliant C++ at all) brings its own bespoke ABI into the picture, that's supposed to be stable but really is not and that one can't easily interoperate with from C. That's a huge non-starter in kernel dev.
Not really, both in Rust and in C++ the C-compatible parts have to be marked explicitly as extern "C". Even the syntax is the same.
C compatibility is marked the same way, but it's not idiomatically used the same way in C++ and Rust. In C++, being C compatible means essentially writing C and for everything else you're expected to rely on the "not really stable" C++ ABI whereas Rust does not even try to give you a stable ABI between stuff that's not part of pretty much the same build. Despite that, you're free to mix C-like and Rustic code in the same crate because the crate is literally the translation unit in Rust.
ABI stability doesn't matter for the kernel since it doesn't have a stable ABI either. You can write a C++ core and C wrappers for C++ classes just fine, the same as what you do in Rust.
I’m a huge Rust fan, but this is a misrepresentation of C++. You can definitely use the vast majority of modern C++ features while also integrating with a C codebase (exotic control flow such as exceptions and coroutines are definitely dangerous, though).
I am assuming that it would work with clang-compiled[1] kernels only at first.
1: https://www.kernel.org/doc/html/latest/kbuild/llvm.html
1: https://www.kernel.org/doc/html/latest/kbuild/llvm.html
Why would it require clang for the C components? The Rust compiler creates native libraries which can be linked to compilation units created by both clang and gcc, unless there's something special going on in the kernel that I'm not aware of.
Not really. Unless you want LTO, the object files generated by GCC and LLVM are compatible, and can mixed without issues.
You didn't read Linus's response, did you?
Formal leaning HN people: Does Rust just have better sales & marketing than C++? Or is it really better for safety and correctness?
* Rust is NOT a programming language with deep, well-integrated support for proof systems or state exploration e.g. TLA+.
* Rust is no Frama-C where static analysis and function contract support is far more apparent and important
* Rust is not ADA
* The previous items also arise and are quite important as a direct result of generality: to the extent Rust imposes (albeit with benefit) language design to avoid certain kinds of functional issues, those features can later become limitations as a pushy, over opinionated language. Therefore to have flexibility in designing fast code that's yet correct code one usually has to have the low-level ability of C/C++ augmented with formal tools.
I'm not seeing Rust do that. Now, I've made a couple of jokes elsewhere in HN about Rust's propensity to show up uninvited to every vaguely language issue and press its case --- ex. I was vacuuming the floor yesterday when Rust somehow got through the front door and was explaining how a Rust vacuum is so much better, or how I'd never need to vacuum again if I used Rust. But I am only half joking.
For the last year I've spent much of my time in GO after getting sick-and-tired of C++ build/language complexity for the previous 10+ years. I am somewhat aligned to others who rightly ask if speed is all that matters? (Recall the Meyers joke about Pavlov training and C/C++ programmers when it comes to speed).
Correctness counts too and if the compiler spends a little bit more I time I'd be fine with that if for distributed/parallel programming I've got good guarantees.
For those who use TLA, SPIN, Frama-C and other formal tools what's your take on this? I think designers of those kinds of formal tools will be surprised to learn some of their correctness aims were solved by Rust.
* Rust is NOT a programming language with deep, well-integrated support for proof systems or state exploration e.g. TLA+.
* Rust is no Frama-C where static analysis and function contract support is far more apparent and important
* Rust is not ADA
* The previous items also arise and are quite important as a direct result of generality: to the extent Rust imposes (albeit with benefit) language design to avoid certain kinds of functional issues, those features can later become limitations as a pushy, over opinionated language. Therefore to have flexibility in designing fast code that's yet correct code one usually has to have the low-level ability of C/C++ augmented with formal tools.
I'm not seeing Rust do that. Now, I've made a couple of jokes elsewhere in HN about Rust's propensity to show up uninvited to every vaguely language issue and press its case --- ex. I was vacuuming the floor yesterday when Rust somehow got through the front door and was explaining how a Rust vacuum is so much better, or how I'd never need to vacuum again if I used Rust. But I am only half joking.
For the last year I've spent much of my time in GO after getting sick-and-tired of C++ build/language complexity for the previous 10+ years. I am somewhat aligned to others who rightly ask if speed is all that matters? (Recall the Meyers joke about Pavlov training and C/C++ programmers when it comes to speed).
Correctness counts too and if the compiler spends a little bit more I time I'd be fine with that if for distributed/parallel programming I've got good guarantees.
For those who use TLA, SPIN, Frama-C and other formal tools what's your take on this? I think designers of those kinds of formal tools will be surprised to learn some of their correctness aims were solved by Rust.
Rust makes it easy to write code that's very expressive (dynamic memory allocation etc) and has strong safety properties. Certainly easier than having to verify with SPARK or Frama-C. Rust's type system, especially traits and affine types, lets you encode a lot of important properties (e.g. typestates).
Rust does need tools to verify Rust code correctness and especially to verify unsafe Rust code. There is a lot of work going on in this area: see https://alastairreid.github.io/rust-verification-tools/ for a sample.
Rust does need tools to verify Rust code correctness and especially to verify unsafe Rust code. There is a lot of work going on in this area: see https://alastairreid.github.io/rust-verification-tools/ for a sample.
I'm gonna look at rust; we're a huge c/c++ shop so there's room from improvement here. Now no program is a spec and implementation in one. Thus rust ought to be careful about keeping all it's hammers in language alone. If rust is smart and particularly wants to move to distributed computing or csps correctness will have to take on more complex meanings that bad index or divide by zero.
Ada doesn't do safe dynamic memory deallocation, right? You'd need Ada.Unchecked_Deallocation.
That's fine in cases where you can statically define everything up front (which you can for many of the use cases that people use Ada for), but for a general-purpose OS kernel, it seems like you wouldn't get the level of safety or correctness from Ada that you would from either Rust or C++.
(Disclosure - I'm one of the authors of the prototype mentioned in the thread. I'd happily use Ada or Frama-C if I thought it would solve the problems needed for getting safe code into the Linux kernel, but it doesn't seem like they actually do.)
That's fine in cases where you can statically define everything up front (which you can for many of the use cases that people use Ada for), but for a general-purpose OS kernel, it seems like you wouldn't get the level of safety or correctness from Ada that you would from either Rust or C++.
(Disclosure - I'm one of the authors of the prototype mentioned in the thread. I'd happily use Ada or Frama-C if I thought it would solve the problems needed for getting safe code into the Linux kernel, but it doesn't seem like they actually do.)
I think this thread is too narrow on correctness. Plenty of memory issues in C/C++ can be solved by library code i.e. not char buf[...] but std::string. Not raw pointers but RC-pointers. The idea that language strictures alone solve a substantial part of code correctness I think is something that people with more experience in particularly CSP computing ala TLA/SPIN would seriously doubt. And even on the nit pickier things ... RUST is gonna help me write lock-free code with CAS, memory fences? Really?
To put this in perspective, though: increasing the number of people paid to work on the Rust compiler by 10x would only mean hiring about 25 people. Compared to the size of the projects that are starting to depend on Rust, that's a rounding error.