I Improved My Rust Compile Times(benw.is)
benw.is
I Improved My Rust Compile Times
https://benw.is/posts/how-i-improved-my-rust-compile-times-by-seventy-five-percent
112 comments
> I’ve found in 10+ years of software development that speed of iteration cycle is highly correlated with productivity.
I agree, though in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker (even if it is sometimes still a little slow).
In most cases, Rust's very strong type system means you spend a lot less time actually running your program.
There are definitely exceptions though. E.g. if you're making a web app and editing CSS or whatever then the Rust compiler isn't going to tell you if you've got the colour wrong.
> a lot of people used to “blame” llvm, but it doesn’t seem to be as big of a culprit
It still is; there was a post here really recently where someone broke down the time spent in various phases. LLVM is still the vast majority, though apparently that is partly due to Rust generating a lot of work for it.
> I would expect incremental build speedup to be prioritized, and steadily improving.
It is.
> But clearly it isn’t moving very fast in that direction. So why not?
Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
I agree, though in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker (even if it is sometimes still a little slow).
In most cases, Rust's very strong type system means you spend a lot less time actually running your program.
There are definitely exceptions though. E.g. if you're making a web app and editing CSS or whatever then the Rust compiler isn't going to tell you if you've got the colour wrong.
> a lot of people used to “blame” llvm, but it doesn’t seem to be as big of a culprit
It still is; there was a post here really recently where someone broke down the time spent in various phases. LLVM is still the vast majority, though apparently that is partly due to Rust generating a lot of work for it.
> I would expect incremental build speedup to be prioritized, and steadily improving.
It is.
> But clearly it isn’t moving very fast in that direction. So why not?
Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
> I agree, though in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker (even if it is sometimes still a little slow).
This is one area that I think Dart did a good job on. They support both a JIT (I am not sure if it’s tiered) and AOT modes of compiling, the JIT targeted for development and AOT for production. This is the ultimate extreme of optimizing for both use cases, obviously it’s a ton of work, but I would love more languages to be like this. (I believe the JIT mode works directly on the source so there isn't an intermediate compilation step like Java, but it’s been a few years since I’ve used Dart)
[1]: https://dart.dev/overview#native-platform
This is one area that I think Dart did a good job on. They support both a JIT (I am not sure if it’s tiered) and AOT modes of compiling, the JIT targeted for development and AOT for production. This is the ultimate extreme of optimizing for both use cases, obviously it’s a ton of work, but I would love more languages to be like this. (I believe the JIT mode works directly on the source so there isn't an intermediate compilation step like Java, but it’s been a few years since I’ve used Dart)
[1]: https://dart.dev/overview#native-platform
This is the best of both worlds, that traces back to Lisp and BASIC environments.
Have all variants available and allow the devs to pick the right one for their current workflow.
Java, .NET, Eiffel, Common Lisp, Scheme, C++ (via stuff like ROOT and Live++, VC++ hot reload isn't that full proof), OCaml, Haskel, are all examples of these mixed tooling approach.
Have all variants available and allow the devs to pick the right one for their current workflow.
Java, .NET, Eiffel, Common Lisp, Scheme, C++ (via stuff like ROOT and Live++, VC++ hot reload isn't that full proof), OCaml, Haskel, are all examples of these mixed tooling approach.
I agree. And Dart's VSCode extension is suspiciously fast. Like it accurately finds compile errors in easily 100ms. I don't know what they've done so differently to pretty much every other system, where you're talking ~1-2s typically. It's very nice though.
> In most cases, Rust's very strong type system means you spend a lot less time actually running your program.
That just sounds wrong to me TBH. IME it never pays off to 'front-load' too much, you might get a working implementation of your original idea without ever running the code, but that original idea will almost certainly be "wrong" once you see it in action and you just wasted a lot of time agonizing over your type structure which turned out to implement a quickly outdated idea.
Nothing beats quick iteration on a running program when it comes to productivity.
> Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
If the a C++ file includes any C++ stdlib header it will also also become very large (tens of thousands of lines of gnarly template code for common headers like <vector> or <string>). If the unit of compilation is a crate in Rust it should actually be faster to build than typical C++ projects which often put the implementation for one class into one source file (and each of those source files is expanded to tens of thousands of lines of code after includes are resolved).
C mostly doesn't suffer from this problem though, since C header typically are only a couple dozen or hundred lines of API declarations.
That just sounds wrong to me TBH. IME it never pays off to 'front-load' too much, you might get a working implementation of your original idea without ever running the code, but that original idea will almost certainly be "wrong" once you see it in action and you just wasted a lot of time agonizing over your type structure which turned out to implement a quickly outdated idea.
Nothing beats quick iteration on a running program when it comes to productivity.
> Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
If the a C++ file includes any C++ stdlib header it will also also become very large (tens of thousands of lines of gnarly template code for common headers like <vector> or <string>). If the unit of compilation is a crate in Rust it should actually be faster to build than typical C++ projects which often put the implementation for one class into one source file (and each of those source files is expanded to tens of thousands of lines of code after includes are resolved).
C mostly doesn't suffer from this problem though, since C header typically are only a couple dozen or hundred lines of API declarations.
This sounds overblown to me, TBH. IME you hardly need to agonize over type structure for common day to day code. The benefits of the type system are highly advantageous to refactoring without causing breakage, so I become more nimble in getting towards the proper structure. This is precisely what allows me to iterate more quickly and is a benefit.
I had lots of coffee pauses when doing builds back in 2000, where our C bindings for the Tcl based server, used as Apache and IIS modules, across Aix, HP-UX, Solaris, Windows 2000/NT, Red-Hat Linux would take one hour per platform on average.
> If the a C++ file includes any C++ stdlib header it will also also become very large (tens of thousands of lines of gnarly template code for common headers like <vector> or <string>). If the unit of compilation is a crate in Rust it should actually be faster to build than typical C++ projects which often put the implementation for one class into one source file (and each of those source files is expanded to tens of thousands of lines of code after includes are resolved).
> C mostly doesn't suffer from this problem though, since C header typically are only a couple dozen or hundred lines of API declarations.
But then, there's no reason why Rust couldn't be like C. The issue comes from the dependencies and libraries bundled, which is not an issue per se of the language.
> C mostly doesn't suffer from this problem though, since C header typically are only a couple dozen or hundred lines of API declarations.
But then, there's no reason why Rust couldn't be like C. The issue comes from the dependencies and libraries bundled, which is not an issue per se of the language.
In C++ it can be a module nowadays, which is similar to being a crate, yet it is faster, from my clang/vc++ experience.
> In most cases, Rust's very strong type system means you spend a lot less time actually running your program.
> There are definitely exceptions though.
I mean.. I know what you mean but this is kinda what I meant with cope. Putting “running code” aside as an edge case seems like a dangerous moving-the-goalpost kind of Rustism that kinda grew out of the “Rust is so safe that if it compiles it’s correct”-meme. Once we’re doing serious and boring good old development, it’s a redeeming quality at best, imo. Not to mention that rust analyzer also suffers slowdowns in similar ways.
> LLVM is still the vast majority
That’s interesting. So why the meager numbers from cranelift then, judging by the post? Granted, I’m not doing Rust day-to-day anymore so I’ve been ootl.
> Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
Isn’t this the same for Go? Sure, it’s a simpler language but we’re talking many, many times on similar sized projects (wish I could link to something but unfortunately anecdotally only). Is it monomorphization, macros, or just everything combined?
> There are definitely exceptions though.
I mean.. I know what you mean but this is kinda what I meant with cope. Putting “running code” aside as an edge case seems like a dangerous moving-the-goalpost kind of Rustism that kinda grew out of the “Rust is so safe that if it compiles it’s correct”-meme. Once we’re doing serious and boring good old development, it’s a redeeming quality at best, imo. Not to mention that rust analyzer also suffers slowdowns in similar ways.
> LLVM is still the vast majority
That’s interesting. So why the meager numbers from cranelift then, judging by the post? Granted, I’m not doing Rust day-to-day anymore so I’ve been ootl.
> Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
Isn’t this the same for Go? Sure, it’s a simpler language but we’re talking many, many times on similar sized projects (wish I could link to something but unfortunately anecdotally only). Is it monomorphization, macros, or just everything combined?
> So why the meager numbers from cranelift then
People are seeing 20-30% improvements thanks to Cranelift. Not sure why you think that’s meagre.
> edge case
It’s not an edge case but it is valid to say that the pattern of development is different. You’re right that people need feedback that what they’re doing is right within 1-2 seconds. They are getting that thanks to the IDE. They need their tests to compile and run within a few seconds, and they’re getting close to that.
I’m not disagreeing with you. In an ideal world all builds and tests would run within 2 seconds, like you’re suggesting. And yet, people are managing to ship software successfully, despite it taking more than 2 seconds. That suggests that maybe this 2 second thing isn’t a requirement and more of a nice to have. And that maybe, there is something to people relying on IDE feedback while coding.
People are seeing 20-30% improvements thanks to Cranelift. Not sure why you think that’s meagre.
> edge case
It’s not an edge case but it is valid to say that the pattern of development is different. You’re right that people need feedback that what they’re doing is right within 1-2 seconds. They are getting that thanks to the IDE. They need their tests to compile and run within a few seconds, and they’re getting close to that.
I’m not disagreeing with you. In an ideal world all builds and tests would run within 2 seconds, like you’re suggesting. And yet, people are managing to ship software successfully, despite it taking more than 2 seconds. That suggests that maybe this 2 second thing isn’t a requirement and more of a nice to have. And that maybe, there is something to people relying on IDE feedback while coding.
It is a single file, but when you include header files which get dropped in they can get to many hundreds of thousands of lines of code in practice per corresponding .o file.
But the level of parallelism possible is downright insane with C++ because of this property... once, out of anger at my build time on something I was working on back in 2016/2017, I set up a AWS Lambda as a compilation server (fully generic: all it had was my toolchain, not the code in question or even the build scripts) for something akin to ccache and successfully managed to build my project with a pipeline (I use that word as I was still doing the preprocessing locally) of at least a hundred simultaneous jobs: good luck ever pulling that off with Rust.
It is also generally very easy to break apart any file or decouple any header files using various techniques that don't affect your project layout or require any mental maintenance once done. In contrast, with Rust, you have to essentially refactor what your project even "means" to pull that off as you now need to be juggling a ton of microcrates (and then it seems like there is some inherent crate overhead so this becomes a painful tradeoff plane you need to optimize; compilation units have to become degenerated small before that matters in C++).
And, even then, the remaining slowest file in my insanely complex C++ project does tend to take a ridiculously long time to build (I have a 30-60 second long compile for one small component) and I have run out of low-hanging fruit; but, I honestly hardly ever actually have to build it: the coupling of header files is massively overstated as when you are in a tight build/update cycle you are generally modifying a single function in a source code file, not changing some header file of basic data types that will affect the code generation of literally the entire rest of the project.
Like, I get it: thinking about code as a merged set of linear forward passes is sometimes annoying, but it doesn't just enable massive gains in compile time... it also encourages more legitimate coupling between components as circular dependencies require extra code and upkeep for prototypes and that's good because circular dependencies are bad. Rust thereby is doing something that frankly feels super awkward: it is elevating behavior that frankly makes more sense only within a single file or at best a single object to a project organizational unit, which suddenly makes this all seem much more complicated to handle correctly.
It is also generally very easy to break apart any file or decouple any header files using various techniques that don't affect your project layout or require any mental maintenance once done. In contrast, with Rust, you have to essentially refactor what your project even "means" to pull that off as you now need to be juggling a ton of microcrates (and then it seems like there is some inherent crate overhead so this becomes a painful tradeoff plane you need to optimize; compilation units have to become degenerated small before that matters in C++).
And, even then, the remaining slowest file in my insanely complex C++ project does tend to take a ridiculously long time to build (I have a 30-60 second long compile for one small component) and I have run out of low-hanging fruit; but, I honestly hardly ever actually have to build it: the coupling of header files is massively overstated as when you are in a tight build/update cycle you are generally modifying a single function in a source code file, not changing some header file of basic data types that will affect the code generation of literally the entire rest of the project.
Like, I get it: thinking about code as a merged set of linear forward passes is sometimes annoying, but it doesn't just enable massive gains in compile time... it also encourages more legitimate coupling between components as circular dependencies require extra code and upkeep for prototypes and that's good because circular dependencies are bad. Rust thereby is doing something that frankly feels super awkward: it is elevating behavior that frankly makes more sense only within a single file or at best a single object to a project organizational unit, which suddenly makes this all seem much more complicated to handle correctly.
> in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker
Rust-analyzer is struggling significantly on larger repos, to the point of being barely usable.
Rust-analyzer is struggling significantly on larger repos, to the point of being barely usable.
> Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.
What do you mean exactly here?
What do you mean exactly here?
In C++ you can compile each file independently from every other file. You can't do that in Rust; you can only compile crates (which are made of many files) independently. Within a crate you can have circular references for example.
So the smallest chunk of work that can be parallelised is much bigger in Rust than in C++. (Unless you are writing atypically small Rust crates and atypically enormous C++ files.)
So the smallest chunk of work that can be parallelised is much bigger in Rust than in C++. (Unless you are writing atypically small Rust crates and atypically enormous C++ files.)
I understand that's not a language limitation, but rather a compiler/toolset/ecosystem issue (like not having a standard ABI?).
I ask becasue I wonder what's the real culprit here and whether it can be solved or if it's due to the language.
I ask becasue I wonder what's the real culprit here and whether it can be solved or if it's due to the language.
It is like you say, mostly compiler/toolset/ecosystem, all 3.
While staying in the same ecosystem you can split your application into smaller crates, which many are doing. It's just not that ergonomic.
While staying in the same ecosystem you can split your application into smaller crates, which many are doing. It's just not that ergonomic.
It kind of is a language limitations. They could have said "no circular dependencies at all!" (which is quite reasonable) and then it would be easier to deal with.
> Is it an unpopular opinion to say 75% of “a lot” is still “a lot”
I don’t know if it’s unpopular, but it may be inaccurate. OP claims that they improved by 75%, so you probably meant “25% of a lot is still a lot”.
> I would expect incremental build speedup to be prioritized, and steadily improving. But clearly it isn’t moving very fast in that direction. So why not?
Is this assessment based on something?
For what it’s worth, people have been working on it for years, with varying approaches and levels of success. The recent spate of articles you’ve seen is precisely because these approaches are bearing fruit. Several folks have worked on making a previously single threaded frontend multithreaded, others have added an option to swap the LLVM backend out for a much faster one (cranelift), and yet others are working on moving to a faster linker by default.
The net gain from all of these once they land could be as much as 30–50%, depending on the project, mode of compilation (clean or incremental) and hardware (single/multiple cores). That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.
If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster. 2023’s report - https://nnethercote.github.io/2024/03/06/how-to-speed-up-the...
Ultimately, all of these improvements might not be enough for the people who need HMR to be productive. That’s possible. But the idea isn’t to get to a point where every line of code written in the world is Rust. There’s plenty of Rust development happening today. When the compile times drop as hoped, more people join and that grows the ecosystem.
I don’t know if it’s unpopular, but it may be inaccurate. OP claims that they improved by 75%, so you probably meant “25% of a lot is still a lot”.
> I would expect incremental build speedup to be prioritized, and steadily improving. But clearly it isn’t moving very fast in that direction. So why not?
Is this assessment based on something?
For what it’s worth, people have been working on it for years, with varying approaches and levels of success. The recent spate of articles you’ve seen is precisely because these approaches are bearing fruit. Several folks have worked on making a previously single threaded frontend multithreaded, others have added an option to swap the LLVM backend out for a much faster one (cranelift), and yet others are working on moving to a faster linker by default.
The net gain from all of these once they land could be as much as 30–50%, depending on the project, mode of compilation (clean or incremental) and hardware (single/multiple cores). That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.
If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster. 2023’s report - https://nnethercote.github.io/2024/03/06/how-to-speed-up-the...
Ultimately, all of these improvements might not be enough for the people who need HMR to be productive. That’s possible. But the idea isn’t to get to a point where every line of code written in the world is Rust. There’s plenty of Rust development happening today. When the compile times drop as hoped, more people join and that grows the ecosystem.
> you probably meant “25% of a lot is still a lot”
Yeah exactly. This was afaik a fairy small sized application (where the majority was probably framework code) which went from 20s to 5s for incremental compilation on a very beefy desktop rig. Those absolute numbers are important to grok the situation.
> That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.
> If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster
It’s not at all that it’s unimpressive. It’s abundantly clear that extremely bright minds are putting a ton of effort into it, consistently and over time. I’ve worked personally with some of these people and they are insanely talented and knowledgeable… Which is precisely why I’m concerned that we’re not seeing the big boosts that comes with picking low hanging fruit, perhaps there isn’t any left. Optimization tends to yield lower results over time (although the recent advances are great to see and I wish it continues). From the outside though, it appears like either inherent constraints that limit how far you can go, or at least initial compiler design reasons that would be cost-prohibitive to change.
Yeah exactly. This was afaik a fairy small sized application (where the majority was probably framework code) which went from 20s to 5s for incremental compilation on a very beefy desktop rig. Those absolute numbers are important to grok the situation.
> That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.
> If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster
It’s not at all that it’s unimpressive. It’s abundantly clear that extremely bright minds are putting a ton of effort into it, consistently and over time. I’ve worked personally with some of these people and they are insanely talented and knowledgeable… Which is precisely why I’m concerned that we’re not seeing the big boosts that comes with picking low hanging fruit, perhaps there isn’t any left. Optimization tends to yield lower results over time (although the recent advances are great to see and I wish it continues). From the outside though, it appears like either inherent constraints that limit how far you can go, or at least initial compiler design reasons that would be cost-prohibitive to change.
> big boosts that comes with picking low hanging fruit, perhaps there isn’t any left. Optimization tends to yield lower results over time
As the link I shared says
> For the first time ever, I’m writing one of these posts without having made any improvements to compile speed myself. I have always used a profile-driven optimization strategy, and the profiles you get when you measure rustc these days are incredibly flat. It’s hard to find improvements when the hottest functions only account for 1% or 2% of execution time. Because of this I have been working on things unrelated to compile speed.
> That doesn’t mean there are no speed improvements left to be made, as the previous section shows. But they are much harder to find, and often require domain-specific insights that are hard to get when fishing around with a general-purpose profiler. And there is always other useful work to be done.
---
> From the outside though, it appears like either inherent constraints that limit how far you can go, or at least initial compiler design reasons that would be cost-prohibitive to change.
Also correct. Rust prioritised faster run time over fast compilation times. For example, monomorphization is something used pervasively in most Rust code. Fast at run time, but slow to compile. The fact that the entire crate needs to be compiled if a line is changed is yet another issue.
Everything indicates that even if Rust improves, it will never reach that 2 second recompile for large projects. But fortunately the experience of the last few years suggests that it's still possible to succeed despite (or maybe because of) prioritising runtime over compile time.
As the link I shared says
> For the first time ever, I’m writing one of these posts without having made any improvements to compile speed myself. I have always used a profile-driven optimization strategy, and the profiles you get when you measure rustc these days are incredibly flat. It’s hard to find improvements when the hottest functions only account for 1% or 2% of execution time. Because of this I have been working on things unrelated to compile speed.
> That doesn’t mean there are no speed improvements left to be made, as the previous section shows. But they are much harder to find, and often require domain-specific insights that are hard to get when fishing around with a general-purpose profiler. And there is always other useful work to be done.
---
> From the outside though, it appears like either inherent constraints that limit how far you can go, or at least initial compiler design reasons that would be cost-prohibitive to change.
Also correct. Rust prioritised faster run time over fast compilation times. For example, monomorphization is something used pervasively in most Rust code. Fast at run time, but slow to compile. The fact that the entire crate needs to be compiled if a line is changed is yet another issue.
Everything indicates that even if Rust improves, it will never reach that 2 second recompile for large projects. But fortunately the experience of the last few years suggests that it's still possible to succeed despite (or maybe because of) prioritising runtime over compile time.
I have a feeling many of the common multithreading optimizations carry the same flaw as incremental compilation: they give you good speed-ups in the short term, but potentially mask deeply flawed architectural decisions. Essentially improving perf by digging yourself into a deeper grave.
[deleted]
I’m not sure I understand why editing an html template requires to recompile the rust project. I implemented a static site generator in rust for my own projects, html templates are simple liquid template files, it’s instant to recompile them. Same for stylesheets. Is the author generating the html from rust directly? If yes that sounds like a painful strategy
75% is really good even if it requires changing the toolchain.
Have you tried https://github.com/firebuild/firebuild ?
It is a caching accelerator and it can cache the linking and the buildscripts, too in Rust builds.
It can make your builds more than 90% faster, especially with low cores counts. https://balintreczey.hu/blog/improve-build-time-of-rust-java...
The Mac port is experimental, though.
The older i get the more i am convinced that there-ain’t-no-free-lunch applies widely, beyond financial markets.
Software development: the art of redistributing aggregate lifecycle pain; who bears what, when and how much.
Software development: the art of redistributing aggregate lifecycle pain; who bears what, when and how much.
> i am convinced that there-ain’t-no-free-lunch applies widely, beyond financial markets
That's amusingly ironic, since it doesn't originate in financial markets anyway!
https://en.wikipedia.org/wiki/No_free_lunch_theorem
That's amusingly ironic, since it doesn't originate in financial markets anyway!
https://en.wikipedia.org/wiki/No_free_lunch_theorem
[deleted]
Very cool, still kind of disappointing the author was only able to get incremental compilation down to 4.5s, no matter the baseline. Hopefully Cranelift lands in stable at some point.
That’s pretty good. Our frontend app built with TS with SWC recompiles in about that time.
Depends on the code size though. Is this 1k lines? 10k lines? 100k lines?
Just looking at the blog this seems like a fully static site that should be able to be built in milliseconds. Some input from the developer here on what takes all this time would be interesting.
Just looking at the blog this seems like a fully static site that should be able to be built in milliseconds. Some input from the developer here on what takes all this time would be interesting.
Their benchmarks are based on a machine with 72GB of RAM and another with 128GB of RAM - not that it does matter, but I'm really curious if they're outliers or if I should bump my computer specs. That's a lot of Electron web apps running in parallel!
I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance.
So no, 72GB or 128GB makes no difference.
So no, 72GB or 128GB makes no difference.
Yes, maybe I didn't make it clear but I'm pretty sure this is not impacting for Rust. I was just wondering if it's normal to have these specs since my computer has probably "just" 32GB of RAM
I agree that memory size isn't going to matter much, but memory speed should be noticeable, especially on large projects.
[deleted]
Definitely outliers. The Steam hardware survey [0] is obviously not a perfect measurement, but in its February results only ~3% had 64GB and only ~0.3% had more than that.
I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB.
[0] https://store.steampowered.com/hwsurvey
I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB.
[0] https://store.steampowered.com/hwsurvey
I have 32GB on my Linux laptop (plus 12GB swap) and regularly run out of RAM with just a handful of VSCode's open (it's generally the language servers using all the RAM) and a couple of dozen Firefox tabs (Firefox seems to be much worse than Chrome RAM-wise). If I run my actual work project, which needs a few gigs then it often just completely runs out of memory and hard reboots.
Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows.
On Linux I wish I had 64GB.
Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows.
On Linux I wish I had 64GB.
You might just be misunderstanding Linux's RAM management strategy. Do you actually start getting out of memory errors, or do you see "free" memory rapidly go to zero?
If it's the latter, that's because Linux deliberately uses as much RAM as possible for caches and such, which means "free" should be near zero most of the time. The number that actually matters on Linux is "available" memory, which is the amount of memory that is currently free or used for optional caches and could be reallocated for a new required chunk of RAM should an application need it. Here [0] is a decent in-depth explanation.
If it's the former, there's something very wrong with your installation, because every Linux system I have makes much better usage of memory than Windows, even running a heavyweight DE like KDE. Even my machine that dual-boots Linux and Windows performs noticeably better on all metrics (including memory usage) when in Linux.
[0] https://linuxblog.io/free-vs-available-memory-in-linux/
If it's the latter, that's because Linux deliberately uses as much RAM as possible for caches and such, which means "free" should be near zero most of the time. The number that actually matters on Linux is "available" memory, which is the amount of memory that is currently free or used for optional caches and could be reallocated for a new required chunk of RAM should an application need it. Here [0] is a decent in-depth explanation.
If it's the former, there's something very wrong with your installation, because every Linux system I have makes much better usage of memory than Windows, even running a heavyweight DE like KDE. Even my machine that dual-boots Linux and Windows performs noticeably better on all metrics (including memory usage) when in Linux.
[0] https://linuxblog.io/free-vs-available-memory-in-linux/
Have you tried zram for instance? There are a lot of things one can do to better utilise RAM on Linux.
I believe Fedora comes ootb with a lot of things to make ram management quite good. Using another distro these days, but Fedora impressed me on that front.
(edit: Try to use zram and zswap, check if your i/o scheduler is correct for your hardware, and increase your swapiness value. These should make your system behave better. For other performance improvements, use ananicy and irqbalance.)
I believe Fedora comes ootb with a lot of things to make ram management quite good. Using another distro these days, but Fedora impressed me on that front.
(edit: Try to use zram and zswap, check if your i/o scheduler is correct for your hardware, and increase your swapiness value. These should make your system behave better. For other performance improvements, use ananicy and irqbalance.)
Yes I have zram enabled. First thing I tried.
Even if it is possible to fix this, it's still a horrible bug because you absolutely should not have to blindly tweak settings to stop a desktop computer from hard-rebooting when it runs out of RAM.
I think this is a really fundamental problem on Linux though - it would require enormous changes and unprecedented cooperation between different groups for a proper fix. Much easier to say "you're holding it wrong".
Even if it is possible to fix this, it's still a horrible bug because you absolutely should not have to blindly tweak settings to stop a desktop computer from hard-rebooting when it runs out of RAM.
I think this is a really fundamental problem on Linux though - it would require enormous changes and unprecedented cooperation between different groups for a proper fix. Much easier to say "you're holding it wrong".
Sounds like you need some swap (?)
Set your swap to double your RAM. Windows has a swap file that grows as much as it needs to.
I'm pretty sure it doesn't. Windows lets you set the swap size limit. The biggest difference is that Windows doesn't over-commit, so you need a big swap file otherwise you'll be running out of memory all the time even before you actually use all your physical RAM.
Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason.
If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.
If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.
You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer.
MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizerFunnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM.
But the 7950X is grossly underutilised, I don't know what to do with it.
But the 7950X is grossly underutilised, I don't know what to do with it.
Shouldn't the quantized LLMs be using the CPU to good effect? I'd imagine your Mixtral is substantially faster than it would be on a weaker CPU.
I have a 4080 with 16GB of VRAM. I experimented with llama.cpp by offloading layers onto GPU and doing the remaining on CPU. I found that it gives me max tokens/sec if I set it to 8 CPU cores as opposed to the 16 available on the 7950X. I guess beyond that, the bookkeeping between the cores might be taking up more time than it is worth.
the answer i found is run gentoo and compile yocto projects.
Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.
If you are like me and opening too many browser tabs (and other applications, of course), you need a lot of RAM. My desktop has only 48 GB of RAM right now and a typical VM usage is around 80--100 GB. I will probably go for 128 GB or possibly 192 GB of RAM for the next desktop.
Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.
Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.
My understanding is that processor performance is more important for rust compilation than available memory, but ultimately the atrocious compilation times are why I struggled so much with Rust.
> Note: Discord User @PaulH(and a couple others) let me know about this bug that prevents the server build from using the dependencies built for the previous compilation of the server/webassembly frontend build due to changes in the RUSTFLAGS. You can fix this by having cargo-leptos > 0.2.1 and adding this to you Cargo.toml under your Leptos settings block. I did not test with this enabled.
> [package.metadata.leptos]
> separate-front-target-dir = true
This option is deprecated since cargo-leptos 0.2.3 (now it's enabled unconditionally), it's going to be removed in 0.3
https://github.com/leptos-rs/cargo-leptos/pull/216
https://github.com/leptos-rs/cargo-leptos/issues/217
https://github.com/leptos-rs/cargo-leptos/commit/b0c19a87cff...
> [package.metadata.leptos]
> separate-front-target-dir = true
This option is deprecated since cargo-leptos 0.2.3 (now it's enabled unconditionally), it's going to be removed in 0.3
https://github.com/leptos-rs/cargo-leptos/pull/216
https://github.com/leptos-rs/cargo-leptos/issues/217
https://github.com/leptos-rs/cargo-leptos/commit/b0c19a87cff...
The sold linker is no longer commercial software; it's been relicensed under the MIT license.
Rui, if I ever get rich, I will seriously give you 1 million US dollars, $100K the minimum, for everything you have done so far, with all your projects.
That's the least I can do to show my appreciation for your incredible work; I can't thank you enough brother!
I also wish to see that chibicc compiler plus the book completed. I want to learn everything around such topics.
That's the least I can do to show my appreciation for your incredible work; I can't thank you enough brother!
I also wish to see that chibicc compiler plus the book completed. I want to learn everything around such topics.
Thanks, I really wish you become a billionaire :)
But it is no longer updated, is it?
Thanks anyway for the work on sold.
Thanks anyway for the work on sold.
> But it is no longer updated, is it?
Essentially, yes, but that’s because Apple’s new linker is so much faster than it was. You can use it instead.
Essentially, yes, but that’s because Apple’s new linker is so much faster than it was. You can use it instead.
Out of curiosity, do you know what technical choices Apple made to be so much faster?
It’s parallelized. https://news.ycombinator.com/item?id=36218330
I don't know the details, but once it is proven that achieving lld/mold/sold-level performance is technically feasible, it shouldn't be extremely hard to achieve similar success for macOS.
Sold was just mold under a commercial license. Mold is still maintained and now licensed permissively.
That's true but mold doesn't have MacOS support.
As a front-end developer dabbling with rust, the compiler being slow is not a problem for me. Whilst I appreciate the benchmark, it's not likely I'll have 72GB of ram laying around to speed up the process.
Yes, hot module reloading is nice and quick. But it reloads with errors included.
The primary benefit of the rust compiler, at least from my point of view, is telling me what's wrong, where. It's a worthwhile sacrifice for a few seconds of my time, when at the end of addressing all of the obvious errors, I have something that works. I find this miles better than HMR.
Yes, hot module reloading is nice and quick. But it reloads with errors included.
The primary benefit of the rust compiler, at least from my point of view, is telling me what's wrong, where. It's a worthwhile sacrifice for a few seconds of my time, when at the end of addressing all of the obvious errors, I have something that works. I find this miles better than HMR.
You’re a front end developer but you don’t need to make small tweaks all the time? “At the end of addressing all of the obvious errors, I have something that works.” is irrelevant to large swaths of front end code, which is basically always in a “working” state (displays something without error), but needs to be continuously expanded and refined.
TFA talks about reload times of 20-30s before their optimization, on a powerful computer. It’s certainly not “a few seconds” especially for people on less powerful hardware. That’s a ridiculous wait time for, say, tweaking a CSS class.
I find this comment highly suspect.
TFA talks about reload times of 20-30s before their optimization, on a powerful computer. It’s certainly not “a few seconds” especially for people on less powerful hardware. That’s a ridiculous wait time for, say, tweaking a CSS class.
I find this comment highly suspect.
JavaScript errors only occur at runtime. It's very common, especially in legacy codebases, for issues to just fly under the radar, even if they are obvious to the seasoned dev. There is a reason why one of the most popular JS books is called 'Javascript: the good parts'.
With rust, it's very refreshing in that it's idiot proof, which JavaScript is not. CSS is not relevant, since it is not actual programming.
With rust, it's very refreshing in that it's idiot proof, which JavaScript is not. CSS is not relevant, since it is not actual programming.
We (at least the author and I) are talking about front end development here where most reloads are tweaking markup and stylesheets and shouldn’t introduce any error. And half a minute or longer reloads are unacceptable for those.
I don’t need a lecture on the history of JavaScript, thank you. Especially since JavaScript/TypeScript has come such a long way that JavaScript the Good Parts is barely relevant anymore.
> Rust… idiot proof
That’s such a… interesting statement that I’m not going to respond further.
I don’t need a lecture on the history of JavaScript, thank you. Especially since JavaScript/TypeScript has come such a long way that JavaScript the Good Parts is barely relevant anymore.
> Rust… idiot proof
That’s such a… interesting statement that I’m not going to respond further.
CSS is actual programming. Nevertheless, he was just stating a common case where one iteratively checks if their assumptions, even if carefully accounted for, conform to their expectations.
Another, very real "programming" situation could be the following: correctly parsing 3D mesh information in a multithreaded fashion and pass that to your preferred API of choice. Is one going to get all edge cases right the first time? Is your program's codebase large enough? Eventually it becomes kinda yucky having to wait.
The compiler may know what is programmatically correct, not what your intention is. I think that's what the other person was trying to argue.
I'm not too concerned about compile times myself, I'm used to them given I mostly dabble with compiled languages (I do a lot of CSS though!), but there's a argument to be made about iteration time and being fast in solving a problem.
Another, very real "programming" situation could be the following: correctly parsing 3D mesh information in a multithreaded fashion and pass that to your preferred API of choice. Is one going to get all edge cases right the first time? Is your program's codebase large enough? Eventually it becomes kinda yucky having to wait.
The compiler may know what is programmatically correct, not what your intention is. I think that's what the other person was trying to argue.
I'm not too concerned about compile times myself, I'm used to them given I mostly dabble with compiled languages (I do a lot of CSS though!), but there's a argument to be made about iteration time and being fast in solving a problem.
I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance.
So no, 72GB or 128GB makes no difference.
So no, 72GB or 128GB makes no difference.
I have a couple of projects where it is measured in minutes, on a dual core with 8GB and a SSD.
If only devs with beefy machines are able to use Rust properly, that hinders adoption.
If only devs with beefy machines are able to use Rust properly, that hinders adoption.
Developers in USA: "nobody in the entire world has less than 32GB of RAM!"
Laptop + 32 gigs of RAM is a recipe for destruction of your wallet. At least for desktops you can kind of scope things out, but companies aren't really in the practice of distributing desktops anymore it seems.
I have the 2022 model of this machine [1] which is a macBook Pro clone from Xiaomi.
It's an absolute beast, build quality is almost on par with Apple (screen is better than what Apple offered in 2022, actually) and the 32GB RAM base model from this year can be had for less than 1,000€.
I dunno where the "destruction of your wallet" threshold lies for you but for me it's way higher.
The 2022 version also used Ryzen APUs and there is excellent Linux support. It's my daily driver for Rust development (I run Pop!_OS on it).
[1] https://a.aliexpress.com/_EygTDml
It's an absolute beast, build quality is almost on par with Apple (screen is better than what Apple offered in 2022, actually) and the 32GB RAM base model from this year can be had for less than 1,000€.
I dunno where the "destruction of your wallet" threshold lies for you but for me it's way higher.
The 2022 version also used Ryzen APUs and there is excellent Linux support. It's my daily driver for Rust development (I run Pop!_OS on it).
[1] https://a.aliexpress.com/_EygTDml
I’m really not wild about buying a Xiaomi laptop (even though my understanding is Xiaomis stuff is all well engineered etc). People will point to the NSA doing shit but I have fairly little faith in Xiaomi devices (especially ones heading towards foreign addresses) not having nasty secret backdoors.
There is a bit of a qualitative distinction between the Lenovo backdoor stuff and that. But that’s just my personal threat model
There is a bit of a qualitative distinction between the Lenovo backdoor stuff and that. But that’s just my personal threat model
"But for me it's way higher"
Point not comprehended.
Point not comprehended.
> Laptop + 32 gigs of RAM is a recipe for destruction of your wallet.
While I'm not a developer of any sort, you couldn't be more on the nose here. I waited and waited for a Surface Laptop that has 32gb on ebay. When it came, price skyrocket to almost double the 16gb model. Twice more it happened and I literally gave up and bought the 16gb. I don't know what the retail premium from Microsoft is, but the reseller premium is truly insanity. Surely, Microsoft spent the time engineering it to fit 32gb and could reap more producing more of them.
While I'm not a developer of any sort, you couldn't be more on the nose here. I waited and waited for a Surface Laptop that has 32gb on ebay. When it came, price skyrocket to almost double the 16gb model. Twice more it happened and I literally gave up and bought the 16gb. I don't know what the retail premium from Microsoft is, but the reseller premium is truly insanity. Surely, Microsoft spent the time engineering it to fit 32gb and could reap more producing more of them.
I recently upgraded my gaming laptop to 32GB DDR4 for very cheap. It doesn’t destroy the wallet if the ram isn’t soldered and you can easily replace it. I paid about 75€ with taxes for two sticks of 16GB. I wouldn’t recommend gaming laptops as laptops though. They are terrible laptops.
How much was the gaming laptop itself though? I’m sure you can make this work of course, just curious about the base machine price
About 2000€ I think. But you can probably find a gaming laptop around 1000€ that is easy to upgrade to 32GB.
What? Laptop + 32G RAM is Laptop + 100€. With a refurbished thinkpad you can get reasonable compile times for as little as 600€.
Stop buying macbooks if you want lots of RAM and performance for a low price.
Stop buying macbooks if you want lots of RAM and performance for a low price.
This has definitely be one big reason for me why I did not get into rust. I don’t want to have to buy a new machine just to get the compile times I’m used to
People are impatient. I expect in a few years someone will”solve” this problem by removing the borrow checker and then by removing the unsafe keyword, then allowing dynamic linking and shared libraries…. And this person will be lauded as a genius. And some group of cranky bitters will make a new language with a new borrow checker,,,, cycle repeats
The borrow checker is not responsible for even 1% of compilation time.
The elephant in the room is monomorphisation and macro expansion. Those generate a ton of code that LLVM has to process. In return you get much faster programs at runtime.
The elephant in the room is monomorphisation and macro expansion. Those generate a ton of code that LLVM has to process. In return you get much faster programs at runtime.
I use Elm at work, and the confidence from really strict compilation and typing is very worth it.
With that said, Elm is super fast to compile too, which is very nice.
With that said, Elm is super fast to compile too, which is very nice.
Working on a couple of ideas that flip that logic on its head, but you’re more likely to be right because of finite time.
The thing about rust is that the overhead is all upfront.
Not to be sacrilegious, but bleeding at the move or &wtf borrow boundary is a price one pays to have confidence, subsequently.
Your cms/customer acronym probably doesn’t need it, but you’ll love the —fast option in some stacks that merely means “compile the rust extension.”
Plus… it’s a gainful hobby. I’m fine if other people have hobbies that don’t resemble x86 in 1998 on slackware ;)
The thing about rust is that the overhead is all upfront.
Not to be sacrilegious, but bleeding at the move or &wtf borrow boundary is a price one pays to have confidence, subsequently.
Your cms/customer acronym probably doesn’t need it, but you’ll love the —fast option in some stacks that merely means “compile the rust extension.”
Plus… it’s a gainful hobby. I’m fine if other people have hobbies that don’t resemble x86 in 1998 on slackware ;)
I will never understand who hated my innocent and light-hearted thought that much. Hope it was worth making me go, “aw” for no reason.
> The primary benefit of the rust compiler, at least from my point of view, is telling me what's wrong, where.
Incredible! Rust developers have invented "IDEs"!
Incredible! Rust developers have invented "IDEs"!
I'm not sure if you're familiar with front-end developer tooling or the development process, but my editor will not tell me what error a browser is going to throw.
> but my editor will not tell me what error a browser is going to throw
And so won't Rust, a runtime error is a runtime error after all.
But even Typescript has pretty good control flow analysis which catches a lot of (what would be) JS runtime errors while typing when properly integrated with an IDE.
It's not like Rust invented static typing, it just has a very strong type system (which can quickly become an annoyance though, that's why there's a wide range of 'strictness' among statically typed languages, because while static typing is generally a good thing, a 'too strong' type system might not be.
And so won't Rust, a runtime error is a runtime error after all.
But even Typescript has pretty good control flow analysis which catches a lot of (what would be) JS runtime errors while typing when properly integrated with an IDE.
It's not like Rust invented static typing, it just has a very strong type system (which can quickly become an annoyance though, that's why there's a wide range of 'strictness' among statically typed languages, because while static typing is generally a good thing, a 'too strong' type system might not be.
Somehow I see a trend on developers educated in scripting languages and using either Go or Rust as their first compiled languages.
Apparently they invented static linking, pattern matching, structural typing, unsafe keyword, and plenty of other stuff already present on compiler toolchains and type systems that predate them...
I wonder if we will ever sort out the issue of developers caring about programing language history, or operating systems for that matter.
I mean in the wider sense, not the geeky community that already does that as a hobby.
Apparently they invented static linking, pattern matching, structural typing, unsafe keyword, and plenty of other stuff already present on compiler toolchains and type systems that predate them...
I wonder if we will ever sort out the issue of developers caring about programing language history, or operating systems for that matter.
I mean in the wider sense, not the geeky community that already does that as a hobby.
Your sarcasm missed the mark. You meant "static typing".
[deleted]
The coolest thing about this to me IMO is the comparison between the two CPUs.
Is there a project that publishes benchmarks compiling projects across different CPUs? I know passmark is kinda the go-to but this would be cool.
Someday I'll feel the need to upgrade from a 2700x...
Is there a project that publishes benchmarks compiling projects across different CPUs? I know passmark is kinda the go-to but this would be cool.
Someday I'll feel the need to upgrade from a 2700x...
Phoronix does a lot of CPU benchmarks, including code compilation, but mostly focused on Linux. More result are also available in the OpenBenchmark page, which is also part of the same project. Take a look at the timed compilation test-suit: https://openbenchmarking.org/suite/pts/compilation
Super interesting! I would have loved to see included in the potential solutions the use of sccache too
Looking at some benchmarks online and the 7900x always beat the 5950x even though the 5950x has more cores.
But in this article the 7900x performed worse than the 5950x, is it because of the core count, or are there some other factors?
But in this article the 7900x performed worse than the 5950x, is it because of the core count, or are there some other factors?
I really wish people wouldn't use AI-generated images for blog posts, it's so distracting! I clicked the link wanting to read the article, but instead spent several minutes looking at the image to find errors: what's shown on the screen (apparently it's "Jot Commeditin") looks like something between a hex dump, git blame, and occult incantations written in a long-forgotten alphabet. Probably you need a keyboard like the one on the desk (with keys arranged in a fractal pattern) to write something like that. And a mouse with a cable going nowhere...
Same, but what’s worse is that people never put in the caption that it’s AI nor what tool is used. Transparency goes a long way.
I did the same! Then didn't read the article.
He got three out of four of “manic pixie dream girl”
[deleted]
From my experience, faster SSD have a suprising strong effect for some (simple) languages (probably not Rust), like Go [0]
[0] https://www.octobench.com/
[0] https://www.octobench.com/
I’ve found in 10+ years of software development that speed of iteration cycle is highly correlated with productivity. Compile times is not the only input into this cycle time, but it’s a big one, and importantly, it’s within the control of the language tooling itself to solve. The human idle attention time of 1-2 seconds should be the gold standard to strive for, even if not always achievable.
There seems to be quite a bit of cope around Rust build times in the community, which was natural a few years ago (a lot of people used to “blame” llvm, but it doesn’t seem to be as big of a culprit) but things are different now, no? Given the maturity, growing ecosystem and corporate investment, I would expect incremental build speedup to be prioritized, and steadily improving. But clearly it isn’t moving very fast in that direction. So why not?