On Proebsting's Law(zeux.io)
zeux.io
On Proebsting's Law
https://zeux.io/2022/01/08/on-proebstings-law/
8 comments
> What we get with LLVM is a large number of people tossing in the things that help the handful of workloads they are currently working on, with limited oversight regarding how that’s impacting compile-time for everyone else. So the compiler gets slower and slower, the compiled code doesn’t get much faster, and overall the compiler grows and grows in complexity.
Is that how you'd describe runtime "speedups in the 10-15% range in optimized builds" vs compile-time "2.2x slower in O2/O3"? Because as a compiler user it sounds totally worth it to me. A lot more CPU time goes into running my code than into compiling it. That's true both when I write large distributed systems software (which occupies many machines) and when I write small-scale software (that runs on less powerful machines than I use for development).
Sure, "15% slower in debug builds" kind of sucks, I might have hoped for more improvement over 10.5 years, maybe complexity did snowball (I'm not a LLVM developer so I wouldn't know), and maybe there were a bunch of changes that didn't carry their weight. I'd still take LLVM 11 over LLVM 2.7 overall.
Is that how you'd describe runtime "speedups in the 10-15% range in optimized builds" vs compile-time "2.2x slower in O2/O3"? Because as a compiler user it sounds totally worth it to me. A lot more CPU time goes into running my code than into compiling it. That's true both when I write large distributed systems software (which occupies many machines) and when I write small-scale software (that runs on less powerful machines than I use for development).
Sure, "15% slower in debug builds" kind of sucks, I might have hoped for more improvement over 10.5 years, maybe complexity did snowball (I'm not a LLVM developer so I wouldn't know), and maybe there were a bunch of changes that didn't carry their weight. I'd still take LLVM 11 over LLVM 2.7 overall.
LLVM has been negligent about compile time for a long time, but they are improving recently. See https://www.npopov.com/2020/05/10/Make-LLVM-fast-again.html for an example.
> This “law” was never really reasonable or sensible.
I took two compilers courses from Dr Proebsting, and while he never discussed it in my classes, based on his sense of humor I'd guess he chose "18 years" mostly as a parallel/pun on Moore's law and "18 months".
In other words, I doubt he ever fit a curve and tried to make predictions. I think the whole point was to cast shade on whether compiler optimizations were all that important past a certain point.
> This is why I’m personally a lot more excited about working on small manageable compiler code bases [...]
It sounds like you came to the same conclusion :-)
I took two compilers courses from Dr Proebsting, and while he never discussed it in my classes, based on his sense of humor I'd guess he chose "18 years" mostly as a parallel/pun on Moore's law and "18 months".
In other words, I doubt he ever fit a curve and tried to make predictions. I think the whole point was to cast shade on whether compiler optimizations were all that important past a certain point.
> This is why I’m personally a lot more excited about working on small manageable compiler code bases [...]
It sounds like you came to the same conclusion :-)
One past thread from long ago:
Proebsting's Law: Compiler Advances Double Computing Power Every 18 Years - https://news.ycombinator.com/item?id=317213 - Sept 2008 (15 comments)
Proebsting's Law: Compiler Advances Double Computing Power Every 18 Years - https://news.ycombinator.com/item?id=317213 - Sept 2008 (15 comments)
The 'benchmarks' would be more interesting if it was performed on a larger more complex codebase. In particular, I'm curious how much the link-time-optimizations that we have today fare against earlier toolchains that don't have this. Basically, I want to interpret "Compiler Advances" as more than version number.
> It is possible in theory that code that’s less carefully optimized exhibits different behavior, or that the benchmarks chosen here are simply not as amenable to compiler optimization as they could be
I think that's very relevant.
A lot of compiler optimisation work is about allowing programmers to write more natural code without losing efficiency. That won't show up as a speed gain in cases where the programmers were writing the "low level" form in the first place.
I think that's very relevant.
A lot of compiler optimisation work is about allowing programmers to write more natural code without losing efficiency. That won't show up as a speed gain in cases where the programmers were writing the "low level" form in the first place.
I’ve been working on optimizing compiler backends for nearly 30 years. The reality is that you hit a wall of diminishing returns pretty quickly, within say 5-20 person-years of effort (so a small team working for say 3-5 years).
You also relatively quickly get to the point where heuristics matter very much and all you do is generate new S-curves as you make changes. Meaning that every change speeds some workloads up, and slows others down.
Disciplined compiler writers will follow the old adage that an optimization needs to pay for itself, meaning that you don’t add things that slow down compilation without improving that S-curve by a relatively comparable amount.
What we get with LLVM is a large number of people tossing in the things that help the handful of workloads they are currently working on, with limited oversight regarding how that’s impacting compile-time for everyone else. So the compiler gets slower and slower, the compiled code doesn’t get much faster, and overall the compiler grows and grows in complexity.
This is why I’m personally a lot more excited about working on small manageable compiler code bases rather than large monolithic ones that try to be everything to everyone.