The square roots of all evil(neilmadden.blog)
neilmadden.blog
The square roots of all evil
https://neilmadden.blog/2024/12/03/the-square-roots-of-all-evil/
51 comments
Those sound to me like the very definition of optimizations that are not premature though. His goal was to teach foundational elements of running programs on computers. Loops and sorts are used all over the place and improvements scale with inputs. I see a pretty big difference between solving for a problem you don't yet understand (the way I've always mentally framed "premature optimization") and establishing reasons for implementing primitives in optimal ways.
In my experience, the people who say"premature optimization it's the root of sal evil" will say that about any and all optimization until the point where all your customers think your product is a slow piece of garbage.
> about any and all optimization that hasn't been measured
Fixed that for you.
Fixed that for you.
When I want some sw to go faster I do profile it, but I also give at least some thought to avoiding unnecessary copying of data or trig functions in tight loops. I regularly have run into people that say considering performance when selecting a data structure is premature optimization. It usually turns out they don't really want to test or profile and just want to churn through tickets. Anything that gets in the way of that bores them.
Measuring would attract even more protest because not only am I using my experience to choose (likely) more efficient algorithms, I am also spending yet more precious company time to measure its performance.
No, broken.
I mean... Maybe? I guarantee that if you do any of the things he discusses in code that is getting reviewed in any company I have ever seen, you will get shot down for premature optimizations.
Just look into Gosper's hack someday and tell me that that would pass most reviews? (Which, I think is probably unfair? But I can't imagine many places being good with using standard int variables to represent subsets.)
Just look into Gosper's hack someday and tell me that that would pass most reviews? (Which, I think is probably unfair? But I can't imagine many places being good with using standard int variables to represent subsets.)
[deleted]
To understand you would have had to be there. Computers got faster (and then some) in the old days you would write something (or not even bother) then see it took way longer than desirable. You would rewrite and itterate over possible ways to rewrite. Sometimes you would see the light, other times you would try different approches brute force. The point where optimization was nesasary was completely obvious and 99% of modern code never needs the consideration.
If communism takes over the world and be given 30 years half the texts wouldnt make sense as it talks about something to do with capitalism? that doesnt exist.
If communism takes over the world and be given 30 years half the texts wouldnt make sense as it talks about something to do with capitalism? that doesnt exist.
I began my programming career on machines with performance, memory, and storage constraints no one today can imagine. Some of the necessary hacks and shortcuts from back then look like premature optimization and stupid coding today.
The Y2K “problem” gives the canonical example. In a world of vast and very cheap and fast storage, it makes no sense to save two characters in a date. But back in the ‘70s and early ‘80s when I implemented dates like that cutting those two characters over a few million records saved significant money. Disk space used to cost a lot, RAM (or core memory) used to cost a lot more.
The Y2K “problem” gives the canonical example. In a world of vast and very cheap and fast storage, it makes no sense to save two characters in a date. But back in the ‘70s and early ‘80s when I implemented dates like that cutting those two characters over a few million records saved significant money. Disk space used to cost a lot, RAM (or core memory) used to cost a lot more.
Computer hardware is very fast now, so why does my computer lag noticeably on OS and browser operations? A facetious question, and perhaps it's not remotely a dealbreaker, but I expect better and would expect the same of my own software. I agree with GGP that too many people seem to take "premature optimization is the root of all evil" as "don't optimize until it's too late and then painstakingly get diminishing returns". There is a comfortable middle ground of the optimizing-delivering tradeoff that I think is wildly missed.
Multiple layers of complex abstractions and sloppy coding account for a lot of it. We write code as if it had no hardware constraints, so we get megabytes-size web pages running a slow interpreted language in a browser built on multiple layers of frameworks and libraries. Over the last several decades inefficient and bulky programming has consumed all of the gains in hardware, to the point that modern applications feel much slower than similar apps from the '80s.
Knuth did not have that in mind when he wrote about premature optimization.
Knuth did not have that in mind when he wrote about premature optimization.
My prediction was that we would create a cpu with a slow and limited opcode set for humans to write on line numbers. Something that is easy to work with with a good static module system with certified sniplets of code. Anything written for it would still run cirkels around higher languages despite slow clock speed. It didnt happen but still could.
I think it was ok to optimize the decrement then jump before publishing 100k hard copies of the code made from trees.
also, great story:
https://www.folklore.org/Saving_Lives.html
also, great story:
https://www.folklore.org/Saving_Lives.html
"book 3" isn't enough for me to identify the textbook you refer to. Could we have a title, perhaps?
This is like asking for the title of the Bible after a psalm was quoted on a forum for religion.
It's "The Art of Computer Programming", volume 3 (Sorting and searching) by Donald Knuth. HTH
It's "The Art of Computer Programming", volume 3 (Sorting and searching) by Donald Knuth. HTH
> This is like asking for the title of the Bible after a psalm was quoted on a forum for religion.
That is…not a great answer, unless you want to reinforce the meme that CS is a cult. Besides, the context you mention is utterly alien to millions of people.
People get born every day, and people learn the basics every day. We should be helping them grow, not gatekeeping. Your second sentence was more than enough.
That is…not a great answer, unless you want to reinforce the meme that CS is a cult. Besides, the context you mention is utterly alien to millions of people.
People get born every day, and people learn the basics every day. We should be helping them grow, not gatekeeping. Your second sentence was more than enough.
This is Hacker News, not Computer Scientist News.
Knuth's work wasn't part of my programming curriculum in an engineering program, either.
Knuth's work wasn't part of my programming curriculum in an engineering program, either.
>This is Hacker News, not Computer Scientist News.
Or Basic Internet Search News.
Or Basic Internet Search News.
Excuse me, but we are hackers, not some fancy elitist people who think for themselves and find out things on their own using some modern internet search tools.
Probably this: https://dl.acm.org/doi/book/10.5555/280635
Rule of thumb. Don't try to generalize until you have at least 3 distinct examples.
This does a lot to find pragmatic tradeoffs between performance and abstraction. It also pushes you away from the over-abstraction that leads to the Second System Effect, as coined by The Mythical Man-Month.
This does a lot to find pragmatic tradeoffs between performance and abstraction. It also pushes you away from the over-abstraction that leads to the Second System Effect, as coined by The Mythical Man-Month.
In most places I’ve worked, there’s a big problem with this approach.
If you have three distinct examples, and you need to do something that requires a fourth, you won’t be able to justify spending time writing an abstraction. You should just copy it again. The three existing examples don’t use an abstraction, and they work just fine, so there’s clearly no need for an abstraction… or so the argument goes. In the unlikely event that you’re allowed to write the abstraction, you definitely won’t be allowed to reimplement the existing examples to use it now, because those modules are not in scope and we don’t have the budget to test them again.
If you don’t create the abstraction up front, you’ve lost the chance to have an abstraction.
If you have three distinct examples, and you need to do something that requires a fourth, you won’t be able to justify spending time writing an abstraction. You should just copy it again. The three existing examples don’t use an abstraction, and they work just fine, so there’s clearly no need for an abstraction… or so the argument goes. In the unlikely event that you’re allowed to write the abstraction, you definitely won’t be allowed to reimplement the existing examples to use it now, because those modules are not in scope and we don’t have the budget to test them again.
If you don’t create the abstraction up front, you’ve lost the chance to have an abstraction.
A fourth hack may still be better than having to develop the four features on top of a wrong abstraction.
If you end up having a poorly fitting abstraction, then you'll be spending extra effort on doing things "properly", but not really getting any benefit out of it. You just loose the agility of hacking things quick'n'dirty, or you end up hacking around the bad abstraction, and get worst of both worlds.
The rest depends on having management that understands trade-offs between doing things properly and efficiently long-term vs getting stuff out of the door quickly (if you're a startup with a short runway, or racing to be the first to win some opportunity, shipping ugly hacks now may be the only way to survive to even have "later" to worry about).
If you end up having a poorly fitting abstraction, then you'll be spending extra effort on doing things "properly", but not really getting any benefit out of it. You just loose the agility of hacking things quick'n'dirty, or you end up hacking around the bad abstraction, and get worst of both worlds.
The rest depends on having management that understands trade-offs between doing things properly and efficiently long-term vs getting stuff out of the door quickly (if you're a startup with a short runway, or racing to be the first to win some opportunity, shipping ugly hacks now may be the only way to survive to even have "later" to worry about).
> A fourth hack may still be better
That may be true, but it’s still a problem if you can never create new abstractions, because that prevents you from creating good abstractions too, not just bad ones.
That may be true, but it’s still a problem if you can never create new abstractions, because that prevents you from creating good abstractions too, not just bad ones.
That's a direct outcome of the contemporary fashion that code changes should be as narrow as possible and only touch existing code where its strictly necessary to their implementation.
It's not the only way of working, but it tends to suit the challenges faced by very large teams with high turnover, diffuse/negligent code ownership, many juniors/early-career contributors, and low trust.
It also produces software that resembles papier maché -- increasingly thick, disordered, and glutinous, with poor visibility and low durability.
If your team isn't subject to the challenges that make papier maché coding necessary, encourage people to escape its cargo cult. Small teams of people who trust each other, where code has long-term owners who can speak to its purpose and direction, can dynamically sculpt and rework code more like clay. And you get better software when that can be done.
It's not the only way of working, but it tends to suit the challenges faced by very large teams with high turnover, diffuse/negligent code ownership, many juniors/early-career contributors, and low trust.
It also produces software that resembles papier maché -- increasingly thick, disordered, and glutinous, with poor visibility and low durability.
If your team isn't subject to the challenges that make papier maché coding necessary, encourage people to escape its cargo cult. Small teams of people who trust each other, where code has long-term owners who can speak to its purpose and direction, can dynamically sculpt and rework code more like clay. And you get better software when that can be done.
> It also pushes you away from the over-abstraction that leads to the Second System Effect, as coined by The Mythical Man-Month.
The Second System Effect has nothing to do with over-abstraction of code - it has nothing to do with code at all. The basic premise is:
First System: We barely did anything we dreamed of and can see 100s of ways to improve things, but we had to get something to market.
Second System: We’ve learned so much! Revenue enables big picture thinking! Everything we ever dreamed of let’s design and build!
It’s fundamentally a mistake at a business/product design level. That feeling of success leads to a release of constraints and then trying to bite off far more than can be chewed.
The Second System Effect has nothing to do with over-abstraction of code - it has nothing to do with code at all. The basic premise is:
First System: We barely did anything we dreamed of and can see 100s of ways to improve things, but we had to get something to market.
Second System: We’ve learned so much! Revenue enables big picture thinking! Everything we ever dreamed of let’s design and build!
It’s fundamentally a mistake at a business/product design level. That feeling of success leads to a release of constraints and then trying to bite off far more than can be chewed.
The second system effect is the same fundamental mistake repeated at every level. From code to business.
The fact that the business mistakes are easiest to spot from outside doesn't mean that the coding mistakes aren't there. Even though that level of detail is only visible from inside of the system.
The fact that the business mistakes are easiest to spot from outside doesn't mean that the coding mistakes aren't there. Even though that level of detail is only visible from inside of the system.
I wonder if they were thinking of the inner platform effect
I call this WET, Write Everything Twice, as opposed to DRY, Don’t Repeat Yourself.
If you need to repeat yourself, first copy and paste. If you need to a third time, then you can pull it into a function to generalize.
But like nearly everything with programming, hard and fast rules are bad, but it’s a good rule of thumb
If you need to repeat yourself, first copy and paste. If you need to a third time, then you can pull it into a function to generalize.
But like nearly everything with programming, hard and fast rules are bad, but it’s a good rule of thumb
Maybe I am out of the lingo loop but I always thought of this as
"We Enjoy Typing"
The point about building the perfect ontology is something I think we all struggle with. I feel like the long lived Cyc project is a classic example.
https://en.wikipedia.org/wiki/Cyc
I'm also surprised that they didn't clarify that the true square root of all evil is 25.8069758011.
https://en.wikipedia.org/wiki/Cyc
I'm also surprised that they didn't clarify that the true square root of all evil is 25.8069758011.
[deleted]
> the true square root of all evil is 25.8069758011.
Ah yes, 665.99999999856098676121, such evil! :P
Ah yes, 665.99999999856098676121, such evil! :P
Haha, but it's a well known fact that Satan rounds up.
The team that wrote the Windows 11 file explorer are clearly scared to death of any optimization whatsoever.
Truth! Many developers believe the coming up with abstractions is the pinnacle of sw engineering. The amount of time and money wasted on arguing about the right abstractions, following by massive refactoring needed when you discover that reality doesn't match your abstractions is immense. I have had engineers abstract all of the way down to Turning machines in an effort to find a model that would fit all of their use cases.
I have encountered the following interface in a real productive code base:
interface ILoadData<In, Out> {
So essentially this says that to implement this interface you must have a function that takes in something (you decide!) and outputs something (you decide!) and it's called Get. Totally useless.
interface ILoadData<In, Out> {
Out Get(In input);
So essentially this says that to implement this interface you must have a function that takes in something (you decide!) and outputs something (you decide!) and it's called Get. Totally useless.
I feel like we can trace further back from premature optimization and premature generalization to:
1. A fear that it won't be possible to alter the program later based on better use-case information. (Sometimes justified, sometimes not.)
2. Excitement, engagement, and the ego-trip of Making A Cool Solution.
Part of the reason I prefer languages with strong static checking is that it makes those kinds of (1) rewrites easier to do.
1. A fear that it won't be possible to alter the program later based on better use-case information. (Sometimes justified, sometimes not.)
2. Excitement, engagement, and the ego-trip of Making A Cool Solution.
Part of the reason I prefer languages with strong static checking is that it makes those kinds of (1) rewrites easier to do.
You're misunderstanding "thinking before you code" vs premature blahblah.
It shares the same mindset as the saying: Fail to prepare is prepare to fail !
The challenges and the evils is in the details of the process of optimization, not that premature optimization is evil.
It shares the same mindset as the saying: Fail to prepare is prepare to fail !
The challenges and the evils is in the details of the process of optimization, not that premature optimization is evil.
Donald Knuth: “premature optimization is the root of all evil”
The article suggests that premature specialization and premature generalization might, instead, be the root of all evil. Really!
Well-written.
The article suggests that premature specialization and premature generalization might, instead, be the root of all evil. Really!
Well-written.
I agree that premature generalisation is a problem, but to tell whether anything is premature, you must have all the information as the person who decided to generalise or optimise. Put simply, these are rules best applied to your own projects, not to others'.
Often I see people criticise some project for being overly generalised or "over-engineered" simply because they don't have as full a picture as the project's maintainers.
Often I see people criticise some project for being overly generalised or "over-engineered" simply because they don't have as full a picture as the project's maintainers.
Wonder what he'd think about my very lispy chess engine I'm writing in Python. A player like that might look like
opener(ender(mcts(ender(ply1(pieces+squares,temperature=25))
We have seen generalization too late -- rather, we have seen the need for it, but by definition, if it is left too late, it is impossible.
I personally have not seen premature optimization, but I may have led a charmed life.
I personally have not seen premature optimization, but I may have led a charmed life.
I've always considered premature optimization to be a different problem than premature abstraction.
Premature optimization is something I struggle with - "hey, if I do this a different way it'll be faster/smaller!" And then I end up spending too much time optimizing one data area of code when I should just do it the easy way and move on to the next problem. I can always come back and optimize later when everything is working and I can see where the problem areas are, but the temptation is often hard to fight.
Premature abstraction is harder because abstracting existing code can be problematic and there's more of a chance to introduce bugs. It takes intuition and planning to get it right the first time around, and by the time you realize you should have abstracted something it's often too late to fix without rewriting a large chunk of your code and your teammates' code.
Premature optimization is something I struggle with - "hey, if I do this a different way it'll be faster/smaller!" And then I end up spending too much time optimizing one data area of code when I should just do it the easy way and move on to the next problem. I can always come back and optimize later when everything is working and I can see where the problem areas are, but the temptation is often hard to fight.
Premature abstraction is harder because abstracting existing code can be problematic and there's more of a chance to introduce bugs. It takes intuition and planning to get it right the first time around, and by the time you realize you should have abstracted something it's often too late to fix without rewriting a large chunk of your code and your teammates' code.
Aren't all of these just variants of incorrect (often due to not being explicitly validated) assumptions?
The fact that everyone uses his premature optimization is the root of evil remark is hilarious to me. The level of optimization in Knuths writing makes it evident that upfront efficiency effort is worthwhile. Just don't overdo it.
But don't trust me on that quote. Just open book 3 (chapter 5 and 6, sorting and searching) and go read the tape drive simulations of merge sort yourself. You can't miss the printouts, they're huge.