In what sense is this true? We understand the theory of what is happening and we can painstakingly walk through the token generation process and understand it. So in what sense do we not understand LLMs?
Expanding on this regarding the difference between abstraction vs encapsulation: abstraction is about the distillation of useful concepts while encapsulation is a specific tactic used to accomplish a behavior.
To continue with the idea of numbers, let’s say you asked someone to add 3 and 5. Is that encapsulation? What information are you hiding? You are not asking them to add coins or meters or reindeer. 3 and 5 are values independent of any underlying information. The numbers aren’t encapsulating anything.
Encapsulation is different. When you operate a motor vehicle, you concern yourself with the controls presented. This allows you, as the operator, to only need a tiny amount of knowledge to interact with an incredibly complex machine. This details have been encapsulated. There may be particular abstraction present, such as the notion of steering, acceleration, and breaking, but the way you interact with these will differ from vehicle to vehicle. Additionally, encapsulation is not concerned with the idea of steering, it is concerned with how to present steering in this specific case.
The two ideas are connected because using an abstraction in software often involves encapsulation. But they should not be conflated, out the likely result is bad abstractions and unwieldy encapsulation.
Computer Science stole the term abstraction from the field of Mathematics. I think mathematics can be really helpful in clearing things up here.
A really simple abstraction in mathematics is that of numeric basis (e.g. base 10) for representing numbers. Being able to use the symbol 3 is much more useful than needing to write III. Of course, numbers themselves are an abstraction- perhaps you and I can reason about 3 and 7 and 10,000 in a vacuum, but young children or people who have never been exposed to numbers without units struggle to understand. Seven… what? Dogs? Bottles? Days? Numbers are an abstraction, and Arabic digits are a particular abstraction on top of that.
Without that abstraction, we would have insufficient tools to do more complex things such as, say, subtract 1 from 1,000,000,000. This is a problem that most 12 year olds can solve, but the greatest mathematicians of the Roman empire could not, because they did not have the right abstractions.
So if there are abstractions that enable us to solve problems that were formerly impossible, this means there is something more going on than “hiding information”. In fact, this is what Dijkstra (a mathematician by training) meant when he said:
The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise
When I use open(2), it’s because I’m operating at the semantic level of files. It’s not sensible to think of a “file” at a lower level: would it be on disk? In memory? What about socket files? But a “file” isn’t a real thing, it’s an abstraction created by the OS. We can operate on files, these made up things, and we can compose operations together in complex, useful ways. The idea of a file opens new possibilities for things we can do with computers.
A peak of 4k/s does not mean they get that for the entire hour. The point I was trying to make is that simply computing the mean over a 24hr period will almost certainly ensure you size things incorrectly.
If a stretch of road was used by an average of 10 cars per minute over a 24 hour period, is it congested?
In both cases, you need more specific traffic data to size things properly.
Broad strokes: absolutely. The practical reality gets tricky, though. All programming abstractions are imperfect in some regard, so the question becomes what level of imperfection can you tolerate, and is the benefit worth the cost?
I think a lot of becoming a good programmer is about developing the instincts around when it’s worth it and in what direction. To add to the complexity, there is a meta dimension of how much time you should spend trying to figure it out vs just implement something and correct it later.
As an aside, I’m really curious to see how much coding agents shift this balance.
> 4. Clarity is seniority. Cleverness is overhead.
Clarity is likely the most important aspect of making maintainable, extendable code. Of course, it’s easy to say that, it’s harder to explain what it looks like in practice.
> 11. Abstractions don’t remove complexity. They move it to the day you’re on call.
This is true for bad abstractions.
> The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise. (Dijkstra)
If you think about abstraction in those terms, the utility becomes apparent. We abstract CPU instructions into programming languages so we can think about our problems in more precise terms, such as data structures and functions.
It is obviously useful to build abstractions to create even higher levels of precision on top of the language itself.
The problem isn’t abstraction, it is clarity of purpose. Too often we create complex behavioral models before actually understanding the behavior we are trying to model. It’s like a civil engineer trying to build a bridge in a warehouse without examining the terrain where it must be placed. When it doesn’t fit correctly, we don’t blame the concept of bridges.
I think clever is being used in two different ways, in that case.
In the original quote, “clever” refers to the syntax, where they way the code was constructed makes it difficult to decipher.
I believe your interpretation (and perhaps the post’s, as well) is about the design. Often to make a very simple, elegant design (what pieces exist and how they interact) you need to think really hard and creatively, aka be clever.
Programming as a discipline has a problem with using vague terms. “Clean” code, “clever” code, “complex” code; what are we trying to convey when we talk about these things?
I came up with a term I like: Mean Time to Comprehension, or MTC. MTC is the average amount of time it takes for a programmer familiar with the given language, syntax, libraries, tooling, and structure to understand a particular block of code. I find that thinking about code in those terms is much more useful than thinking about it in terms of something like “clever”.
(For anyone interested, I wrote a book that explores the rules for writing code that is meant to reduce MTC: The Elements of Code https://elementsofcode.io)
I suppose it has moved from “what are you working” to “what have you worked on” territory, but since I wrapped up the website just about a week ago it still feels quite fresh.
Always interested in feedback and what folks find useful! It’s focused on the mechanics of writing understandable software, which I think is especially important in the age of AI slop.