The five orders of ignorance (2000)(cacm.acm.org)
cacm.acm.org
The five orders of ignorance (2000)
https://cacm.acm.org/opinion/the-five-orders-of-ignorance/
24 comments
"product is the knowledge in the code, not the code itself".. and other interesting observations. That might be relevant in current to-AI-or-not-AI questions
Published as book - The Laws of Software Process: A New Model for the Production and Management of Software , 2003, Phillip G. Armour
https://www.amazon.com/Laws-Software-Process-Production-Mana...
Published as book - The Laws of Software Process: A New Model for the Production and Management of Software , 2003, Phillip G. Armour
https://www.amazon.com/Laws-Software-Process-Production-Mana...
Essentially the idea of a context window in modern LLM models, there is implicit domain knowledge to every task in which no matter how capable the model may be, if not in the context, the software will not be functional.
I think this is the cause for the division in the perception of how useful AI is.
If you work in a field with mostly proprietary implementations of solutions, the top model aren't going to be all that helpful. The models won't have the domain knowledge, because open source code doesn't exist for most domains, because there's very real competitive advantage in keepings code/processes, that aren't trivially implemented, a secret!
I think proprietary data is the new moat, because that's where the vast majority of useful domain knowledge exists.
If you work in a field with mostly proprietary implementations of solutions, the top model aren't going to be all that helpful. The models won't have the domain knowledge, because open source code doesn't exist for most domains, because there's very real competitive advantage in keepings code/processes, that aren't trivially implemented, a secret!
I think proprietary data is the new moat, because that's where the vast majority of useful domain knowledge exists.
Incredibly prescient given what’s happening now 25 years later. This message resonates with me quite strongly! Thanks for sharing.
There's quite some more of the kind, and of about that time.. 80ies-90-ies.
Like the 3 levels of audience (known for milennia as shu-ha-ri):
https://wiki.c2.com/?ThreeLevelsOfAudience
and 4 levels of knowledge:
https://wiki.c2.com/?FourLevelsOfCompetence
or this from Dijkstra, 1989:
Surely, our computers would unscramble all the secret code of all our enemies and guide our missiles with unfailing precision right on their targets. Robots would take over the tedium of production, guaranteeing a positive balance of trade for all nations. Office automation would multiply the productivity of the white-collar worker, information systems would enable management to avoid waste and to make the right strategic decisions, and finally, the giant brains would not only relieve us from the tedium but also from the obligation to think about hard problems and from the painful responsibility to take difficult decisions. In short: computers were tolerated because they promised a well protected and prosperous paradise for the lazy, the incompetent, and the cowardly.
https://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD104...
Like the 3 levels of audience (known for milennia as shu-ha-ri):
https://wiki.c2.com/?ThreeLevelsOfAudience
and 4 levels of knowledge:
https://wiki.c2.com/?FourLevelsOfCompetence
or this from Dijkstra, 1989:
Surely, our computers would unscramble all the secret code of all our enemies and guide our missiles with unfailing precision right on their targets. Robots would take over the tedium of production, guaranteeing a positive balance of trade for all nations. Office automation would multiply the productivity of the white-collar worker, information systems would enable management to avoid waste and to make the right strategic decisions, and finally, the giant brains would not only relieve us from the tedium but also from the obligation to think about hard problems and from the painful responsibility to take difficult decisions. In short: computers were tolerated because they promised a well protected and prosperous paradise for the lazy, the incompetent, and the cowardly.
https://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD104...
The intro is really good and stands alone. I'd point any outsider to this as a decent description of hacking, programming, software engineering, prototyping in general.
Coding agents let me build and throw away prototypes extremely fast. A major value, for me, is that they help me understand early what users truly want and need — rather than relying on assumptions or lingering in abstraction. They help me discover and reduce my ignorance.
I've been building an EMR as a solo dev for a few years now while working at a healthcare agency. The main stakeholders are the CEO (super tech friendly, business type), the owner (nurse for longer than I've been alive) and the other clincians I work with.
What I learned very early on after having direct access to the users was how difficult it was to describe a future state of the application (or fish for pain points) without having something tangible to show/compare. A lot of them have a hard time thinking abstractly about software (and I don't blame them).
A few weeks back, I showed Bolt.new to the CEO and ever since then, our workflow has sped up tremendously. He has the technical know-how and desire to sketch out ideas he thinks will be useful (in lieu of me spending a week to build something up, getting it knocked down, and repeating over and over again). I told him to instruct it to use mock data and it's already using the exact same stack I use (React/Tailwind/React Aria). He knows enough about the process that it's not as simple as building it in Bolt, but also knows how valuable it's been to me.
I'm constitutionally incapable of building a decent UI. So bad that I can take a well designed system and completely screw it up. I just can't extrapolate on designs well (and even got into an argument during an interview with a designer because I mentioned that as one of my weaknesses). Having the ability to go back and forth with a "designer" and not get angry that I'm asking for EXACT examples has been insanely refreshing.
Our goal is to get enough of the app together (while also being mindful of stuff like accessibility) and then bring in the professionals at the end. We've burnt so much money bringing in designers too early, and now we can get to a baseline before asking for help.
I truly believe that we are witnessing another renaissance in software dev. Instead of development being relegated to the big dev companies and FAANG's, the economics of a small company bringing on a software developer are changing enough that it could turn the tide. Instead of one-size-fits-all behemoths, we can now tailor software to the client.
What I learned very early on after having direct access to the users was how difficult it was to describe a future state of the application (or fish for pain points) without having something tangible to show/compare. A lot of them have a hard time thinking abstractly about software (and I don't blame them).
A few weeks back, I showed Bolt.new to the CEO and ever since then, our workflow has sped up tremendously. He has the technical know-how and desire to sketch out ideas he thinks will be useful (in lieu of me spending a week to build something up, getting it knocked down, and repeating over and over again). I told him to instruct it to use mock data and it's already using the exact same stack I use (React/Tailwind/React Aria). He knows enough about the process that it's not as simple as building it in Bolt, but also knows how valuable it's been to me.
I'm constitutionally incapable of building a decent UI. So bad that I can take a well designed system and completely screw it up. I just can't extrapolate on designs well (and even got into an argument during an interview with a designer because I mentioned that as one of my weaknesses). Having the ability to go back and forth with a "designer" and not get angry that I'm asking for EXACT examples has been insanely refreshing.
Our goal is to get enough of the app together (while also being mindful of stuff like accessibility) and then bring in the professionals at the end. We've burnt so much money bringing in designers too early, and now we can get to a baseline before asking for help.
I truly believe that we are witnessing another renaissance in software dev. Instead of development being relegated to the big dev companies and FAANG's, the economics of a small company bringing on a software developer are changing enough that it could turn the tide. Instead of one-size-fits-all behemoths, we can now tailor software to the client.
> the real job is not writing the code, or even building the system — it is acquiring the necessary knowledge to build the system.
Not only very true, but the grammar will trigger those who insist on forcing the "that's written by AI" meme. I love it.
Not only very true, but the grammar will trigger those who insist on forcing the "that's written by AI" meme. I love it.
In this case, I think the grammar is just wrong: an em dash with spaces around it! Although I'm not so much of a stickler that I'd personally consider this to be a problem.
An em-dash (in this use, there are others where the normal style differs) set with regular spaces around it isn’t a grammar problem; it is a less common style preference (usually they are set closed—without spaces—or surrounded by thin spaces, or an en-dash surrounded by regular spaces is used instead.)
>As a development life-cycle model, prototyping acknowledges that our job is not to build a system, but to acquire knowledge.
So if there is any hope in making software development faster, we need to focus more on the specification part - to get it right faster.
So if there is any hope in making software development faster, we need to focus more on the specification part - to get it right faster.
Calling it specification makes it sound like someone knows what should be developed but didn’t put the time and effort to specify what they wanted.
In my experience, most people don’t know what should be developed. Even users don’t know if you ask them.
As the article outlines: you need to acquire that knowledge. And they’re many ways to do that. Talking to customers, testing with customers, having smart people who understand and care (!) what outcomes people want to achieve and so on.
In my experience, most people don’t know what should be developed. Even users don’t know if you ask them.
As the article outlines: you need to acquire that knowledge. And they’re many ways to do that. Talking to customers, testing with customers, having smart people who understand and care (!) what outcomes people want to achieve and so on.
> In my experience, most people don’t know what should be developed. Even users don’t know if you ask them.
That's also my experience. The most productive use of coding agents I've found so far is rooted in this.
I wanted to build a tool, for my own use, to let me specify a simple 2D geometry using a textual representation that looks like a simple version of the geometry itself. I'm the user and the implementer, so it should be easy right? I had some ideas for the basics of the textual representation, but the details were far from obvious.
I spent ~10 hours having AI give me suggestions and refinements for the textual representation along with evolving code to parse the representation. It turned out to be a highly productive way to explore the space of trade-offs. This representation looks really good, but it's ambiguous in some cases. That one is not ambiguous, but parsing is tricky. That one is too annoying for the user. It was all about quickly gathering the knowledge and therefore understanding what to build.
When I finally settled on a representation that had a good set of trade-offs, the code was pretty ugly. It worked, but it was about 1,200 lines of code, with weak tests. I tried to have the AI refactor it, and even restart from the final representation choices. Its best version was 1,000 lines of code that were difficult to understand.
I was getting on a plane with no internet the next day. So, armed with all the gathered knowledge, I decided to see what I could do with the code on the plane. It was too complicated to refactor as I went. So I rewrote it from scratch, employing the knowledge I'd built up. In 2-3 hours, I had a complete implementation that was simple to understand and less than 300 lines of code. ~40% of those lines of code were tests that were quite robust.
That amount of iteration in the knowledge gathering step would have taken me closer to a couple of weeks without the AI. And, by employing Fred Brooks' "build one to throw away" (a concept that I think is largely about the same thing as the article), I had a solid implementation without much more work.
I'm sure this workflow is not for everyone. But it (accidentally) leaned into the topic of the article, and I think that's exactly why it worked so well for me.
That's also my experience. The most productive use of coding agents I've found so far is rooted in this.
I wanted to build a tool, for my own use, to let me specify a simple 2D geometry using a textual representation that looks like a simple version of the geometry itself. I'm the user and the implementer, so it should be easy right? I had some ideas for the basics of the textual representation, but the details were far from obvious.
I spent ~10 hours having AI give me suggestions and refinements for the textual representation along with evolving code to parse the representation. It turned out to be a highly productive way to explore the space of trade-offs. This representation looks really good, but it's ambiguous in some cases. That one is not ambiguous, but parsing is tricky. That one is too annoying for the user. It was all about quickly gathering the knowledge and therefore understanding what to build.
When I finally settled on a representation that had a good set of trade-offs, the code was pretty ugly. It worked, but it was about 1,200 lines of code, with weak tests. I tried to have the AI refactor it, and even restart from the final representation choices. Its best version was 1,000 lines of code that were difficult to understand.
I was getting on a plane with no internet the next day. So, armed with all the gathered knowledge, I decided to see what I could do with the code on the plane. It was too complicated to refactor as I went. So I rewrote it from scratch, employing the knowledge I'd built up. In 2-3 hours, I had a complete implementation that was simple to understand and less than 300 lines of code. ~40% of those lines of code were tests that were quite robust.
That amount of iteration in the knowledge gathering step would have taken me closer to a couple of weeks without the AI. And, by employing Fred Brooks' "build one to throw away" (a concept that I think is largely about the same thing as the article), I had a solid implementation without much more work.
I'm sure this workflow is not for everyone. But it (accidentally) leaned into the topic of the article, and I think that's exactly why it worked so well for me.
No that workflow isn’t for everyone and everything.
But something like that is exactly what you need to do.
Somehow experience / test reality and with that feedback go back and build.
But something like that is exactly what you need to do.
Somehow experience / test reality and with that feedback go back and build.
> As the article outlines: you need to acquire that knowledge. And they’re many ways to do that.
Domain-driven design is all about this.
Domain-driven design is all about this.
This would be so useful of a model, in personal development, life and more! Incredible take on this.
Not so much, actually. The better-than-default "process" for their 3rd level is to interview the customers, users, or domain experts, which is something you should do already in a sane software development process. Transposed and generalized to everyday life, this just means talk to people, ask questions and listen. This is generally called being "open-minded".
Ok so he’s got KK, KDK, and DKDK, but he’s missing DKK.
Funny how something written in 2000 can sound so modern.
... And here's the first three orders mentioned in a famously quoted press conference from 2002:
https://www.youtube.com/watch?v=REWeBzGuzCc
https://www.youtube.com/watch?v=REWeBzGuzCc
This is one reason why artificial general intelligence is impossible. It is because most of the knowledge needed would require knowledge that does not already exist in text form.
https://pages.cs.wisc.edu/~remzi/Naur.pdf
Not to say the article doesn't have value, as great foundational ideas are always worth repeating and revisiting.