I like sunsets, long walks on the beach, functional programming, type systems, interpreters and compilers, and many other things which don't pay the bills.
No, in my opinion as a parent this is lunacy. By the time my little kids "need" AI or the Internet in a meaningful way the AI labs will have either delivered the dream of "just talk to the computer" in which case there's nothing to learn to use, or they will be smoldering Pets.com-esque craters. Either way, for us right now it's books and a little bit of "old fashioned" movies and video games; no algorithmic feeds, all human-created content, and a focus on enjoying narratives and experiences together. We can look things up together on the Internet if we need to, and if that routes to an LLM Dad may groan a little but it's OK. Mostly we learn by trying things out, making guesses and talking about them, or looking in books.
I understand the calculus may change with middle school and up, but I still think that despite the "rat race" dynamic of grades and homework, kids who learn to think the pre-2023 way will come out ahead in the long run, even if it's only in life satisfaction.
I don't buy the superintelligence package, but I think uncritical LLM adoption poses plenty of threats to things I care about, in a mundane human-scale way.
Anyhow, I think you're (absolutely! ugh) right about the politics and I try to make the same point to people: whether you love or hate LLMs, accepting the "inevitabilism" framing is just ceding control of the Overton window. For better or worse, technology adoption can be and has been slowed by politics. We don't have nuclear plants everywhere. We don't have Project Orion starships colonizing Mars. We still have very strong social stigmas against genetic selection for human embryos, etc. This all can change in a heartbeat, and I'm not sure that policing the hardware rather than holding specific humans accountable for bad LLM outcomes is productive, but fundamentally: yes, we can stop it.
I think at this point the containment board is the entire Internet. I have no idea how, but we all need to "Atlas Shrugged" this shit and start over somewhere else with something else.
I think you may have missed a subtle point: there is an especial risk from automation which almost always works correctly. The aviation industry calls the phenomenon "automation fatigue". It's very difficult for humans to stay alert and monitor systems like these, and the use of the systems tends to lead to de-skilling over time in the very skills required to monitor them and fix the (rare but fatal - at least in aviation) error cases when they occur.
I have a question for all the "humans make those mistakes too" people in this thread, and elsewhere: have you ever read, or at least skimmed a summary of, "The Origin of Consciousness in the Breakdown of the Bicameral Mind"? Did you say "yeah, that sounds right"? Do you feel that your consciousness is primarily a linguistic phenomenon?
I am not trying to be snarky; I used to think that intelligence was intrinsically tied to or perhaps identical with language, and found deep and esoteric meaning in religious texts related to this (i.e. "in the beginning was the Word"; logos as soul as language-virus riding on meat substrate).
The last ~three years of LLM deployment have disabused me of this notion almost entirely, and I don't mean in a "God of the gaps" last-resort sort of way. I mean: I see the output of a purely-language-based "intelligence", and while I agree humans can make similar mistakes/confabulations, I overwhelmingly feel that there is no "there" there. Even the dumbest human has a continuity, a theory of the world, an "object permanence"... I'm struggling to find the right description, but I believe there is more than language manipulation to intelligence.
(I know this is tangential to the article, which is excellent as the author's usually are; I admire his restraint. However, I see exemplars of this take all over the thread so: why not here?)
It's getting bad here; I've seen at least three obviously AI-written "anti-AI" or "AI critical" pieces hit and remain on the front page in the last week. I can't help but think about Bill Hicks on marketing: "Everyone here who’s in marketing is now thinkin' the same thing: 'Oh, cool. Bill's going for that anti-marketing dollar. That's a huge market.'"
It’s ok, Harlan, you save up for that baby and don’t let the cult browbeat you over it.
(People are fuzzy and while game theory is fun and even sometimes useful, this kind of stuff skeeves me out and I think it lures “smart” people in and gets them to empower the SBFs of the world. My altruism is ineffective and I’m happy that way.)
The formal methods people may yet have the last laugh. I did not have Lean becoming a hyped programming language / proof assistant on my bingo card for 2025-26 and yet here we are, because these tools help us close the validation loop for LLM agents. That is not dead which can eternal lie...
But yes, I think the best rebuttal to Dijkstra-style griping is Perlis' "one can't proceed from the informal to the formal by formal means". That said I also believe kind of like Chesterton's quote about Christianity, they've also mostly not been tried and found wanting but rather found hard and left untried. By myself included, although I do enjoy a spot of the old dependent types (or at least their approximations). There's an economic argument lurking there about how robust most software really needs to be.
> A number of these phenomena have been bundled under the name "Software Engineering". As economics is known as "The Miserable Science", software engineering should be known as "The Doomed Discipline", doomed because it cannot even approach its goal since its goal is self-contradictory. Software engineering, of course, presents itself as another worthy cause, but that is eyewash: if you carefully read its literature and analyse what its devotees actually do, you will discover that software engineering has accepted as its charter "How to program if you cannot.".
- Edsger Dijkstra, 1988
I think, unfortunately, he may have had us all dead to rights on this one.
For me, “interestingly wrong” becomes just “wrong” without human thinking behind it. I wasn’t bowled over by the prose, I just thought it was an uncommon take and didn’t twig the signs it was Claude product.
As a couple sibling comments said, I took it for an insight into the way an optimistic writer might see AI software development becoming a new form of "end-user programming" or "citizen developer" tooling. I'm personally too deep in the weeds to ever see it becoming empowering in that way (if nothing else, this will be an incredibly centralizing technology and whoever wins the "arms race" [assuming we we're not in a bubble destined to pop soon] will absolutely have the possible Toms and Megans of such a future by the short hairs). But I love end-user programming, or whatever we're calling it now! (I was partial to "shadow IT" - made it sound really cool.) So I enjoyed the idea that somebody saw AI as a "bicycle for the mind" in that sense, even if I feared they'd end up disappointed.
But there was nobody there, and I'm only disappointed in myself for not noticing.
It's a major bummer. When I first read the story (a few days ago, maybe?) I thought it was an interesting metaphor that didn't quite line up with the observed details of software development with AI. I assumed the writer was a journalist or author with a non-technical background trying to explore a more "utopian" vision of where trends could go.
Without the inferred writer, it's much less interesting to me, except as a reminder that models change and I can't rely on the old tics to spot LLM prose consistently any more.
I'm sorry you've had that experience, and I agree there are a good share of "skeptics" who have latched on to anecdata or outdated experience or theorycrafting. I know it must feel like the goalposts are moving, too, when someone who was against AI on technical grounds last year has now discovered ethical qualms previously unevidenced. I spend a lot of time wondering if I've driven myself to my particular views exclusively out of motivated reasoning. (For what it's worth, I also think "motivated reasoning" is underrated - I am not obligated to kick my own ass out of obligation to "The Truth"!)
That said, I _did_ read your comments history (only because you asked!) and - well, I don't know, you seem very reasonable, but I notice you're upset with people talking about "hallucinations" in code generation from Opus 4.6. Now, I have actually spent some time trying to understand these models (as tool or threat) and that means using them in realistic circumstances. I don't like the "H word" very much, because I am an orthodox Dijkstraist and I hold that anthropomorphizing computers and algorithms is always a mistake. But I will say that like you, I have found that in appropriate context (types, tests) I don't get calls to non-existent functions, etc. However, I have seen: incorrect descriptions of numerical algorithms or their parameters, gaslighting and "failed fix loops" due to missing a "copy the compiled artifact to the testing directory" step, and other things which I consider at least "hallucination-adjacent". I am personally much more concerned about "hallucinations" and bad assumptions smuggled in the explanations provided, choice of algorithms and modeling strategies, etc. because I deal with some fairly subtle domain-specific calculations and (mathematical) models. The should-be domain experts a) aren't always and b) tend to be "enthusiasts" who will implicitly trust the talking genius computer.
For what it's worth, my personal concerns don't entirely overlap the questions I raised way above. I think there are a whole host of reasons people might be reluctant or skeptical, especially given the level of vitriol and FUD being thrown around and the fairly explicit push to automate jobs away. I have a lot of aesthetic objections to the entire LLM-generated corpus, but de gustibus...
Well, no, not with that attitude there won’t! I am not trying to insinuate that there is a conspiracy, or that posts like yours are part of it, but there has been a huge wave of posts and comments since February which narrow the Overton window to the distance between “it’s here and it’s great” and “I’m sad but it’s inevitable”.
Humanity has possessed nuclear weapons for 80 years and has used them exactly twice in anger, at the very beginning of that span. We can in fact just NOT do things! Not every world-beating technology takes off, for one reason or another. Supersonic airliners. Eugenics. Betamax.
The best time to air concerns was yesterday. The next best time is today. I think we technologists wildly overestimate public understand and underestimate public distrust of our work and of “AI” specifically. We’ve got CEOs stating that LLMs are a bigger deal than nuclear weapons or fire(!) and yet getting upset that the government wants control of their use. We’ve got giddy thinkpieces from people (real example from LinkedIn!) who believe we’ll hit 100% white collar unemployment in 5 years and wrap up by saying they’re “5% nervous and 95% excited”. If that’s what they really think, and how they really feel, it’s psychopathic! Those numbers get you a social scene that’ll make the French Revolution look like a tea party. (“And honestly? I’m here for it.”)
So no, while I _think_ you’re correct, I don’t accept the inevitability of it all. There are possibilities I don’t want to see closed off (maybe data finally really is the new oil, and that’s the basis for a planetary sovereign wealth fund. Maybe every man, woman, and child who ever wrote a book or a program or an internet comment deserves a royalty check in the mail each month!) just yet.
I am going to try to put this kindly: it is very glib, and people will find it offensive and obnoxious, to implicitly round off all resistance or skepticism to incuriosity. Perhaps to alienate AI critics even further is the goal, in which case - carry on.
But if you are genuinely confused by the attitudes of your peers, try asking not "what do I have that they lack" ("curiosity"?) but "what do they see that I don't" or "what do they care about that I don't"? Is it possible that they are not enthusiastic for the change in the nature of the work? Is it possible they are concerned about "automation complacency" setting in, precisely _because_ of the ratio of "hundreds of times" writing decent code to the one time writing "something stupid", and fear that every once in a while that "something stupid" will slip past them in a way that wipes the entire net gain of AI use? Is it possible that they _don't_ feel that the typical code is "better than most engineers can write"? Is it possible they feel that the "learning" is mostly ephemera - how much "prompt engineering" advice from a year ago still holds today?
You have a choice, and it's easy to label them (us?) as Luddites clinging to the old ways out of fear, stupidity, or "incuriosity". If you really want to understand, or even change some minds, though, please try to ask these people what they're really thinking, and listen.
I think this is coming, alongside professional licensure for "software engineers". Every public-facing project will need someone to put a literal stamp of approval on the code, and regardless whether Claude or Codex wrote the bulk of it, it'll be that person's head on a pike when something goes wrong.
This isn't what many of us probably would have wanted, but I think the public blowback when "AI-coded" systems start failing is going to drive us there. (Note to passing hype-men: I did not say they will fail at higher rates than human-coded systems! I happen to believe this, but it is not germane to the argument - only the public perception matters here.)
I like sunsets, long walks on the beach, functional programming, type systems, interpreters and compilers, and many other things which don't pay the bills.
blog: https://usethe.computer