- The computer is plugged to an infinite source of unlimited power
This was big before the mobile era and is true to this day to an extent. Many mainstream languages created in the 1990s (I call them "the children of the 1990s") were designed with this fallacy plus the ones you listed as a basis: JavaScript, Python, Ruby, Java, etc.
I think it depends on both the complexity and the quality bars set by the engineer.
From my observations, generally AI-generated code is average quality.
Even with average quality it can save you a lot of time on some narrowly specialized tasks that would otherwise take you a lot of research and understanding. For example, you can code some deep DSP thingie (say audio) without understanding much what it does and how.
For simpler things like backend or frontend code that doesn't require any special knowledge other than basic backend or frontend - this is where the bars of quality come into play. Some people will be more than happy with AI generated code, others won't be, depending on their experience, also requirements (speed of shipping vs. quality, which almost always resolves to speed) etc.
Ask your Chinese room what left and right means. Also: near and far, heavy and light, hot and cold. When it comes up with definitions through other concepts, ask what those things mean, etc.
However, a correct, pretty and fast solution may exist that neither of you have found yet.
But yes, the scope and breadth of their knowledge goes far beyond what a human brain can handle. How many relevant facts can you hold in your mind when solving a problem? 5? 12? An LLM can take thousands of relevant facts into account at the same time, and that's their superhuman ability.
> But what makes a human mind more "understanding"?
If you view understanding as knowledge + the ability to apply it, everything falls into its place. The Chinese room can't apply the knowledge that it has, even theoretically.
Excellent post. I share the author's sentiment which is essentially "to hell with Figma, at least fix Sketch". Been feeling very lonely in may hatred towards Figma, which is for a whole bunch of reasons (among others, it's an incredibly shitty, memory and CPU hungry Electron app that looks and feels worse than any more or less well designed web site), but now after reading this I realize the number of reasons has doubled.
> Which means we need people like Alice! We have to make space for people like Alice, and find a way to promote her over Bob
The solution is relatively simple though - not sure the article suggests this as I only skimmed through:
Being good in your field doesn't only mean pushing articles but also being able to talk about them. I think academia should drift away from written form toward more spoken form, i.e. conferences.
What if, say, you can only publish something after presenting your work in person, answer questions, etc? The audience can be big or small, doesn't matter.
It would make publishing anything at all more expensive but maybe that's exactly what academia needs even irrespective of this AI craze?
15 years ago I was thinking about switching my career to a different industry altogether, just didn't know what it would be. One thing I knew was that I was so tired of building web sites and backends. Boring, repetitive, uninspiring.
Then a friend asked me to write a simple iPhone app. I had no idea what development for Apple platforms would be like...
Fast forward to 2026, I'm 57 now, still in tech, building apps for Apple platforms, still enjoying it very much.
> AI should learn to say two things: ‘I don’t know’ and ‘you’re wrong.’
My guess is, the next evolutionary step of LLM's should be yet another layer on top of reasoning, which should be some form of self-awareness and theory of mind. The reasoning layer already has some glimpses of these things ("The user wants ...") but apparently not enough to suppress generation and say "I don't know".
You don't even need to reinvent a walkable city, just look at any medieval historical town that is say ~500 years old, almost untouched, and has restricted traffic today (possibly with no public transport whatsoever). These towns are a pure joy to live in, they are walkable with no other options, quiet, pleasant and overall healthy to live in in all respects.
I'm a big fan of Swift (and SwiftUI), such a concise and elegant language. Beauty.
Also I appreciate how you made all backend calls just static functions which they always should be. People tend to overcomplicate these things and add a lot of boiler plate and unnecessary bureaucracy.
Absolutely. Even worse, when you ask AI to solve a problem it almost always adds code even if a better solution exists that removes code. If AI's new solution fails, you ask it to fix, it throws even more code, creates more mess, introduces new unnecessary states. Rinse, repeat ad infinitum.
I did this a few times as an experiment while knowing how a problem could be solved. In difficult situations Cursor always invariably adds code and creates even more mess.
I wonder if this can be mitigated somehow at the inference level because prompts don't seem to be helping with this problem.
Being an AI skeptic more than not, I don't think the article's conclusion is true.
What LLM's can potentially do for us is exactly the opposite: because they are trained on pretty much everything there is, if you ask the AI how the telephone works, or what happens when you enter a URL in the browser, they can actually answer and break it down for you nicely (and that would be a dissertation-sized text). Accuracy and hallucinations aside, it's already better than a human who has no clue about how the telephone works or where to even begin if the said human wanted to understand it.
Human brains have a pretty serious gap in the "I don't know what I don't know" area, whereas language models have such a vast scope of knowledge that makes them somewhat superior, albeit at a price of, well, being literally quite expensive and power hungry. But that's technical details.
LLMs are knowledge machines that are good at precisely that: knowing everything about everything on all levels as long as it is described in human language somewhere on the Internet.
LLMs consolidate our knowledge in ways that were impossible before. They are pretty bad at reasoning or e.g. generating code, but where they excel so far is answering arbitrary questions about pretty much anything.
So the market is going to be flooded with this type of soulless books that have no distinct character or style, just pure dry facts?
In a sense, "I wrote a book about it" is disingenuous and I agree the author's bullet list would probably be more interesting and would save us a lot of time.
This was big before the mobile era and is true to this day to an extent. Many mainstream languages created in the 1990s (I call them "the children of the 1990s") were designed with this fallacy plus the ones you listed as a basis: JavaScript, Python, Ruby, Java, etc.