You’re right that automation has historically expanded what we can build. But the real work of software engineering isn’t coding. It’s architecting solutions: defining the problem, making trade-offs, and designing for maintainability and change. That’s significantly harder to automate, and I explained why in another reply on this thread.
Even if AI becomes competent at engineering (my two-part series covers what “System Definition” would be needed to get there), it still has a deeper limitation: AI is a consensus machine. It excels at recombining what’s already known and rewarded. Paradigm shifts and black swans almost always come from people willing to challenge the current consensus, which is exactly what AI consensus tends to suppress.
That’s the real risk to technological progress: how do we move in new directions when AI defaults to following the majority view from past experience?
https://hackernoon.com/who-dares-to-be-the-first-cassandras-...
This matches the exact point being discussed. I actually just published a two-part article on HackerNoon digging into this exact issue—and the consequences of ignoring it.
Hopefully, this adds some useful context to the discussion: