I think the article has a point. There seem to be two reactions among senior engineers atound me these days.
On one side, there are people who have become a bit more productive. They are certainly not "10x," but they definitely deliver more code. However, I do not observe a substantial difference in the end-to-end delivery of production-ready software. This might be on me and my lack of capacity to exploit the tools to their full extent. But, iterating over customer requirements, CI/CD, peer reviews, and business validation takes time (and time from the most experienced people, not from the AI).
On the other hand, soemtimes I observe a genuine degradation of thinking among some senior engineers (there aren’t many juniors around, by the way). Meetings, requirements, documents, or technology choices seem to be directly copy/pasted from an LLM, without a grain of original thinking, many times without insight.
The AI tools are great though. They give you an answer to the question. But, many times making the correct question, and knowing when the answer is not correct is the main issue.
I wonder if the productivity boost that senior engineers actually need is to profit from the accumulated knowledge found in books. I know it is an old technology and it is not fashionable, but I believe it is mostly unexploited if you consider the whole population of engineers :D
So far, the best reference for software engineering research appears to be R. Glass et al.'s 2002 work, Facts and Fallacies of Software Engineering. I haven't found a better or more comprehensive reference.
What do a crowded airport in Beijing, parenting, the risk-return frontier, and micromanaging bosses have in common? Everything—if you look at the world through the right lens.
In Through the Geek’s Lens, you’ll embark on a personal, humorous, and yet thought-provoking journey through some fundamental trade-offs and models from mathematics, psychology, economics, and engineering. Through the eyes of a textbook lover, Through the Geek’s Lens opens unexplored mental horizons and connects the dots between accumulated scientific knowledge and our private and professional lives.
The book introduces and explains complex subjects, their factual basis, and their limitations by contextualizing them within a specific time and place—a moment in the author's personal life. Each section offers a small dose of actionable insight. Among the many topics explored are the trade-off between freedom and security, the diminishing returns faced by particle accelerators and large machine-learning models, and the No Free Lunch Theorem, which explains why a self-confident explorer like the author can get lost in an unknown forest. It applies paraconsistent logic to interpret Eastern philosophies and uses unbiased statistical sampling to show how to see your loved ones. It discusses the Law of Small Numbers and why it’s just as important as the Law of Large Numbers, the determination of Service Level Agreements using basic probability inequalities, and the Shannon-Hartley Theorem to decide when to shout (or not) at your kids. And that’s just the beginning—there’s much more to discover when you look your day to day through a geek’s lens.
I think the problem is not just software estimation, but project cost and time estimation is difficult across many domains (construction, transportation, IT, defense, and even the organization of events).
The word "mess" seems to indicate that the uncertainty is easily fixed, but if it has happened across domains for several millennia, it could indicate that there is something fundamentally challenging.
The situation worsens with factors such as project size, requirement changes, methodology (too much or too little of it), technology (specially new technologies), the nature of the delivering institution (public institutions performing worst than private ones), and organizational culture (e.g. waterfall being in many cases detrimental by making adaptations more difficult).
There are certainly bad ideas in the field of estimation, like assuming a Gaussian distribution, but the problem is far from trivial.
See for example:
* Defense: Bolten, Joseph G., et al. Sources of weapon system cost growth: Analysis of 35 major defense acquisition programs. Rand Corporation, 2008.
* Public works: Flyvbjerg, Bent, Mette Skamris Holm, and Soren Buhl. "Underestimating costs in public works projects: Error or lie?." Journal of the American planning association 68.3 (2002): 279-295.
* Transportation: Cantarelli, Chantal C., et al. "Cost overruns in large-scale transportation infrastructure projects: Explanations and their theoretical embeddedness." arXiv preprint arXiv:1307.2176 (2013).
* Olympic Games: Flyvbjerg, Bent, Alexander Budzier, and Daniel Lunn. "Regression to the tail: Why the Olympics blow up." Environment and Planning A: Economy and Space 53.2 (2021): 233-260.
As a baseline on what is know and not know about managing software projects, I always recommend Robert L. Glass Facts and Fallacies of Software Engineering.
More speculatively, I think that the following biology readings are inspiring:
- Gerald Jay Sussman, Building Robust Systems an essay. In general, biology should be a source of inspiration for engineers.
On one side, there are people who have become a bit more productive. They are certainly not "10x," but they definitely deliver more code. However, I do not observe a substantial difference in the end-to-end delivery of production-ready software. This might be on me and my lack of capacity to exploit the tools to their full extent. But, iterating over customer requirements, CI/CD, peer reviews, and business validation takes time (and time from the most experienced people, not from the AI).
On the other hand, soemtimes I observe a genuine degradation of thinking among some senior engineers (there aren’t many juniors around, by the way). Meetings, requirements, documents, or technology choices seem to be directly copy/pasted from an LLM, without a grain of original thinking, many times without insight.
The AI tools are great though. They give you an answer to the question. But, many times making the correct question, and knowing when the answer is not correct is the main issue.
I wonder if the productivity boost that senior engineers actually need is to profit from the accumulated knowledge found in books. I know it is an old technology and it is not fashionable, but I believe it is mostly unexploited if you consider the whole population of engineers :D