I'm happy to make that bet. Just not for money. I don't gamble at all anywhere in my life.
But I'm happy to write something publicly like "Simianwords was right about this prediction and I was wrong". Also happy for you to suggest alternatives as well.
>you took a single report that agreed with your statistics
These are not my statistics. I'm not affiliated with Faros at all. I built an analysis on top of their reporting.
And, it's also not one report. DORA has tracked statistics with respect to throughput and quality as well. Those indicators are flat for throughput and negative for quality. The throughput flatness is also supported by the shovelware data.
>you suggest that net value is lost simply because there are more incidents. this is a big jump
I don't think it's a big jump at all. Incidents and bugs drive rework. Rework has to be subtracted from throughput. Product throughput is the only thing people pay for.
This type of analysis is done all the time in manufacturing and devops. Here's a link for you: https://reworkcost.com/benchmarks. I'm not bringing novel intellectual ideas to the table here.
Faros reports a 16% throughput improvement on PRs. They also report an 860% code churn increase. If you assign only 9% of that increase to wasteful rework, then the absolute throughput improvement disappears. This is a very simple, straightforward analysis of the operations data reported by Faros.
> - you say that historically different technological improvements may have had similar patterns but this specific one is different because AI is stochastic
I'm saying LLMs are unreliable. I think we agree on that front, you say:
>I agree AI is stochastic and I'll put it this way: it is a high variance bet but it pays off.
What I'm disputing is the "pays off" statement. That statement is amenable to validation with data. In my view, the data is saying it doesn't pay off. I think it says that very clearly. Across distinct lines of evidence.
>if you are so sure this won't lead to enterprise level productivity, how do you think this will show in macro trends? Surely you must believe that the valuations must drop wouldn't you? Can you come up with a concrete future scenario that would vindicate your opinion that AI doesn't make enterprises more productive?
I think LLMs can deliver value in the enterprise. I think the way to do that is to use them as quality checks and not as primary authors of intellectual work - like writing code.
Unfortunately, this use case would not support the expected 2-10x productivity increases that current valuations depend on. I do expect a major market correction in the near future. It would not surprise me if OpenAI or Anthropic are acquired. I think we're at risk of that happening within the next 1-7 months.
What would invalidate my beliefs?
1. Actual micro or macroeconomic data indicating economic productivity is increasing.
2. A Faros like observational study demonstrating sustained throughput improvement with significantly less rework and quality impacts.
I think I could be swayed against the market correction if the financials of OpenAI or Anthropic are strong. I'm anticipating they will be quite bad. I think Mythos was very expensive to train and I think the improvements in capability are sublinear. The inference costs are incredibly high.
I also have ideas about how Anthropic and OpenAI are trying to change their business models into enterprise transformation plays. Similar to Palantir. But this comment is already long.
>If they don't build it, someone else might do it.
No other players in the market other than US tech companies have the capital or the technology to train the models of the power of Fable. The way the Chinese model builders are building their models is by distilling from US models. So Anthropic, by building Mythos with all this bio data, has created the possibility that other actors can distill their models and do harm with them. (Not to say the Chinese are seeking to build weapons, but actors with their models might).
>Understanding what is going on with AI productivity is … frustrating to say the least.
Agreed. I think one of the hardest things about it is that productivity != value. You can push all the code you want, but if it's not driving revenue up or cost down, it doesn't matter economically.
Here is the best data I've been able to find. An observational study of 4000 teams over 2 years across many different organizations. Data gathered from their task management, version control, and CI/CD tooling. Critically - this is not survey data. It's much more direct measurement.
I'd love to understand why. This would be valuable feedback for me as I try to make my writing and exposition better. Also, if you have other data, that also would be valuable for me to know.
>if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts. Do you agree?
I think you misunderstand my beliefs. On net I think how we're using LLMs destroys value. That doesn't mean no one ever gets value from LLM use.
My particular point about trillion dollars is - the main place Anthropic, OpenAI, and - hilariously - SpaceX think they will drive value creation is in enterprise applications. In that domain I think the evidence is very convincingly negative. I'm certainly not the only person who thinks this. It's pretty well accepted in economics right now that there is no observed organizational level productivity improvement. Lines break down on whether it will show up eventually or whether we will wait forever.
My belief about LLM value is that it's most useful for individuals and small teams. Places where coordination and trust are easily established and feedback loops to value creation are tight. They are "short range" as it were.
Their value starts to erode as soon as a user becomes disconnected from the point of direct value creation. Which is pretty much everyone who works inside of a large organization. It becomes negative at pretty small scale, IMO. I do think there are patterns of use that could drive value at these scales. I talk about that in my post.
On Bioweapons in particular, I could see small teams of people working to build something very dangerous. Having spent my formative academic years in a biochemistry and microbiology lab though, I do think the danger is overstated. Papers are not know-how or equipment. There's a lot of tacit knowledge that can't get written down that is super hard to acquire.
But, I'd be happy for us to regulate AI for dangerous applications.
My question would be - why would Anthropic build something they so clearly think is dangerous? If they were really building something deserving of the valuation they have, why build applications like this?
To my eyes - it's super weird that a company would build something they think is dangerous and turn around and beg the governments of the world to stop them. That's really strange behavior from my perspective.
Maybe. There was a great comment in the thread on Fable 5 yesterday about benchmark comparisons between Fable and the latest opus models. here it is: https://news.ycombinator.com/item?id=48464600.
You could be right, but this is the most direct benchmark comparison I could find and it's not that strong.
>the "destroying value" conclusion flips sign on an assumed 15% baseline rework rate. The report's most direct metric is +16% merged PRs per dev.
I discuss this directly in my analysis. There's also an 860% code churn increase ratio. You only need 9% of that to be allocated to wasteful rework to drive throughput flat to the 15% rework baseline. Not to an assumed ideal state where there was no rework.
But even if it were not true, a 16% throughput improvement is pretty weak given the investment - especially given the direct evidence of quality degradation. IMO.
I appreciate you reading my stuff and taking the data seriously. Thank you.
I think this is pretty uncharitable, especially when I've provided you with a dataset you can evaluate yourself and an argument you can review for logical inconsistency.
I have worked quite hard to locate data that supports your thesis, I can't find it. I've at least gone to the effort of documenting that search. Before you throw around such strong convictions, I suggest you actually look for yourself.
Are we plotting against cost? How is the capability advancement vs dollars paid for development?
By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit.
If the value prop of LLMs doesn't prove out, that won't last. I'm of the opinion there is no data that shows actual economic value being delivered by models. The best data shows that LLM use might be destroying value [3].
>So from a holistic perspective, I think intentionally limiting your own AI usage is the best approach for maximum long-term productivity.
I think this is right. They are much better applied as editors than authors, IMO.
The key thing is stay in control of your output. i.e. understand it thoroguhly. I think you let the LLM make decisions you don't really understand, you're increasing the likelihood of introducing defects that are expensive to address.
By “break” I don’t mean “won’t function”. I mean “won’t deliver on their value proposition”. A functioning product is a necessary, but not a sufficient condition for technology to have utility. I would defend vigorously that the generative value in LLMs is derived from their unreliability. Which is what the argument ultimately rests on.
The report was not paywalled for me. It just required a work email. Which is totally fair from my perspective. Faros is providing a ton of value with the report. People do deserve to get paid - even if in collected emails.
You're right my analysis is at variance to what Faros.ai says. I think they interpret their data trying to rescue utility for the dominant patterns of LLM use.
But I think to anyone who is experienced with process improvement or queuing theory, their interpretation is clearly weak. Rework is a huge problem in queue systems, and they mostly just elide the throughput impact of an 860% increase in code churn coupled to a massive spike in bugs.
Obviously draw your own conclusions. But I don't think because I disagree with the interpretation of the people who originated the data makes me wrong.