Nice summary. Reading this reminds me about the strong encryption discussion.
> We optimize what we can measure, not what we actually want to achieve. We hope and pray that these are the same thing, but they often aren’t.
He points out the core problem with LLMs. I believe it is impossible (or extremely expensive) to ensure that the models are aligned safely for everyone and any intention. And 'safe' can mean different things for a different audience.
I strongly doubt it will even provide us with a roof over our heads. In an unconstrained market, the pressure to extract as much as possible from the UBI will be enormous. The amount of UBI will probably always lag years behind the actual amount required to create a liveable situation, and increasing its amount will be a constant political struggle.
UBI in an unconstrained market is nothing else than enslavement.
Fair and progressive taxation and proper social systems are far more efficient. UBI is just an excuse to get rid of social systems and leave everyone individually stranded with problems no one can solve alone.
Those price increases will increase the pressure to use cheaper / free models (commoditization), thus cutting into the revenue projections of the frontier model vendors. Its going to be exciting to see what happens to these huge investments and valuations.
Because what is aligned, how and for whom? And who decides how that alignment should look like? There are probably many domains in which required alignment is in conflict with each other (e.g. using LLMs for warfare vs. ethically based domains). I can't imagine how this can be viable on the required scale (like one model per domain) for the already huge investments.
Compute in science was already subsidized by public funding or by donations. Most supercomputers are financed this way. And that's a good thing. If you have a good science problem that can be computed, apply for compute time. There is nothing wrong to apply that to LLMs as well, like I wrote in my initial post. The human is still required to identity problems that are worth to be computed, to create prompts that the LLM can act on, and to verify results. But, OpenAI providing compute for basically free is still tied to a different incentive: to fuel the hype and to capture the market, while distorting/obfuscating the real costs. That's also the reason for why we cannot claim that 'economics on LLMs is just unbeatable'. It depends on the problem, the reason for a prompt.
"After 16 minutes and 41 seconds, it came back" ... "further 47 minutes and 39 seconds" ... "After 13 minutes and 33 seconds" ... "After 9 minutes and 12 seconds" ... "After 31 minutes and 40 seconds" ... plus other computations
Anyone spotting the issue here? What did that really cost?
I am not against compute being used for scientific or other important problems. We did that before LLMs. However, the major LLM gatekeepers want to make all industries and companies dependent on their models. And, at some point, they need to charge them the actual, unsubsidized costs for the compute. In the meantime, companies restructure in the hopes that the compute costs remain cheap.
I wonder how this figure was settled. Is it based on consumer pricing? Can't Microsoft and OpenAI just make a number up, aside from a minimum to cover operating costs? When is the number just a marketing ploy to make it seem huge, important and inevitable (and too big to fail)?
True. I didn't expect it to provide novel designs. Maybe Anthropic should find a better replacement for 'Design'.
In my example, I expected it to create UI elements for a business application / expert system. And it did fine. In fact, I believe its perfect for creating average and functional designs. Its a better way to test variations of UIs for expert systems. But I want to know what the actual costs are.
Good for crunching out some prototypes, ideas and getting inspirations I guess. Two prompts - the initial one and one refinement - took about ten minutes and used up 90% of the token budget. I wonder what the real costs are. After the IPO, they will no longer be able to subsidize token costs. The question will then be whether it's still cheap enough just for prototypes, ideas and inspiration.
They are trying to optimize the circus trick that 'reasoning' is. The economics still do not favor a viable business at these valuations or levels of cost subsidization. The amount of compute required to make 'reasoning' work or to have these incremental improvements is increasingly obfuscated in light of the IPO.
I can slow down the compute by a factor of a thousand. It would not change the result. But it changes the economics. We only call it intelligent, because we can do the backpropagation, the inference (and training) fast enough and with enough memory for it to appear this way.
I don't trust anyone who claims that LLMs today are superhumanly intelligent. All they do is perform compute-intensive brute-force attacks on the problem/solution space and call it 'reasoning', all while subsidising the real costs to capture the market. So much SciFi BS and extrapolation about a technology that is useful if adopted with care.
This technology needs to become a commodity to destroy this aggregation of power between a few organizations with untrustworthy incentives and leadership.