monodeldiablo

letzten Monatdiscuss

You annihilated your own argument with the inclusion of radiology. The only successfully deployed "AI" in use by radiologists (that I'm aware of) are bespoke image analysis models, not LLMs. And that space is rapidly fragmenting as there's a frustrating and seemingly irresolvable tension between sensitivity, generalizability, and accuracy.

monodeldiablo

letzten Monatdiscuss

A few hundred billion in salary and benefits reductions equates to millions of layoffs. At minimum, we'd be looking at something about the same magnitude as the 2008 financial crisis. That scale of workforce reduction would have profound implications for the broader economy.

In a consumption-driven economy, businesses need consumers. Any gains from these layoffs would be short term at best.

monodeldiablo

vor 3 Monatendiscuss

Leaks from within OpenAI have made it pretty clear that they've been struggling to achieve significant improvements lately by simply scaling up parameter size. Experts like LeCunn have also been vocal that blindly scaling up is a dead end.

(Incidentally, the line of skill improvement isn't "exponential". It's been incremental in improvements per generation, but generations have been coming thick and fast of late, and have grown in parameter count exponentially since 2017.)

Speaking more broadly, LLMs don't have to "hit a wall" in scaling to become uneconomical. If incremental improvement continues to come at exponential cost, however, then the fundamental value argument falls apart.

Setting all that aside, even presuming that model performance scales linearly with dimensionality, there are just fundamental limits to the size of the training corpuses. Quality training data is not unbounded and infinite. Given the same size corpus of training data, there's a hard theoretical limit to how much meaning and inference a model can wring out of it.

And then there are other issues with the whole business model. For one thing, it's predicated on continual full scale retraining to achieve even modest gains in skill and relevancy. Topical and targeted learning requires a full retraining. Etc cetera.

I think that the next generation of AI will lean more heavily on RL to be useful beyond a few months. I also think that the energy requirements of a particular technology are a good proxy to whether it's got a realistic future.

monodeldiablo

vor 3 Monatendiscuss

I played a role in China's shift to renewables. It's been decades in the making.

monodeldiablo

vor 3 Monatendiscuss

I do run open models locally, but let's not fool ourselves into thinking that they're functionally competitive. I'm extremely skeptical of anybody claiming they've obviated a $22/hr job with an open model. Qwen is a big step down in capability. I can play with something like k2.5 for awhile, but if I want real work done I'm going back to a frontier model, which has significant runtime requirements for inference.

You're also ignoring the cost of purchasing and amortizing dedicated hardware in your local model example.

It's not an apples-to-apples comparison.

monodeldiablo

vor 3 Monatendiscuss

It's not really even a question. It's an obvious boondoggle. The forecasted net new energy requirements for the AI buildout over the next couple of years are roughly equivalent to all of Western Europe's power demand today.

That's absurd. It's a physical impossibility to bring that much power online that quickly. And the cost to get even close would make AI more expensive than just hiring knowledge workers to do the same tasks.

And it's all predicated on a tower of wobbly or broken assumptions -- chief among them that increasing the size of these models yields better performance.

We're going to look back on this era and wonder why anybody took any of the outrageous claims of tech CEOs seriously.

monodeldiablo

vor 3 Monatendiscuss

$0.18/hr is the (massively) subsidized price of AI services. Once these companies are required to turn a profit for their investors, they'll raise the price. Then the math doesn't look so lopsided. We're already seeing this process unfold with token windows and ad rollout.

monodeldiablo

vor 4 Monatendiscuss

His word choice about semicolons is problematic for other reasons, so I won't quote it here, but Vonnegut made his views on punctuation and story structure very known. An internet search will provide it to you readily. And anybody who's read his works is familiar with his love of the em dash.

But more to his -- and my -- point: He also regularly encouraged people to flout rules and standards. His famous quote about semicolons, when read in its original context, is followed by a sentence with a semicolon!

He was a subversive author who abhorred mindless compliance and begged us to remain inquisitive. Subversion of accepted standards lies at the heart of all creativity. And as creative works enter the broader discourse, they themselves shape new standards. It's why our languages are always changing.

Your point about Citizen Kane and Independence Day 2 is nonsensical, and presumes we all have the same goals when consuming entertainment. I'm not going to engage in that argument.

monodeldiablo

vor 4 Monatendiscuss

You and Kurt Vonnegut seem to disagree here. He made liberal use of em dashes and hated semicolons.

And he was a giant of literature.

The problem with your definition of "good writing" is that it's entirely subjective. Just like Vonnegut's.

comments