That looks interesting. I can't comment on the validity of the methodology of testing coding skill used here. But in general, I'd say we desperately need some way to track changes in our own skills caused by AI usage.
However, this would of course be a financial threat to OpenAI & anthropic in case serious, long-term atrophy of skills can be proven consistently. So I'd expect this to be buried quickly, unless supported by an academically significant institution.
They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense.
Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.
Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.
Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.
Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.
You really don't seem to understand what a model is. A model is essentially an approximation of reality capable of making predictions. If those predictions get confirmed, the model will be accepted.
Barely anybody with reasonable education in physics would confuse a model built for humans to understand reality with actual reality. Models are not reality, they are a tool used to describe it, nothing more and nothing less.
Let's take Newton's laws as an example. Obviously, there is not a literal force vector manifesting and pushing objects. But it's a good predictor of behavior and allows building of understanding.
The Higgs Particle did exactly what it was expected to do and works just like Higgs predicted. If you look at click-focused press and think their bad explanations properly describe the state of physics, you're simply mistaken.
However, to understand that, you have to actually learn. And, if you intend to revolutionize physics, you also need to be able to make better predictions. Which is ... unlikely, to say the least.
Again: Models in physics approximate reality. They do not actually exactly describe fundamental reality, only how it works within the limits of the model.
This really reads like some american lawyer used an llm and never questioned whether legal precedent is even a thing in germany aside from the highest courts.
Have seen several like this in the last months, though in much more niche areas and with barely any publicity.
He was right about the cost changes, which he predicted quite some time ago. People shouted at him that he was making it all up - yet it was correct.
He was also right about AI-video and sora in particular being a fundamentally flawed idea.
He was also right about the dangers and problems with the general inaccuracy of LLMs and people relying on it.
Also about the expected triggering of ROI-checking in companies, such as Uber is doing now. His prediction is, ROI is negative. And I'm awaiting the society's consensus on that.
The general direction seems correct to me.
He's not a technical guy and does not have the knowledge to critique models on a factual basis. I do wish he'd just focus on the stuff he _does_ know about, which is the financial side of things.
He is a much needed counterweight to the unhealthy hype going around, imho.
The PR-thing was always openly communicated by him and is not some secret or gotcha. It's essentially "fleecing the boosters", which I fully approve of and do similarly myself.
I'll gladly tell my customers all the most glorious stuff about AI and big tech while spending a significant chunk of the money they pay me on supporting AI-/tech-counterculture, such as doctorow, zitron and quite a few other writers, journalists and activists.
It's the old "you live in a society" counter-point against anti-capitalist activism. Needing to make ends meet does not imply that your points or principles are meaningless, it just implies that you have no interest in being homeless and that way losing your chance to actually change things.
So that's fine to me. But: I stated it for a reason, because I know others don't agree. I, personally, consider him trustworthy. You do not, and that's fine. I suspect we both await anthropic's Z.1, which will be able to settle a big chunk of the debate.
However, it needs to be said that he received those numbers. I personally have quite a few issues with him, but there's no reason to doubt his journalistic integrity. Because of that, I believe he reports truthfully on data he receives by informants.
Additionally, none of the frontier models actually publicly talks about inference costs in anything but broad, "let's just forget that"-like takes. Which does not exactly spark confidence.
I'm eagerly awaiting anthropic's public disclosure of their financial details. That should be rather interesting in any case and finally put the inference-discussion to rest.
The problem here is looking at a substance in isolation, instead of comparatively.
The actual question is: would drinking that stuff with sugar have caused more damage to health? And the answer will likely be yes. Because we _know_ just how bad sugar is for you. Particularly diabetes, microbiome changes, addictive behavior, obesity of course, cardiovascular issues...
If you'd look at sugar in isolation, as a new substance that stuff would never be allowed in any country at all.
Not technically but practically. The decrees are effectively considered law by the executive. Yes, you'll likely win in court later on, but you'll lose your job, get sent to prison, have your bank accounts and vehicle seized, etc., in the meantime.
Legality isn't really of much practical concern anymore. It's about what gets/can be enforced immediately.
The thing is that the IPOs necessitate a full release of their actual costs for inference and training. This by itself should be enough to pop the bubble, if the occasional bits of it we get are anything to go by.
There is a reason anthropic is still hiding those details:
> key details typically included in that form about a company’s operations — like potential risks to its business, executive compensation, and other financials — won’t become public until later on in the process
This is conjecture. There is a reason both openai and anthropic refuse to comment on inference costs. If it were falling so much, they would use it to brag.
I really don't understand why so many people keep repeating it without any actual data for the frontier models.
Apart from that, I'm not sure if focusing on tokens is even a good idea, because they are so different from model to model. I'd almost consider them a red herring now.
We could look at tasks instead.
Is there anything even remotely suggesting that your typical task you give an LLM now costs less in inference than before?
This is not about america. Not everything has to be turned into a discussion about some US internal issue.
The medium author has this in their bio: "healing, self-improvement, meditation, manifestation". Well, does not seem like the best source to me.
Aside from that, next you're probably going to post the protocols? Because that's where this line of thinking usually seems to take people. It's really nonsensical to focus on individual people, it's much more important to talk about systems and incentives. And, especially, compare to how it works in other countries.
Did they get to a similar place without person x? Then person x is probably not the primary issue here, but rather something on the system level.
Just like how the story of epstein is not the story of one evil person, it's the story of a part of society which deliberately enabled him and a system with no real safeguards in place.
Generally, with the regular in-IDE agents you have the ability to easily intervene, correct and live-check. Considering the high fail rate of agents (depending on software complexity of course), that's required if you want to get anything done and not be slowed down by it.
Otherwise you'd always have to context switch, consider which git state it's actually working from, etc. - rather than just letting the code directly before you change in your IDE.
It's significantly lower cognitive load and has a higher success rate, in my experience.
But, of course: Highly depends on the software being written and the general code infrastructure.
Problem here being that those terms aren't used as defined in regular discourse. Language changes and casual use differs from academic use.
When on an american-centric board anybody writes about "communism", I assume they refer to anything from marxism to stalinism to socialism to democratic socialism to social democracy up to anything non-hyper-capitalist. Not great, but sadly something to be taken into consideration.
Especially when looked at in context - parent was criticizing the EU initiative by essentially claiming something like that leads to a kind of monoculture like in a planned economy reminiscent of "communism", here probably meaning stalinism, from what I assume is a radical libertarian position. Which tells me the person is likely american, implying a rather ... minimal awareness of the nuance here.
Please, look at the actual comment chain and it should be rather trivial to make out what everybody is talking about. Does your comment really add value here?
However, this would of course be a financial threat to OpenAI & anthropic in case serious, long-term atrophy of skills can be proven consistently. So I'd expect this to be buried quickly, unless supported by an academically significant institution.