The Openrouter providers the GP referenced were never at the "free money party". The actual cost of running something like GLM5.2 is well understood and tokens from those providers are not sold at a loss.
Obviously running things locally is more expensive but that all comes down to economies of scale. GLM5.2 is as expensive as it will ever be, barring an increase in demand that forces/allows providers to realise windfall gains disconnected from their underlying costs (always possible, but not the point).
> It's always integers unless you have a VERY good reason to do otherwise
I don't really agree. This seems like one of those outdated "greybeard rules" which people love to cargo cult, but to me it just comes with its own set of trade-offs, like now you have to think about exponents everywhere and getting them wrong comes with orders-of-magnitude consequences.
If you have a modern DB and your languages handle its decimals well then I'd use them. You can translate as needed for imports and exports, but the core of your system is always sound. Switch to the more basic formats if and only if there was some perf problem with decimals, which for most fintechs will be a VERY long time with modern DBs.
What really pushes me towards decimals is the consequence of getting something wrong. Relying on comparing raw ints, with variable exponents baked in, can lead to catastrophic errors with multi-order-of-magnitude gaps if that implicit exponent isn't carried along correctly. It's a massive footgun always waiting to happen. Decimals avoid that whole class of problem. So the goal is to maximise the "safe" decimals as source of truth everywhere you can, and have very well-tested "in and out" paths for any producers or consumers with different representation preferences. To me, that's good system design. And when, not if, you're manually inspecting DB records to track down a bug, having everything normalized in a glanceable, obvious format like decimals will let you recognise errors faster and more intuitively.
This goes extra for crypto, especially stablecoins, where one USD stablecoin might be exp-6 and another might be exp-9. And you're representing them in the same DB! Off by a factor of a thousand if you miss that exponent! Decimalize that immediately says I.
Well, I guess this is the silver lining to the price increases. I'd been thinking about an M5 128GB for local inference (eg DS4), probably off the table now given that it jumped $2k overnight. But I was on the fence about it for a long time given that even the M5 is not that good compared to even a 4090. It would have been good, but not "omg" good.
If they are pulling out all the stops to make the M7 more competitive.. guess I can wait for that?
I'm going to leave my above comment for embarrassment/posterity, but since writing it I've driven GLM5.2 much more extensively and I take it back. 5.2 is surprisingly, even shockingly good. It seems MUCH better than when I tried it on day one, and I probably shouldn't have drawn solid conclusions from that as models often struggle a little on launch.
I take it all back, 5.2 is very much competitive with any Opus and Fable seems very much in reach if they can continue making these leaps. It seems ZAI has quite a lot more up their sleeve than I had surmised.
But the question was about whether the Chinese labs will have fable-equivalence in 1 year. I am by no means some kind of insider, but knowing the vaguest outlines of what went into Mythos, they just can't do it. The compute is not there. The Chinese engineers are incredible, but they're not literal magicians.
Of course there could be something incredible to come out of left field and overturn the apple cart yet again, but that's speculation. It would be awesome, sure! But I wouldn't bet too heavily on it.
And FWIW - again, no disrespect at all to the Chinese engineers but I don't rate GLM5.2 as being even close to opus 4.6. It can hit a few benchmarks, sure, that's the top edge of the "jag". But filling in the rest of the capabilities - again, it takes compute and data the OSS labs just don't have, that anyone knows about at least.
I don't think we will. The open model labs are too resource constrained to approach Fable or even Opus on the general case and I don't see that changing within a year.
Right now, due to profound shortfalls in both data and hardware compared to the US labs, the OSS models are IMO basically technology demonstrators that in practise are even more jagged than the US labs' efforts. The high points of the jaggedness are close - but number of happy paths is many times fewer, and their behaviour inside the harness is far less refined. Barring some incredible breakthrough I don't think that is changing without a much higher level of resources - which seems impossible given the current hardware environment.
I have no reason to think that Anthropic or OpenAI are in possession of some secret sauce that the Chinese labs can't duplicate given the right resources, but the fact remains that absent those resources they'll remain behind. Barring some incredible bombshell reveal from Huawei I don't think this asymmetry resolves in a year. In three years it may well be a different story.
What a great analogy. And IG/Tiktok reduce it into an even purer state - endless random videos, barely if at all connected, ephemeral stimulation you can't even remember 30 seconds after seeing it.
I know 50 year old adults who can spend entire hours just in this mesmerized state of flicking through these random feeds, seeing but not seeing, like some kind of drug induced hypnosis. I wonder what Postman would write today, were he still with us.
I think when you follow this stuff every day it's easy to lose perspective of the rate of change and these leads seem more profound than they really are when you zoom out a bit.
I'm no super-insider, I only hear industry scuttlebutt like everyone else, but I have about a 95% confidence that the last 18 months has just been about more and better, without any kind of real leap or breakthrough. More hardware, more data, better technique. Well, technique diffuses as people change companies, hardware can be built, and data can be gathered (or stolen!).
From my admittedly outsider perspective, the only years-long moat there is who has the most hardware. If you have the hardware, you can give away the compute to get the data (hello, subsidized subscriptions!). Technique can simply be hired. The only durable, multi-year advantage is the hardware.
So is that a moat? Sure, but it doesn't have a whole lot to do with the leading model companies of the moment. ASML is the real moat, and so it's ASML China is besieging, correctly (IMO) identifying that everything else can be caught up easily enough.
Ed Zitron certainly was right that a constant firehose of denialist AI doom would get him clicks and views from the type of audience who yearns to have their biases confirmed and their fears validated. He's made a lot of hay off that excellent prediction.
Can't really think of anything else he's been right about, though. I don't think "right" is what he's going for anyway, it's all about that validation and a coherent, testable hypothesis takes a very distant second place.
it's down 99% since that peak. But let's compare to it anyway.
It's pretty useless to compare raw FLOPS, but as a general hand-waving guesstimate, F@H is currently doing about 25 petaflops in a mix of FP16 and 32. AI usually trains at FP8, but to keep things fair the H100 is quoted at 60 FP64 teraflops per unit, so that's 12 FP64 exaflops given its 200k count.
So F@H at its peak did 2.43 exaflops@FP16/32. Colossus 1 does 12@FP64. These numbers are very hand-wavy, but I think the point is made.
By the way, I'm not trying to crap on F@H - I think it's an outstanding project and I've run it in the past. But a volunteer group simply cannot compete with well-funded, concentrated effort like what's going into AI.
Not sure what you are referring to, unless you don't think h100/h200/b200 are "AI hardware"
> Superpods aren't really power efficient
Maybe not compared to a specialized rig with multiple 4090s, but that is the best case for consumer hardware - the vast majority will be dramatically less efficient than that
Anyway, I agree the interconnect is by far the biggest obstacle and seems insurmountable, I should probably have led with that.
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.
The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.
And even if electricity was free, having those GPUs spread over the world with internet-level latency will slow everything down by factors of thousands to millions - if it's feasible at all. Regardless, you're not getting fable-oss this decade, maybe even not this century.
It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.
If folding@home is a useful yardstick by which we might estimate the amount of GPU-ish capability that civilians might be coaxed into donating to a shared enterprise, yeah, it doesn't look pretty. This is extremely rough napkin math but comparing to xAI's Collosus 2 for example, for training workflows you're probably looking at 4-5 orders of magnitude the capability of all of folding@home combined. That's 100,000 times faster.
Very rough math like I said but I doubt it's directionally wrong.
And even if you did force literally everyone on earth with some sort of GPU to max it out 24/7 in service of an open source AI training enterprise - you would waste so much power trying to use that inefficient consumer hardware with the worst latency imaginable that it would be cheaper and faster to get everyone to instead chip in some cash to buy a datacenter with blackwell chips instead! So the idea has no legs whatsoever.
I don't think insulting people is a great way to contribute. Not everyone who sees things differently than you has "psychosis".
Your reflexively negative comments on anything relating to AI are as insight-free as they are numerous; it's all just vague shitting-on without even a hook or argument that could be engaged with and debated. It's pretty tiring, honestly. If you really think your point of view is valuable and others should pay attention to it, rather than just filtering it out like the trollish noise it usually is, why don't you put a little more effort in?
An enduring, confounding quality of LLMs is that even minor differences in prompting content and style, harness type and environment can lead to radical differences in the output and perceived performance and ability. In my environment and in my "style", Fable has been a huge step up, to the extent that I am seriously considering paying for a second $200/m account just to get more usage out of the next 10 days. I'm also starting to prepare my organization for what I now see as the completely inevitable end of human-written code.
All that said, considering Anthropic's heavy-handed nerfing I'm not surprised Fable did poorly in a security-focussed benchmark. And this benchmark seems poor anyway - penalising a model for "cheating" by knowing the answer from its training data? That's not the model's fault, that's a lazy benchmark.
Same story with me. To be clear, I am a subscriber, though I tend to hold out for the ultra-cheap last ditch retention deals they through at you. But I take them with a grain of salt these days. They have a narrative like anywhere else, and they don't let the full facts get in its way.
Michael Crichton said it best:
“Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them.
In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.”
I 100% agree with you, but I've been convinced over the last year that it's a time and scale issue, not anything fundamental.
The Chinese models right now are in a weird spot. Compared to the frontiers, both their pre and post training is woeful - tiny, resource constrained in every dimension including human, slow. I'd compare it to OpenAI 5 years ago except I think even then OpenAI had way more!
But they "cheat" quite a lot in distillation and very benchmark-focussed RL and that's where you get this superficial quality in the leaderboards that doesn't match up when you go off-script. Arc is a great example in that it really belies an "inferior soul" at the heart of it all.
What gives me great hope though is that those same scaling laws that Altman and others have been hyping forever will absolutely kick in for the Chinese labs just as they did for the US ones, and I don't think anything can stop that process now. So they will catch up. It won't be tomorrow, but it's not going to be 10 years either. 3-5 would be my reasonably educated guess.
And the final risk, that China itself might try to restrict availability of the tsunami of GPU or other AI hardware it will inevitably produce - well, I just can't really imagine a country that has been configuring itself for the last 40 years as a single purpose export machine deciding that actually, no, it doesn't want to export something.
About the model restrictions - absolutely. I've been trying to do security research on my own software and the frontier models immediately get suspicious. I've been playing with the local ones much more this year basically because of this. They have deficiencies, for sure - they feel very "hollow" compared to the major labs. But I've talked to a lot of people, and the consensus is pretty clear - just a matter of time.
What bleeding? Anthropic wants as much of that "bleeding" as possible. The interaction data gathered from genuine human CC subscription usage of their models goes directly into their RL training, it's invaluable and they are more than happy to lose money on the inference to get it. That data is what xAI was recently willing to pay $10b to cursor to get.
They want you to use Claude Code. They hate other UI surfaces like OpenCode etc purely because they lose control over that data, so they're subsidizing the inference without getting what they actually want, the data (they still get some of it of course, but it's much less ergonomic for them. Those tools often abstract away the subagent calls, for example). OpenCode can collect that data themselves, so by allowing subscription there, Anthropic sees itself as subsidizing another org getting that data. Hard no.
And tools like OpenClaw are useless because they're mechanical and don't represent actual users interacting with the service - again, subsidizing but not getting the reward.
It's all very simple once you understand their motivations.
I am no-where near as concerned by this as I was a year ago, when I was expecting the axe to fall at any moment before the Chinese labs achieved some sort of escape velocity. I now think it's too late, all the cats are out of all the bags, there's no moat except maybe a temporal one of a few months, the genie is out of the bottle.
There is no secret sauce the US labs have that the Chinese ones don't, or won't have soon enough. Deepseek 4 and Kimi 2.5 are not quite Claude 4.5/GPT5.5 but there's no fundamental principle missing - they are strong evidence that there's no real advantage the "frontier" labs possess that isn't related to scale, which they will gain in time (if they even need to). The RL post-training techniques that work are widely known and easily copied. All Deepseek is really lacking is data, which they're getting - and the harder Anthropic/the USG makes it to access claude in china, the more of that precious data they'll get!
I used to sort of entertain the "fast take-off breakaway" scenario as being plausible but not really anymore. The only genuine moat the frontier labs have is their product take-up, which isn't nothing, far from it, but it's not some unbreakable technological wall. Too late guys - it might have been too late for quite some time.
> The thing is that even if I was wrong (I'm not) and AI was somehow helpful for software engineering (it isn't), I still wouldn't want to use it.
So even if you were wrong on the facts (you are) you still wouldn't change your mind? In other words, you're unreasonable and know you're unreasonable and think that's totally fine?
The Openrouter providers the GP referenced were never at the "free money party". The actual cost of running something like GLM5.2 is well understood and tokens from those providers are not sold at a loss.
Obviously running things locally is more expensive but that all comes down to economies of scale. GLM5.2 is as expensive as it will ever be, barring an increase in demand that forces/allows providers to realise windfall gains disconnected from their underlying costs (always possible, but not the point).