The article seems to be the result of deep research by an LLM with almost zero user input or review, something like "write an article about running LLMs on Strix Halo vs. RTX 5090": the article is from July 2026, but it uses 2024-2025 models as examples (Llama 2, Llama 3, Qwen3), and talks a lot about 70B models, which haven't been a thing for like two years.
Usually, when I talk/research about running models locally, even GPT-5.5 will always start by mentioning Llama 3/Qwen3 (knowledge cutoff somewhere in 2025); it will claim Qwen3.6 doesn't exist if web search is disabled.
The author didn't even bother to prompt it properly (i.e. guide the LLM to find more recent information as of 2026), didn't bother to review.
Qwen3.6 below Q8 often can't exit a reasoning loop (until it hits max output token count), forgets to insert a tool call, often mistakenly inserts them inside the thinking block... It's still usable though.
The thing is, everyone has their own variant of "qwen3.6 27b" depending on the launch parameters, ranging from "SOTA in its class" to "completely broken"
We've been self-hosting GitLab for about a year now, and I don't remember it ever going down or being unavailable. We self-host almost everything else too (except for online meetings), and it's all been pretty stable as well. Some of the tools we self-host do go down occasionally, but it's usually just a matter of restarting the VM or adding more storage.
It converges to "almost deterministic" on highly predictable outputs (i.e. code) with the right sampling params (say, you only sample the most probable token without randomness/high temperature) and with self-correction loops
Did you compare the code before/after? It's a mechanical line-by-line port, and most of the code is identical to the old version, just with Rust syntax. They have an example in the blog post.
Judging by the examples, if I understand it correctly, J-space supports higher-order logical / multihop transformations, but it is limited in size because of the limited network depth (max number of layers). When we emulate "reasoning," we basically extend J-space and allow the higher-order transformations to continue for longer, toward a more logical conclusion.
It sounds like instead of generating reasoning tokens end-to-end, we could probably only loop the middle layers (the ones most related to J-space) while skipping the first and last layers (less related to J-space) It probably explains why [0] worked. OP accidentally extended J-space? Also reminds of looped transformers.
Fable 5 was released on June 9 and removed on June 12. GLM-5.2 was released on June 13. It would be an amazing feat to make a model SOTA in just 3 days but I highly doubt it. It's more like z.ai released an existing checkpoint earlier than planned to capitalize on the news
GLM 5.2 is "almost Opus," and it needs at least 8xH200s for comfortable inference (so it's closer to $400k than $40k).
They suggest using this modified model:
>A REAP-pruned (≈22% of experts removed), Int8-mix NVFP4 quantized version of GLM-5.2, ≈594B parameters.
I wonder how it behaves in practice outside of benchmarks. Qwen3.6, even at 6-bit quantization, often gets stuck in loops while reasoning. And here they've also removed some experts. I mean, sometimes an 8-bit or 16-bit small model can be smarter than a lobotomized large model. I heard the consensus is you shouldn't go below 8 bit for coding.
Also, it's not clear what is left of the available context when you try to fit a lobotomized model into 4 RTX 6000s. Anything below 100k is barely usable because it often hits compaction before it's able to gather the necessary context
P.S. found in the repos, 240k context
I run a corporate AI server and coding peak hours here are 1PM-5PM judging by AI usage stats. My guess is that people spend 9AM-12PM in meetings and at lunch, and the actual coding starts around 1 PM.
From the perspective of LLM inference, you currently mostly care about:
- Memory bandwidth; BUT the requirements are currently capped because models have stopped growing at around 1-1.5 trillion parameters for quite a while now. You only need more bandwidth if you're optimizing for the highest possible concurrency (i.e. you're a cloud provider). Also, MoE exists.
- Support for native low-precision math (like FP4 and FP8); BUT once your GPU supports native FP4 (Blackwell+), there's generally no reason for GPUs to go lower because of the obvious quality degradation.
- VRAM capacity - just like memory bandwidth, it's practically capped by 1-1.5 trillion parameter models and is unlikely to need much more in the near future. Also, the current trend is toward miniaturization: modern 30B-class models (which require far less VRAM), now completely destroy 200B-class models from just two years ago on most tasks. We also have better understanding now how to compress contexts.
Most model improvements currently seem to come from RL/harness-based methods, not from scaling models or running new algorithms that require fundamentally new GPUs.
So I don't see why GPUs that exist today must become "outdated" in a few years. They'll be seen as outdated by hyperscalers because they need to serve the maximum number of users as cheaply as possible, so of course they'll replace their GPUs with newer ones that have higher memory bandwidth or more tensor cores. But you don't need that for local inference.
Yeah people don't realize these "toy models" now completely destroy gpt-4o on most tasks, and no one called gpt-4o a toy model back in the day... It was OpenAI's flagship model from 2024 to 2025.
Some countries and jurisdictions still have laws that allow for the involuntary confinement of tuberculosis patients, I guess dating back to the times when tuberculosis was rampant in those countries? And most professionals seem to be okay with the policy:
>17% said that, as a matter of principle, the involuntary incarceration of TB patients was inappropriate on any grounds.
>Regionally, members from Europe Region had the highest percentage of respondents objecting to the policy as a matter of principle (26.2%) while the North America Region had the lowest (3%).
The emergence of multi-drug resistant tuberculosis in the 1990s is probably one of the reasons:
>Respondents most strongly supported the policy of incarceration for patients known to have multidrug-resistant TB (49.7%)
See my answer in this same subthread. I was perplexed myself as to why I was diagnosed based on just one radiology report. But the moral of my story is that you can always try to obtain a second opinion from another doctor. I'm not saying doctors shouldn't be trusted in general.
The article seems to be the result of deep research by an LLM with almost zero user input or review, something like "write an article about running LLMs on Strix Halo vs. RTX 5090": the article is from July 2026, but it uses 2024-2025 models as examples (Llama 2, Llama 3, Qwen3), and talks a lot about 70B models, which haven't been a thing for like two years.
Usually, when I talk/research about running models locally, even GPT-5.5 will always start by mentioning Llama 3/Qwen3 (knowledge cutoff somewhere in 2025); it will claim Qwen3.6 doesn't exist if web search is disabled.
The author didn't even bother to prompt it properly (i.e. guide the LLM to find more recent information as of 2026), didn't bother to review.