Yes... And, perhaps the political comments here are a reasonable reaction to the misguided political influence that shaped grok.
How can we trust grok after this? At least it will take a while.
I need a model that can answer questions in an unbiased way and do what it is told. If I need a specific political opinion I can find that myself - thank you very much.
What Google is doing is shameful. One of the promises of Android was being more open than the restrictive Apple ecosystem.
Now that they reached penetration they do the switch - under the guise of security.
Just let me do with my hardware what I want to do it. Let it be my responsibility to install whatever I want (and stop calling it "side-loading", as if I am doing something shady from the "side").
We need to resist this! Alas, from the broader response it seems that most people just do not care.
I soooo wish that to be true. Alas, in my experience it is not... Yet.
What is true is that it gets easier and faster to run local models.
With QAT (quantization aware training), turboquant (or similar) K/V compression; what used to be impossible to run is now fairly easy.
I can run gemma4:26b-a4b-qat on my laptop with 20-30 tokens/s with a 256k context window. That was unthinkable just 6 months ago.
So the local models are "OK" for small'ish projects.
But it does not at all(!) compare to the frontier models. For a large project Claude's Opus 4.6+ just work, whereas local gemma tangles itself up, makes weird mistakes, and just can't handle it (for those cases it is faster if I do it myself).
If the trends continues, with 1.58bit QAT models, even better K/V compression, faster multi-token prediction et al, maybe soon it will be comparable.
This is really awesome.
I tried my test-task (generate a Python wrapper for a fairly complex C-interface) with gemma4:26b-a4b-it-qat. And for the first time it would just do it, without prodding and without errors. (Of course Claude just did it a heartbeat, too.)
My optimal local setup now is gemmma4-qat and Q8_0 K/V cache quantization with 256k context windows. And that runs fine with 12GB VRAM and another 10GB in RAM.
Previously I tried with gemma4:26b-a4b-it-q4_K_M and qwen3.6:35b-a3b-q4_K_M, and they both would tie themselves into knots (especially qwen3.6 can take forever with incessant "but wait..." thinking loops.) More often than not, they would not finish the task.
It seems true these 4b QAT models are as precise as Q8_0 quantization (which is supposedly indistinguishable from bf16).
I am really excited about the prospect of local LLM inference.
Adding these to the index immediately would force passive index funds (multiple trillions of $) to buy this stock, and thus not allow the market to make performance based decisions.
It's truly a shame that the NASDAQ caved and I will definitely reduce my position in such index funds (I have less trust in it now).
I use Claude every day. Often for multiple hours a day.
Basically doing my job not worrying how many tokens I spend (as in too many or too few). This is a pretty complex code base (database optimizer and related).
Just looked at spent for the past 30 day, didn't even come to $600. 95% of my tokens are from cache. If I were to reach even $1500 I have to let claude run unsupervised over night (and with the amount of mistakes it still makes and guidance it needs, I do not believe we are there yet.)
My son is prepping for the SAT and I am helping him. I studied physics and computer science, and was a advanced math A+ student...
IMHO: The SAT is useless, solving equations under artificial time constraints is something that only happens in these kind of tests. The focus is on solving problems fast and getting a good score, and nobody really cares if you understand the math behind it.
So, please, if you go back to testing, find something more useful than the SAT.
Other countries paying $10,000's to educate people who then want to apply this knowledge in the US. US reaction: "Nah." Besides, we are talking about legal immigration here.
> Wrong the way it would be wrong to predict that if you set your kitchen on fire, the result will be a renovation.
This might be favorite metaphor ever, and one I'll quoting in the future! :)
I think the author conflates social media with other inventions like a portable GPS device, an electronic map, a music player, or indeed a cell phone.
As far as social media goes the author is (IMHO) spot on. You do not have to look far to see how that is at least harming democracy around the globe. For democracy to flourish you need reflective voters who can entertain multiple viewpoints and make informed decisions. That is what social media - in its most common current form - discourages and rather optimizes for attention-time (which is money).
And of course (some) anonymity paired with global reach would not bring out the best in people. Anger and flames spread faster than conciliatory messages and get you more dopamine posting those.
I work on database optimizers and other database related stuff, and I can assure Claude Code - with all the highest settings - does make mistakes. It will generate a test that does not actually test what it "thinks" it tests. It will confidently break stuff.
Do not get me wrong. It is still awesome! It takes much of grunt work off me. It can game out designs decisions even when that needs to refactor a lot of code.
If you point out a mistake more often than not it can fix it itself.
It's just for a critical project I would never ship it without understanding every line of code - with the exception perhaps of some of the test code. Maybe in a year or two that will be different.
The size of the KV cache (context stored) is proportional to the number of layers of the model and number of "hidden dimensions". For a 400B model it could be 30-60GB for just an 8K context window (depends on the model, etc, just a ballpark).
So shrinking that by 6x (from fp16), would be big win for larger models.
True, while TurboQuant can also be applied to model weights, it won't save size over q4 compression, but will have better accuracy.
I am fascinated by this and similar research (RotorQuant, etc). It seem by next year we will be able to run this year's largest models on last year's hardware. :)
Maybe we won't need as many data centers and as much power as we thought. Maybe we can run more powerful models locally.
Hmm... Sure, if you do not need a database then do not use a database.
Don't use a sports-car to haul furniture or a garbage truck as an ambulance.
For the use case and scale mentioned in the article it's obvious not to use a database.
Am I missing something? I guess many people are the using the tools they are familiar with and rarely question whether they are really applicable. Is that the message?
I think a more interesting question is whether you will need a single source of truth. If you don't you can scale on many small data sets without a database.
I will say this before I shut up with my rant: If you start with a design that scales you will have an easier to scale when it is time without re-engineering your stack. Whether you think you will need to scale depends on your projected growth and the nature of your problem (do you need a single source of truth, etc.)
Open Source was always open to "many eyes" in theory exposing itself to zero-day vulnerabilities. But the "many eyes" go for the good and the bad actors.
As far as I am concerned... Way to go Cal.com, and a good reminder to never use your services.
Good. Now leave TikTok and Facebook as well. People who care will find out what you are up to, and people who don't won't see you on social media anyway.
I left Twitter, Facebook, et al about a decade ago. And I can assure you: You will never miss any important development.
The notion that we need to plugged into Twitter, X, whatever, to stay up to date is simply false.
How can we trust grok after this? At least it will take a while.
I need a model that can answer questions in an unbiased way and do what it is told. If I need a specific political opinion I can find that myself - thank you very much.