I feel similarly. But texting became the primary form of inter-personal communication in the last couple of decades, in part because that was the primary modality technology could handle.
So, now that we can talk to computers, maybe we will feel more comfortable talking to people too? Wishful I know, but one can hope right?
I can't find it. Can you state your performance versus comparable 3-bit quantization from Unsloth/Bartowski? Edit: I appreciate that you seem to have open-sourced the quantization pipeline. This is not to question your work, but to understand where the outputs stand relative to the SoTA for quantization.
Have you benchmarked against other 3-bit dynamic quants like Unsloth? I am sorry but this framing against a full precision, newer, smaller MoE just seems misleading. Also, Gemma-4-26B-A4B is not the SOTA for edge. Even at launch, that would be the 31B.
Anthropic better get that IPO out soon. Their incredible revenue run-up was basically a result of botched Gemini releases and OpenAI having their hands-tied behind their Azure backs.
Anthropic models were quite literally the only viable serverless API (i.e. Bedrock) models on AWS. They didn't even bother releasing the recent Qwen 3.5/3.6 series. Combined with the token efficiency/ROI focus, I would really like to see how Antrhopic ends Q3.
Are you comparing single-user requests or multiple concurrent requests when you say comparable to rented GPU? Most of the cost efficiencies kick in with concurrent/batch requests. A single H100 node can provide like 5k input + 2k output tok/s on a model like Qwen 3.6 35B-A3B with 30+ concurrent requests.
Let's say the government can't care for 100M people because of lack of doctors. Now they could train one over 10 years, or you could have one of the smartest doctors in the world come be 100M+1. Would you take that?
Now expand that across socio-economic spectrum (not enough plumbers, teachers, AI experts, researchers etc). That is what legal immigration is meant for.
Subjective, but if we compare to compute not everyone needs the most expensive laptops or super computers for their work.
I think frontier models will be invaluable for scientific research, defense, financial analysis and such. But the average person probably would be reasonably well-served with a local model.
If you're in sales, customer service, product management and such - the leading open models at the 30B mark are already good enough.