it is funny because nobody ever bother points out that they overcharge you for text input token price.
sure it was pretty resource intensity a few years before, but with turbo quant, sparse attention and various techniques, plus the advancing of hardware (dedicated prefill machine, memory pool for kv caching) the cost should be drastically reduced, and yet they still keep the same cost formula.
I can't help but laugh whenever someone proudly share how many billion input tokens they spent in their code sections and how much they saved with the subscription, meanwhile it is pretty much just electricity cost for the providers.
That's for the long term. Anthropic only needs short term solutions for the sake of IPO. They will do whatever they can to sabotage other companies (specially the Chinese ones) to reach the same parity with best claude models.
I doubt you can do that. MTP magic happens because for texts, we have a lot of low value fixed tokens that almost always get generated in the sequence (like punctuation, function words, language keywords etc). for most important ones (the entities, the content words, variables) you still need the full model.
so there is alwasy a maximum limit for how well MTP can do.
edit: now I read the article fully, seems like they utilize some very effective MTP algorithm. and somehow the quality is still decent enough.
though, I doubt that the quality really only drip a bit like they claimed. maybe for the benchmarks, but for general uses the heavily quantized models very often so worse result.
Is this somehow satire? This is just the dgx spark with keyboard and monitor in a convenient format. Since it has more stuff, I'm sure that the price mark up will increase too.
Up to $5000 because why not?
With that money you can build a real PC with rtx 5090!
from what I understand, it's because unlike the other models, MAI models haven't yet fine-tuned against the synthetic datasets specifically designed to boost the benchmark scores.
I personally do not like Microsoft, but congrats them to release this model.
While the scores are not good compare to other open weight model, the important thing to note is their training data (as they claimed) is very clean, without any synthetic datasets.
I bought one AMD MI50 32GB back then when they were sold rather cheap (around $150-$170). it can easily generate over 70 tokens per second for gemma 4 26B moe model (q4).
I have no doubt that we will have another wave of cheap retired server gpus just like before. And that is the time when everyone will have their own models at their home.
Or we can just buy the newest medusa halo mini pc. they will be pretty decent, too, albeit pricey.
we all know it is impossible goal to make. surely AI will be even more useful in the future, but as long as china exists and continue to undercut the price, the goal will be never meet.
> We're talking about a world where you need 5% of every knowledge workers salary to go into tokens. 20% if you're a developer.
with that much money, the companies can easily buy their own hardware and hosting free public models, no need for those expensive subscriptions.
Finally some good news. There are a lot of niche products (like handheld emulators or pocket devices) are on verge of collapse right now due to the ram price.
I was waiting for a new GPD win max with amd hx 385 or newer CPU. But they are holding the production plans right now, it sucks.
because for most people they don't need what deno promises.
me for example only use nodejs or bun to run a basic sveltekit server, so it can render the html for the first time. all core functionalities are delegated to backend services written in crystal or rust. I don't need some bloated js runtime that hoard 500MB of ram for that purpose (crystal services only take 20+ MB each).
bun promised a lean runtime, every essential functionality is written in zig to increase the speed and memory footprint. and javascriptcore also uses less memory compare to v8. the only thing we expect is for bun to stabilize and can run 24/7 without memory leaking or crashing.
The premise is if they stop training new models then it will become pure profit after 2 years when the hardware finished paying for itself.
It's pretty funny that everyone say that this business is unsustainable, but I have yet seen anyone bankrupt, even the pure hardware providers who are renting out a100 b200.
sure it was pretty resource intensity a few years before, but with turbo quant, sparse attention and various techniques, plus the advancing of hardware (dedicated prefill machine, memory pool for kv caching) the cost should be drastically reduced, and yet they still keep the same cost formula.
I can't help but laugh whenever someone proudly share how many billion input tokens they spent in their code sections and how much they saved with the subscription, meanwhile it is pretty much just electricity cost for the providers.