If compute is not the bottleneck, memory is easy-ish to produce (the hard part is mostly on the fab side); what stops a Chinese NVIDIA (huawei) from being 10x cheaper?
They are usually the same family, LPDDR is used for amd and macs, but the fabs are the same as the most expesive HBM memory, if they have a choice they are going to produce the ones that they can sell for more $$.
I'm writing my own inference engine for Strix Halo and the same model. I already have 30%+ performance plus a more graceful decay over long contexts; that said, their point stands: memory bandwidth is what you really want.
> This is very literally what already happens, it's called a EULA.
Yes, but they "reserve the right" to update whenever, making it pointless
> "In favor of the customer over anything else" is not a legally viable clause.
I'm sure that legislators could put the principle down in a much clearer way. What's lacking is the will.
Yep, that's me. the only real blocker is that American companies don't trust Chinese providers, but i could just find a good American provider that hosts DeepSeek and/or GLM. I would at least be able to choose my own agent instead of a quite mediocre one that wastes time and output nonsense verbs in a pathetic attempt to gain sympathy.
The only reason that stopped me from doing it is the absence of a subscription, and I did believe I couldn't get the same value with API pricing, but I'm starting to see that it's a blatant lie and true only for anthropic and openai...
A simple law: everything the customer buys must always behave *in favor of the customer over anything else*. If the product/service contradicts this, it must be fully stated before the purchase and cannot be updated. <= This would be a sane balance.
It isn't a promotion, it's 2x the parameters of opus and we are paying with 2x the consumption rate.
They just want to get rid of the subscription model.
you can absolutely use it for some workloads, but as soon as you have some extra complexity for a big repo it'll take forever and the economics are so silly to the point that the electricity bill would be comparable to a subscription. I love having the possibility of running things locally if some random dude decide to pull them plug, and give me solice the fact that i can have 100% private inference, but as the main driver during the day? shoot me
i used to mix remote and local minimax 2.7(q3) on my strix halo, it run at 30 tg and 220 tokens pp... it was a bit painful slow, but it was a good feeling i could stay offline. unfortunately m3 which is at opus .8 levels is 460b parameters and doesn't even fit in 128gb of memory, let alone a big context. strix halo feels like a toy for ai purposes. https://kyuz0.github.io/amd-strix-halo-toolboxes/
I did develop my own agent around MiniMax. I did see weird behavior when I messed up the loop, like omitting pieces of remove thinking; maybe it's an agent bug, some models/providers just ignore/normalize the broken input, some don't.
>It's such a weird "Gotcha" that seems to only assume that Chinese LLMs might censor something.
We are not assuming anything; it is illegal, and you will get prison time just for talking about it. Yeah, sure, everyone distorts reality, but there is a huge gap between hiding and enforcing.
So yeah, having models respond accordingly is unexpected. There are probably multiple variants tuned differently.