I lead Enterprise Data Science and Media Digital Product development teams across APAC, Middle East and Africa for a Fortune 50 consumer products company.
srinathh at Gmail.
Well OpenAI is offering equity to the US Government (and who knows who else privately) Tim Apple famously refused to bring manufacturing back to the US when the current president asked and play hardball on infosec. While this is a civil case, increasingly judiciary seems to be an extension of the executive. So it will be interesting to see how this plays out.
Materials science fundamentally has a math & computability problem. Rigorous materials simulations at a nanometre scale seem now feasible - so you could model physical properties of small groups of atoms - eg. Modeling molecular reactions at a sub-atomic level. Bulk property simulations at a centi and up scale also work and so you can do first principles based design and use tricks like finite element methods to deal with the underlying stochasticity to a degree. Materials properties however largely arise from features between nano and micro scale - grain boundaries, dislocations etc. These are computationally infeasible today - there are neither engineering solutions, not math tricks that make this tractable and so materials science and engineering becomes a grind of experimentation and metrology.
This is interesting in that it seems to be making the grind more efficient. I think the true breakthrough will likely be proper scale quantum computing to make the first principles design feasible
For those unfamiliar with the reference, in Hitchikers guide to the galaxy series, Mice are projections of hyper intelligent extra terrestrial beings from a higher dimension who commission the creation of planet earth complete with fossils as a giant organic computer
Used both. DeepSeek-4 Flash Q2 - last 6 layers Q4 quant with DwarfStar which just about fits in 128Gb is definitely superior IMO - my contexts tend to run typically 50-100k. Throughput tends to be about 12-13k tok/sec - just about acceptable.
Is this advice coming from personal experience of significant diabetes and weight reversal or just online reading informed opinion? Physiology & psychology are quite complex and individualized and "just do this.." doesn't quite work at any sustainable level when people have jobs, families, mortgage, sick children & sick parents to also manage at the same time.
You do see this working in many places. Singapore is probably the best example with an explicit scholar programme and competitive pay in much of the upper rungs of the government. This comes with its own problem culturally but does result in very well thought through programs and infra that actually helps people.
Even in the top echelon of the Indian government (Indian Administrative Services), a similar culture exists and has interestingly been strengthend as the power has fragmented across political parties. I personally know several of my top B School/Engineering college mates who joined and any of them would do very well in top management in private industry. You can see the result in the rapidity with which India is pulling people out of poverty and modernizing. Certainly alower than China, but pretty amazing seei living outside for a decade now - many urban services are already far superior in quality to what you get in Europe, US or even Singapore. The challenge in India is the bulk of the bureaucracy below them isn't held to the same standard.In India it's still a prestige thing only in the top echelon - the lower echlons are more motivated by job security, pensions and avenues for corruption.
The difference in Singapore is the government pays well enough to attract and retain higher quality talent deeper.
> We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs.
This is nonsense that AI providers want to peddle. Inference is wildly gross margin profitable - likely 90%+ gross margins. It's very easy to work out the cost structures bottoms up. All providers can drop costs to a third and still keep positive gross margins.
The problems are
1. It possibly still doesn't pay out on training investment in a reasonable time frame without a massive expansion of the 90% gross margin.
2. There is no moat. As we see Mac Mini & High End GPUs stock outs and the pricing offered by DeepSeek and Qwen, the performance of Open Weight models are good enough that people can and are already shifting many inference workloads out of these 90% margin players
Inference I estimate runs 90% plus gross margins. Just work out the math on these servers. I am pretty sure any player can price down. It wouldn't look good on an IPO prospectus.
So a lot depends on your specific use case but mid-sized open weight models are pretty actually good now, so this is realistic [1]
The first question to ask is does your use case require handling personal or sensitive data.
If you're using the LLM for OpenClaw or you want to handle sensitive or medical data, a local model generally is necessary.
if it's not so sensitive - Cloud providers with some sort of user agreement guarantee on not using your data for training would be the next bet. I personally generally use Gemini or Sonnet as my cloud backup. As I understand, OpenAI, Cloudflare (which bought replicate) and Qwen also seem to provide such guarantees and make SOTA models available. Others like DeepSeek seem to have an opt-out setting. Open router & co I avoid except for benchmarking models with public or dummy data as there is absolutely zero guarantee or ability to enforce terms on providers where your data might be sent.
Gemini and Anthropic (and OpenAI) tend to be expensive - it's very easy to run up 15 dollars a day or so bills which puts you solidly in 1 year pay out on Mac Mini territory - at this point I decided to buy. Gemini Flash Lite 3.1 is however surprisingly good value.
the next question is Mac or CUDA. If your expected use is serving LLM models for inferences, the latest large memory Macs give pretty good inference speed (better than DGX Spark) at a reasonable cost - I think there offer much better value than CUDA if the only use case is LLM inference & harnesses.
if you plan to also fine tune models, experiment with other types of ML on GPUs, do computer vision stuff etc. the development tooling on CUDA is far in advance of all other platforms.
Lastly if you choose CUDA, the question is GB10 family (DGX Spark - cluster able with 128Gb RAM et all) or dedicated GPUs workstations. What I found is practically any serious models weighs in requiring at least 96GB VRAM - Antirez's 2 bit quant of Deepseek 4 flash (my current daily driver) [2] , the Qwen 3.5 122B A10B 4-bit quant, the Qwen 3.6 27B Dense and 35B A3B 8 but quants etc. So you're well out of the consumer GPU territory into 1 or more RTX 6000 Pros or Data center grade devices. Yes you can try to hack away with multiple consumer cards or SSD streaming but it's very fiddly and you probably have better things to do with your life.
The GB10 system - which I ultimately went with - is certainly much cheaper and can be clustered through the Special NVLink cable to get 256, 384 or 512 GB setups but comes with severely constrained bandwidth. The Pro GPUs blast these out of water on performance but are expensive.
Lastly, renting a cloud GPU machine doesn't really make sense except to run already debugged fine tuning workloads. You'll probably spend at least 4 dollar an hour for sufficient capacity which if it's personal use, will mostly sit idle.
And companies from outside the US will outcompete those from the US forcing more protectionism, higher prices in US etc. I think there have been several cycles of several industries that have gone through this cycle (cars, shipping?), and mostly forced to roll back.
The problem here is that open weight models are already good enough for a majority of process automation and intelligence tasks and that is where a good chunk of efficiency corporate dollars are. So there's an ever shrinking slice of inference that will hit the frontier models and inference is where the insane margins are. Now to be fair, Claude co-work and Claude code/Codex do seem magical today and these potentially will continue to be high margin/leverage plays. Frontier models are also likely to push towards decision making - so we'll have to see how it shakes out but the bottom end is already commoditized and it is getting bigger and bigger.
As an update, turns out Antirez created a brilliant 2 bit quant of Deepseek to fit into 128Gb systems along with a custom highlight tuned server. I've been running this the last few days and if I turn off envelope on OpenClaw, the performance is brilliant. Still to try with coding harnesses. It's a bit slow compared to the other models but so good that I'm willing to put up :-) https://github.com/antirez/ds4
I think Israel's leadership has very unwisely lost the Strategic plot here in favour of tactical political advantages of survival.
The history of middle East for the last 5000 years (since Sargon of Akkad) is replete with 'the king X "pacified" (the most commonly used euphemism) the people in the conquered territory'. It has never gone well for the victors under successors of king X, often within a generation or two.
In today's age when access to technology and information is such that any small sufficiently competent and motivated group can cause massive destruction, is it wise to keep creating motivated enemies and expect they will somehow never become competent or that the competent won't become motivated? It is doubly ironic given Israel's own defense industrial complex is filled with such small motivated and competent groups and the evidence of Ukraine/Russia conflict is staring in the face. This situation will blow up I fear within a generation unless Israeli society chooses different leaders.
Pretty sure this ruling will be used as a precedent in cases against the other providers really soon. As I understand it, there is a legal cottage industry that brings digital related cases (eg. Copyright) in Germany.
Deepseek is too large for me to self host on Spark. I was actually using Deepseek as my cloud backup and it performed well but then read the T&C which doesn't give as strong data protection guarantees unlike Google and Alibaba. Kimi is again massive and cloud hosted APIs are fairly expensive compared and it also has weak T&C, so have only benched but not tested. In general I found that with OpenClaw it works better to turn Reasoning off.
I think there's possibly value to try fine tuning Qwen 3.5 on my OpenClaw turns log to see if performance improves. The one recent model I haven't tested yet is Nemotron 3 Super which I might bench soon.
Qwen is definitely the model to beat as of Mid 2026. While I didn't benchmark with SWE as my use cases are OpenClaw [1]. I found both Qwen 3.6 35B A3B and more impressively Qwen 3.5 122B A10B starting to be competitive with closed flash models. The NVFP4 quant of the latter is what I'm running now on DGX.