The article does answer the question, in detail. What you're reading is just the first half is freely available. The 2nd half is on the same web page and for subscribers.
That number is wrong. I have a number from googler, not livestats which cannot have google internal data
- Not every query has to run on LLM. Probably only 10% would benefit from it
Agree, i have something different coming up that looks into this more, 10% may be too low. I know i used 100% which isn't right, and explicitly say that
- This means 10,000 queries per second, each needing 5 A100s to run, so 50,000 A100s are sufficient. Cost for that is $500MM, quadruple that to $2B with CPU/RAM/storage/network. That is peanuts for Google.
50k A100s networking ramp cost way more than $2B HW utilization rate
- Latency, not cost, is a bigger issue. This should be addressed soon by H100 and newer chips.
Thats discussed in the subscriber section. It's both, but yes latency is bigger issue. H100 helps but doesn't solve.
Thanks! Ya, the archive link is useless, it only captures the free part, and I think I am very generous with what I keep in the free section on the ad-free website.
It has noticed. Look at Apple architects, validation, layout, etc engineers moving to Nuvia + Rivos + Google + Amazon + Microsoft + Meta + Intel + Nvidia + AMD + Apple + Qualcomm.
If you don't want to believe it. Go ahead. The major claims are true, Qualcomm moving away. Nvidia moving away. DRAM node ramps being pitiful. DRAM engineering efforts come directly from a source there.
The cultural issues being the cause of these issues is the substantial claim.
MediaTek is killing it. They hired a lot of people from TSMC's process development kit teams which vastly improved their capabilities in integrating IP. They were the first to develop AV1 decode, their modems are pretty decent. They even have leading teams in wifi.
Most folks blame the culture shift on Paul actually. He was the cause of it all. Brian being a horrible symptom of what he created, and Brian definitely propagated it far.