Many of us grew up in the PLD era, k-maps, etc. Woz pushed early HW to the limit, with SW APIs that delivered real value. Woz made astute design trade-offs based on full stack knowledge that his peers lacked. The world’s moved on to the GPU (low precision, accelerated parallel compute?) era, but the Woz view point still holds. You can see it in the AI kernel optimizations, or rematerialization methods to push GPU HW to the new limits, and trade-offs need to be made. GPU HW for 4-bit QAT or even 2-bit will dramatically affect the SW (AI) of this era. What trade-offs do you make?
I saw Woz on Northbound 280 “driving” his cherry red Model S, using FSD. He was looking down at the screen the whole time I watched him. Swear he had ssh’d into it.
[edit - Gabe responded]. See this Cloud Run spending cap recommendation [0] to disable billing, which potentially irreversibly deletes resources but does cap spend!
Not sure what “official” means but would direct you to the GCP MaxText [0] framework which is not what this GDM paper is referring to but rather this repo contains various attention implementations in MaxText/layers/attentions.py
Please don’t do this, Zach. We need to encourage more investment in the overall EDA market not less. Garry’s pitch is meant for the dreamers, we should all be supportive. It’s a big boat.
Would appreciate the collective energy being spent instead towards adding to amor refining Garry’s request.