Would you use a 10-ton truck to carry groceries to your car?(morlockelloi.substack.com)
morlockelloi.substack.com
Would you use a 10-ton truck to carry groceries to your car?
https://morlockelloi.substack.com/p/the-gpu-fable
2 comments
ofc not
https://morlockelloi.substack.com/p/the-gpu-fable
Yeah if your usecase is chat sure, it can run faster than you can type. For anything useful, like code autocomplete, or agentic coding, the context is always in the hundreds of thousands of tokens. And usually the new prompt is going to be 50 to a few thousand tokens (if you are including error tracebacks for example). And the user will typically expect hundreds of thousands of tokens to be generated. If you don't believe me use cerebras with claude code and watch it generate millions of tokens in a few minutes.