As a serial successful field-hopper, I agree that I'm not the right person to be making these estimates.
But the external view is that college courses roughly expect you to do what I'm claiming, in roughly the time investment that I'm claiming -- and undergrads are typically in 4+ classes at a time. So is it that the whole educational system is delusional? (I fully acknowledge it might be so!)
I could have sworn that sickly sweet smell was the smell of various phosphine reagents? Just my vague recollection of my time in lab from 15 years ago.
Does this handle, e.g., water of hydration CaSO4 . 2H2O? states of matter H2O(g)? does it preserve subunit information, as in (C6H5)CH2COOH? Writing a parser for basic formulae is such a tiny tiny part of the actual problem... deciding the scope of what you want to handle and how is the real problem
Great write-up, but I admit that I found the interweaving of human and AI-written content/headlines/summaries pretty distracting. I kept on wanting to scroll past, but had to keep on backtracking to find the human thread again.
I think if you want to give your reader a quick intro to, e.g., what is the Adam optimizer, a simple link to Wikipedia is fine. No need to copy-paste an AI tutorial on Adam into the blog post.
This throughput assumes 100% utilizations. A bunch of things raise the cost at scale:
- There are no on-demand GPUs at this scale. You have to rent them for multi-year contracts. So you have to lock in some number of GPUs for your maximum throughput (or some sufficiently high percentile), not your average throughput. Your peak throughput at west coast business hours is probably 2-3x higher than the throughput at tail hours (east coast morning, west coast evenings)
- GPUs are often regionally locked due to data processing issues + latency issues. Thus, it's difficult to utilize these GPUs overnight because Asia doesn't want their data sent to the US and the US doesn't want their data sent to Asia.
These two factors mean that GPU utilization comes in at 10-20%. Now, if you're a massive company that spends a lot of money on training new models, you could conceivably slot in RL inference or model training to happen in these off-peak hours, maximizing utilization.
But for those companies purely specializing in inference, I would _not_ assume that these 90% margins are real. I would guess that even when it seems "10x cheaper", you're only seeing margins of 50%.
Just like the thesis of the slime mold talk, the talk itself is a result of the slime mold and doesn't particularly represent any company-authoritative take on the situation.
As a serial successful field-hopper, I agree that I'm not the right person to be making these estimates.
But the external view is that college courses roughly expect you to do what I'm claiming, in roughly the time investment that I'm claiming -- and undergrads are typically in 4+ classes at a time. So is it that the whole educational system is delusional? (I fully acknowledge it might be so!)