Microbatch size is a hyperparameter, it can be set to 1 and work just as effectively. With gradient accumulation it's equivalent even. Large batch sizes are used to increase parallelism, and sometimes to reduce variance in the loss signal (at the cost of increased bias).
Batch size is frequently limited by compute bottlenecks well before memory.
No one wants to be around an alcoholic. But it's not to be nice, but to understand them, their struggles, and their perspectives.
Anecdote - I was at a concert with my brother, outside in the smoking area. A homeless woman came by, didn't say anything but was kinda just looking around. My brother is the only one who spoke to her, he offered his cigarette butt. Which is exactly what she was looking for.
Never would have occured to me- but I would guess it made her feel more seen and human.
He wasn't the only one who saw the problem computationally. Famously, the mathematician Michael Waterman sat on the other-side of the race for the human genome.
not really. early deep learning models were run on single consumer-grade GPUs. the inflection occured _right_ when parallel computing became fast enough to do backprop in a reasonable amount of time with performance better than tree methods.
at that time all the compute resources in the world would not have been enough to train the models from even the last ~6 years or so, probably more.
3x increase in compute for a 1.5x increase in tdp is pretty good considering the underlying process had barely changed. In anycase, consumer GPUs aren't a good metric as they operate with different economic constraints.
H100 to GB200 saw a 50x increase in efficiency, for example.
You make excellent points -- the point related to culture is the most common I've heard from people outside Portland (particularly non-white people).
I think the only things you omitted are the differences in weather and the size of the cities.
- Portland is a fairly small city (especially inner portland) with a long history of anti-growth sentiment. The result is very rural agricultural areas _just_ outside of Portland.
- The weather patterns bring different wildlife and kinds of beauty. While the forests outside SF are quite pretty, they are also very different than the wet douglas fir forests up north. Beauty is in the eye of the beholder -- Portland also extends into the hills and provides famous views of Mt. Hood from the SW hills. The portland zoo for instance, is like going from a city into a rain forest.
The oregon coast is cold and rainy 95% of the year, not the most fun place to visit.
agreed -- portland is right there with very similar recreational activities and literally 25% the rent. why rent a 1b1b in SF when you can rent a mansion right on washington park for the same price?
Batch size is frequently limited by compute bottlenecks well before memory.