LLM from scratch, part 32k – Interventions: gradient accumulation(gilesthomas.com)

https://www.gilesthomas.com/2026/04/llm-from-scratch-32k-interventions-training-our-best-model-locally-gradient-accumulation