The annotated PyTorch training loopidlemachines.co.uk81 points·by smaddrellmander·20 วันที่ผ่านมา·19 comments
Mae vs. MSE: more than just the mean vs. median debateidlemachines.co.uk1 points·by smaddrellmander·24 วันที่ผ่านมา·0 comments
DiffusionGemma: Discrete diffusion in a large language modelidlemachines.co.uk3 points·by smaddrellmander·เดือนที่แล้ว·1 comments
Reading MAI's efficiency gain. How to pick architectures like serious peopleidlemachines.co.uk9 points·by smaddrellmander·เดือนที่แล้ว·0 comments
MAI-Thinking-1: Building a Hill-Climbing Machine [pdf]microsoft.ai3 points·by smaddrellmander·เดือนที่แล้ว·0 comments
Are contrastive losses just cross entropy all along?idlemachines.co.uk2 points·by smaddrellmander·เดือนที่แล้ว·0 comments
Every token, everywhere, all at onceidlemachines.co.uk2 points·by smaddrellmander·2 เดือนที่ผ่านมา·0 comments
The cut in the Mixture of Experts compute graphidlemachines.co.uk1 points·by smaddrellmander·2 เดือนที่ผ่านมา·0 comments
Softmax, can you derive the Jacobian? And should you care?idlemachines.co.uk134 points·by smaddrellmander·3 เดือนที่ผ่านมา·47 comments
Gemma 4 is not your standard transformeridlemachines.co.uk2 points·by smaddrellmander·3 เดือนที่ผ่านมา·0 comments
smaddrellmander·4 วันที่ผ่านมา·discussLove this! Cute graphics and the way the balls combine is really fun. Great job.
smaddrellmander·2 เดือนที่ผ่านมา·discussA similar vein, asking why in PRs. Most people don’t do the above, and asking why when it doesn’t make sense really helps understanding what your colleagues are actually thinking. Helps create a unified perspective on the codebase.