SpaceX has almost finished writing v1.0 of an in-house AI training stack in C(xcancel.com)
xcancel.com
SpaceX has almost finished writing v1.0 of an in-house AI training stack in C
https://xcancel.com/elonmusk/status/2059884150187053488
2 comments
C is next to godliness.
> SpaceX has almost finished writing V1.0 of an in-house AI training stack in C that exact-maps to 220k GB300s with 800G NICs, making heavy use of pipeline parallelism and getting as close to bare metal as possible.
> The potential speed improvement vs JAX for large training runs is over an order of magnitude