Tracing tokens through Llama 3.1 8B inference on H100s(krithik.xyz)
krithik.xyz
Tracing tokens through Llama 3.1 8B inference on H100s
https://krithik.xyz/what-is-inference-actually
1 comments
yeah, architecture and serving efficiency matter just as much as model quality once you leave the demo phase