cerebras: 450 tokens/sec llama 3.1 70B(theregister.com)
theregister.com
cerebras: 450 tokens/sec llama 3.1 70B
https://www.theregister.com/2024/08/27/cerebras_ai_inference/
2 comments
- 1,800tps on llama 3.1 8B
- 450tps on llama 3.1 70B
free chat interface is at: https://inference.cerebras.ai (requires login)
- 450tps on llama 3.1 70B
free chat interface is at: https://inference.cerebras.ai (requires login)
Going to be interesting to see the speed and accuracy keep increasing, cant imagine how fast/accurate things will be in a decade. Cant wait.