razorguymania

2 anni fadiscuss

2 anni fadiscuss

3 anni fadiscuss

yes

3 anni fadiscuss

I think this post is output from the chatbot itself

3 anni fadiscuss

GPUs can't scale single user performance beyond a certain limit. You can throw 100s of GPUs at it but the latency will never be as good.

3 anni fadiscuss

Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.

3 anni fadiscuss

Its not using GPUs. Its using Groq's own HW - LPUs

comments