HackerTrans
TopNewTrendsCommentsPastAskShowJobs

immortal3

no profile record

Submissions

Improving LLM Inference with Continuous Batching: Orca Through Tinyorca

junupark.xyz
2 points·by immortal3·3 เดือนที่ผ่านมา·0 comments

[untitled]

1 points·by immortal3·4 เดือนที่ผ่านมา·0 comments

Bits-per-Byte (BPB): a tokenizer-agnostic way to measure LLMs

dipkumar.dev
1 points·by immortal3·9 เดือนที่ผ่านมา·0 comments

comments

immortal3
·9 เดือนที่ผ่านมา·discuss
There's another angle to this comparison. Groq and Cerebras use custom chips, but I'm not sure about Together. In this case, Together is sharing results based on the B200 GPU. Another important point is the accuracy of these speed-ups compared to the baseline model. It's known that such tricks reduce accuracy, but by how much? Kimi has already benchmarked several providers. https://x.com/Kimi_Moonshot/status/1976926483319763130