Don't Build a Router. Train the Small Model to Know When to Deferdistillabs.ai2 points·by kkm·11 hari yang lalu·1 comments
The gap between open weights LLMs and closed source LLMsblog.doubleword.ai306 points·by kkm·15 hari yang lalu·250 comments
2678x Faster Matrix Multiplication with a GPU0mean1sigma.com2 points·by kkm·25 hari yang lalu·0 comments
UCCL-EP: DeepEP-style expert parallelism on any NIC, no GPU-initiated commsfergusfinn.com9 points·by kkm·26 hari yang lalu·0 comments
No Token Left Behind: Demystifying Token-in-Token-Out in Mileslmsys.org2 points·by kkm·bulan lalu·0 comments
MoE expert co-activations: Reordering inputs yields easy throughput gainsblog.doubleword.ai2 points·by kkm·bulan lalu·0 comments
Speculative KV coding: losslessly compressing KV cache by up to ~4×fergusfinn.com155 points·by kkm·bulan lalu·48 comments
Brave AI privacy:LLMs on NEAR AI Nvidia-Backed Trusted Execution Environmentsbrave.com1 points·by kkm·8 bulan yang lalu·0 comments
kkm·bulan lalu·discussAlso the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword