Don't Build a Router. Train the Small Model to Know When to Deferdistillabs.ai2 points·by kkm·il y a 11 jours·1 comments
The gap between open weights LLMs and closed source LLMsblog.doubleword.ai306 points·by kkm·il y a 15 jours·250 comments
2678x Faster Matrix Multiplication with a GPU0mean1sigma.com2 points·by kkm·il y a 24 jours·0 comments
UCCL-EP: DeepEP-style expert parallelism on any NIC, no GPU-initiated commsfergusfinn.com9 points·by kkm·il y a 26 jours·0 comments
No Token Left Behind: Demystifying Token-in-Token-Out in Mileslmsys.org2 points·by kkm·le mois dernier·0 comments
MoE expert co-activations: Reordering inputs yields easy throughput gainsblog.doubleword.ai2 points·by kkm·le mois dernier·0 comments
Speculative KV coding: losslessly compressing KV cache by up to ~4×fergusfinn.com155 points·by kkm·le mois dernier·48 comments
Bringing Up DeepSeek-V4-Flash on AMD MI300Xfergusfinn.com120 points·by kkm·le mois dernier·25 comments
Brave AI privacy:LLMs on NEAR AI Nvidia-Backed Trusted Execution Environmentsbrave.com1 points·by kkm·il y a 8 mois·0 comments
kkm·le mois dernier·discussAlso the vllm patch accompanying the blogpost: https://github.com/doublewordai/vllm-amd-blog-doubleword