The Anatomy of an Instruction Pipeline Hazardhiraditya.github.io5 points·by somnial·vor 3 Stunden·0 comments
Width vs. Depth: Speculating on the Marginblog.doubleword.ai17 points·by somnial·vor 12 Tagen·1 comments
Pushing memory bound CUDA kernels past the speed of light with data compressionfergusfinn.com2 points·by somnial·vor 2 Monaten·0 comments
Speculative KV coding: ~4× losslessly compressed KV cache using a small modelfergusfinn.com2 points·by somnial·vor 2 Monaten·0 comments
LLM powered data structures: A lock-free binary search treefergusfinn.com1 points·by somnial·vor 6 Monaten·0 comments
Parallel Primitives for Multi-Agent Workflowsfergusfinn.com1 points·by somnial·vor 6 Monaten·0 comments
somnial·vor 17 Tagen·discussthis is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
somnial·letzten Monat·discusshttps://fergusfinn.com/blog/economics-of-speculative-decodin...good point tho - plus for Deepseek the shared expert increases the overlap slightly
somnial·letzten Monat·discusstrue, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches