The Anatomy of an Instruction Pipeline Hazardhiraditya.github.io7 points·by somnial·8 ore fa·0 comments
Width vs. Depth: Speculating on the Marginblog.doubleword.ai17 points·by somnial·12 giorni fa·1 comments
Pushing memory bound CUDA kernels past the speed of light with data compressionfergusfinn.com2 points·by somnial·2 mesi fa·0 comments
Speculative KV coding: ~4× losslessly compressed KV cache using a small modelfergusfinn.com2 points·by somnial·2 mesi fa·0 comments
LLM powered data structures: A lock-free binary search treefergusfinn.com1 points·by somnial·6 mesi fa·0 comments
somnial·17 giorni fa·discussthis is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
somnial·mese scorso·discusshttps://fergusfinn.com/blog/economics-of-speculative-decodin...good point tho - plus for Deepseek the shared expert increases the overlap slightly
somnial·mese scorso·discusstrue, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches