Pushing memory bound CUDA kernels past the speed of light with data compressionfergusfinn.com2 points·by somnial·2개월 전·0 comments
Speculative KV coding: ~4× losslessly compressed KV cache using a small modelfergusfinn.com2 points·by somnial·2개월 전·0 comments
LLM powered data structures: A lock-free binary search treefergusfinn.com1 points·by somnial·6개월 전·0 comments
somnial·17일 전·discussthis is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
somnial·지난달·discusshttps://fergusfinn.com/blog/economics-of-speculative-decodin...good point tho - plus for Deepseek the shared expert increases the overlap slightly
somnial·지난달·discusstrue, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches