The Anatomy of an Instruction Pipeline Hazardhiraditya.github.io5 points·by somnial·قبل 3 ساعات·0 comments
Width vs. Depth: Speculating on the Marginblog.doubleword.ai17 points·by somnial·قبل 12 يومًا·1 comments
Pushing memory bound CUDA kernels past the speed of light with data compressionfergusfinn.com2 points·by somnial·قبل شهرين·0 comments
Speculative KV coding: ~4× losslessly compressed KV cache using a small modelfergusfinn.com2 points·by somnial·قبل شهرين·0 comments
LLM powered data structures: A lock-free binary search treefergusfinn.com1 points·by somnial·قبل 6 أشهر·0 comments
somnial·قبل 17 يومًا·discussthis is a blog post from a company that hosts open weights LLMs (https://www.doubleword.ai/). I think its possible it might have been tongue in cheek
somnial·الشهر الماضي·discusshttps://fergusfinn.com/blog/economics-of-speculative-decodin...good point tho - plus for Deepseek the shared expert increases the overlap slightly
somnial·الشهر الماضي·discusstrue, but no reason the predictor model couldn't use linear attention (i.e. mamba, GDN etc) to predict KV caches