TurboQuant: Redefining AI efficiency with extreme compression(research.google)
research.google
TurboQuant: Redefining AI efficiency with extreme compression
https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/
1 comments
This looks great, but I'm wondering how effective this would be for full model weights rather than just the KV cache. Their paper only gives results for the KV cache use case, which strikes me as strange since the algos are claimed to be near optimal.