Show HN: TurboQuant for mlx-lm (Apple Silicon)(github.com)
github.com
Show HN: TurboQuant for mlx-lm (Apple Silicon)
https://github.com/pythongiant/mlx_turboquant
1 comments
Hey guys, i'm especially interested in feedback on the kernel design and integration onto mlx lm.
Its on pypi as well as a simple pip install mlx-turboquant :P
Its on pypi as well as a simple pip install mlx-turboquant :P
I built mlx-turboquant, an implementation of Google's TurboQuant KV-cache compression algorithm for Apple's MLX framework.
The repository includes quality benchmarks, memory benchmarks, and a modular implementation so individual pieces (PolarQuant, QJL, packing, codebooks) can be studied independently.