256K context with 72MiB of KV cache on the GPU(github.com)
github.com
256K context with 72MiB of KV cache on the GPU
https://github.com/Luce-Org/lucebox-hub/tree/main/optimizations/kvflash
https://github.com/Luce-Org/lucebox-hub/tree/main/optimizations/kvflash