Running LLMs with 3.3M Context Tokens on a Single GPU(arxiv.org)
arxiv.org
Running LLMs with 3.3M Context Tokens on a Single GPU
https://arxiv.org/abs/2410.10819
3 comments
Their demo looks really cool: https://github.com/mit-han-lab/duo-attention
[deleted]
[deleted]