Bonsai 8B: a 1-bit LLM that fits in 1.15GB(firethering.com)

4 pointsby steveharing13 miesiące temu1 comments

https://firethering.com/bonsai-8b-1bit-llm/

2 comments

lifecodes3 miesiące temu

If this holds, does it unlock 100B+ models running locally in ~tens of GB RAM? Or does accuracy collapse before that point?

steveharing13 miesiące temu