I packed 16 GB of GGUF quants into 1.8 GB, losslessly: every file comes back bit-identical
A publisher ships 15-25 quant variants of a model, and every single one is a deterministic function of one F16 source — run `llama-quantize` with a given type (plus an imatrix for k-quants) and you get that exact file back. So storing all 15-25 near-duplicate blobs is wasteful. Store the F16 once, plus a tiny recipe per file and a small correction delta, and regenerate on demand.
I ran it on a real published repo (`bartowski/Llama-3.2-1B-Instruct-GGUF`, llama.cpp `b3821`): 19 files, 16.0 GB -> 1.8 GB, 8.7x, manifest 12.4 KB. Then I deleted the originals and regenerated all 17 quants from the pack: 17/17 sha256 identical to the original HF files, in 283 seconds.
It's lossless by construction. Each file is stored as EXACT (recipe only), NEAR (recipe + a zstd delta), or a plain zstd blob if it can't be regenerated. Every plan is actually executed and hash-checked at pack time before it's recorded, and on unpack the output is sha256-verified and refused on any mismatch. So it can't silently hand you a wrong file.
While building this I hit something I didn't expect: GGUF quantization isn't reproducible across machines. Same F16, same imatrix, same tag, same quant type — a Linux/x86_64 build and a macOS/arm64 build differed in 113/147 tensors (0.196% of bytes). It's perfectly deterministic locally; it just diverges across machines. I traced it to FP contraction (FMA) in the k-quant scale-search loops, and one build flag (`-ffp-contract=off`) makes quantization bit-identical across OS/arch/compiler — proven on public CI. That's literally why the correction deltas exist: they patch the gap between the machine that built the published file and yours. Upstream PR: [llama.cpp PR link placeholder]. Evidence repo: github.com/theadamdanielsson/gguf-quant-determinism.
Honest limitations, up front: packs are machine/build-scoped in v0 (portable packs wait on that upstream deterministic mode), the F16 has to be in the directory, and regeneration isn't free — roughly ~17s per file on an M-series laptop. Validated bit-exact on 40/40 files across two families so far (Llama-3.2-1B and Qwen2.5-1.5B); bigger models should behave the same but I haven't measured them yet.
A publisher ships 15-25 quant variants of a model, and every single one is a deterministic function of one F16 source — run `llama-quantize` with a given type (plus an imatrix for k-quants) and you get that exact file back. So storing all 15-25 near-duplicate blobs is wasteful. Store the F16 once, plus a tiny recipe per file and a small correction delta, and regenerate on demand.
I ran it on a real published repo (`bartowski/Llama-3.2-1B-Instruct-GGUF`, llama.cpp `b3821`): 19 files, 16.0 GB -> 1.8 GB, 8.7x, manifest 12.4 KB. Then I deleted the originals and regenerated all 17 quants from the pack: 17/17 sha256 identical to the original HF files, in 283 seconds.
It's lossless by construction. Each file is stored as EXACT (recipe only), NEAR (recipe + a zstd delta), or a plain zstd blob if it can't be regenerated. Every plan is actually executed and hash-checked at pack time before it's recorded, and on unpack the output is sha256-verified and refused on any mismatch. So it can't silently hand you a wrong file.
While building this I hit something I didn't expect: GGUF quantization isn't reproducible across machines. Same F16, same imatrix, same tag, same quant type — a Linux/x86_64 build and a macOS/arm64 build differed in 113/147 tensors (0.196% of bytes). It's perfectly deterministic locally; it just diverges across machines. I traced it to FP contraction (FMA) in the k-quant scale-search loops, and one build flag (`-ffp-contract=off`) makes quantization bit-identical across OS/arch/compiler — proven on public CI. That's literally why the correction deltas exist: they patch the gap between the machine that built the published file and yours. Upstream PR: [llama.cpp PR link placeholder]. Evidence repo: github.com/theadamdanielsson/gguf-quant-determinism.
Honest limitations, up front: packs are machine/build-scoped in v0 (portable packs wait on that upstream deterministic mode), the F16 has to be in the directory, and regeneration isn't free — roughly ~17s per file on an M-series laptop. Validated bit-exact on 40/40 files across two families so far (Llama-3.2-1B and Qwen2.5-1.5B); bigger models should behave the same but I haven't measured them yet.
`pip install ggufpacker` (MIT). Repo: github.com/theadamdanielsson/ggufpacker.
Please try to break it: if you get a file back that doesn't match its original sha256, that's the bug I most want to hear about.