> *2-bit quantization produces \name\ instead of "name" in JSON output, making tool calling unreliable.
I was wondering about that statement. Shouldn't it restrict sampling to only tokens that produce valid JSON matching the schema during a tool call? On the other hand, I have heard a lot about how even production LLM providers don't always call tools accurately, so I suppose either it's hard to implement what I described or there's something I haven't thought of that makes it impossible.
You would think, but the initial implementation of this stripped ICC profiles (https://github.com/oven-sh/bun/issues/30197), and while that was fixed in time for the release, it still currently doesn't understand EXIF rotation metadata (https://github.com/oven-sh/bun/issues/30235) or high-bit-depth images (https://github.com/oven-sh/bun/issues/30462)