I run Llama 3.2 3B locally for latency-sensitive classification (sub-50ms, so no room for bigger models). At that scale Q2_K vs Q4_K_M isn't just smaller — Q2 starts flipping yes/no answers that Q4 gets right. Not often, but enough to notice in production.
So the KL divergence numbers here are more useful to me than the MMLU tables honestly. I've had MMLU hold steady while the output distribution drifted enough to break things downstream.
Does the calibration dataset make much difference at 3B though? There's so little redundancy that I'd expect it to hit a floor pretty fast regardless of how good the calibration data is.
This is great timing — I've been putting off making animated diagrams for a blog post because the Manim setup was too much friction for what I needed.
Tried the live demo and the 3D orbit scene is surprisingly smooth. Curious about a couple things:
- How are you handling the animation interpolation? Manim's rate functions (smooth, there_and_back, etc.) have some quirks that are easy to get subtly wrong. Did you reimplement those from scratch or find a way to match the Python easing curves exactly?
- For the py2ts converter — how far does it get on real-world scripts? I have a few older Manim CE scripts with custom VMobjects and I'm wondering if it handles subclassing or if it's more of a "simple scenes only" thing.
One suggestion: it'd be really useful to have an export-to-GIF or export-to-MP4 option directly in the browser (maybe via MediaRecorder API). A lot of the Manim use case is generating assets for slides/posts, not just live playback.
The deduplication problem is the part I haven't worked out cleanly. The hierarchy in veronica-core sidesteps it as long as you declare parent-child relationships upfront — B's spend rolls directly into A's ceiling without a separate aggregation step. But in a dynamic pipeline where you don't know the call graph until runtime, that assumption breaks.
The fire-and-forget tradeoff makes sense. I went with blocking enforcement because the original use case was preventing runaway agents, not auditing after the fact. For RAG you're probably right that eventual consistency is the better fit — you care more about the trace than cutting off a half-finished response.
The flip side is enumerable IDs. Back when I was scraping a site for a side project, sequential photo IDs were basically a free sitemap. YouTube's random-ish IDs aren't just branding — they at least make bulk harvesting annoying.
.git/hooks is underrated. I have a pre-push hook that runs my test suite — annoying to set up the first time but I've probably avoided a dozen broken CI runs by now.
So the KL divergence numbers here are more useful to me than the MMLU tables honestly. I've had MMLU hold steady while the output distribution drifted enough to break things downstream.
Does the calibration dataset make much difference at 3B though? There's so little redundancy that I'd expect it to hit a floor pretty fast regardless of how good the calibration data is.