Pretty cool, rendering PowerPoint files to an image is probably the only way for LLMs to make sense of them.
Does this work in Cloudflare’s workerd environment? Would be nice to have a cheap serverless render -> LLM (GLM-OCR / PaddleOCR) -> Markdown pipeline for the various MS Office formats.
> While OpenAI originally pioneered Codex (which went on to power GitHub Copilot), Google’s direct answer for dedicated, native code completion and natural-language-to-code generation is CodeGemma.
If you give every agent an isolated container to use, you’re going to be paying for the reserved memory while the container is active, even if the agent isn’t doing anything.
Wow, that’s very interesting. I wish more benchmarks were reported along with the total cost of running that benchmark. Dollars per token is kind of useless for the reasons you mentioned.
Lots of comments about the price change, but Artifical Analysis reports that 3.1 Flash-Lite (reasoning) used fewer than half of the tokens of 2.5 Flash-Lite (reasoning).
This will likely bring the cost below 2.5 flash-lite for many tasks (depends on the ratio of input to output tokens).
That said, AA also reports that 3.1 FL was 20% more expensive to run for their complete Intelligence index benchmark.
The overall point is that cost is extremely task-dependent, and it doesn’t work to just measure token cost because reasoning can burn so many tokens, reasoning token usage varies by both task and model, and similarly the input/output ratios vary by task.
Very interesting blog post. I’d never seen that method for quickly computing the patterns. I thought I had done a lot of research on bloom filters, too!
Yeah, I agree with this. I think there are open addressing hash tables like Swiss Table that do something similar. IIRC, they have buckets with a portion at the beginning with lossy “fingerprints” of items, which kind of serve a similar purpose as a bloom filter.
This article is a little confusing. I think this is a roundabout way to invent the blocked bloom filter with k=2 bits inserted per element.
It seems like the authors wanted to use a single hash for performance (?). Maybe they correctly determined that naive Bloom filters have poor cache locality and reinvented block bloom filters from there.
Overall, I think block bloom filters should be the default most people reach for. They completely solve the cache locality issues (single cache miss per element lookup), and they sacrifice only like 10–15% space increase to do it. I had a simple implementation running at something like 20ns per query with maybe k=9. It would be about 9x that for native Bloom filters.
There’s some discussion in the article about using a single hash to come up with various indexing locations, but it’s simpler to just think of block bloom filters as:
1. Hash-0 gets you the block index
2. Hash-1 through hash-k get you the bits inside the block
If your implementation slices up a single hash to divide it into multiple smaller hashes, that’s fine.
I wonder if the “spawn” API is ever preferable over “fork”. Do we really want to remove context if we can help it? There will certainly be situations where we have to, but then what you want is good compaction for the subagent. “Clean-slate” compaction seems like it would always be suboptimal.
That's a good find, but I was unsure of whether "program" is semantically equivalent to "code" here. Plus I'm tempted to leave codex since it sounds so good.
It's been seven years since I've tried writing any Latin, so you should assume this is butchered. (edit: I think it's less butchered now)
codex bonus a discipulo prendatur
codex magnus a novo prendatur
codex optimus nullus est
Part of the problem is I couldn't find any good word for "code". "Codex" sounds cool but may not be the best fit here.
EDIT: Forgot a word. Also, I think "prendere" is better for "understood" here than "scire", which is more like "to know".
EDIT2: My friend suggested using the subjunctive for "comprehend" so that it's "may be comprehended" instead of "is comprehended". Also I got the tense wrong initially and I think that's fixed now.
EDIT3: "Ablative agents" are a thing. This is a rough language. Thanks, James.
EDIT4: prendar -> prendatur; aka "oops, should have used third person"
Does this work in Cloudflare’s workerd environment? Would be nice to have a cheap serverless render -> LLM (GLM-OCR / PaddleOCR) -> Markdown pipeline for the various MS Office formats.