Full disclosure, I worked briefly with Ello / Catalin some years back.
They've done a lot of work on at least two dimensions: (1) handling the nonstandard sounds and habits of speech of very early readers, who might be as young as four, and (2) connecting this to a specialized teaching system based on the science of reading, e.g. decodable readers.
The explanation I've heard is that the national curriculum switched away from phonics and to a teaching method called "balanced literacy", and this went terribly [1]. IIRC it involved teaching kids to recognize words by their overall visual shape. I believe this is what mature readers do, but the problem is that for this to work, one first needs to have bootstrapped a robust vocabulary. And that happens via being able to sound out words, i.e. phonics.
(I'm not sure whether Cato is generally reliable, but FYI there are lots of other writeups online on the same topic. It was the first non-paywalled and reasonably complete one I found)
Full disclosure: I worked on a small project with Ello / Catalin a few years ago.
At the time of writing, the sentiment in this post is that this is a terrible idea, and that kids need human tutors. The latter is 100% true. But also, you might want to know some facts about the state of children's literacy in the US (Ello is a math and reading tutor):
1. We're in crisis. As of 2025, 40% of fourth graders are reading below basic levels [1].
2. There's a massive teacher shortage. 2025 US state data shows ~400k teacher positions either unfilled or underqualified [2] – over 10% of the workforce.
3. Bloom's 2-sigma shows that 1-1 tutoring delivers outcomes at the 90th percentile of classroom teaching. Early research is finding that AI can deliver some of this benefit [3].
4. This can't always be solved by parents: 54% of US adults have a literacy below a 6th-grade level, and 20% are below 5th-grade level [4].
At Ello, I heard stories of children figuring out they were behind at school, and when given the app, they holed themselves up in their room and used it to get themselves caught up. And then they could read! Can you imagine falling behind at this critical juncture, and being stuck illiterate while your friends grow past you? We're currently setting kids up for lives of shame and deprivation.
My take: this really is a life-changing technology. And we need this problem solved. Democracy doesn't function without an educated populace.
Hello! I'm one of the main authors of the paper. Thanks for engaging with our work so thoughtfully – that's a very clear and valid question.
We didn't get around to addressing this within the paper itself – 80 pages is a lot, and deadlines, etc. But I have unpublished experiments that show that in a reasonably broad setting I'm doing some work in, verbalized probabilities are restoring a distribution that looks almost identical to the base distribution. It is not possible to demonstrate this on frontier models, since their public models are already mode-collapsed, and they don't share the base model or logprobs anyway. But I've established this to my personal satisfaction on large local models which offer base / post-trained pairs.
To share some intuition on why one might believe this is occurring: there are a bunch of tasks implicit in the pre-training corpus that encourage the model to learn this capability. Consider sentences in news and research articles like: "Scientists discover that [doing something] increases [some outcome] on [some population] by X%". It seems quite natural that the model might learn a pathway by which it can translate its base probabilities into the equivalent numeric tokens in order to "beat" the task of reducing loss on the "X%" prediction. I can even almost visualize how this works mechanically in terms of what the upper layers of an MLP would do to learn this, i.e. translating from weights into specific token slots. And this is almost certainly more parameter-efficient than constructing an entire separate emulated reality for filling in X. Although I'm not ruling out that the latter might still be happening – perhaps some future interp research might be able to validate this!
I'm actually working on a paper that packs up some of the above findings in passing. But if helpful in the meantime, this is also building on related work by Tian et al. 2023, "Just Ask for Calibration" [1] and Meister et al. 2024, "Benchmarking Distributional Alignment of LLMs" [2], that give some extra confidence here. Their findings indicate that whether or not they are rooted in the model's base probabilities, they seem to be useful for the purposes that people care about. (Oh, and you can probably set up an experiment to verify this independently with vLLM in a few Claude Code requests!)
Hope that was helpful – feel free to ping with follow-ups! (Although replies might be a little delayed, I happened to see this at a good time; having quite a crunchy week)