From the chat APIs alone, we can extract two useful oracles:
Token length oracle: Given any string s, return len(tokenize(s)).
Prefix token oracle: Given a string s and integer n, return the string decoded from the first n tokens of tokenize(s).
The first oracle (token count) is straightforward. The second oracle's main purpose is to resolve ambiguities in merge order. For example, without it we couldn’t distinguish between the segmentations (a, bc) and (ab, c). By leveraging the model’s ability to repeat text in a controlled way, we can construct this second oracle from the regular chat interface.
With only these two oracles, is it possible to reconstruct an equivalent tokenizer — one that produces the exact same token boundaries/segmentation on any input (up to token ID remapping)?
From the chat APIs alone, we can extract two useful oracles:
Token length oracle: Given any string s, return len(tokenize(s)).
Prefix token oracle: Given a string s and integer n, return the string decoded from the first n tokens of tokenize(s).
The first oracle (token count) is straightforward. The second oracle's main purpose is to resolve ambiguities in merge order. For example, without it we couldn’t distinguish between the segmentations (a, bc) and (ab, c). By leveraging the model’s ability to repeat text in a controlled way, we can construct this second oracle from the regular chat interface.
With only these two oracles, is it possible to reconstruct an equivalent tokenizer — one that produces the exact same token boundaries/segmentation on any input (up to token ID remapping)?
I saw the earlier HN discussion about Claude’s encrypted reasoning blobs and found it interesting.
After some experiments, I think I may have found a way to get the original reasoning from signature.
I built a verification test. The idea is simple: you generate a random secret locally, make Claude place it only inside private reasoning, extract only the returned signature, and then submit only that signature. If the secret comes back while the visible answer never contained it, that is evidence that I am correctly recovering content from the encrypted CoT, rather than guessing or re-deriving it.
Another signal that makes me take this seriously: in most cases, without giving the recovery side any hint about the hidden reasoning, the recovered CoT has exactly the same length as the CoT reported by the API.
One of the motivations for open-sourcing this is exactly to see it grow beyond macOS. I personally don’t have much development experience on Windows or Linux, so it’s great to see people picking up the idea and trying it on other platforms.
Interestingly, the original spark for this project actually came from my dad. He mostly uses CAD to review architectural design files, and there are quite a few repetitive steps that are fairly mechanical.Many operations don’t seem to be accessible through normal shell automation and end up requiring GUI interactions.
So one of the next things I want to try is experimenting with similar ideas on Windows, especially for GUI-heavy workflows like that, and see how far it can go.
I really like the Claude Chrome extension, but unfortunately it has too many limitations. Not only is it restricted to Chrome, but even within Chrome some websites especially financial ones are blocked.
That’s true. The demo I showed was somewhat cherry-picked, and agentic systems themselves inherently introduce uncertainty. To address this, a possible approach was proposed earlier in this thread:
currently, after /teach is completed, we have an interactive discussion to refine the learned skill. In practice, this could likely be improved when the agent uses a learned skill and encounters errors, it could proactively request human help to point out the mistake. I think this could be an effective direction.
That sounds like a good idea. During the use of a skill, if the agent finds something unclear, it could proactively ask the user for clarification and update the skill accordingly. This seems like a very worthwhile direction to explore.
In the current system, I have implemented a periodic sweep over all sessions to identify completed tasks, cluster those tasks, and summarize the different solution paths within each cluster to extract a common path and proactively add it as a new skill. However, so far this process only adds new skills and does not update existing ones. Updating skills based on this feedback loop seems like something worth pursuing.
My own view is that the bigger long-term opportunity is actually Windows, simply because more desktop software and more professional workflows still live there. macOS-first here is mostly an implementation / iteration choice, not the thesis.
That’s exactly the hard part, and I agree it matters more than the happy path.
A few concrete things we do today:
1. It’s fully agentic rather than a fixed replay script. The model is prompted to treat GUI as one route among several, to prefer simpler / more reliable routes when available, and to switch routes or replan after repeated failures instead of brute-forcing the same path. In practice, we’ve also seen cases where, after GUI interaction becomes unreliable, the agent pivots to macOS-native scripting / AppleScript-style operations. I wouldn’t overclaim that path though: it works much better on native macOS surfaces than on arbitrary third-party apps.
2. GUI grounding has an explicit validation-and-retry path. Each action is grounded from a fresh screenshot, not stored coordinates. In the higher-risk path, the runtime does prediction, optional refinement, a simulated action overlay, and then validation; if validation rejects the candidate, that rejection feeds the next retry round. And if the target still can’t be grounded confidently, the runtime returns a structured `not_found` rather than pretending success.
3. The taught artifact has some built-in generalization. What gets published is not a coordinate recording but a three-layer abstraction: intent-level procedure, route options, and GUI replay hints as a last resort. The execution policy is adaptive by default, so the demonstration is evidence for the task, not the only valid tool sequence.
In practice, when things go wrong today, the system often gets much slower: it re-grounds, retries, and sometimes replans quite aggressively, and we definitely can’t guarantee that it will always recover to the correct end state. That’s also exactly the motivation for Layer 3 in the design: when the system does find a route / grounding pattern / recovery path that works, we want to remember that and reuse it later instead of rediscovering it from scratch every time.