I run a proofreading benchmark that tests how well models can find and fix errors in English text. They get several passes in a simple agent loop. Sonnet 5 is definitely better than Sonnet 4.6, but inferior on both quality and cost to GLM 5.1, GLM 5.2, Gemini 3.1 Flash, and Gemini 3.1 Pro. https://revise.io/errata-bench
Trivial to simulate basic keystrokes. But I don't think it's trivial to simulate the natural process of drafting something. There's no concrete heuristic or algorithm (yet) for judging these types of replays, but I'd be impressed if someone can actually make a program that reliably simulates a natural keystroke-by-keystroke thought process which appears human when replayed in this way.
I agree. I have gotten an incredible amount of work done iterating with 5-30 minute long agent tasks. But it requires I stay engaged, and not go chill on the beach, which I guess is a lot of agentmaxxers’ goal.
Yeah I don’t really see the backpressure analogy here - it implies that the agent is constantly producing new stuff, which isn’t really possible since the solution is very detailed specs/goals.
am I the only one who always replies to people manually? I don't think I've ever done the "send back a chatGPT screenshot" or copy paste a response from chatGPT to a message I know was from a human.
I'm someone who is trying to build a subscription-based business to cover underlying LLM costs, and very hopeful I can one day just sell a permanent license to the software instead with customers using local LLMs to power it.
https://art.cx
https://revise.io