Interesting read! Creating tests is highlighted as something Claude did well, but it strikes me that all the weaker rejected solutions could have been avoided if it were really good at designing intelligent tests for itself. For example, the first solution “was very specific to the reported bug and wouldn’t have fixed the general case” and the third suggestion “prevented the perfectly valid use of as conversion expressions in go commands as well”. I imagine both of these cases could have been noticed and avoided by the agent if it had planned out adequate tests ahead of time.
The “correct”, elegant way for AI to interact with existing software would take decades and billions of dollars to build. Someone would have to do the hard work of building new APIs, solving decades of accessibility issues, etc.
Or you can show an AI screenshots and ask it where to click.
The team with the most star power and hype tends to attract the best young talent.
If the next big breakthrough in AI comes from Anthropic, good chance it comes from some genius you’ve never heard of who decided to work there because of [famous researcher].
I wish them all the best and hope they succeed, but can’t help but suspect they’ve fallen into deep LLM psychosis. Even if you assume they can build this thing and it works as described and then get past all the regulatory hurdles, the scale of infrastructure they’re talking about is enormous.
I actually think “explore Claude’s understanding of colors” is an interesting concept. A lot of fascinating cultural information gets compressed into LLMs.
Unfortunately for the people mad about this, I predict the only thing they will accomplish by pressuring the rsync maintainers, is to discourage everyone else from responsibly disclosing their use of AI. You’re just going to make people disable Claude attribution on their commits to avoid drama.
Somewhat useless article. To summarize, we have anecdotes suggesting they may work but no one has figured out how to prove or disprove it in a study, and the author has some doubts. Meanwhile supplements can be dangerous if you take too much or have a liver condition, or if you buy them from an unreputable source, as with every other substance on earth. Author confirms it tastes good in milk.
I was really unimpressed by the free Codex (for nodejs/react dev). I think it must be using a less powerful model or they’re limiting it in some other way.
You’re probably right in a literal technical sense, but a very large number of people (maybe most?) would choose “no” if properly informed and asked for consent, and lots of people are morally opposed even in principle to downloading a large AI model onto their computer. I’m not one of them, but they’re out there. So in a cultural sense, it is different.
The models are primitive right now, but we’re clearly heading toward “AI as sound synthesis, human as artist” - much like how producers currently use a DAW to assemble premade loops and sounds from Splice, but with the producer now able to prompt any sound, filter, or effect they can imagine into existence and then rearrange them into a song.
See for example Suno Studio, which is not very good in my opinion, but shows the direction they’re going.
I have the opposite experience: random HN/Reddit comments saying “this sucks” or “whoa this is a huge improvement” are the only benchmark that means anything. Standard benchmarks are all gamed and don’t capture the complexity of the real world.
Stars have been useless as signals for project quality for a while. They’re mostly bought, at this point. I regularly see obviously vibe-coded nonsense projects on GitHub’s Trending page with 10,000 stars. I don’t believe 10,000 people have even cloned the repo, much less gotten any personal value from it. It’s meaningless.
Ape thinking is a cognitive practice where a human deliberately solves problems with their own mind. Practitioners of ape thinking will typically author thoughts by thinking them with their own brain, using neurons and synapses.
The term was popularized when asking a computer to do it for you became the dominant form of cognition. "Ape thinking" first appeared in online communities as derogatory slang, referring to humans who were unable to outsource all their thinking to a computer. Despite the quick spread of asking a computer to do it for you, institutional inertia, affordability, and limitations in human complacency were barriers to universal adoption of the new technology.
Their design approach wasn’t particularly unusual, so I’m not sure what that sentence means.
I do miss the days when technical reports were clear and concise. This one has some interesting information, but it’s buried under a mountain of empty AI-written bloat.
> but the number of problems requiring deep creative solutions feels like it is diminishing rapidly.
If anything, we have more intractable problems needing deep creative solutions than ever before. People are dying as I write this. We’ve got mass displacement, poverty, polarization in politics. The education and healthcare systems are broken. Climate change marches on. Not to mention the social consequences of new technologies like AI (including the ones discussed in this post) that frankly no one knows what to do about.
The solution is indeed to work on bigger problems. If you can’t find any, look harder.
I’m honestly surprised LLMs are still screwing up citations. It does not feel like a harder task than building software or generating novel math proofs. In both those cases, of course, there is a verifier, but self-verification with “Does this text support this claim?” seems like it ought to be within the capabilities of a good reasoning model.
But as I understand the situation, even the major Deep Research systems still have this issue.