Yeah, I think so. So far, Claude Opus is the only model I found that doesn't fold under the minimal pressure and can push back, but still - push just a little bit harder and it's back to "appear productive and useful to the user". I don't even have an idea how to balance it in LLMs to keep their business alive :D
I had exactly this between two LLMs in my project. An evaluator model that was supposed to grade a coaching model's work. Except it could see the coach's notes, so it just... agreed with everything. Coach says "user improved on conciseness", next answer is shorter, evaluator says yep great progress. The answer was shorter because the question was easier lol.
I only caught it because I looked at actual score numbers after like 2 weeks of thinking everything was fine. Scores were completely flat the whole time.
Fix was dumb and obvious — just don't let the evaluator see anything the coach wrote. Only raw scores. Immediately started flagging stuff that wasn't working. Kinda wild that the default behavior for LLMs is to just validate whatever context they're given.
The examples in the article are all big scary wipes, But I think the more common damage is way smaller and harder to notice.
I've been using claude code daily for months and the worst thing that happened wasnt a wipe(yet). It needed to save an svg file so it created a /public/blog/ folder. Which meant Apache started serving that real directory instead of routing /blog. My blog just 404'd and I spent like an hour debugging before I figured it out. Nothing got deleted and it's not a permission problem, the agent just put a file in a place that made sense to it.
jai would help with the rm -rf cases for sure but this kind of thing is harder to catch because its not a permissions problem, the agent just doesn't know what a web server is.
The compounding booboos bit is the key insight here. Humans are a bottleneck and that bottleneck is actually load-bearing. You feel the pain of bad decisions slowly enough to course correct.
I've been building the same AI product for months - a coaching loop that persists across sessions. Every few weeks someone ships a "competitor" in a weekend. Feature list looks similar. The difference is everything that breaks when a real user comes back for session 3 or 4. Context drifts, scores stop calibrating, plans don't adapt. None of that shows up in a demo. You only find it after sitting in the same codebase for weeks, running real sessions, getting confused by your own data. That's the friction the post is talking about and I don't think you can skip it.
Raw audio is a cool idea! I've seen a similar approach in other domains, "keep the source of truth accessible so you can verify the AI output against it".
I wouldn't go as far as "no place in medicine" though. The Heidi scribe tool mentioned in the article is a good example, because in the end it's the doctor who reviews and signs off.
IMO the problem is AI doing the work with no human verification step, but I can 100% agree I don't want to have vibe-doctor for my next surgery/consult :D
One thing from practical experience - the quality gap between model sizes shows up in a way benchmarks don't capture. I have a system where a smaller model generates plans and a larger model can override them. On any single output they look comparable. The difference shows up 3-4 steps later — small model makes a decision that sounds reasonable but compounds into a bad plan. Perplexity won't catch that, KL divergence won't either. They both measure one prediction at a time.
The union rep gets it - people improvise when you cut their tools and then threaten discipline for improvising.
That memo is how you make staff hide things instead of asking for help.
The scarier part though is that LLM-written clinical notes probably look fine. That's the whole problem. I built a system where one AI was scoring another AI's work, and it kept giving high marks because the output read well. I had to make the scorer blind to the original coaching text before it started catching real issues. Now imagine that "reads well, isn't right" failure mode in clinical documentation.
Nobody's re-reading the phrasing until a patient outcome goes wrong.
Yeeah, LLMs are the perfect procrastination tool because they feel productive. You're "learning", you're "exploring", you're having this great conversation about the problem. And then you close the tab and realize you never actually wrote anything yourself.
The best procrastination device ever built because it validates you the entire time. Great post, even beyond the algorithms example.
Your "no compiler" rule on day 3 taught you more than the LLM did. The LLM made concepts click. But the binary search vanishing under interview stress proves that understanding something and being able to produce it under pressure are totally different skills. Nobody talks about this enough in the "just use ChatGPT to learn" discourse.
I wrote this after going through Fabric HQ's data. The 61% stat surprised me - the cheating actually works, at least for now. Happy to discuss the implications.
I built this because it's easier for me to treat interview & job search as a math game, and math needs numbers(lol?). The original component works with voice, because engineers delivery collapse under interview pressure and it's hard to practice with just text.
The tool scores your answer on Structure, Completeness, Clarity, and
Conciseness (0-10 each), then gives you one specific fix. No signup required.
Built with Laravel + Vue + Claude Sonnet 4.6. The scoring rubric is visible on the page + OG image.
Looking for feedback on the scoring calibration especially. Does it feel
accurate to your experience?