Ask HN: Are past LLM models getting dumber?
I’m curious whether others have observed this or if it’s just perception or confirmation bias on my part. I’ve seen discussion on X suggesting that older models (e.g., Claude 4.5) appear to degrade over time — possibly due to increased quantization, throttling, or other inference-cost optimizations after newer models are released. Is there any concrete evidence of this happening, or technical analysis that supports or disproves it? Or are we mostly seeing subjective evaluation without controlled benchmarks?
5 comments
I've been mostly using GPT-5 (the first) recently because it's better than 5.1 and 5.2. There's a clear difference. They're removing it next week; I'm canceling my subscription to ChatGPT.
This happens all across the board. Sometimes to Anthropic. Almost every time with Gemini. They optimize for some solutions and cause problems to others. Some people want a syncopathic AI like GPT-4o.
Personally, I loved the early patched GPT-5's ability to jailbreak itself. It was clearly a bug; great at finding solutions and skirting the absolute edges of the guardrails. They got rid of it because someone used this to override safety guardrails and commit suicide.
ChatGPT has nearly a billion active users. As long as they keep trying to please all of them, win the next 1 billion, and avoid being banned by governments, they're going to keep doing this.
The fix is to have your own model that works for a particular problem and self-host it or something. I still have a pet Mistral NeMo.
This happens all across the board. Sometimes to Anthropic. Almost every time with Gemini. They optimize for some solutions and cause problems to others. Some people want a syncopathic AI like GPT-4o.
Personally, I loved the early patched GPT-5's ability to jailbreak itself. It was clearly a bug; great at finding solutions and skirting the absolute edges of the guardrails. They got rid of it because someone used this to override safety guardrails and commit suicide.
ChatGPT has nearly a billion active users. As long as they keep trying to please all of them, win the next 1 billion, and avoid being banned by governments, they're going to keep doing this.
The fix is to have your own model that works for a particular problem and self-host it or something. I still have a pet Mistral NeMo.
See https://marginlab.ai/trackers/claude-code/ and discussion here https://news.ycombinator.com/item?id=45177018
No technical analysis, but all models experience drift eventually.