"Nerf"ing models isn't real in the vast majority of reported cases. Benchmarks like this or the 100 other "let's see if nerfing is real" copies would have shown it by now if it was.
I made a graphic to explain why people feel like the models get nerfed:
The idea is that new models can handle up to a certain level of complexity, at which point they fall apart. Every new model can handle more complexity, so there's a wonderful time upon release when you feel like you can do anything, only for you to hit the complexity ceiling a few days later when you saturate it. Rinse and repeat for the next model.
Anthropic has admitted to nerfing in the past. There have also been inference bugs. On top of that, model performance changes as they move compute to schwaggier providers as well.
Two postmortems, neither quite "admitted to nerfing":
Sept 2025, infra bugs: "A small percentage of Claude Sonnet 4 requests experienced degraded output quality" [0], alongside "We never reduce model quality due to demand, time of day, or server load." [1]
April 2026, Claude Code: default reasoning effort was lowered from high to medium, plus a caching bug and a verbosity prompt. Per Anthropic, "The models themselves didn't regress, and the Claude API was not affected." [2]
So users were right that quality dropped, but the confirmed causes were bugs and a product default, not deliberate model degradation.
I mean there is a direct link two comments down from here from 30 minutes before your comment: <a href="https://news.ycombinator.com/item?id=49902477">https://news.ycombinator.com/item?id=49902477
I’m typing from my phone and im not going to review the semantics of Anthropic’s storied history of performance issues.
It’s not just ant. There are so many small knobs that providers can claim isn’t nerfing but “load management” or “improving user experience”. One example from OpenAI is reducing juice to reduce time to first token.
There was this:
<a href="https://www.reddit.com/r/Anthropic/comments/1sl5wfh/the_degradation_of_claude_opus_46_people_are/" rel="nofollow">https://www.reddit.com/r/Anthropic/comments/1sl5wfh/the_degr...
does it really matter wherever its the harness or the model for the vibe coder user claude code?
fwiw, i think almost all regressions are down to a/b testing in the harness by anthropic, but it is objectively indistinguishable beyond "the coding agent ceases to be usable" and i'm back to traditional coding for a few hours until its back to normal again
It absolutely matters because something like Claude Code has no guarantee that there won't be changes between updates but a model pinned at the API version level that is getting enterprise traffic absolutely does have that guarantee and would be a much more widespread problem...
Yes. The agent allows you to switch models, so you could sidestep a bug in one model by temporarily using other models.
The /r/antigravity SubReddit is full of users who very much notice bugs with the tool/agent. We should be thankful that Claude Code is pretty stable by comparison.
johnfn · · focus · HN ↗
I made a graphic to explain why people feel like the models get nerfed:
<a href="https://x.com/thesilenceturns/status/2103551351825543610" rel="nofollow">https://x.com/thesilenceturns/status/2103551351825543610
The idea is that new models can handle up to a certain level of complexity, at which point they fall apart. Every new model can handle more complexity, so there's a wonderful time upon release when you feel like you can do anything, only for you to hit the complexity ceiling a few days later when you saturate it. Rinse and repeat for the next model.
prodigycorp · · focus · HN ↗
Anthropic has admitted to nerfing in the past. There have also been inference bugs. On top of that, model performance changes as they move compute to schwaggier providers as well.
Your chart is wrong.
simonw · · focus · HN ↗
Where?
prodigycorp · · focus · HN ↗
consumer451 · · focus · HN ↗
Two postmortems, neither quite "admitted to nerfing":
Sept 2025, infra bugs: "A small percentage of Claude Sonnet 4 requests experienced degraded output quality" [0], alongside "We never reduce model quality due to demand, time of day, or server load." [1]
April 2026, Claude Code: default reasoning effort was lowered from high to medium, plus a caching bug and a verbosity prompt. Per Anthropic, "The models themselves didn't regress, and the Claude API was not affected." [2]
So users were right that quality dropped, but the confirmed causes were bugs and a product default, not deliberate model degradation.
[0] <a href="https://status.claude.com/incidents/72f99lh1cj2c" rel="nofollow">https://status.claude.com/incidents/72f99lh1cj2c
[1] <a href="https://anthropic.com/engineering/a-postmortem-of-three-recent-issues" rel="nofollow">https://anthropic.com/engineering/a-postmortem-of-three-rece...
[2] <a href="https://texxr.com/handle/claudedevs" rel="nofollow">https://texxr.com/handle/claudedevs
source: <a href="https://claude.ai/share/4435bbcf-d6df-44a0-b1db-f08a11858bc2" rel="nofollow">https://claude.ai/share/4435bbcf-d6df-44a0-b1db-f08a11858bc2
what · · focus · HN ↗
There are no bugs, just happy little accidents.
consumer451 · · focus · HN ↗
Spooky23 · · focus · HN ↗
That just means they don’t reduce model quality for those reasons.
They didn’t mention other reason, for example, “Make more money”.
p-e-w · · focus · HN ↗
erinnh · · focus · HN ↗
p-e-w · · focus · HN ↗
Christ this forum has become intellectually dishonest.
jibalt · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
jibalt · · focus · HN ↗
> There are recorded cases of real regressions, but they're better characterised as incidents, not nerfs
erinnh · · focus · HN ↗
So I found this incident, as they called it, to still be relevant and why benchmarks such as the OP are useful.
prodigycorp · · focus · HN ↗
It’s not just ant. There are so many small knobs that providers can claim isn’t nerfing but “load management” or “improving user experience”. One example from OpenAI is reducing juice to reduce time to first token.
winwang · · focus · HN ↗
computerex · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
QwenGlazer9000 · · focus · HN ↗
Unintentional tbf.
weird-eye-issue · · focus · HN ↗
ffsm8 · · focus · HN ↗
fwiw, i think almost all regressions are down to a/b testing in the harness by anthropic, but it is objectively indistinguishable beyond "the coding agent ceases to be usable" and i'm back to traditional coding for a few hours until its back to normal again
weird-eye-issue · · focus · HN ↗
It absolutely matters because something like Claude Code has no guarantee that there won't be changes between updates but a model pinned at the API version level that is getting enterprise traffic absolutely does have that guarantee and would be a much more widespread problem...
thephyber · · focus · HN ↗
The /r/antigravity SubReddit is full of users who very much notice bugs with the tool/agent. We should be thankful that Claude Code is pretty stable by comparison.
weedfroglozenge · · focus · HN ↗
Rapzid · · focus · HN ↗