"Nerf"ing models isn't real in the vast majority of reported cases. Benchmarks like this or the 100 other "let's see if nerfing is real" copies would have shown it by now if it was.
I made a graphic to explain why people feel like the models get nerfed:
The idea is that new models can handle up to a certain level of complexity, at which point they fall apart. Every new model can handle more complexity, so there's a wonderful time upon release when you feel like you can do anything, only for you to hit the complexity ceiling a few days later when you saturate it. Rinse and repeat for the next model.
That’s right, and it’s been like this ever since we stopped programming in assembly language. Programmers’ brains used to grow manly and strong on a strict diet of manual memory management and custom stack frame handling. Once we transitioned to soft, weak modern languages like C it’s been all downhill.
johnfn · · focus · HN ↗
I made a graphic to explain why people feel like the models get nerfed:
<a href="https://x.com/thesilenceturns/status/2103551351825543610" rel="nofollow">https://x.com/thesilenceturns/status/2103551351825543610
The idea is that new models can handle up to a certain level of complexity, at which point they fall apart. Every new model can handle more complexity, so there's a wonderful time upon release when you feel like you can do anything, only for you to hit the complexity ceiling a few days later when you saturate it. Rinse and repeat for the next model.
physicallyIllfr · · focus · HN ↗
[dead]
mwigdahl · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
physicallyIllfr · · focus · HN ↗
[dead]
kdkdjcjejxowjdj · · focus · HN ↗
[dead]
physicallyIllfr · · focus · HN ↗
[dead]
cheevly · · focus · HN ↗
physicallyIllfr · · focus · HN ↗
[dead]
[deleted] · · focus · HN ↗
[deleted]
cindyllm · · focus · HN ↗
[dead]