‹ BackHN Continuity

Thread

Livenerf: Has Opus 5.5 been nerfed yet?

922 points · 392 comments · bryan0

  1. jug · · focus · HN ↗
    We also have Nerf Bench:

    <a href="https:&#x2F;&#x2F;www.bridgebench.ai&#x2F;nerf-bench" rel="nofollow">https:&#x2F;&#x2F;www.bridgebench.ai&#x2F;nerf-bench

    They test it on launch day, then benchmark it against that. A deviation of above 10% is considered a change. They&#x27;re currently tracking Opus 5.5 and GPT-6 Astra.

    This bench famously detected a degradation of Opus 4.6 which Anthropic later blogged about. I personally think people sense nerfs more often than they happen and that it&#x27;s often about honeymoon effects.

    1. scrollop · · focus · HN ↗
      There&#x27;s also this one which has been around for a while

      <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;claude-code&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;claude-code&#x2F;

      1. sscaryterry · · focus · HN ↗
        The tracker for Codex resonates with me. Its thick as pig shit the last few days: <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;
        1. phoghed · · focus · HN ↗
          Seems like a harness change (maybe bug) rather than a model change, the input tokens dropped quite a bit right when the degradation happened.
        2. lxgr · · focus · HN ↗
          &gt; We use the latest available Codex release with GPT-6 Sol.

          This alone makes the benchmark unsound.

        3. shawabawa3 · · focus · HN ↗
          &gt; &quot;We are collecting a new GPT-6 Sol&#x2F;high baseline from runs beginning September 24, 2026. Degradation detection is paused.&quot;
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.