Thread

1. Gareth321 · 1790064431 · focus · HN ↗

   OpenAI usage limits have been severely cut, and intelligence appears to be markedly declining, so I'm going to start trying these Chinese models seriously now. I don't mind if it takes longer. I just need the intelligence to predictably work the same way from day to day.

   1. phoghed · 1790073913 · focus · HN ↗

      > and intelligence appears to be markedly declining

      Serious question: does anyone have evidence of this?

      It’s something that’s constantly asserted, and has been since 2023. Every time someone posts a site that tries to track this though, I look at it and it’s just a flat line.

      1. conception · 1790081511 · focus · HN ↗

         <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;

         By and large they don’t. I have seen this drop a few times, eg before fable came out opus dropped a lot probably due to less compute available.

         My guess is it’s a combination of getting used to the new cliff models fall off on and forgetting that model performance drops significantly when context fills up.

         So new model comes out, people try it and it’s amazing on a task or two. Then they start using it, context window fills up and it gets a lot worse.

Unofficial Hacker News client; not affiliated with Y Combinator.