Thread

1. the_duke · 1790703020 · focus · HN ↗

   The GPT 6 release was ... not great.

   Sol 6 was so bad that I switched over to Opus 5.5 exclusively.

   Huge regression compared to Sol 5.6, often doing really dumb things. Same for Luna.

   Even Astra is very unreliable for coding. Brilliant for vision, sometimes just great, but it also often does very stupid things.

   I'm a bit sour on OpenAI right now and skeptical that 6.1 will be much different.

   (Note: this is after preferring and shilling Codex/OpenAI models for the last half year)

   1. jstummbillig · 1790703586 · focus · HN ↗

      Eh. What? Is this common sentiment?

      I mean Opus 5.5 is absolutely fantastic, unreasonably and unexpectedly so, but Astra was great and as far as I can tell SOTA until, when was it, 3 days ago, no?

      (Sol 6 idk, have not used it much for coding really. Seemed to work just fine when Astra used it in Codex as subagents.)

      1. the_duke · 1790704049 · focus · HN ↗

         On r/codex the sentiment seems to be quite wide-spread.

         1. phoghed · 1790796372 · focus · HN ↗

            <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;

            Codex itself seems to have a regression. You can see clearly the token use changing significantly coincides with a score drop

      2. nicce · 1790704257 · focus · HN ↗

         When GPT 6 Sol &amp; Luna were released, everything went down. I have been running Sol at max thinking and it is about the same as old Luna with max thinking, give or take. Sometimes feeling even dumber. I can&#x27;t trust it to do anything big alone anymore without babysitting.
      3. copperx · 1790704278 · focus · HN ↗

         Opus 5.5 is so good that I don&#x27;t want it to be replaced anytime soon. Stop training models, Anthropic, and just serve this thing without regressions for a year or three, can you?

         1. Marha01 · 1790705017 · focus · HN ↗

            They should etch it into an ASIC. The first model worthy of that honor.
      4. Eridrus · 1790705038 · focus · HN ↗

         Sol 6 definitely feels kind of dumb and worse than 5.6

         Astra seems better though.

         Showing one potentially saturated benchmark doesn&#x27;t necessarily fill me with a lot of confidence in the coding results.

      5. phoghed · 1790705560 · focus · HN ↗

         In my experience, no. There’s no way to know though. The whole conversation and industry are a combo of benchmaxing, faith, and mysticism.

         Since like last December I haven’t had any issues getting work done with whatever the latest Anthropic or OpenAI models at the time were. Tooling and models have only gotten better since then.

Unofficial Hacker News client; not affiliated with Y Combinator.