Thread

1. the_duke · 1790703020 · focus · HN ↗

   The GPT 6 release was ... not great.

   Sol 6 was so bad that I switched over to Opus 5.5 exclusively.

   Huge regression compared to Sol 5.6, often doing really dumb things. Same for Luna.

   Even Astra is very unreliable for coding. Brilliant for vision, sometimes just great, but it also often does very stupid things.

   I'm a bit sour on OpenAI right now and skeptical that 6.1 will be much different.

   (Note: this is after preferring and shilling Codex/OpenAI models for the last half year)

   1. nxc18 · 1790703303 · focus · HN ↗

      How does this jive with the exponential growth claims? Theoretically sol models are better than the 4 series models I was using at the beginning of the year, but in practice the results don’t seem to be much better. They always nerf the models over the course of the release so it _looks_ like the next version is better but I haven’t seen actual capability growth since ~January, and I’m pretty sure that was all tooling/harness improvements.

      1. user43928 · 1790706203 · focus · HN ↗

         They never nerfed any model after release.

         The lackluster GPT-6 Sol has been superseded by this apparently much better 6.1 Sol within a week.

         I am very skeptical of claims that old models weren't much worse. Compare this to February's GPT-5.3.

         1. Hammershaft · 1790794699 · focus · HN ↗

            <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;

            Am I misinterpreting this, or did OpenAI clearly nerf GPT-6 Sol on the 23rd.

            1. user43928 · 1790860014 · focus · HN ↗

               It released on that date, the data before is a different model.

               The chart shows GPT-5.6 Sol and a surprisingly large drop in performance when the switched it over to GPT-6 Sol.

Unofficial Hacker News client; not affiliated with Y Combinator.