Thread

1. the_duke · 1790703020 · focus · HN ↗

   The GPT 6 release was ... not great.

   Sol 6 was so bad that I switched over to Opus 5.5 exclusively.

   Huge regression compared to Sol 5.6, often doing really dumb things. Same for Luna.

   Even Astra is very unreliable for coding. Brilliant for vision, sometimes just great, but it also often does very stupid things.

   I'm a bit sour on OpenAI right now and skeptical that 6.1 will be much different.

   (Note: this is after preferring and shilling Codex/OpenAI models for the last half year)

   1. nxc18 · 1790703303 · focus · HN ↗

      How does this jive with the exponential growth claims? Theoretically sol models are better than the 4 series models I was using at the beginning of the year, but in practice the results don’t seem to be much better. They always nerf the models over the course of the release so it _looks_ like the next version is better but I haven’t seen actual capability growth since ~January, and I’m pretty sure that was all tooling/harness improvements.

      1. user43928 · 1790706203 · focus · HN ↗

         They never nerfed any model after release.

         The lackluster GPT-6 Sol has been superseded by this apparently much better 6.1 Sol within a week.

         I am very skeptical of claims that old models weren't much worse. Compare this to February's GPT-5.3.

         1. nxc18 · 1790708083 · focus · HN ↗

            I am comparing to GPT-5.3 and 5.2, and I perceive that things have not been noticeably better since then. I also know that I can predict new model releases with high accuracy when my coding agent suddenly becomes regard-level at following instructions and completing simple tasks. This is how I knew 6.0 was about to be released - 5.6 suddenly got unusably bad.

            I could point out that I said 6.0 seemed good only in comparison to nerfed 5.6 - people would say I’m just a RSI denialist - but now it is in vogue to accept that 6.0 sucked now that 6.1 is out.

            1. holbrad · 1790723843 · focus · HN ↗

               I've heard very little positive press around Sol 6, with a ton of people preferring Sol 5.6 instead.

               I haven't used it much yet, but I have much higher hopes for Sol 6.1, as it seems to be based off of a completely different base, it's not just a tune.

            2. sebzim4500 · 1790724243 · focus · HN ↗

               I never used sol-6.0 in part because everyone kept talking about how bad it was.

               Astra is clearly far better than anything prior though, so I'm not sure what you mean really.

         2. Hammershaft · 1790794699 · focus · HN ↗

            <a href="https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;marginlab.ai&#x2F;trackers&#x2F;codex&#x2F;

            Am I misinterpreting this, or did OpenAI clearly nerf GPT-6 Sol on the 23rd.

            1. user43928 · 1790860014 · focus · HN ↗

               It released on that date, the data before is a different model.

               The chart shows GPT-5.6 Sol and a surprisingly large drop in performance when the switched it over to GPT-6 Sol.

Unofficial Hacker News client; not affiliated with Y Combinator.