Thread

1. kristofferR · 1790007505 · focus · HN ↗

   What's with the deceptive graph on top? Not including Astra can't have been an oversight, did the model compare poorly to it?

   1. Iolaum · 1790007716 · focus · HN ↗

      I wonder if that means that SpaceX evals show that they consider astra better than fable or that they hate Sam&co so much they don't want to show their stuff.

      1. Jcampuzano2 · 1790007920 · focus · HN ↗

         <a href="https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;our-decision-on-cursor-following-its-acquisition-by-spacex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;index&#x2F;our-decision-on-cursor-following-it...

         Its because of this. You can&#x27;t use Astra in Cursor, and cursorbench uses cursor as the harness. They can&#x27;t actually benchmark it using their harness hence why its not included.

         1. babelfish · 1790007968 · focus · HN ↗

            They have Astra in other benchmarks lower on the page. They just don&#x27;t want to show it winning

            1. Jcampuzano2 · 1790008063 · focus · HN ↗

               The chart is cursorbench though and they asked about the &quot;deceptive graph&quot;
         2. ryeguy · 1790010656 · focus · HN ↗

            They can benchmark it because you can use an openai api key with cursor. Astra is just not included in the cursor plan.

Unofficial Hacker News client; not affiliated with Y Combinator.