Thread

1. user43928 · 1790024175 · focus · HN ↗

   I don't trust any of the benchmarks where Opus 5 surpasses Astra or Fable 5.1.

   Maybe Terminal Bench 4.0 and ExploitGym are reasonable.

   Terminal Bench 4.0

     GPT 6 Astra             59.6
     Claude Fable 5.1        55.1
     Claude Opus 5           49.0
     MiMo-V2.6-Pro           34.9
     MiMo-V2.6-Flash         28.8
     DeepSeek V4.1 Flash     26.8
     MiMo-V2.5-Pro            1.5
   

   ExploitGym

     GPT 6 Astra             42.4
     Claude Fable 5.1        30.4
     Claude Opus 5           22.1
     MiMo-V2.6-Pro           17.8
     MiMo-V2.6-Flash          6.0
     MiMo-V2.5-Pro            0.1
   

   DeepSWE v1.1

     DeepSeek V4.1 Flash     74.2
     Claude Opus 5           74.0
     GPT 6 Astra             74.0
     MiMo-V2.6-Pro           71.9
     Claude Fable 5          70.0
     MiMo-V2.6-Flash         67.9
     MiMo-V2.5-Pro           19.0

   1. 3abiton · 1790077030 · focus · HN ↗

      We're past the one model fits them all kind of LLM. Most of the recent release actually regress on world knowledge for example, but optimize for something different: tool usage, thinking process, and agentic approach. And yes, in my own usage, some usecases Opus beats Fable.

Unofficial Hacker News client; not affiliated with Y Combinator.