Thread

MiMo v2.6

1130 points · 483 comments · volf_

user43928 · 1790024175 · focus · HN ↗

I don't trust any of the benchmarks where Opus 5 surpasses Astra or Fable 5.1.

Maybe Terminal Bench 4.0 and ExploitGym are reasonable.

Terminal Bench 4.0

  GPT 6 Astra             59.6
  Claude Fable 5.1        55.1
  Claude Opus 5           49.0
  MiMo-V2.6-Pro           34.9
  MiMo-V2.6-Flash         28.8
  DeepSeek V4.1 Flash     26.8
  MiMo-V2.5-Pro            1.5

ExploitGym

  GPT 6 Astra             42.4
  Claude Fable 5.1        30.4
  Claude Opus 5           22.1
  MiMo-V2.6-Pro           17.8
  MiMo-V2.6-Flash          6.0
  MiMo-V2.5-Pro            0.1

DeepSWE v1.1

  DeepSeek V4.1 Flash     74.2
  Claude Opus 5           74.0
  GPT 6 Astra             74.0
  MiMo-V2.6-Pro           71.9
  Claude Fable 5          70.0
  MiMo-V2.6-Flash         67.9
  MiMo-V2.5-Pro           19.0

varispeed · 1790025817 · focus · HN ↗
They match my experience. Astra and Fable I rate below Sonnet. They are incredibly poor. They were excellent for a couple of days after release and then plummeted.
Maybe I am being routed to more quantised versions or less capable models with system prompt to fake Astra or Fable.
1. [deleted] · 1790027307 · focus · HN ↗
  [deleted]

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.