Thread

Qwen3.8 Max now ranked as the best overall model by agentic index

403 points · 261 comments · apitman

drnick1 · 1786043769 · focus · HN ↗
Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index. Since you can't presumably run this on your own hardware given the model size and hence gain other things like privacy, I don't see any reason to move away from GPT at this rate.
1. eli · 1786043818 · focus · HN ↗
  It's not enough that it's better?
  Many providers will host it and will compete on price. It also can't easily be taken away because one company (or one government) decides they don't want it around any more. People can fine-tune it for particular workloads.
  1. drnick1 · 1786044982 · focus · HN ↗
    > It's not enough that it's better?
    It's barely better, and barely cheaper, not really enough to challenge the status quo IMO. Half the price for basically the same performance would be a much stronger value proposition.
    ux266478 · 1786046996 · focus · HN ↗
    What status quo? Just look at Openrouter's rankings: <a href="https://openrouter.ai/rankings" rel="nofollow">https://openrouter.ai/rankings
    Things change radically month to month. Nobody is remotely close to capturing the market or having any kind of stability over time. People move around quite a lot, often to sidegrade within a generation. Just playing fly on the wall with discourse would be enough to tell you all of this, even without the data to back it up.
    eli · 1786048158 · focus · HN ↗
    That's got a significant selection bias. Claude and ChatGPT and Gemini and other subs do not go through openrouter.
    ux266478 · 1786049699 · focus · HN ↗
    Not really, because that's not a unique aspect of any of those. It's true of all subscription services (that I'm aware of), as well as all of the free models. The selection bias primarily will be against models which be an outlier in the difference between openrouter users and total users, which is a much harder position to argue for any given company except for maybe Twitter.
    You can argue there's a selection bias that openrouter users are less likely to display model loyalty, but it would still be a visible confounding factor if it was a statistically significant behavior. And it's not. Nor is there a visibly meaningful indication that people don't sidegrade between models. With every single data set, you're going to see that. You're also going to see it reflected in discourse, as I mentioned. Fact of the matter is there isn't a status quo in AI any more than there's a status quo in cars.
    SwellJoe · 1786049960 · focus · HN ↗
    If anybody has, it's DeepSeek. But, with the promised price hikes, I'm sure that'll change. I'm guessing they're raising prices not because they're not making a profit at those prices, but because they're running into capacity problems and need to slow down until they've got more or risk providing poor service. For now DeepSeek Flash is the best deal going for API usage and its popularity makes sense.
    Also, OpenRouter misses most of the usage of the US models, as most people are getting those from the vendor directly via subscriptions.
  2. Art9681 · 1786045784 · focus · HN ↗
    They cherrypicked benchmarks. The ONE weighed benchmark where is beats Opus5 by 0.1 points is what was linked because that's how propaganda works. The Agentic Index that includes the full benchmark suite has it in 5th place.
    Might as well use gpt-sol.
    iAMkenough · 1786050305 · focus · HN ↗
    The whole industry cherry picks benchmarks.
    I stopped paying attention to self-published benchmarks when Apple started including those non-sensical performance graphs with "relative performance" as a vertical axis.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.