‹ BackHN Continuity

Thread

Mercury 2.5 LLM hits 770 tokens per second

151 points · 92 comments · Retro_Dev

  1. walrus01 · · focus · HN ↗
    Pricing at $0.25 and $0.75 already puts its cost well above reasonably reputable inference providers for deepseek v4 flash or qwen 3.8-flash-next or similar class of open weight LLMs that fit in under 170GB of RAM, so I don't see the point. I think this is probably also stupider than laguna s 2.1 which can also be very cheap to serve.
    1. RussianCow · · focus · HN ↗
      The point is the speed.
      1. _aavaa_ · · focus · HN ↗
        If the model provides me with bad results because it's dumb, I don't care how quickly it does it.
        1. jgalt212 · · focus · HN ↗
          fast results that you need to verify are better than slow (allegedly better) results that you still need to verify. REPL vs batch.
          1. RussianCow · · focus · HN ↗
            Depends on how many times you need to iterate to get the result you want. If you need to run the fast model 5 times to get the results you need, compared to 1-2 times for a smarter but slower model, you've just eroded any advantage that the speed gave you.

            So, as always: it depends on the use case.

            1. jgalt212 · · focus · HN ↗
              fair enough, but in my experience fast models seem to be 5-10X faster than slow models.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.