Thread

Mercury 2.5 LLM hits 770 tokens per second

151 points · 92 comments · Retro_Dev

bearjaws · 1790205736 · focus · HN ↗
If you care about speed Cerebras gpt-oss-120b is 1400tk/s and "just as smart" in ranking.
I've used it on a few for fun projects and its decent but the speed is crazy to watch.
1. ford · 1790210663 · focus · HN ↗
  Also kimi 2.6 at 1000tps (as of may), though when we reached out they had a >12 month waitlist and minimum 7-8 figure annual token spend.
  [0] <a href="https://www.cerebras.ai/blog/cerebras-kimi-k2-Enterprise" rel="nofollow">https://www.cerebras.ai/blog/cerebras-kimi-k2-Enterprise
  1. walrus01 · 1790220213 · focus · HN ↗
    7-8 figures annual spend will buy a hell of a lot of capable local inference hardware you can own, though it won't be at the absurd token/s rate, you'll be able to run almost anything on it... And it'll still have a good residual resale value after 4 years the way things are going now.
    podocarp · 1790235370 · focus · HN ↗
    Feel like you could spend 6 figures building out a team and the rest renting compute for a whole year, and get the team to create a local inference solution with that kind of budget…

Unofficial Hacker News client; not affiliated with Y Combinator.