Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

ContinuityLab · 1791173095 · focus · HN ↗
Achieving 100 tokens/sec on consumer hardware for a massive model like this is an incredible engineering feat. Pushing high-throughput local inference forward is crucial for decentralized edge stacks

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.