Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

snehesht · 1791118313 · focus · HN ↗
I tried it and it worked surprisingly well. On my machine (Nvidia 4090, 128GB DDR5, Ryzen 7950x3d) I'm getting 124 tokens per sec, thought to share it here.
<a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next" rel="nofollow">https://huggingface.co/Qwen/Qwen3.8-Flash-Next
1. proc0 · 1791119012 · focus · HN ↗
  Do you know how it compares to Qwen 3.8 27B? I really want to compare the distilled ones with harness versus the full MoE versions.
  1. incognito124 · 1791120233 · focus · HN ↗
    Qwen 3.8 flash next is way better than 27B. It's so good I dont even use claude anymore
    mickeyp · 1791120591 · focus · HN ↗
    I have not tried Flash Next yet; but 27B is a cracking, little model. It is the first small model that I, as someone with 30 years of experience, can finally say is good enough to hand off small and mid-sized tasks and expect a pretty good result.
    It is also a competent tool caller when quantised to NVFP4 for use with ninfer; my own harness only reports the occasional hiccup and it is only because the model will sometimes emit tool calling tokens in its reasoning loop.
    snehesht · 1791121364 · focus · HN ↗
    This is interesting, thanks. - <a href="https://github.com/Neroued/ninfer" rel="nofollow">https://github.com/Neroued/ninfer

Unofficial Hacker News client; not affiliated with Y Combinator.