Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

SuperV1234 · 1791126129 · focus · HN ↗
We're getting closer and closer to the day we can have an Opus-like model running locally. The dream!
1. stymaar · 1791127600 · focus · HN ↗
  It depends on what you mean by “Opus-like”, because if you mean “as strong as Opus 4.6 for agentic coding” then Qwen3.8-27B has been there for the past two months.
  But if you mean “as strong as current-gen Opus” then it's probably never gonna happen, but it doesn't really matter since we're long into the diminishing returns for performance improvements: I haven't notice any major leap between 4.6 and 5.5 in my daily usage, and I'm convinced that with a fact enough piecs of hardware I would be using local Qwen exclusively (I'm using it daily but only at night for long running tasks because they take much more time than Opus due to the compounding effects of my slow GPU and Qwen's verbosity).
  1. system2 · 1791129512 · focus · HN ↗
    I would be forever happy with Opus 4.8.
    bitexploder · 1791130983 · focus · HN ↗
    Flash next is /really/ close. It is at parity with 4.7 as far as I can tell and basically where Opus 4.8 was. It is a genuinely good model. And I run it at home on $1500 of GPU at 125 t/s :)
    gruturo · 1791132390 · focus · HN ↗
    Seconding this. Flash-next (and let's not forget, it's a PREVIEW of the 4 architecture - with the "real" 4 rumored coming later this month) is the first model I can run on reasonable hardware (2 thoroughly obsolete P100s off ebay at ~$100 each plus the RAM I could scavenge from other PCs at home) at a reasonable speed (22, with GPUs in layer-split due to llama-cpp's limitation on qwen4-exp arch, and no MTP. Strata could double these numbers).
    It's.... the real thing, for the first time. If you cut me off cloud models today, I would get plenty of utility out of this thing.
    (Others may have had the same feeling from GLM5.3 or Deepseek 4.1 flash but I never had a chance of running those.)

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.