‹ BackHN Continuity

Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

  1. cjdell · · focus · HN ↗
    This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck.

    For the Nix inclined: <a href="https:&#x2F;&#x2F;github.com&#x2F;cjdell&#x2F;nixos-config&#x2F;blob&#x2F;main&#x2F;hosts&#x2F;zen3-nixos&#x2F;ai&#x2F;strata.nix" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;cjdell&#x2F;nixos-config&#x2F;blob&#x2F;main&#x2F;hosts&#x2F;zen3-...

    Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t&#x2F;s). Fast enough to leave it with a prompt before I go to bed and wake up to a solution.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.