‹ BackHN Continuity

Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

  1. fsiefken · · focus · HN ↗
    I wonder if a higher Qwen3.8-27b quant could beat or match these lower < 16/24/48/64G Qwen3.8-Flash Next quants given similar quality.

    What speed are you willing the sacrifice to debug/program for more complex jobs faster?

    Then there are also these quants; <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;IsValorum&#x2F;Qwen3.8-35B-A3B-Distill-MLX-APEX-NanoPlus-Abliterated" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;IsValorum&#x2F;Qwen3.8-35B-A3B-Distill-MLX...

    1. latentsea · · focus · HN ↗
      The benchmark indicates the IQ3_XXS quant beats 27B. I&#x27;ve switched to that now and am ditching 27B. Genuinely better results so far.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.