Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

panny · 1791121753 · focus · HN ↗
I'm far less interested in how good a big expensive model is on hardware 99% of people can't afford and would rather see what runs best on a chromebook or mobile phone with 8GB of RAM.
1. throwawayffffas · 1791126502 · focus · HN ↗
  The 1% can afford to run these models without quantization.

Unofficial Hacker News client; not affiliated with Y Combinator.