Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

panny · 1791121753 · focus · HN ↗
I'm far less interested in how good a big expensive model is on hardware 99% of people can't afford and would rather see what runs best on a chromebook or mobile phone with 8GB of RAM.
1. somenameforme · 1791123106 · focus · HN ↗
  The card in question here had an initial MSRP of $1600. It's been bumped up by the market, probably because it turned out it's nice for things like this, but it's hardly in the 99% can't afford domain, especially if you're using it to replace a never-ending rent at which point it will pay for itself very rapidly, especially for heavy LLM users.
  In any case, we've gone from requiring supercomputers, to requiring very high end computers, to requiring $1600 video cards. It's tracking the exact same path that image rendering systems took (which if you haven't been keeping up there, now run excellently on pretty much any plain old computer), and we'll probably be there within a couple of years if not much sooner.
  1. panny · 1791135802 · focus · HN ↗
    >The card in question here had an initial MSRP of $1600.
    63% of Americans can't come up with $400 in an emergency.
    <a href="https://www.investopedia.com/here-s-how-many-americans-can-t-afford-a-usd400-emergency-the-numbers-may-shock-you-11814788" rel="nofollow">https://www.investopedia.com/here-s-how-many-americans-can-t...
    The richest country in the world. Where all 50 states consume more than any other country in the world.
    <a href="https://x.com/cremieuxrecueil/status/2102889196000256219" rel="nofollow">https://x.com/cremieuxrecueil/status/2102889196000256219
    Can't come up with %25 of that in an emergency. (Even though the real price is something like 2-3x more than MSRP)
    It must be nice, up there where you are so incredibly disconnected from reality.
    somenameforme · 1791174093 · focus · HN ↗
    You misread your article, which really shouldn't pass a sniff test. 77% could cover the $400, with 14% of that being from people selling/credit. It also mentioned that 70% have enough saved away to cover all expenses for at least 3 months, including 15% that could do so by selling/credit. Also the US is far from the richest country in the world in terms of the population. We're 28th in terms of median wealth. [1] And that's nominal - measure it PPP and we'd be much lower. For the times they are a-changin'.
    [1] - <a href="https://en.wikipedia.org/wiki/List_of_countries_by_wealth_per_adult" rel="nofollow">https://en.wikipedia.org/wiki/List_of_countries_by_wealth_pe...

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.